The Leitmotif of this Quarter's SWG Meeting: Validation versus Confirmation

The Kepler Science Team is scattered all over the US (excepting our two colleagues that help to run the Asteroseismology Consortium… from Denmark!).  Because of the distance, teleconferencing technology and platforms like webex for sharing computer desktops are as essential as the computers used to analyze the data.  But sometimes no technology can replace a room full of brainpower all attacking the same problem or contemplating the same question.  Maybe it’s just the smell of coffee and donuts or the inspiration derived from watching your colleague’s facial expressions.  The reality is that more gets done when we’re all in the same room.  So a few times each year, the team members pack up their laptops, hop on planes, and gather at the home institution of whomever happens to be taking his or her turn as the designated host.  This week, the team gathered at NASA Ames Research Center in Mountain View, California.  We call these our “Science Working Group” meetings,  more lovingly referred to as SWG (“swig”) meetings.
The first order of business is to hear status reports.  How is the instrument performing?  What does the budget look like going forward (as if we don’t already know the answer to that question)? What can we expect from future software releases and when can we expect them?  We learned that the spacecraft has been free of safe mode events for almost 8 months now.  This is very good news.  With more than a year of data under our belt, we’ve learned a lot about the performance of the instrument.  We see seasonal changes in the focus, for example, that are driven by the amount of sunlight striking the backside of the spacecraft.  Varying sunlight levels induce slow temperature changes that affect the focal length of the photometer.  The amplitude of the variation and its effect on the data is well within our tolerances.  There is no indication that we’ll lose any more modules.  
The various working groups then share what they’ve accomplished over the last months.  We heard a report, for example, from the Eclipsing Binary Working Group.  Their goal is to produce an updated binary catalog using four quarters of data within the next 6 months.  We are counting on them to tell us, statistically, how many binary stars of a given period and apparent brightness we expect to see — a number which helps us to understand how often we expect to find astrophysical signals in our data that mimic planet transits.  We must understand this number well because most of our planet candidates will NOT be confirmed in the traditional sense — by way of high precision radial velocity  measurements that give us the mass of planet.  The catalog of planets that Kepler produces will be probabilistic in that every candidate will have an associated likelihood that the planet interpretation is correct.  This is new territory we’re charting.  It will lead to a new type of discourse on exoplanets in our galaxy.  Someone wondered out loud how the public will react to the announcement of a new world… “with a 95% likelihood”.  As he asks the question, I’m picturing the fine print on my wireless phone bill, and I’m hearing the guy at the end of the commercial who uninteligibly rattles off the litany of horrible side-effects that said medication can produce.  That’s not a path we want to take.  We’ll be asking the public to contemplate the Universe in a statistical sense, and it might get ugly.
The team devoted a good fraction of its time discussing how these “likelihoods” will be computed.  There are challenges ahead.  A small committee will meet with the folks over at the Ames supercomputing facility to talk about ways to speed up the number crunching that will be required.  We’ll likely have to revisit our priorities for utilizing highly competitive time on telescopes like Keck, WIYN, and the MMT.  These discussion were kicked off by a talk that Jack Lissauer gave.  The title of his talk became the leitmotif of the meeting: “Validation versus Confirmation”.  We realize that there are issues we are only beginning to fully appreciate.  This does not discourage us.  To the contrary, it motivates us.  This is the scientific method at its best.  As we each head back to our home institutions, we know what to do.
Natalie Batalha
Deputy Science Team Lead, San Jose State University
For two days, members of the Kepler Science Working Group filled the air in the meeting room with descriptions of their work on the pixels, the pipelines, and the planets.   What impressed me most was the long hours and technical expertise everyone was contributing toward the still-absurd goal of finding and characterizing the first Earth-sized planets.
The folks on the team are critically examining and improving the instruments, software, and performance of every aspect of the Kepler mission, down to truly invisible minutia.   Some members are devoted to studying and minimizing electronics noise in the 94 megapixel light detector.  Others are optimizing spacecraft stability.  Some are dedicated to improving Kepler’s phenomenal photometric performance, working toward Kepler’s amazing goal of 20 parts per million.   And some are observing Kepler “Objects of Interest” at ground-based telescopes by night, attending the meeting by day, and then going back to the telescope each night, all to verify and measure the planet candidates Kepler has identified.   Just archiving all the information about the hundreds of “Kepler Objects of Interest” is a monumental task.
In the end, I felt fortunate to be a part of the camaraderie.  The meeting room was filled with a warmth of shared purpose.  Everyone is working with mutual appreciation of the enormous challenges and personal sacrifices needed to get the job done. And everyone was swept along by the unprecedented hope of placing our home planet in its proper cosmic context.
Geoff Marcy
Co-Investigator, UC Berkeley

It was exciting to see how much work has been accomplished and overwhelming to see how much work still needs to be done.  It was surpising to see so many new faces – the number of collaborators and contributors continues to grow – which is a very good thing.  On Wednesday, the day after the science team meeting, there was an all-day session to discuss the follow-up observations of individual candidates.  The participants  rolled up their sleeves and worked extremely hard all day.  By mid-afternoon it looked like the group had been locked up for a week – there were computer cords, styrofoam cups, jackets and sweaters, half-eaten doughnuts, empty soft-drink cans, and miscellaneous debris strewn everywhere, as people had not even left to get lunch.  The single-mindedness, dedication, and teamwork demonstrated by the scientists as they struggled to interpret the myriad observations on a plethora of candidate planets was a joy to behold.

Michael Haas
Science Office Director, NASA Ames Research Center


As a member of the Follow-up Observing Team, my days(and nights) are focused on collecting astronomical data that contributes to our understanding of the potential planets that Kepler finds. Since my work is focused on extending what Kepler finds, I forget that dozens of people are working on every detail of the Kepler project. Nearly all of the people at the SWG meeting need to complete their work, before I can do my own. From satellite communication, to public outreach, to nitty gritty pixel-level analysis, the amount of work that goes into Kepler is absolutely amazing.
Watching all the entire Kepler picture be painted was remarkable.
Howard Isaacson
Kepler Collaborator, UC Berkeley

Kepler Memorabilia – Original Kepler Star Plate

And now, for a bit of Kepler history…

Over the holidays at the end of 1992, Lloyd Robinson, the CCD guru at UCO Lick in those days, had just retired and offered to set up a CCD system in the clean room in Santa Cruz to see what could be learned.  This star plate was a key component in that original setup.  
Original Kepler Star Plate

Original Kepler Star Plate
This star plate is an important Kepler relic. It was used in the first laboratory experiments to determine whether CCDs could produce very precise differential photometry.
When Jon Jenkins (Co-Investigator and Analysis Lead for Kepler) first set eyes on this plate in June, 2010, he recognized it immediately, even though he had never seen it before. This was because one of his first tasks for the mission was the analysis of an untold number of images from a variety of tests using this plate. These tests were invaluable in discovering all sorts of ways for test setups to go wrong, from sagging light bulb filaments to liquid nitrogen fills. Based on what was learned, more sophisticated laboratory test systems were developed using a variety of other star plates. This painstaking laboratory work culminated in the very successful Kepler Test Demonstration (KTD), which was instrumental in selling the Kepler Mission to NASA.