Adieu from Palmdale, CA, for now, until returning for SOFIA commissioning flights


We completed night 2 of line ops for the mid-IR cameraFORCAST on SOFIA. Our tests included script validation of the main observationmodes for the imaging and spectroscopic (grism) channels, optimization ofdetector bias, and exploring a new way to improve the flat fields for theimaging modes. During part of the night, we were slowed down by debugging ofscripts, an activity we were glad we found issues with now, rather than inflight. As a result of the delays, not all the planned tests were completed,but May 28th is the next line ops for this instrument.

We did manage to get our first grism spectral tests in, butmore testing remains for May 28th where my colleagues will take over the testing. I will be on call for data analysis.

I head back to my home in Boulder, CO, today, after a powernap. I will be returning to experience a flight on SOFIA on Jun 11th.Until then another line ops is scheduled for May 28th, followed bycommissioning flights on May 30, June 4, June 6, and June 11th.  I’ll be on call for dataanalysis and wishing my colleagues clear skies, good script runs, and completion of the readiness tests. SOFIA soon enters formal science flights at the end of June, and we want to give the larger community a high performing instrument with an observing strategy to optimize time and signal-to-noise.


SOFIA on the runway at NASA’s DOAF, Palmdale, CA during FORCAST line ops May 2013.

Who's Sitting in the Driving Seat when you Fly the Infrared Friendly Skies?


Note: The pictures in this blog are taken from a recent line operations (when SOFIA is not in flight, but being operated on the runway) on May 23, 2013. As the program is in its iterative operations phase, these pictures capture the inside of SOFIA on this date. There will be additional preparations to enable the required safe readiness for the flights, the first set for May 30th.

At one level SOFIA is quite simple: you want to point the telescope at for target, hold there on the instrument sensor array for a set amount of time, maybe repeat the observation to allow for better signal-to-noise, and then move on to the next target. At another level, SOFIA is quite complicated as you have a moving observatory (in several degrees of freedom, i.e., forward, left, right) that is trying to target, “peak up” and stay on target for several minutes by which the observatory and/or the target has moved enough.

Mission Director: He/she is responsible for ensuring the flight meets the success criteria and is safe. They run the readiness reviews and summary the milestones for the flight campaigns. They also make decisions if needed to deviate from plan. They keep track of how well we are executing the plan.

The Flight Planner: On the plane he/she sits next to the Mission Director. Lots of work is done ahead of the actual flights to map out an optimized series of “flight legs” to maximize the time spent on target. As the telescope has a fixed position within the aircraft (aft-port), a leg towards the West (in the Northern Hemisphere) means the sky target is in the south; a leg towards the North means the sky target is in the west, etc. When SOFIA flies in the Southern Hemisphere, this gets reversed.

More information about SOFIA Flight Legs can befound at SOFIA Flight Plans
Description of SOFIA console layouts

The Telescope Operators. Sitting close to the telescope, they monitor the telescope set-up and operations and real-time interfaces with the science instrument. Each science instrument will have different requests for the telescope assembly. They also perform the Line Of Sight rewinds periodically. Normally this is scoped out in advance but this is envisioned to be a manual operation performed with agreement the Instrument Scientist’s okay. For example, you don’t want to interrupted an observation, so you are watching the clock and the angles in real time to know when this activity is needed.

More information about SOFIA Line of Sight can befound at SOFIA Line Of Sight Rewind

Instrument Scientist. Each science team will have an Instrument Scientist who knows the “ins and out” of the instrument and the subtleties of changes to observations or techniques. He/she is in constant communication with the Telescope Operator as well as the Principal Investigator of the observations. Science Instrument observations are run mainly by pre-written and pre-tested scripts, but sometimes there are some manual observations that the Instrument Scientist can execute.

The many roles to complete a SOFIA Science Flight

There is also a lead for the MCCS, the Mission Controls and Communication System, to supervise the performance of that critical subsystem. This software controls the communication between the science instrument and telescope as well as all the archiving of any data taken during the line ops or flight.


There is the Science Team who is on board for the in-real-time data reduction and assessment, in case an observation needs to be redone again or done differently,  a principal investigator who decides the priorities of the science observations for the flight and directs the science team and  two pilots and a flight engineer for flying the aircraft.

Finally, there is space allocated for ride-along teachers and other guests who can participate in SOFIA science.
 
To learn more about the SOFIA Airborne Ambassador program check out SOFIA Airborne Ambassadors Program

'To Chop, Nod, or not Chop, Nod. That is the question.' SOFIA FORCAST May 23, 2013, line-ops.


I am out here in Palmdale, CA, not for a SOFIA flight (yes Iknow that’s where most people’s interest peaks) but for a critical step calledline-ops, or operations on the flight line. Essentially we are going throughexactly what we plan to while the plane is at altitude and work on end-to-enddata testing, assessing observation timing, and communication, both among thedifferent people needed to complete the observation and also between we humansand the highly complex software subsystems.

At 2130h May 22, crew briefing. We covered the mainreadiness topics: Weather (winds, humidity), Required Personnel, AircraftStatus and Configuration (System Engineer reported out), Telescope Status,Mission Systems Status (Flight Systems reported out), Operational Timeline(roll out, people on, telescope door open, telescope door closed, people off,roll back to hangar), Mission Rules (don’t connect laptops to the internalsystem and wireless at same time, bring drinks in closed containers, getpermission before entering roped off areas, etc.), Safety & EmergencyProcedures (exit doorway locations, footwear required), and Test Summary.

Being on SOFIA is not like flying on a normal 747 jet. Ihope from the various photos in this blog entry and others, you’ll see it’s got“other things” like computer racks, a whole data collection and archivingserver farm on board (the MCCS), conference tables, and various electricalpanels needing access for maintenance or operation. It’s got airline seats(with the normal seatbelts) for takeoff and landing and places to store your laptopbags, but the similarities end there.

So last night we got through some key tests. We did a pupilcheck (to optimize alignment of the FORCAST instrument to the telescope). Nextwere a series of inspections of the telescope boresight (telescope centered ona star) and how that appeared on all the imaging (all filters) and spectroscopy(for all grism and slit combinations) modes. We learned we had a systematicoffset in our slits, but we updated the .ini file to address this. Then we didsome testing of the basic modes. We tested chop-nod-dither in the SIRF and ERFcoordinates. SIRF=Science Instrument Reference Frame (rows & columns on thedetector array). ERF=Equatorial Reference Frame (RA/DEC on the sky). There isalso a third coordinate system, the TARF=Telescope Assembly Reference Frame (elevation,x-elevationangles). Yes, astronomers love their coordinate systems.

Below is a photo of one of the chop-nod tests, on a bright targetstar. It’s chop-nod-match mode. Left is the Science Instrument Console withquick look software showing a reduced subtracted image (you see the positiveand negative star images). The right image shows a series of display for thetelescope guide camera and telescope display.

Examples of short throw chop-nod testing SOFIA FORCAST

With the remaining hours for this night, we started probingthe space of the chopping throw vs. angle. Below is an example of a large chopthat was bumping up against a hardstop of the secondary, so we spent the restof the night investigating that issue. The scale bar on the lower left of thatguide star camera image is 1 arc minute.

Example large chop testing SOFIA FORCAST

The telescope door was closed at 0500h. Sunrise was at0545h. We’ll regroup later tonight to address the series of tests for tonight.There will be a crew briefing at 2130h to assess readiness for tonight.

Oh, surprise to me, we had internet on lineops, soI was tweeting away in near-real time we did our testing and I also got someIDL coding done for the pipeline end-2-end tests.

Stairway to the stars. Climbing aboard the SOFIA Airborne Telescope


We got the “go” to proceed with line ops. SOFIA, a 747SP,was towed out of its hangar onto a side-runway, and away from any air traffic. Theheading is 130.5. This is important as it tells us what view angles areavailable from the telescope. The telescope looks out the aft-port side of theaircraft. (Aft=back of wing, Port=left side, when viewed from the back, facingthe front). So at this heading, we are looking at the N-E portion of the sky.Our calibration targets include TDra, NSV25184, RUCyg, muCep, and TCass, allpretty bright stars.

We walked out from the hangar to the craft and can come/gofrom the craft during the night. Of course, this is not what will be likeduring the flight. Below is a picture of our ingress/egress path on the plane, 

a “true stairway to the stars.”  I learned that for the flights, we would do a similar activity, meaning we donot board within the hangar, but board after the craft has been towed out tothe runway.

Using the waiting time wisely to make the best use of the remaining ops ahead.


Line ops last night were cancelled due to a “no-go” by thetelescope assembly subsystem. A problem had been found that could not enableobservations tonight. It was a call the science team did not want to hear, butit was the right call. This exercised the reason why there is a “readinessreview” before going out to execute a complex activity. A plan was put in placefor the 1st shift when they get in at 7am (0700h) today (Wed) to address theproblem and report back during the day. If all goes well, a crew-briefing willbe scheduled again at 2130h tonight and we can resume lineops at 2300h.

If we were observing using a ground-based telescope, wewatch the weather. A seasoned ground-based observer watches the humidity. Youcan often get obsessed looking at trends in pressure, temperature, etc. It’simportant as you may need to replan your allotted observation time if you losea night  (or nights) to the weather-gods.When I assisted with a balloon launch last summer at Ft. Sumner, NM, we’dgather daily to address the winds. Winds were most stable at dawn so we’d haveour “crew briefing” at 3 or 4am with readiness to roll out at 5am with the hopeto launch in the next hour or so (it would take nearly an hour to do the roll-outof the balloon and the He fill). Yes, sometimes the call would be made at 3amfor a “no-go” or even as late as right before the fill. And then you roll backthe balloon to the hangar. Last Sept, we launched on the 3rd attempt. All rocket launches also watch the weather and have various sub-system “go/no-go” checks.

SOFIA ops are not so different from those other examples.

So, we replan again. We have three remaining nights left inthe schedule, two this week and one contingency night next week, which nowseems to be required. Also, we’ve started looking at the flights scheduled fornext week, to see what tests planned in flight would supersede the line opstests to allow to compress our “line ops” schedule. Now, this is a calculatedrisk since the purpose of line ops is to test the system end-to-end beforeflight. So essentially you want to run the key components you plan to test inflight on the ground first.

What are line ops anyway? It’s not as “dramatic” as theactual flight, but it serves very important purposes to follow our observationplan end-to-end, address timing issues, and most importantly, communication betweenpeople and communication between people & machines. The plane is towed outon the runway to a viewing position safe from any active runway traffic, andpreferably in a location far from buildings or lights to obstruct viewing angles.We operate on plane-provided power. We command the telescope door to open,configure the telescope, check it out, power the science instrument, and startrunning through a series of discrete tests, some of which are to be run exactlyon the flights, and other diagnostic tests that are needed that would otherwisetake up the valuable flight time.

One of the tests we want to do is test the “nod” function ofthe telescope and how the data sets we collect affect our observing strategyoptimization (ahem, improve signal to noise). In mid-IR astronomy, the skybackground is “brighter” than our targets. In fact, we often cannot see ourtargets in the original raw data until we do a “background subtraction.” So weuse the telescope’s secondary mirror to “chop” a source back & forth (as itwould appear on our detector) at a fast rate. And then we would command thetelescope to “nod” to a different part of the sky. And repeat the process of“chopping” and “nodding” over a pre-planned orientation, both “throw distance”and “angle.”

You can read more about Chopping at Nodding at Why Chopping & Nodding is needed for SOFIA FORCAST Observations

An example taken from PDF on Signal to Noise Improvement by Chop/Nods sums it up nicely.

So we’ll be exercising things like this during the line ops,exploring the same technique for different roll angles because when it comes toyour science target which can be anywhere in the sky, we’d like to understandthe system performance and, if any, limitations.

We have other tests planned like assessing the detector biasperformance, looking at flexure of our alignment, particular for our grism modewhere we have narrow slits, optimizing a new flat field technique, and runningthrough the science scripts to checking for timing and fix any commandingerrors.

So fingers crossed, we will get on sky tonight, on the tarmacat Palmdale, CA. The skies have been clear the last two nights, so we theweather gods have been kind. We now need the electrical-power-subsystem gods tobe kind.

Science enabled by the platforms of Air & Space


I’m out here at NASA Dryden’s Aircraft Operations Facility,the DAOF, to support line operations for the Stratospheric Observatory forInfrared Astronomy, SOFIA. I’m normally a spacecraft science instrumentbuilder, having previously tested detectors for astronomy space telescopes Spitzerand JWST and building, testing and operating a 10 instrument payload for LCROSSthat impacted the moon in 2009 detecting water within a permanently shadowedcrater. And since 2011, I am working instrument calibration operations for theen-flight probe to Pluto, New Horizons.

Thus, SOFIA, being an aircraft, is a very different experiencefor me, coming from the spacecraft side of the house.

Sitting in the DAOF with SOFIA are some of the world’spremiere aircraft used for Earth Science observations, measuring in-situmolecules in our planet’s atmosphere, capitalizing on a mobile platform thatcan go monitor fires, or survey ice sheets at the poles, or observe transientphenomena like meteor showers or spacecraft or space-sample return capsules.

Check out this amazing suite of aircraft and theirobjectives at NASA’s Airborne Science Program:

NASA’s Airborne Science Program

Tonight we roll out ~8pm local time for first night of lineops from the 11pm-5am shift. I’m very eager to experience this important prep-activityfor SOFIA commissioning science flights which start next week.

More information about SOFIA's unique science can be found at NASA SOFIA Web Page

Firm Flexibility


Tonight’s line operations were cancelled due to open issuesrecertifying work on reworked parts of the telescope assembly (TA) powersubsystem. There are no show-stoppers, just the need for more time for testingand integration. Progress continues to be made. The cautious step was to makethe decision to start line ops tomorrow, and there is a contingency day nextweek to make up time if needed. The schedule for the remaining three nights ofline ops will remain tight, but there is a plan. Creative re-ordering of taskswill be the “philosophy” these next three days. Having worked operations on twospace missions, I can say that operations of any craft, air or space, is askill of “firm flexibility.”

This evening, I experienced a Technical Readiness Review(TRR). This consisted of getting all the leads around a table and walkingthrough the status of each subsystem, who is needed where and when, what typesof testing will be done during the next few days, and when the daily crewbriefings will be held. Also addressed were questions posed by the visitingscience team to the operations team, to fill in some gaps. Today was the firsttime the group had re-assembled since the last line & flight ops, which forthe FORCAST instrument, had been back in March. Since then, two otherinstruments (HIPO/FLITECAM and GREAT) had been installed, tested, and removed,and there have been software upgrades to both the telescope and telescope toscience instrument communications. This phase of operations is pretty complex,folding in highly dynamic items that may seem be changing a lot, but it’sactually normal. And the job of operations is to keep to schedule while stillachieving the tasks. Sometimes the path is different from the exact originalconcept, but if the goals are met, it was a successful journey. At tomorrow’screw briefing at 2130h, open items from today’s TRR will be addressed andclosed before line ops begins, set for 2300h-0500h.

I’m still a bit on the sidelines, watching and learning fromthe experienced SOFIA observers who have worked with SOFIA operations before.During a lull this afternoon, I got a glimpse into the AORs, or AstronomicalObservation Requests, which is how an end-user communicates her requests toenable an observing plan via scripted observational tasks. The AORs for ourupcoming lineops have been written, and one of my roles will be quick look dataanalysis to confirm they executed as expected. My colleague Luke Keller, from Ithaca College, is shown below crafting some new slit-stepping observations.

Oh, I got to step inside SOFIA today. She’s bigger on theinside (compared to what I had expected, that is.).

Being in the presence of a cool lady, a 747SP named the Clipper Lindbergh

I have arrived here in Palmdale, CA. This is a new place forme, so it has a share of expectations. Palmdale, just 50 miles north-east-ishof Los Angeles is home to the Dryden Aircraft Operations Facility or DAOF, forshort.  Upon arrival, I learned that NASADryden Flight Research Center itself is about another 40 minute drive away, sotime permitting, I’d like to check out that sister center.

I’ve rendezvoused with two colleagues from Cornell andIthaca College who have both flown on SOFIA and also have put in so many hoursto make the FORCAST instrument a success. They are eager to get back tooperations & science observations again. I’ve also met two graduate students, one who has flown already andanother, just as green-as-me, this being his first time to Palmdale andchecking out the *Stratospheric Observatory for Infrared Astronomy* forhimself.

 Today marks a specialoccasion for me to see SOFIA in all her shiny-white-paint with an organized crewgetting her ready for this week of line operations, or line ops. The reality isintense. One can read about things on the internet or in papers, but toactually see the physical metal,glimpse at her sleek curves, observe the crews keeping her safe and airworthy,is something else. And that’s just the outside.

The scienceinstrument FORCAST, a mid-infrared instrument, is already installed and had itslatest cryogen fill this morning.

Tonight, line operations are scheduled from 11pm-5am and Ican share what I learn.  Until then,pieces of the complex set of what goes into operating a facility such as SOFIA,are slowly coming into place.

For now, I just cannot help staring at this amazing beauty.

SOFIA at the Dryden Aircraft Operations Facility getting ready for a weight and balance test

747SP, the SP means “Special Performance.”