Monthly Archives: July 2009

Introduction to Cruise Phase

Posted on by .

The most frequent questions I get about LCROSS go something like, “You’ve been flying for ‘x’ days…aren’t you at the moon already?”, “Why does it take so long to reach the moon?”, or “What are you doing with all the extra time between now and impact?” Admittedly, the answers are hard to communicate, because the LCROSS mission profile isn’t as straightforward as one might guess. 

Between Transfer Phase and Impact Phase is the longest phase of the LCROSS mission – Cruise Phase.  Transfer and Impact Phases are the most intense and busy, and therefore get the most attention.  Cruise Phase is when the Flight Team performs long-term preparations for Impact in every respect – we calibrate our science instruments, we better characterize our spacecraft, we finalize our lunar impact target and refine our impact trajectory.  Many of us on the Flight Team envisioned Cruise to be a lot less busy than Transfer, but our team has been steadily busy since Lunar Swingby.

Over the next two posts, I’ll introduce you to Cruise Phase, and get you all up to speed on what’s happened since Lunar Swingby so far.  Maybe in doing so, I’ll answer those questions that you’ve wanted to ask about Cruise Phase that seems, at first glance, to be nothing more than a long delay between the really cool parts of the LCROSS mission.

The Real Reason for Cruise Phase: Impact Energy

Cruise Phase is first and foremost about trajectory and impact energy, so I’ll start there.  So, why does it take so long to get to the moon, and why are we still orbiting the Earth?  Actually, it doesn’t take very long to get to the moon.  LCROSS passed by the moon on Day 5 of our mission, way back on the morning of June 22.  We could have impacted the moon back then, but our objective was to impart as much energy into our impact as possible, and it just wasn’t possible to do that on our first lunar encounter.  I’ll explain.

On our initial outbound trajectory from the Earth in the days after launch, we could have hit the moon’s equatorial region very directly, but the only way we could have impacted the lunar south pole is at a very low, grazing angle.  A low impact angle results in relatively little energy release.  Our goal is to hit our south pole target as steeply as possible, transferring all of our kinetic energy into the impact site, thereby raising the greatest amount of lunar material from the surface for analysis.  The only way to do that is to approach the moon roughly perpendicularly to its orbit plane, to smack the moon from “below”, at the south pole.  At Lunar Swingby, we used the moon like a slingshot to throw LCROSS from its Trans-Lunar Orbit, roughly in the same plane as the moon’s orbit about the Earth, into a steeply-inclined Cruise Phase orbit that would allow us to do just that.  So, we got to the moon in five days, but we never entered an orbit around the moon.  Instead, we kept on orbiting the Earth, but in a much different orbit than we started in, thanks to the help of lunar gravity as the moon flew by us.  That’s right…you read this right: when you look at the animations of the LCROSS trajectory, the moon had the larger component of velocity (as compared to LCROSS) relative to the Earth, and actually flew by LCROSS rather than the other way around.  LCROSS was at the right place at the right time to be thrown into its Cruise orbit!

 

This figure illustrates the Transfer Phase and Cruise Phase segments of the orbit, as seen for July 22, 2009.  The moon’s gravitational influence threw LCROSS into the Cruise Phase trajectory where it will remain, orbiting the Earth, until October 9.  The moon has already made a full revolution around the Earth, and LCROSS is (at the time of posting) far below the moon’s orbital plane, not quite 3/4 of the way around its first orbit around the Earth.  At the time of Impact, LCROSS will rise and intersect the moon’s orbit just at the time the moon crosses the same point.  Courtesy of NASA Ames Research Center.

Now that we’re in the Cruise Phase orbit, we’ll orbit the Earth three times before we impact.  As of today’s post, we haven’t even made it around the Earth once yet, though the moon has made one orbit around the Earth since Lunar Swingby.  Among other things, Cruise Phase is a waiting period for when the orbits of the moon and LCROSS cross paths again on Impact day on October 9.  Next time, the moon will be squarely in our sights!

Trajectory Correction Maneuvers: Fine-Tuning Impact

At special positions in the orbit during Cruise, LCROSS will perform additional TCM’s (Trajectory Correction Maneuvers) to converge on the impact target parameters – position, timing, and impact angle.  Originally we had 10 Cruise Phase TCM’s planned: TCM 4a, 4b, 5a, 5b, 5c, 6, 7, 8, 9, 10.  Some of these “maneuvers” are mandatory, others are optional.  The mandatory ones cannot be skipped if we want to hit the moon.  In the LCROSS mission, these are TCM 1 (executed during Transfer Phase) and TCM 5a.  The others are optional, and are only performed if errors remain from previous burns, or stemming from other influences (e.g. solar radiation pressure, disturbances caused by our own attitude control thruster firings, etc).  It’ll be rare that we skip a burn opportunity, but it has happened already.  We skipped the TCM 4a and 4b opportunities because we were so close to our target trajectory after Lunar Swingby that performing the burns wasn’t worth the extra risk and effort involved in doing such maneuvers. 

These burns vary dramatically in magnitude.  Earlier burns change the orbit more significantly.  By TCM 10, we’ll really be polishing things.  To give you a feel for the difference, TCM 5a was designed to alter the LCROSS velocity by 21.1 meters/second (47 miles per hour).  By comparison, our final burn will be changing the velocity by less than 10 cm/s (0.22 miles per hour), perhaps even less.  However, when you think about it, this kind of refinement is necessary.  Imagine the effect of 1 cm/s of velocity error over 10 hours of drift time prior to Impact.  The distance adds up fast: 0.01 m/s x 3600 s/hr x 10 hr = 360 meters of position error!

The fewer burns we have remaining in the mission, the more “locked in” we’ll be to a very specific impact target position and timing.  We started the mission with quite a bit of flexibility in both, but that will soon change.  In fact, the LCROSS science team has to make its final impact target selection by 60 days prior to Impact.  Any later, and the propellant cost to change to a different target becomes prohibitive. 

Science Instrument Calibrations

Cruise Phase provides some nice opportunities to continue calibrating our science instruments before Impact.  Our most valuable calibration opportunity was Lunar Swingby, executed on Day 5 as the close of Transfer Phase.  The close range to the moon meant that the moon filled a high percentage of the fields of view of our instruments, thereby improving our resolution, etc.  As I mentioned, I’ll have the Science/Payload team provide a guest posting sometime soon to discuss those results so far. 

Cruise Phase never gets another opportunity like Lunar Swingby.  We only get that close to the moon once more – on our final approach before Impact.  It’s far too late to be worrying about calibrations by then.  However, there are times on each orbit when LCROSS gets closest to the moon and Earth (making them look bigger in the instruments), and these are valuable opportunities for follow-on calibrations. 

Our mission has three nominal calibrations planned: two “Earth Look” calibrations, and one “Moon Look” calibration.  As the names indicate, we’ll train the instruments on the Earth and moon, and perform a series of sweeps back and forth across their distant shapes to improve our knowledge of instrument pointing accuracy, radiometric response and spectral response to known inputs.  The Science team’s lessons from Lunar Swingby will guide the specific measurements and focus of these activities.

Omni Pitch Maneuvers: Staying in Contact

The Cruise Phase orbit takes LCROSS high above and far below the moon’s orbital plane on each revolution.  Our standard spacecraft attitude for Cruise places our long axis (the one common to LCROSS and the Centaur) perpendicular to the ecliptic plane – the plane of the Earth’s orbit around the sun.  There’s nothing particularly special about this orientation, but it makes things a lot easier to think about operationally, and that’s worth a lot.  The catch is that our primary low-gain (omni-directional) antenna points in one direction off one side of the spacecraft, and it’s not always pointed toward the Earth.  It enables communications over about a full hemisphere of angle, but as we orbit the Earth without changing orientation, the Earth will move into and out of the antenna pattern.  When Earth is out of the pattern, we can’t communicate with LCROSS (note that we have a secondary omni antenna on the opposite side of the spacecraft, but it’s less effective, so we try to stay on the primary whenever possible). 

 

This figure depicts the LCROSS shepherding spacecraft and the Centaur, shortly after separation.  The spacecraft depiction shows the location of the primary omni-directional antenna and the solar array, relative to the standard body reference frame axes.  Omni Pitch Maneuvers rotate the spacecraft about the Pitch axis to re-orient the primary omni antenna.  The Cold Side Bakeout maneuvers rotate the spacecraft 180 degrees about the Roll axis to orient the cold side of the spacecraft (and Centaur) towards the sun.  Artwork courtesy of Northrop Grumman.

The remedy is to flip the spacecraft 180 degrees, keeping our solar array pointed at the sun, but turning the omni antenna pattern to the opposite direction.  At the time of the flip, neither the starting nor the ending orientation is very good for communications.  However, if left in the starting orientation, the Earth would move out of the antenna pattern.  By flipping the antenna (and the spacecraft) over, the Earth will tend to move deeper into the pattern over time.  These flips are rotations about the spacecraft “pitch” axis, and so are officially called “omni pitch maneuvers”.  During Cruise Phase, we have to perform these roughly every two weeks, though the timing depends on the specific orbit geometry.

Cold Side Bakeout: Getting the Water (and Other Stuff) Out of the Centaur

The Centaur upper stage (our impactor) is covered with material that is naturally absorbent and was expected to absorb some water from the atmosphere when sitting on the pad in the humid conditions in Florida.  In fact, if you watch the LCROSS launch video closely (the Centaur-mounted camera view), you can see ice accumulated in ridges on the vehicle surface, frozen by the extremely cold temperatures caused by the cryogenic contents (liquid oxygen and hydrogen) of the Centaur propellant tanks. 

In space, this accumulated water might melt or sublimate (a straight conversion from ice to vapor) if exposed to the extreme heat of the sun.  This is what we expected the water to do on the side of Centaur aligned with our spacecraft’s solar array, since that side is almost always pointed toward the sun.  After several days, we expected all of the water on the sun side of the Centaur to be removed naturally. 

Water trapped within the very cold foam on the opposite side of the Centaur, however, might remain for a lot longer, perhaps through the end of the mission.  In fact, Lunar Swingby spectrometer measurements show a significant water signal (along with other exotic hydrocarbons) – given the dry mid-latitude targets we were measuring, the water must have come from the spacecraft, and the strongest possibility is that it was from ice debris floating around the spacecraft.  There are two problems with water trapped on the Centaur.  The more obvious of these, but actually the less problematic, is that LCROSS is trying to find water on the moon.  Bringing water to the moon from the Earth will only confuse the issue!  Our Science team is actually not so worried about this, because we’re talking about only 10 kg of water or so, and they wouldn’t expect to “see” such a small amount relative to the hundreds of tons of material LCROSS is expected to loft and the percentages (albeit small) of that material expected to be water.  Still, it’s a concern.

Interestingly, the larger problem is that water escaping from the Centaur is predicted to cause a disturbance to its trajectory (via conservation of momentum).  The worry is that in the hours before Impact, as the Centaur is descending alone to the lunar surface, its shadowed side will be exposed to sunlight, causing a release of water and a notable trajectory disturbance.  A significant release of water could turn a perfectly-aimed impactor into a poorly-targeted dud.  We’re talking potentially 100’s of meters error or more!  It’s hard to believe that water escaping the surface could cause that much harm, but welcome to space physics.

To prevent this from happening, we’ve scheduled a Cold Side Bakeout maneuver that rotates the spacecraft about its long axis to point the solar array directly away from the sun, and bakes the water right out of the Centaur in full sunlight.  Over enough exposure time, the water should vent out of the Centaur.  The challenge is that LCROSS is designed to spend all but short periods of time pointing its solar array toward the sun.  Thermal and Power subsystem flight rules prevent long dwells in this opposite orientation.

Other Engineering Stuff

Aside from giving us time to conduct all of the above activities, Cruise Phase is a chance to get more familiar with our spacecraft so that we can totally predict how it will behave at Impact.  Recall that we came out of Transfer Phase with a few “anomalies”, or unexpected problems.  The Flight Team needs time to sort through all of the issues, starting with the most severe ones, and gradually moving down to the more benign ones.

Our goal is to figure out why each of the problems is occurring, and then to come up with ways to work around the problems, or to show that their behaviors are completely benign and not worth worrying about.  Each must be considered in terms of risks against achieving our mission objectives.  Our team can only bring an anomaly investigation to closure by proving to our mission stakeholders (NASA Ames, LPRP and NASA Headquarters management) that our mitigating strategies maintain our risk of failure to acceptable levels.

Flight Team Planning, Training and Rest

As Flight Team Lead, I’m responsible for creating and modifying the short and long-term plans for the mission, including Cruise Phase.  Before launch, most of our time was spent preparing for the first week of flight.  Once Lunar Swingby came and went, I had to drop into overdrive to get our Cruise Phase planning in order.  This meant (and continues to mean) nailing down times for TCM’s and other maneuvers (courtesy of our MMDS team – see the post on “LCROSS Flight Team Breakdown”), nailing down DSN contact times (via our CLASS team at Ames and JPL), putting together staffing schedules, etc. 

Then there’s planning for Impact.  Again, our team spent the 2-3 months prior to launch working out final details on Transfer Phase operations.  We last practiced Separation and Impact in February.  We have things worked out pretty well, but now we need to practice some more so that we nail our Impact in October.  As Flight Team Lead, I am responsible for setting down the rehearsal schedule, working with the CLASS team to schedule DSN readiness tests, and refining our procedures as needed.

When we’re not planning for our next activity, coordinating on anomaly resolution, working out schedules, or training for Impact, the Flight Team has a little extra time to rest than it did before launch, and during Transfer Phase.  We’re certainly not avoiding work – it’s what you as a taxpayer are paying us to do.  But the Flight Team has been working full steam ahead for six months or more, and now that the schedule is a little more relaxed, we can afford to take some hours and days off here and there.  It’s not much, but it sure helps!

This is it for now.  In the next post, I’ll present a brief diary to catch everyone up on all the activities we’ve performed since Lunar Swingby! 

Lunar Swingby: Development of a Procedure

Posted on by .

All of you keeping up with LCROSS have undoubtedly already heard the good news – that our team at NASA Ames successfully performed our capstone event on Day 5 of the mission – the Lunar Swingby.  In execution, it worked just as we had hoped.  But getting it to work wasn’t easy.  It took us many months of development, testing and updates to get the procedures right, and throughout our months of training, our team rehearsed this event far more than any other. 

Lunar Swingby (LSB for short) is one of two close lunar encounters during the mission – in many ways it was a dress rehearsal for impact.  It’s an intricate coordination between orbit dynamics, attitude control, communications, thermal, power and of course, the science payload.  In this post, I’d like to walk you through the evolution of the procedures and command sequences that we ultimately flew on Day 5.

An Introduction to Lunar Swingby

So, what was LSB designed to do?  There were two primary goals – one, to throw LCROSS into its phasing orbit, called the LGALRO, or Lunar Gravity Assist Lunar Return Orbit.  As that name suggests, our mission uses the gravitational well of the moon to drastically change the Transfer Phase orbit without any propellant cost.  The post-LSB orbit is designed to hit the moon with one additional Trajectory Correction Maneuver (ideally), enabling the steep impact angle at the lunar south pole that otherwise would have been impossible to achieve using the outbound Transfer Phase (trans-lunar) orbit we were provided by the Atlas/Centaur. 

The second goal of LSB was to calibrate the science instruments, taking advantage of the relatively close range of LCROSS to the moon.  The calibration was divided into two phases: one to measure lunar surface targets, and the other to sweep the payload over the “limb” of the moon (the “limb” is the edge of the apparent disk of the moon).  Looking at lunar surface targets with known mineralogy and optical properties (albedo) allows our science team to calibrate the cameras and spectrometers.  Having “ground-truth” means they can compare the actual instrument outputs against expected outputs, knowing how the instruments should respond to the known surface properties. On the other hand, sweeping the payload over the lunar limb allows the science team to calibrate the payload alignment.  The LSB incorporated “slews”, or changes of orientation, that alternately pointed the instrument boresights off the moon, then on the moon, back and forth, such that the boresight would inscribe a path perpendicular to the edge of the lunar disk.  Since we know the spacecraft orientation accurately, and we know the precise positions of the moon and the spacecraft, we can predict, for a given alternating sequence of slews, the precise timing of when the boresight should cross onto and off of the lunar disk.  A consistent bias between the predicted and measured timing indicates an alignment offset. 

LSB was designed to measure three surface targets for 5 minutes each (three craters: Mendeleev, Goddard C, and Giordano Bruno), and two limb targets, one in a “left-right” slewing orientation, and a second using an “up-down” orientation to detect misalignments in both possible directions.  Each of these limb sequences was also 5 minutes long.  Sounds pretty straightforward, right?  Well, the details make things tricky.

For reference in the next few sections, check out the link to an animation of the “as-executed” Lunar Swingby. The animation is sped up so that you can see the various sub-events (slews from Cruise attitude to initial sampling attitude, surface samples, limb slews, etc). Vectors coming out of the spacecraft model indicate the direction to the Sun (yellow) and Earth (Blue), and the body-fixed vectors representing the LCROSS body coordinate frame (Red).  The projected square represents the field of view of the LCROSS visible camera, and the narrow cone projection represents the 1.0 degree field of view of the nadir-pointing Near-Infrared Spectrometer, the most important instrument for LSB science return.  The little bullseyes are our surface targets.  See if you can identify the “setup” and “cleanup” attitude changes from those between targets.

LSB Constraints

To get into LSB constraints, let’s continue with the discussion of pointing the spacecraft to the targets.  The requirement was to point to within 1 degree of the intended target, and to slew no faster than 0.15 deg/s during measurements to avoid “smearing” our images (since we don’t have a super-fast effective “shutter speed”).  For LSB, we designated an attitude control mode that would maintain our orientation to within 0.5 degrees of a given target orientation.  The trick is that our spacecraft cannot autonomously identify the targets in an image, nor can it track them automatically once we point it in the right initial direction.  So, we had to tell the spacecraft where to point each few seconds, relying on knowing the precise position and orientation of the moon, as well as the position of the spacecraft (via orbit predictions) to predict the right set of orientations that would point the science instruments to the targets.  This means the whole sequence of orientation targets had to be planned in advance, but no earlier than several hours prior to the LSB event, since with TCM 3 happening the day before LSB, we couldn’t possibly know our orbit accurately enough to meet the pointing accuracy requirements.

To make matters slightly more complicated, instrument pointing was not our only concern with spacecraft attitude.  LSB and Impact both call on our highest available communications downlink rate – 1 megabit per second (Mbps).  This rate is only possible via one of two Medium Gain Antennas (MGA’s), and typically would require the use of a DSN 70 meter antenna.  For lower rates, our Omnidirectional antennas provide adequate link margin in almost any orientation.  The MGA’s, however, need to be pointed to within 20 degrees of the Earth to enable a strong signal to the ground, with more precise pointing providing better results.  With two attitude constraints, one for the instruments and the other for communications, our attitude was fully constrained.  Any other orientation requirement would have to be balanced against sacrificing one of the first two.  So, for example, we’d have to deviate from our normal “happy” orientation pointing the solar array at the sun during all of these observations.

This brings me to thermal and power constraints.  Both the Power and Thermal subsystems were designed to take only short deviations from the nominal spacecraft sun-pointed orientation.  Prior to flight, our Power subsystem was predicted to allow two hours off-sun.  Our Thermal subsystem’s constraints were a little more complicated, but it was predicted to enable up to one hour in some orientations, and two hours in other orientations.  We’d have to follow the one-hour limit since we couldn’t rely on satisfying the two-hour orientation constraint.

What about the science instruments?  The Data Handling Unit (DHU) is a separate computer that controls the science payload.  The DHU includes 10 buffers in non-volatile memory to store instrument command sequences.  The Science team defined different sequences for different purposes.  Originally, the Science team defined three sequences for the nominal LSB – one for surface target sampling, and another two for the limb measurements.  Later, as I’ll describe below, they had to define an additional three to cover contingency cases.  Another job of the operations team was to perform power-up and power-down of the DHU and operational heaters.  Thermal constraints dictate that payload operational heaters must be enabled at least 4 hours prior to using the science instruments to warm them to their operational temperature range.  Furthermore, the DHU, running with the full set of instruments, cannot run for more than 60 minutes without risking overheating.  Finally, the spacecraft command sequences, had to coordinate the execution of the LSB science command sequences with the commands to control the spacecraft attitude so that everything happened at the right times.

The following is a depiction of our surface and limb targets for LSB.


Wringing Out the Nominal Procedure

Last September, we thought we had a pretty good handle on flying Lunar Swingby (recall that our original launch date was October 28, 2008, so we had to be ready).   Due to the precise timing required to get this right, all of the LSB commands were originally packaged into an onboard command sequence, including several hundred attitude commands and other commands to power the DHU on and off, to start DHU command sequences, and to configure our communications system.  We had a nominal procedure that worked well in simulation, and both shifts had practiced it enough to gain confidence that we could do this in flight.  We had already tackled many of the issues, but there were problems. 

For one, we were exceeding our one-hour thermal time constraint.  The science calibrations were predicted to require a total of 45 minutes (five targets, five minutes each, plus four five-minute slews between targets), leaving only 15 minutes for other preparation and cleanup steps.  The initial reorientation from our Cruise attitude (which started the thermal clock ticking), the communications configuration steps, and the return to Cruise attitude (ending the thermal clock) required another 25-35 minutes, so we exceeded the limit by 10-20 minutes.  Ultimately, this forced our lead Thermal Engineer to revisit the thermal subsystem analysis, and to prove that the Thermal system could actually handle up to two hours off-sun in almost any orientation. 

Another LSB problem was with attitude control fault management.  As I mentioned earlier, to achieve the accuracy needed for science calibrations, LSB needed to employ an attitude control mode that enabled ]0.5 degree pointing precision.  Early tests of our command sequences worked just fine.  However, this mode was not originally designed with LSB slews in mind.  One day in the middle of an LSB simulation, our simulated spacecraft suddenly reset and dropped into Survival State, terminating LSB.  As we later discovered, we had exceeded the maximum attitude rate limit for the ACS mode we were using (the limits were tailored for near-stationary operation), causing a significant spacecraft fault management response.  Later versions of our procedures added steps to temporarily adjust the rate limits for that control mode to allow the spacecraft to slew a little faster without tripping fault management.

While testing LSB, the Flight Team also was temporarily stymied by a bug on the spacecraft that seemed to be activated when flowing high-rate data from the science payload (as happens during LSB).  In some runs, we found that the bug prevented telemetry and science data flow, and even prevented most commanding – a really bad problem to have in the middle of such an important event.  Over many trials, we found the only effective Flight Team response was to reboot our spacecraft processor – which would also put the spacecraft into Survival State and totally remove hope of successfully collecting our calibration data.  This behavior had been observed in earlier, lower-level tests of the payload and spacecraft, but had never reared up to affect operations.  Our operational discovery placed LSB in serious jeopardy.  After a lot of very careful debugging, our Northrop Grumman partners ultimately discovered the cause of the problem – an FPGA (Field Progammable Gate Array) that encoded the communications protocol had a programming bug in it, sourcing from a faulty specification of the protocol.  Once the problem was isolated, NG wrote software workarounds that enabled us to completely avoid the problem.

Contingency Responses

Once our original launch date had slipped briefly to December 2008, and ultimately into 2009, the Flight Team began focusing on “risk buy-down”.  With every aspect of our mission design, we asked ourselves, “with this new-found time, how could we make our mission less risky?”  With all nominal procedures worked out, we began to work on possible contingencies – things that might go wrong and cause LCROSS to have a bad day, either through loss of important data, or worse, a loss of spacecraft.

What were the big ones for Lunar Swingby?  LSB payload calibration data is really important to the Science Team.  I mentioned the high downlink rates required that we use our Medium Gain Antenna, and a DSN 70 meter ground antenna.  What if one of those two elements failed?  Could we still get enough data back to make our Science Team happy?

In November and December 2008, we worked on two low-rate contingency options, one assuming an MGA failure that utilized the Omnidirectional to 70 m antenna, the other assuming a DSN 70 meter antenna failure, utilizing the MGA and a 34 m antenna.  We spent hours working through how to address these failures, and it became very clear that the structure of the nominal procedures and command sequences was not well-suited to the contingencies.  Our challenge: at the time the failure is detected, the contingency response was to reconfigure the spacecraft for a lower downlink rate (256 kbps), then to run a specially-tailored low-rate DHU instrument command sequence that would approximate the higher-rate data return, with minimal sacrifice.  The trick was that things had to be done pretty quickly (within 10-15 minutes in some cases) and, depending on when the failure happened, it required different responses.  For example, if the DHU and instruments already running, the response was very different than if they were still several minutes from powering up.  Doing the wrong thing could get the team tied in knots and make a bad problem worse.  The nominal commanding structure made it very difficult to do this in a timely way. 

We went back to the drawing board to re-structure our commands and procedures to suit both nominal and contingent execution.  Ultimately, the changes we came up with were significant, but simple, and the result was far cleaner.  However, in practice, things weren’t yet up to snuff.  January Operational Readiness Tests (ORT’s) proved our initial contingency solutions were technically correct, but too slow – they required lots of human detective work and decision making under time pressure.  The right decision made too late was not much of a help.  In February, after multiple full-team tests, and hours more re-work, we finally arrived at a solution that addressed both the MGA and DSN 70 meter failures under a common approach that streamlined the decision process dramatically.  As a side bonus, the same approach provided a contingency response option for similar failures for the Impact procedure.  We finally had this one nailed down.

Aside from the low-rate contingency responses, we developed a few other major contingencies, including a general abort response that would enable our team to return to nominal Cruise state in the event of an unspecified problem aboard the spacecraft, and another response that would, in the event of a fire in our building or an evacuation, would enable the Flight Team to leave the control room and still collect LSB data successfully.

Details, Details: Deep Space Network Scheduling 

As we waited for launch, now moved to April, May and then to June, our team worked with JPL to finalize our Deep Space Network schedule.  Before launch, every mission that uses the DSN has to negotiate antenna time to support early-mission communications between the mission operations team and the spacecraft.  Typically, negotiations happen between the new mission team and teams representing other spacecraft that are already flying.  Newly-launched spacecraft tend to get high priority, since every team needs to have a lot of dedicated time at the beginning to get into orbit, for commissioning and to work out problems.  Our launch was a little different because there were two new spacecraft (LRO and LCROSS), both going to the moon, and both relying on DSN for critical periods.  As we worked through the details, we discovered that LRO and LCROSS were in conflict for two big LCROSS events: TCM 1 and, you guessed it, Lunar Swingby. 

Lunar Swingby coincided with LRO’s Lunar Orbit Insertion burn (LOI), which would place LRO into orbit about the moon.  Even though LSB was quite important to LCROSS, LOI was mission-critical for LRO.  Missing LOI would result in a total mission failure for LRO, so LCROSS had to take back seat for DSN scheduling in this case.  To ensure LOI success, LRO requested dual-antenna coverage for the burn, in case one antenna went down at a critical time (a standard request for mission-critical events).  Each site has only two 34 meter antennas, so dual-coverage is pretty restrictive for other missions.  Depending on our specific time of launch, different DSN sites – either Canberra or Madrid – would cover LOI and LSB.  In addition to their 34 m dishes, both complexes typically operate a 70 meter antenna.  However, to make matters more complicated, Madrid’s 70 meter antenna was scheduled for servicing over May and early June, making it unlikely that LCROSS could use it for LSB.  Canberra also has a single LCROSS-compatible 26 meter antenna that could be brought to bear if needed.

Here’s how the negotiations ended up.  Under most scenarios, during LRO’s LOI burn, the LCROSS Flight Team would have no direct contact with the spacecraft – our first loss-of-contact for the mission.  If the LRO’s LOI went off successfully, LRO would release its backup antenna for use by LCROSS.  With problems, LRO was entitled to hold onto its backup antenna far longer.  The story was far more complicated because each launch time (three per day for our mid-June window) resulted in significantly different geometry and timing.  But under many circumstances, an early release of the DSN backup would provide only 20 minutes before the required start of onboard command sequences – not enough time to load the sequences first.  A late LRO release would mean the command sequences would have to start “in the blind”, before LCROSS actually was able to start them.  This took weeks of planning meetings to work out, right up until the week before launch.  Our JPL scheduling team deserves a lot of praise for supporting LCROSS and LRO in this intricate and time-constrained effort.

Our final negotiated schedules caused us to re-think how our team prepared for LSB.  Originally, all of our command sequences were to be loaded in the two hours immediately prior to the LSB.  Now, it was clear we’d need to load our command sequences, and potentially start them, prior to the LRO LOI, and hours before LSB.  That way, even in the worst case, even if LRO’s burn took longer and overstepped the beginning of the LSB sequence, once we got an antenna, we’d have a shot at getting LSB data.  This meant we’d have to have the full LSB command sequence much earlier than originally thought, and that significantly affected our Day 5 timeline and staffing plan.  We made the procedural and timeline changes, and familiarized ourselves with all of the possible scenarios.  Perhaps most complicated – what once was a single, complicated sequence now split into several possible sequences, depending on our launch time and on the success of LRO’s insertion burn.  Some of these plans came into place in the week before launch, so we only had time to review them on paper!

Doing it for Real: Lunar Swingby in Flight

On Day 5 of the LCROSS mission, our team was getting the hang of flying this new spacecraft.  All of the events had gone really well, and despite the various problems, the team was still going strong.  Shift B was slated to perform Lunar Swingby, the capstone event for the week, and certainly the most difficult event to that point.  We were all very excited, and a little nervous.

My Shift A team worked hard to get everything perfectly planned to give Shift B the best shot at success.  Our Command Approval Meeting went off without a hitch, and soon we were handing the spacecraft off to our friends on Shift B.  Timing for LSB command upload was extremely tight – as it turns out, for our launch time, we had just over one hour to get our LSB command sequence loaded to the spacecraft before a 5-hour DSN commanding outage over Madrid (consumed by LRO performing the LOI).   Thankfully, the DSN team at Madrid worked hard to complete the refurbishing of the Madrid 70 meter antenna, and it was ready for unofficial service in time for LCROSS.  It couldn’t provide commanding, but it enabled us to receive telemetry during the LOI. 

To add to the complication and urgency, the Flight Team was still actively commanding the spacecraft to keep its thrusters warm.  Recall that in previous days, some of the LCROSS thruster heaters were not activating, forcing our team to resort to commanding non-standard attitudes and unplanned thruster firings to keep them from freezing.  Well, with the gap in commanding schedule to enable the LRO LOI, we’d have to endure several hours of cooling without commanding.  We warmed the thrusters before going out of contact, and then relied fully on our biased attitude and sunshine to keep things warm.  All we could do is watch.

I stayed past my shift to watch Shift B successfully load the LSB commands to the spacecraft, then left to catch some sleep.  Even though I was scheduled to return hours after LSB, I knew that I’d be missing a once-in-a-lifetime experience if I slept through the night.  I couldn’t resist, and planned to return early.

I returned with perhaps 30 minutes to spare prior to LSB.  The spacecraft thrusters stayed warm, and LRO’s LOI apparently went off perfectly, because they released their backup DSN antenna even earlier than we thought possible!  We had the best scenario possible.  Shift B confidently prepared for and executed LSB.  A number of us on the off-duty shift watched things unfold from a conference room outfitted with large-screen monitors and full voice loop audio below the control room.  The onboard command sequence started right on time, and transitioned our attitude to the first target direction.  Shift B commanded the communications reconfiguration and successfully achieved our 1 Mbps downlink rate via an MGA, the first major hurdle.  Then, we waited on the edge of our seats as the command sequence neared the DHU power-up commands, and LCROSS’s first photos of the moon.  It all happened just like it was supposed to – the DHU powered up, our Payload Engineer reported he saw good DHU telemetry, and then, our first image of the moon appeared on our monitors – not the cardboard image that we used for our spare simulator payload, but the real moon!  Within minutes, we saw images from the Visible Camera, the Near-IR Camera, the Mid-IR Cameras, and spectra from our various other instruments.  We sat rapt, quietly taking in this live data feed.  Watching these images live from the moon was the culmination of all we had worked for – these photos were living proof that the year of work had paid off.  We were absolutely thrilled!

Forty five minutes later, without fanfare, the DHU powered off again, the communication reconfigured for low-rate Omni downlink, and the spacecraft rotated back to Cruise attitude.  Though the spacecraft returned to Cruise state after LSB, our team’s demeanor had changed subtlely.  The pressure of Transfer Phase had lifted.  Our success had set in and, exhausted from a week of very long hours on console, we celebrated with smiles and a shared sense of satisfaction.  There was a lot yet to be done, but we’d made it through the most intense period of the mission…save Impact! 

Incidentally, the Science and Payload team has volunteered to produce a guest posting on the details of the science findings, so I promise some details on Science findings in a future posting.  Stay tuned!