E10002 – Liquid Rocket Engines (J-2X, RS-25, general)

J-2X Progress: November 2013 Update

It’s been a few months since we talked about J-2X development progress. So, let me bring you up to date. Here’s the short version:

Testing for engine E10002 is complete
Engine E10003 has been installed in test stand A-2 and has successfully completed its first test (a 50-second calibration test on 6 November)
Engine E10004 is in fabrication

Okay, so that’s it. Any questions?

Oh, alright, I’ll share more.

Engine E10002 is the first J-2X to be tested on both test stand A-2 and on A-1. It saw altitude-simulation testing using the passive diffuser on A-2 and it saw pure sea-level testing on A-1 during which we were able to demonstrate gimballing of the engine. Below is a cool picture from our engineering folks showing a sketch of the engine in the test position on A-2.

The clamshell shown in the sketch effectively wraps around the engine in two pieces and the diffuser comes up and attaches to the bottom of the clamshell. This creates an enclosed space that, while the engine is running, creates the simulated altitude conditions. I’ll show some more pictures of the clamshell when we talk about engine E10003 below. Next is a cool picture of engine E10002, hanging right out in the open, while testing on A-1.

This next table gives a history of the engine E10002 test campaign across both test stands:

So, let’s talk about the three times that we didn’t get to full duration.

The first time, on test A2J022 we had an observer cut. Just like that sounds, there was actually a guy watching a screen of instrumentation output and when he saw something that violated pre-decided rules, he pushed the cut button. We use such a set-up whenever we’re doing something a little unusual. In this case, we were making an effort to reduce the amount of cooling water that is pumped into the diffuser. It was our general rule-of-thumb to “over cool” the diffuser. After all, who cares? It’s a big hunk of facility metal that we wanted to preserve for as long as possible despite the fact that it always gets a beating considering where it sits, i.e., in receiving mode for the plume from a rocket engine. However, one of the objectives for our testing was to get a good thermal mapping of the conditions on the nozzle extension. What we’d found with our E10001 testing was that all of the excess water that we were pumping into the diffuser was splashing up and making our thermal measurements practically pointless. Thus, we had to take the risk of reducing the magnitude of our diffuser cooling water. As I’ve said many times, there are only two reasons to do engine testing: collect data and impress your friends. If our data was getting messed up, then we had to try something else. Eventually, through the engine E10002 test series we were able to sufficiently reduce the diffuser cooling to the point where we obtained exceptionally good thermal data. This first test on which we cut a bit early was our first cautious step in getting comfortable with that direction.

The second early cut was caused by some facility controller programming related to facility instrumentation. Here is a little tidbit of neato information that I’ve probably not shared before about our testing: in the middle of longer runs, we transfer propellant from barges at ground level upwards and into the run tanks on the stand. The run tanks are pretty big, but they’re not big enough to supply all of the propellants needed for really long tests. When I’ve shown pictures of the test stands in the past, you’ve seen the waterways that surround and connect all of the stands. These are used to move, amongst other things, barges of propellant tanks. Liquid oxygen is transferred using pumps and liquid hydrogen, being much lighter, is transferred by pressure. In the picture below, you can see a couple of propellant barges over to the left. This is an older photo of a Space Shuttle Main Engine Test on stand A-2.

Thus, in addition to monitoring the engine firing during a test, you also have to watch to make sure that the propellant transfer is happening properly. The last thing that you want to happen is have your engine run out of propellants in the middle of a hot fire test. On test A2J025, there was an input error in the software that monitors some of the key parameters for propellant transfer. Thus, a limit was tripped that shouldn’t have been tripped and the facility told the engine to shut down. Other than some lost data towards the end of this test (data that was picked up on subsequent testing), no harm was done.

On test A2J027, there was something of an oddball situation. We have redlines on the engine. What that means is that we have specific measurements that we monitor to make sure that the engine is functioning properly. During flight, we have a limited number of key redline measurements and these are monitored by the engine controller. During testing we’ve got lots more redline measurements that we monitor with the facility control system. When we’re on the ground, we tend to be a bit more conservative in terms of protecting the engine. The reason for this is that when we’re flying, the consequences of an erroneous shutdown could mean a loss of mission. Thus, we have different risk/benefit postures in flight versus during ground testing. [Trust me, the realm of redline philosophy is always ripe for epic and/or sophist dissertation. Oh my.] Anyway, with regards to test A2J027, when doing ground testing we shutdown not only when a redline parameter shows that we may have an issue (as happened erroneously on test A2J025) but also if we somehow lose the ability to monitor a particular redline parameter. Thus, we did not shutdown on test A2J027 because we had a problem or because we had a redline parameter indicating that we might have a problem. Rather, we shut down because we disqualified a redline parameter.

On J-2X, wherever we have a critical measurement (meaning that it is a parameter that can control engine operation, including redline shutdown) we have a quad-redundant architecture. In the sketch below, I attempt to illustrate what that actually means.

Thus, we have two actual measurement ports and each port has two independent sets of associated electronics. We are doubly redundant in order to ensure reliability. However, does a man with two watches ever really know what time it is? No, he doesn’t because he cannot independently validate either one. We have a similar situation, but in our case we simply want to make sure that none of the measurement outputs that we are putting into our decision algorithms are completely wacky. So we do channel-to-channel checks and we do port-to-port checks to make sure, at the very least, some level of reasonable consistency. Thus, we cannot know the exact answer in terms of the parameter being measured, but we can decide if one of the measurement devices themselves is functioning improperly. This process is called sensor qualification. On test A2J027, our sensor qualification scheme told us that one port was measuring something significantly different than the other port, different enough that something was probably wrong with at least one of the sensors. That resulted in disqualifying the measurements from one of the two ports. In flight we would have kept going unless or until the remaining port measurements notified us of a true problem, but on the ground, as I discussed, we are more conservative.

When we investigated the apparent issue, what we discovered was that we should have predicted the port-to-port offset. It turns out that due to the engine conditions that we’d dialed up for that particular test, we were running the gas generator at a mixture ratio higher than we’d yet run on the engine. When we went back and examined some component testing that we’d done with the workhorse gas generator couple of years ago, that data suggested that yes indeed, when we head towards higher mixture ratio conditions, our two measurements tend to deviate. This suggests, perhaps, a greater amount of localized “streaking” in the flow at these conditions. Localized effects like this are not uncommon in gas generators or preburners. Because of the particular configuration of the J-2X, with more mixing available downstream of the measurements, the impact due to such variations on the turbine blades is minimized. This too was shown in the component level testing. Thus, the sensors were fine and the engine was fine. It was just out qualification logic that needed reexamination. Sometimes, this is how we learn things.

So that tells you all about those handful of cases where we didn’t quite get what we intended. Overall, however, the engine E10002 test campaign was truly a rousing success. Here are some of the key objectives that were fulfilled:

Conducted 13 engine starts – 10 to primary mode, 3 to secondary mode – including examinations of interface extremes for a number of these starts.
Accumulated 5,201 seconds of hot-fire operation.
Performed six tests of 550 seconds duration or greater.
Conducted eight “bomb” tests to examine the engine for combustion stability characteristics. All tests showed stable operation.
Characterized nozzle extension thermal environments.
Characterized “higher-order cavitation” in the oxidizer turbopump.
Demonstrated gimbal operation (multiple movement patterns, velocities, accelerations) with no issues identified.

Hot on the heels of the success of engine E10002, we have engine E10003 assembled and ready to go. I love this picture below. This is the engine assembly area. We have three engine assembly bays and, on this one special occasion, we happened to have each bay filled. Engine E10001 is all of the way on the left. It is undergoing systematic disassembly and inspection in support of our design verification activities. Engine E10002 had just come back from its successful testing adventure. And engine E10003 is all bundled up and ready to travel out to the stands to begin his adventure. [I’m not sure why this is the case, but E10003 has a male persona in my mind so the possessive pronoun “his” seems to fit best.]

In the picture below you see E10003 being brought into the stand on A2. Note the water of the canals in the background. See the concrete pilings over to the left in the background. Those are where the docks are for the propellant barges that we discussed above.

In the pictures below, you can see E10003 installed into the test position. The picture on the left shows half of the clamshell brought down into place. Compare this picture to the sketch at the beginning of this article. The picture on the right shows what the engine looks like with both halves of the clamshell brought down into position.

So that’s where we stand. Engine E10003 has begun testing in November 2013 and continue on into 2014. As always, I will let you know how things are going and if anything special pops up, you can be sure that we’ll discuss it here at length. After all, there’s not a whole lot that’s more fun than talking about rocket engines.

LEO Progress: J-2X to Test Stand A1

“Nothing behind me, everything ahead of me, as is ever so on the road.”
– Jack Kerouac, On the Road

Recently, J-2X development engine 10002 was on the road. If you remember, E10002 went through a six-test series on test stand A2 that began in February and finished up in April. The next planned phase of E10002 testing is on test stand A1. In between these series, the engine was back in the assembly area of NASA Stennis Space Center Building 9101.

This respite between test series allowed for a complete series of inspections of the engine hardware. This is vital piece of the learning process for engine development. The basic truth is that a rocket engine is just darn tough on itself when it fires. The reason that we test and test and test is to make sure that our design can stand up to the recurring brutal conditions. The chance to look for the effects of that testing through detailed inspections away from the test stand is an opportunity to collect a great deal of useful information.

Also, even before the engine arrived at the assembly area, the stub nozzle extension was removed. This was done while the engine was still installed in the test stand. Remember, the testing on test stand A2 was performed with a passive diffuser and so we were able to use an instrumented stub nozzle extension to examine the nozzle thermal environments. On A1, there will be no diffuser. We’ll be firing straight into the ambient Mississippi afternoon and so we’ll not have any nozzle extension attached. The other change made to the engine — this one made while in the assembly area — was that we swapped out the flexible inlet ducts so that we can use our specially instrumented ducts for the gimbal testing on A1. These ducts will provide a great deal of unique data when we gimbal the engine and force the inlet ducts to twist and bend and they are designed to do.

Below is my favorite picture from the recent assembly activities for E10002 back in Building 9101.

“FOE” stands for “foreign object elimination.” I love this picture because it is a demonstration of how dedicated and meticulous are our assembly techs and how much basic integrity they bring to their job every single day. In the rocket engine business we tend to be fanatics about foreign objects (i.e., random debris of unknown origin) hanging around. The reason for this is that if you spend enough time in the business, you will eventually have a story of what happens when trash gets into the engine. The rocket in question might be an amazingly powerful beast pumping out five hundred pounds of propellant per second generating 300,000 pounds of thrust, but all it takes is one bit of junk in the wrong place to destroy the whole thing in fractions of a second.

In the picture above is a small nut that was found in the periphery of the assembly area. Shoot, if this was my garage you’d be lucky to find a clean patch not strewn with various bits of stuff like nuts, bolts, wads of duct tape, old hunks of sandpaper, that lost pair of pliers, a “Huey Lewis and the News” cassette, or, well, who knows what. But the rocket engine assembly area is NOT my garage (thank goodness). When something is found like a stray nut in the picture above, that sets off an investigation. Where did it come from? How did it get loose? What procedure allowed this nut to escape control? This is serious stuff. Yet, just think about how easy it would have been for a tech to see that stray nut, pick it up, and stick it in his pocket. They could have avoided the whole minor investigation thing entirely. But that’s not what they did and that is not what they do. Because they know that if they do not show the necessary integrity and methodical approach to continue to learn and perfect our procedures, then the next stray nut could be lodged where it could do terrible damage.

Here are the techs moving E10002 out of its assembly bay and unpacking it for transport.

And a couple more pictures of the process in Building 9101.

Here’s the engine ready for the road, then being lifted up the side of the test stand, and then sitting in the porch area while sitting on the engine vertical installer. I really enjoy the pictures of the engine trucking about sunny NASA SSC. That picture was the inspiration for including the Jack Kerouac quote at the beginning of the article. It’s bright and shiny and full of so much thrill and promise.

All of this should look reasonably familiar. It is the process that we follow, more or less, whenever we take an engine out to the stand. Getting the engine into and out of A1 is a bit easier since you don’t have to deal with moving the diffuser out of the way, but they’re really quite similar. The slightly different flavor that we’ve got for this testing is the addition of the thrust vector control system. In the picture below you can see where these hang. The engine mounts up with the gimbal bearing to the stout, yellow thrust take-out structure.

The two hydraulic actuators are also attached to the thrust take-out structure but slightly outboard and at 90 degrees apart. These actuators are what will swing the engine around as if we were steering a vehicle. Here, below, is another, closer view of the thrust take-out structure and the actuators.

In the picture below, E10002 is mounted up to the thrust take-out structure. The gimbal bearing is the shiny object towards which the arrow is pointing. If you look over to the right side of the picture, you can see one of the “scissors ducts,” i.e., the flexible propellant inlet ducts.

The next picture shows one of the hydraulic actuators hooked up to the engine. As you can see the tolerances are awfully tight. That’s an important vehicle integration consideration. If this was a vehicle stage rather than a test stand to which we were attaching the engine, the thrust structure and the actuators would be the responsibility of the stage manufacturer. Making sure that the stage and the engine can work together in such close quarters takes a great deal of vigilance between the two teams.

So, you’re probably asking yourself, to what do these actuators connect on the engine? That’s a very good question. It certainly isn’t obvious from the assembly pictures. The actuators connect to the forward manifold of the main combustion chamber (MCC). Below is a computer model of the MCC with the two actuator attachment points shown.

The MCC is really the heart of the whole engine, the sturdy framework stuck right in the middle, so it makes sense that when you want to push the engine around, this is what you’d have to push. This final picture below of the engine completely mounted into the stand. Again, it’s amazing to think about that whole thing being able to move about and not having one component run into another component or the actuators or the stand. It is quite the integration miracle.

Testing for E10002 on test stand A1 will commence in June. So, if all goes well, for the next J-2X update, I’ll be able to link in a video of the engine firing and twisting about.

BTW, NASA is in the process of swapping software used for posting blog articles and comments and such. As part of this process, they have to shut down the capability to accept input comments for a short time, specifically the first two weeks of July 2013. Sorry about that. But after that, it ought to be up and running as normal.

Liquid Engines Extra — Introducing LEO

The following picture is a test. Find me amidst the mess…

That’s right. I’m the handsome chap with the snazzy specs. See, I’m just oozing with exactly the vitality that you’d expect from your typical government-trained civil servant! Well, okay, so maybe I’m not exactly Milton Waddams. All personal resemblance and charm aside, I don’t actually have quite that much paper clutter in my office. No. Instead, I just have electronic clutter. Indeed, if someone spent the time to actually print out all the stuff on my computer here at work, then we wouldn’t need a rocket to get beyond the moon. We could just build a staircase from the resulting gargantuan pile of paper.

Why is this the case? It could be that I’m just a data hoarder. Some people hoard clothes (that’s not me). Some people hoard automobiles (that’s not me). Some people hoard books (okay, that is me). And some people hoard data. All data. All of the way back to class notes from Aerodynamics I (AERSP 311, junior year, professor Bill Holl — great teacher, great guy). Yes, okay, so maybe I’m a little guilty of all that. In my defense, however, I will simply say that there’s a lot that goes into making rocket engines and much of it is quite removed from the exciting cutting-metal stuff or the making-smoke-and-fire stuff. And I get to stick my nose into much of it. For this article, I am going to reveal super-duper, deep-and-dark secrets that they won’t even teach you in college. I am going to tell you a little about …

…wait for it…

…project management. Yes, it’s true: I am going to invite you into our little piece of office space and I will do so to tell you about how the office has recently evolved. Also, towards the end as a reward, I’ll give you an update of J-2X testing progress to date.

So, let me introduce you to LEO —

No, none of those LEOs. Instead, let me introduce you to the Space Launch System (SLS) Program Liquid Engines Office (LEO). The LEO is responsible for development and delivery of the liquid rocket engines for the core stage and upper stage of the SLS vehicle. For more information about the SLS Program in general, I highly recommend the following site: https://www.nasa.gov/exploration/systems/sls/

(Oh, and I want to say something briefly about the term “liquid engines.” Probably every single person who would bother to read a blog about engines or rockets or space travel in general knows that this is just a shorthand term. No, we are not talking about engines made of liquid — although that would be really cool. “Liquid engines” is a quick and easy way to denote “liquid-propellant rocket engines.” In case I’ve disappointed anyone, I’m sorry. If ever we are able to make an engine out of liquid, I promise to be the first to report it. Probably the most far-out thing that I once heard was the suggestion to make a hybrid rocket motor using solid hydrogen and liquid oxygen. I cannot even imagine what the infrastructure would be to make the use of solid hydrogen plausible, but you never know…)

For the SLS vehicle, the upper-stage engine is the rocket engine so near and dear to our hearts after several years of design and development and fabrication and assembly and test: J-2X. The core-stage engine is the RS-25. No, the RS-25 is not a brand new engine. Rather, it is the generic name for that workhorse of the last thirty years, the Space Shuttle Main Engine (SSME).

At the end of the Space Shuttle Program, there were fourteen SSMEs that had flown in space on the Shuttle and that still had usable life remaining. I’m not sure that everyone knows this, but rocket engines have limited useful lives. I guess that most things do, but with rocket engines it’s often pretty short. Think of them like cherry blossoms (popular motif in Japanese tattoos): amazingly beautiful and quickly gone too soon. The stresses within an operating rocket engine are tremendous. For example, the J-2X has an official, useful life of only four starts and less than 2,000 seconds of operational run time after the engine has been delivered for use as part of the vehicle. No, the engine doesn’t crumble into dust after that, but based upon our certification strategy and on our analysis of margins, that is the official life for our human-rated launch system. After that point, depending on the proposed usage and risk considerations, and based on the likely reassessment of our margins with the proverbial “sharper pencil,” we can and do routinely talk ourselves into longer active lives for engine hardware. On the test stand, we can test the J-2X upwards of 30 times and for lots of run time, but that is a lower risk situation. Nobody is riding the test stand into space.

Thus, when you come to the end of a program and you have fourteen engines with remaining, usable life, then you’ve got one heck of a residual resource. In addition, there was one SSME assembled and ready to go, but it never made it to the test stand or the vehicle. So it’s brand new. And, on top of that, there were enough leftover pieces and parts lying around of flight-quality hardware to cobble together yet another engine. And, there’s more! (Yes, I feel like the late-night infomercial guy, “and if you call in the next 10 minutes you will get this special gift!”) There are also two development SSMEs. These are not new enough to fly, but they are useful for ground testing and issues resolution. That means that there are a total of sixteen RS-25 flight engines and two RS-25 development engines available to support the SLS Program.

However, before your excitement bubbles over, you have to understand that when you see a sign for “free puppies,” you probably shouldn’t take that whole notion of “free” too literally. As in, well, not at all. Yes, we still have an extraordinary asset in the residual RS-25 engines. No question. But, we have work to do to integrate them into the SLS Program. In a future article, I will discuss the multiple facets of this work. By the way, I cannot claim to be immune from the “free puppies” thing myself. Meet Ruugie –

The Liquid Engines Office (LEO) was formed to manage both the J-2X and the RS-25. This office will also manage other liquid rocket engines used to support the SLS Program as it matures. It was decided from a project management perspective that it would be best to have one office manage both engines. In this way, we can be more efficient by leveraging the expertise across various disciplines and components. For example, do we really need two turbomachinery subsystem managers? No, Gary Genge is our turbomachinery subsystem manager and in that position he can understand and evaluate the relative programmatic and technical risk across all of the various turbomachinery pieces under his purview. If in some utopian future our office responsibilities expands to three or four or eight different engine development or production efforts, we would, in theory, maintain the same structure but provide Gary with the support necessary to effectively manage turbomachinery across so many activities.

So, for LEO we have subsystem managers for Engine Systems (effectively systems engineering and integration), Engine Assembly and Test (also includes asset management, logistics, and operations), Engine System Integration and Hardware, Valves and Actuators, Engine Control Avionics, Turbomachinery, and Combustion Devices. LEO is supported by a Chief Engineer, a Chief Safety and Mission Assurance (S&MA) Officer, Program Planning and Control (i.e., the business office), and Procurement. Plus, of course, we have support from the engineering and S&MA organizations across the many technical disciplines. The structure is really quite similar to how we’ve been managing J-2X for these past several years. We’ve just expanded our responsibilities.

So, that’s LEO and I’ll be talking more about RS-25 and SLS in the future.

Now, while I’ve been off doing my little part to get the foundation of LEO solid, including refreshing and getting into place our prime contracts for both J-2X and RS-25, how has J-2X been doing? Well, in short, J-2X has been just cruising along. E10002 has gone through six tests on NASA Stennis Space Center (SSC) test stand A-2. Below are a series of images showing what an E10002 start looks like if you stood in view of the flame bucket (which I would very strongly advise against, by the way):

First, all you see is the facility water being pumped into the flame bucket. Then you can see the ignition and everything glows orange. Then the whole flame bucket is filled with exhaust. And, finally, the exhaust coming barrelling down the spillway and eventually engulfs the camera. The final step is not shown since there’s nothing to see but solid whitish grayness.

Here are the stats on the six tests:
     • Test:          A2J022          2/15/2013          35 seconds duration
     • Test:          A2J023          2/27/2013          550 seconds duration
     • Test:          A2J024          3/07/2013          560 seconds duration
     • Test:          A2J025          3/19/2013          425 seconds duration
     • Test:          A2J026          4/04/2013          570 seconds duration
     • Test:          A2J027          4/17/2013          16 seconds duration

So the total accumulated time is 2,156 seconds. Tests #22, #25, and #27 all experienced early cuts, but all three were instigated by different flavors of instrumentation or monitoring system issues or oddities. The engine is fine and running well. Some of the key objectives included gathering additional data about the nozzle extension cooling characteristics, additional samples of the turbomachinery design, and main chamber combustion stability trials. Something else that we did for this test series is that we tested a very special fuel turbopump port cover. Here’s a picture of it:

Now, port covers are not something about which one usually says anything at all. What makes this one special is that it was made by using a process known as Selective Laser Melting (SLM). That is a fabrication method that is somewhat analogous to “3-D printing.” A long time ago, I wrote a blog article about a gas generator discharge duct that we made for component-level testing using this technique. This, however, is an engine test and this small, seemingly innocuous, piece of engine hardware may be the humble harbinger of a revolution in rocket engine fabrication. The fact that we systematically stepped through the process of validating this port cover as a piece of hardware for an engine hot fire demonstration paves the way for pursuing other parts in the future, more complex parts, and, hopefully one day, regular production parts as part of a human-rated launch architecture.

E10002 was removed from NASA SSC test stand A-2 on April 30th. It is currently being retrofitted with instrumented inlet ducts and other hardware in preparation for the next phase of testing that will occur on NASA SSC test stand A-1. As you’ll remember, in the past A-1 was used for the PowerPack Assembly testing. Well, the talented and productive folks at NASA SSC remodeled the stand back to the configuration for engine testing. The current plan is to install E10002 into A-1 by mid-May and to perform a series of five to seven tests through probably August. The reason for using A-1 for the next series is because that stand does not have a diffuser. That means that we can gimbal the engine, i.e., twist it around as if we were providing steering for a vehicle. The thrust vector control (TVC) system composed of the hydraulic push-pull actuators that will be performing the gimballing is a component belonging to the stages element of the vehicle. This testing will be providing those folks with data to inform their system design for the SLS Program. See, it’s all win-win when we play nicely together.

And, finally, right on the heels of E10002, the assembly of E10003 will commence in June with scheduled installation into NASA SSC A-2 in September. That’s my report for where things stand. To finish up, I’ll leave you with a purely gratuitous glamor shot of the J-2X. Isn’t she pretty?