LEO Extra: The Engine as a House

Posted on by .

Imagine that you want to build a house.  I have never built a house or had a house built for me but I know a number of people who have.  They have all used different words to describe their experience, but if I could encapsulate their descriptions into a single expression it would be this, “Building a home from scratch is an adventure.”  That word, “adventure,” is useful here since it can mean a whole spectrum of positive and negative experiences.  For this article, I would like to describe the analogous adventure of rocket engine development.  While the processes of building a house and developing a rocket engine do not mirror each other exactly, they do have some similarities from the perspective of project management and procurement considerations.

BlBill1

Why does one build a house?  Because one perceives a need.  Is it an absolute need?  Are there not other houses around that might fit it bill?  Yes, perhaps.  But a need is perceived and that need is translated to an overall objective with an array of attendant requirements.  I once talked with a technician at a small company in California that was doing some very specialized testing for us out in the desert.  This was during development of the X-33 vehicle.  In years past, he had worked for a housing contractor and he was describing some of the more odd custom configurations for houses that they’d been contracted to build.  One that I remember him describing was a 20,000 square-foot home (over ten times the gross size of my own home) with all kinds of specialized rooms, seven full or partial bathrooms, but only one bedroom.  I assume that such a home fit the bill for someone with particular needs but I cannot imagine that it would do very well on the re-sale market.

The point, however, is that you start with your objective, “I want a new home built to my needs,” and then you list out your needs.  Maybe you want a game room.  Maybe you have a large family and need seven bedrooms.  Maybe you or your spouse is a gourmet cook and so you want a kitchen that fulfills that talent.  Or maybe you have a specific, scenic plot of land you want something that fits into the terrain a la Frank Lloyd Wright and the Fallingwater house.  Whatever it is that is your combination of requirements, you have decided that there is nothing on the market that meets these needs and so you are going to build a new house from scratch.

water

That is not too different from what we do with rocket engines.  It starts with the mission and the mission is translated to a vehicle and then the needs of the vehicle become propulsion needs and therefore rocket engine requirements.  In the past, I’ve talked about process of requirements decomposition along these lines.  In some cases, you can find something “on the market” that fits the bill.  You can think of the RS-25 for the Space Launch System (SLS) along these lines.  The RS-25 was the Space Shuttle Main Engine and now, with some adaptations, it will become the core-stage engine for the SLS Program.  It’s like a house on the market that would meet your needs with a few renovations.  In other case, you look around and nothing quite fits the bill.  That is like J-2X.  That effort has been like building a house from scratch.

So, you want to build a custom house?  What’s next?  Well, you hire an architect to design the thing.  Interestingly, on the subject of NASA and rocket engines, we at NASA sometimes do some of the architectural design work ourselves.  This was especially true in the past.  It was as if you were yourself the architect and you could design much of the house yourself.  These days, however, it more often becomes a collaborative process for the architecture of the engine and that’s probably also quite true when building a house in that the architect and the owner work together in developing the final plans.  Who then is responsible for building the house?  Well, that’s your general contractor.  On the rocket engine side, that is what we call our prime contractor.  They are responsible for delivering the final product.  In some cases, the architect and the general contractor function as a unit when building a house.  Other times, they are separate (and that, I’ve been told, can lead to all kinds of “fun”).  For a rocket engine, there is rarely a separation between architect and contractor other than the degree to which NASA assumes the role of architect.

owner

Above is a simplified sketch of how a new custom house comes to be (I am really quite the artist, aren’t I?).  For a rocket engine, everything to the right of “owner” is what we contract out.  These days, depending on the project, we contract out anywhere from 70% to 90% of the expense of developing a rocket engine.  Thus, our “government engine” is almost always, in reality, the product of a commercial company.

Over to the right of the little diagram, I show various entities listed as “subcontractors.”  For a home construction effort, these could be plumbing and electrical subcontractors.  They could be the roofers or perhaps the landscape folks.  It is rare that a general contractor maintains all of these disciplines within his own shop.  Sometimes the general contractor is nothing but a manager and all of the real construction work is done by hired subcontractors.  Sometimes, the general contractor does have his own capabilities and only hires out for a couple of specialized things.  It all depends upon the business model that is found to work best.

The same is true for rocket engines.  Our prime contractor for J-2X and RS-25, Aerojet Rocketdyne, has a number of exceptional in-house capabilities.  But there are also a number of things that are obtained by going through subcontractors or vendors.  In our world, we typically think of “subcontractors” as a company that is working on and delivering a large piece of the engine like, for example, the engine controller whereas we typically think of “vendors” or “suppliers” as those who supply smaller pieces (but, admittedly, the lines across these definitions sometimes blur).  Of the latter, vendors and suppliers, we have many examples.  One is a small company that specializes in the fabrication of high-quality metal bellows.  This is an extremely important and difficult skill.  There is another, somewhat larger company that specializes in making tubes.  While that sounds straightforward, consider the tubes that make up the RS-25 nozzle.  These are about six-feet long with a cross-sectional profile that changes in size and shape along its entire length.  And the necessary tolerances are extremely tight.  There probably are not many companies on the planet who can deliver what is necessary make the RS-25 nozzle a reality.  Or in some cases, as with the main combustion chamber, while the prime contractor does much of the machining and assembly work in house, they are dependent upon a supplier for the specialized alloy that is used for combustion chamber liners.  It’s not something that you can get off the shelf of the local hardware store.  The truth is that we depend upon a broad array of exceptional subcontractors, vendors, and suppliers to make rocket engines a reality.

chamber

 The next issue to consider is what you do as owner as your house is being designed and built.  On the one hand, you could walk away, take a trip to Nepal, and come back to find your house complete, but if you’re anything like me, that’s not what would happen.  As I said, I’ve never been through the process personally, but if I was going to do it, I would be checking in on the progress on a regular basis.  Now, I am quite sure that there is a balance to be maintained such that insight doesn’t become interference, but it is, after all, my money being spent to pour the concrete and put up the framing and hang the doors, etc., so I ought to be able to have some access and right to judge the effort as it goes along.  And, should we run into schedule delays and cost overruns, then I ought to have the right to raise an indignant ruckus.

Again, the same is true for our rocket engines except that we are spending a great deal more money than the cost of your typical house and so our processes for contract performance surveillance are a bit more involved and formal than showing up at the construction site with a level and a plumb line at the end of the workday.  We have continuous insight into what the contractor is doing and why.  While we don’t control the subcontracts, we have access to the records, insight to the activities, and, for the larger ones, the responsibility to review and authorize contract placement.  Quite simply in this case, we don’t want large chunks of our money to go to shoddy subcontractors.  Thankfully for us, our prime contractor does an excellent job of ensuring that their subcontractors and vendors are up to snuff thereby helping to ensure their own success.  All of this is on a day-to-day basis.

We also have monthly project reviews where we assess technical progress coupled with business considerations using a system called “earned-value management” or EVM.  EVM is a budgeting and scheduling tool that continuously charts progress and expenditures against the approved plan.  With EVM, the contractor can identify, and we can see, potential problem areas within the overall project down to the detailed task level, if necessary.  With an effort as extensive as developing a rocket engine, you cannot expect everything to go smoothly at all times so you don’t use EVM as a scream-and-yell tool for punishment.  Rather, it is properly used as a way to re-prioritize allocated resources, to prioritize the application of reserves (if available), or to initiate additional management oversight or whatever else might be necessary to address the problem.

chart

 And, finally, we have an extremely formal process for contract oversight built right into the federally regulated procurement process.  It is called, simply, “performance evaluation.”  On a regular basis, a formal report is compiled and submitted wherein the contractor is evaluated on a range of established criteria spanning technical, business, and managerial considerations.  A board of high-ranking officials reviews this report and the contractor receives what is the equivalent of a report card.  Depending on the structure of the contract, this report card can determine how well the contractor is paid and, therefore, it can have significant repercussions from a corporate perspective.  In other words, this becomes very serious stuff.

The key thing to remember at all times when engaged in contractor oversight – and I suspect that this is just as true with building a house as it is for building a rocket engine – is that you are successful only to the degree that everyone is successful.  While there may be a momentary sanctimonious thrill in identifying that the plumbing subcontractor is an unreliable scoundrel, if you don’t actually find a way fix the problem in an efficient manner, then your house is still going to have lousy plumbing or you are going to go broke with cost overruns or, possibly, both.  Thought of properly, the relationship between the entity spending the money and the entity doing the building is a form of partnership.  Yes, the different parties have different fiduciary responsibilities and long-term objectives, but you want a good house and your general contractor wants to be recommended for potential future work for your friends who might be building their own houses in the years to come.  So a good final result is in everyone’s best interest.

No, the analogy between building a home and building a rocket engine is not perfect, but it does help in understanding the important aspects of identifying requirements, establishing an organization of an architect, a general contractor, and an array of capable subcontractors, and the issue of good performance surveillance.  People often ask me what I do for a living and I say, “I’m part of management for rocket engine development,” which doesn’t really say much that is helpful.  If I had the time, however – and if they were truly interested – I’d explain the job of the Liquid Engines Office to them with this analogy of building a house.  I would also tell them that my job is also sometimes an adventure.

hammer

 

 

 

4 thoughts on “LEO Extra: The Engine as a House

  1. John Carter

    This was an interesting read, me being a 2nd year IT student trying to learn Rocket engineering at my spare time. I have few questions , if you don’t mind me asking about the earned-value management. Can I ask here or shoot you an email? thanks for the very insightful post!

    Reply
  2. w.d.greene

    Ask here and I will do my best to answer (although I might have to call in some EVM heavy-hitters if the questions are too tough). That way others who might have similar or related questions could also see the exchange.

    Reply
  3. RussellN

    The compromises in building a J-2X to a budget would be considerably more critical than in building a house to budget. But what happens if you realize during the design process that the engine specs can’t be met within the budget? In an IT project, you either get more budget or reduce functionality – but you can’t reduce functionality on a rocket engine built for a purpose. You could spend more of the precious budget chasing down potential but unlikely solutions.
    I suppose what I’m asking is: who sets the original budget and what happens if that is found to be insufficient for the requirement? Who has to do the cap-in-hand bit?

    Reply
  4. w.d.greene

    It is convenient to think of project management as having three intertwined, mutually dependent elements: cost, schedule, and technical content. If you decrease my budget, then either you are going to get the product later than originally planned or you are going to get a lesser product. If you want more functionality, then your product will cost more and, possibly, take longer. In essence, this model relies on a couple age-old adages: time is money and nothing is free.

    That, however, is pure theory. It is useful, but it is not a perfect model. In practice, given the realities of a constrained industrial base, the truth is that for any given technical content, there is a corresponding and linked best possible cost and best possible schedule. If you want it faster, then it will cost more. If you want it slower, then it will still cost you more because you have to carry a fixed-cost component along for the ride. This latter effect is caused by the fact that while everyone in the country might have an automobile (or two, or three), very few people buy multi-million-dollar rocket engines. For industries operating with an unusually small customer base, the typical, textbook rules get distorted.

    By the way, the only time that any of this changes is when you have some revolutionary, game-changing technology come into use. When that happens, the whole process get reset to a different equilibrium point but the general relational rules would still apply after that.

    Now, to your question. What happens when you discover that your costs are escalating? Well, this happens all of the time. Sometimes, your original estimate was not very good. Sometimes, you face entirely unforeseen circumstances. Whatever the reason, you come to realize that your three-legged stool of cost, schedule, and technical content is broken. The solution to this situation can be distinctly and uniquely characterized as this: It depends.

    Some programs (in our glorious past) had sufficient budget to simply throw money at problems and resolve them via brute force. I don’t think that I’m being a pessimist when I suggest that this will not be the model for any of our big current or future programs. That’s just reality.

    In other programs, when you run into examples where you can’t achieve the technical content with our budget according to the current schedule, you delay the schedule. Usually what happens in this case is that, in the long term, you end up spending more money than originally planned, but since your immediate concerns are living to an annual budget line while still accomplishing the technical content, you find a way to convince your financiers that the greater full-program cost is still worth the investment. To be honest, this is the most common practice that I’ve observed.

    The other alternative approach to which you alluded was that you could reduce the technical content. If you can’t make what you’d originally promised, then make something else, something with less functionality or less performance. And, yes, in the world of rocket engines this is possible to some extent. For example, the J-2X is not nearly as neato as we could have made it. It could have separate, independent closed-loop thrust and mixture ratio controls. We considered that and discussed that in the early days of J-2X development but there was no way that we could meet our original budget and schedule constraints, so we were forced to go less-neato and make it open loop. This was not, however, a unilateral decision since open-loop versus closed-loop control on the engine impacts directly the margins on the stage. Thus, our more simple engine caused the stage and therefore the vehicle and the program as a whole to have less payload-to-orbit performance. This was a cost/benefit question raised early in the development and a decision was made.

    But, what if instead we’d decided to go to an independent, closed-loop system up front on J-2X and found out along the way that, oops, we completely misestimated how much effort we’d need to get there. Our choices might be to delay the whole program or to pursue open-loop control on early flights and then do a block upgrade later where we’d incorporate the closed-loop control. Perhaps we could live with less performance early and then, eventually, demonstrate the full capability later.

    My rather long-winded response is meant to show that it is all very dependent on the circumstances and that we do have some flexibility with regards to rocket engine features. Please note, however, that everyone that I know who works on rocket engines understands exceptionally well what is meant by the “badge-on-table” moment. This moment – thankfully only hypothetical in most people’s careers – is where you refuse to endorse the safety of the rocket engine project going forward and walk away from the program. Whether this moment is caused by budget constraints or by other issues, living with the responsibility to be brave enough to take that step is something that we all take very seriously whether we work in project management or engineering or safety and mission assurance. Thus, there is a bottom line below which we cannot go and we sometimes spend a good bit of time figuring out where that line is.

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *