USAF Certification a possible hindrance to future F9 Development?

[Llian Rhydderch]

If they want to be certified they can stop complaining about how the process is long winded, expensive, and unnecessary and either let it occur or not go after the launches, simple.

Right.  No one should ever question a review process that costs more than the rocket being examined... especially if it has been long-established and proven.  No one should be so bold as to 'suggest' that this process is less-than-perfect.

Correct. You don't question a potential customer's certification process. You put up or shut up.

Thank you!  You just provided a beautiful example of the gross distortion in any market by the existence of a monopsony buyer.

When there is only one buyer that has significant market power, as the US government does in the space "market", you darn well better question the actions of that monopsonist.

Moreover, in the US, as tattered as our Constitutional Federal Republic is with its government of (supposedly) limited powers, there remains some kernal of an idea that we still have, or should have, a government of the people, for the people, by the people.  Therefore, it is of course right that the government be questioned.

And especially so when our "captalism" has devolved into a form of state capitalism rather than market capitalism.

The USG is, shall we say, a very unusual customer.

So I say, Questions all around.  Here, here.   

[Darkseraph]

If they want to be certified they can stop complaining about how the process is long winded, expensive, and unnecessary and either let it occur or not go after the launches, simple.

Right.  No one should ever question a review process that costs more than the rocket being examined... especially if it has been long-established and proven.  No one should be so bold as to 'suggest' that this process is less-than-perfect.

Correct. You don't question a potential customer's certification process. You put up or shut up.

Thank you!  You just provided a beautiful example of the gross distortion in any market by the existence of a monopsony buyer.

When there is only one buyer that has significant market power, as the US government does in the space "market", you darn well better question the actions of that monopsonist.

Moreover, in the US, as tattered as our Constitutional Federal Republic is with its government of (supposedly) limited powers, there remains some kernal of an idea that we still have, or should have, a government of the people, for the people, by the people.  Therefore, it is of course right that the government be questioned.

And especially so when our "captalism" has devolved into a form of state capitalism rather than market capitalism.

The USG is, shall we say, a very unusual customer.

So I say, Questions all around.  Here, here.   

Not sure why you would say that capitalism has "devolved" into state capitalism, as if its some lower form. That's actually an improvement if you're interested in space access. In a pure market environment, Space X would probably not exist, or not for very long. Musk won't even IPO the company in the next 10 years because markets are anathema to his goals. Space exploration hasn't been held back because the government is the only customer, its been held back by no one being able to make a business case that closes outside of government contracts and communication satellites. There's the old saying "How do you become a millionaire at space.?...start of as a Billionaire!" I'm generally glad the government funds this stuff with all its own imperfections, because the alternative is zero.

[Llian Rhydderch]

If they want to be certified they can stop complaining about how the process is long winded, expensive, and unnecessary and either let it occur or not go after the launches, simple.

Right.  No one should ever question a review process that costs more than the rocket being examined... especially if it has been long-established and proven.  No one should be so bold as to 'suggest' that this process is less-than-perfect.

Correct. You don't question a potential customer's certification process. You put up or shut up.

Thank you!  You just provided a beautiful example of the gross distortion in any market by the existence of a monopsony buyer.

When there is only one buyer that has significant market power, as the US government does in the space "market", you darn well better question the actions of that monopsonist.

Moreover, in the US, as tattered as our Constitutional Federal Republic is with its government of (supposedly) limited powers, there remains some kernal of an idea that we still have, or should have, a government of the people, for the people, by the people.  Therefore, it is of course right that the government be questioned.

And especially so when our "captalism" has devolved into a form of state capitalism rather than market capitalism.

The USG is, shall we say, a very unusual customer.

So I say, Questions all around.  Here, here.   

Not sure why you would say that capitalism has "devolved" into state capitalism, as if its some lower form. That's actually an improvement if you're interested in space access. In a pure market environment, Space X would probably not exist, or not for very long. Musk won't even IPO the company in the next 10 years because markets are anathema to his goals. Space exploration hasn't been held back because the government is the only customer, its been held back by no one being able to make a business case that closes outside of government contracts and communication satellites. There's the old saying "How do you become a millionaire at space.?...start of as a Billionaire!" I'm generally glad the government funds this stuff with all its own imperfections, because the alternative is zero.

Agree with you on Musk and taking SpaceX public on the stock market.

But don't confuse the "stock market", with market relations and the market process more generally.  Musk has ownership of whatever part of SpaceX he owns (a majority, I believe).  His choice to retain that ownership, and act with the ordinary property rights this affords him, is absolutely a sterling example of market process in action:  he has no doubt looked at the expected benefit, and the expected cost (including likely redirection of company focus on strategic objectives he thinks important, increased focus on short term results and less on long term results, etc.), and made a determination not to exchange his property right in SpaceX under the rules of the game of a public IPO under US law.  That is a market choice, and part of the working out of a market process.

Whether the little bit of private market action that is happening in NewSpace today is sufficient, and can continue long enough to make a difference, is an open question.  I hope it does.  Some have different views.

But I certainly can see a huge difference in the kinds of technological innovation made under (largely) private market relations vs. those that are made (and frequently, not made) under state control, whether the state is run by a monarch, an autocrat, or a democratic majoritarian political regime.  I believe that the empirical evidence is that we see much more innovation under private market relations--where property rights, the rule of law, freedom to contract, freedom of new entrants to enter the market, freedom to leave the market, and a few other key institutions are in place--will tend to significantly outperform the state capitalism model.

So relative to the OP topic, the growth or extension of excessive regulatory control by the state over launches will tend to reduce innovation and increase costs on F9 innovation by SpaceX.  That's the principal argument I've been making. 

[Lar]

Debates about capitalism, the stock market, patents, etc are off topic. No more.

[john smith 19]

On their brochures, they claimed that the Falcon 1 having only one main engine was good for reliability

That's exactly the argument made for the EELV designs. Which were not meant to be human rated

and that the Falcon 9 having 9 engines was good for reliability (engine out)!

And that's the argument that's made for crew rated vehicles.

Reliability statistics say more parts --> more parts to fail --> less reliable.

But smart engineers know 2 things. 1) Bigger engines have much worse combustion instability problems 2)All parts are not  equally prone to failure.

The math says the one big perfect engine is better than the 7 dwarfs.

Except the math does not take into account the consequence of failure.
1 engine fails on 1 engine LV --> Loss of Mission
1 engine fails on multiple copies of same engine --> Possible continuance of mission.

That qualification on engine type is because an engine failure on Shuttle takeoff (especially the SRB's) and certainly within the first 2 minutes would have been a LOC.

I know what I'd prefer to be on.

In respect to the thread title it'd be interesting to see given that the USAF was intimately involved with the EELV designs if they'd allow a 9 engine 1st stage or rule "It's too risky. Our statistics estimate one of your engines would fail on nearly every flight" when F9 was being designed. 

[Jim]

That's exactly the argument made for the EELV designs. Which were not meant to be human rated

But smart engineers know 2 things. 1) Bigger engines have much worse combustion instability problems 2)All parts are not  equally prone to failure.

The math says the one big perfect engine is better than the 7 dwarfs.

Except the math does not take into account the consequence of failure.
1 engine fails on 1 engine LV --> Loss of Mission
1 engine fails on multiple copies of same engine --> Possible continuance of mission.

A lot is wrong here

a.  Not originally designed to be is not the same as meant to be.  There is nothing that says EELV's can't be human rated.

b.  Smart engineers know that bigger does not equate to large.  Large engines have combustion instability problems.  EELV engines are not large nor do they have combustion instability issues.

c.  But more of the same parts are have more chances to fail

d.  The multi engine scheme is just a marketing ploy.  More than a 2/3's of a Falcon 9 flight time has the same consequence from a engine failure as the EELV's.  It uses a single engine (which is also significantly different than the first stage engines) for second stage flight.   Find the time that a first stage engine on a US vehicle (other than SpaceX) cause a LOM.  It will be in the 90's and an Atlas with a bad set screw.  You have to go further to find an engine that let go.

[llanitedave]

Wow, Jim!  That was positively loquacious!

And I agree with the argument.

[john smith 19]

A lot is wrong here

a.  Not originally designed to be is not the same as meant to be.  There is nothing that says EELV's can't be human rated.

AFAIK "Can be man rated in later versions" was not a design criteria of the EELV programme. It happens that crew rating the Atlas V and the RD180 were not particularly difficult. IIRC the big item was adding a 2nd hydraulics system. If space had been tighter on the engine layout that could have gotten very tricky to do.

b.  Smart engineers know that bigger does not equate to large.  Large engines have combustion instability problems.  EELV engines are not large nor do they have combustion instability issues.

I'm not sure the theory on CI is still good enough to guarantee that in the design phase. I'd say bigger chamber and nozzle --> more resonance modes to excite --> probability of triggering a CI goes up.

That said IIRC the scarfed bell nozzles on the Shuttle also suffered from CI. Prior to that it was believed that CI in such small engines was simply impossible.

IOW smaller chambers improve the odds of not encountering CI in development, but they do not guarantee it. 

c.  But more of the same parts are have more chances to fail

Which was my question about if the USAF had the sort of insight into the F9 development that they had to Delta IV and Atlas V would they have allowed it?

d.  The multi engine scheme is just a marketing ploy.  More than a 2/3's of a Falcon 9 flight time has the same consequence from a engine failure as the EELV's. 

An argument that could be made of the Saturn V. Engine out helped save the crew of Apollo 13.

It uses a single engine (which is also significantly different than the first stage engines) for second stage flight.   

I'm a fan of engine clusters but recalling cases of asymmetric start on a Centaur when the thrust imbalance broke up the stage I'm not sure anything less than 5 engines is viable. I'll also note that a Merlin 1d Vac is (AFAIK) still about 90% of a Merlin 1d. An RL10 (or two) is not an RD180 or even close to an RS68.

Find the time that a first stage engine on a US vehicle (other than SpaceX) cause a LOM.  It will be in the 90's and an Atlas with a bad set screw.  You have to go further to find an engine that let go.

I presume you're talking about the Orbital secondary payload on the maiden F9 2.0 flight?

AFAIK no one held a gun to Orbcomm's management to force them to fly on the F9.

They could of course have flown as a primary on the Pegasus XL. How much is an XL these days?

Or waited for an EELV ride.

Or maybe an Ariane 5 ride.

But they didn't. 

I'll hazard a guess that price played a big part in their decision, as I would guess it does for nearly every secondary payload. 

IOW they took a calculated (well I presume someone did a PRA  ) risk based on cost Vs benefit. On the day it didn't go their way.

They're not the first (and I doubt they will be the last) secondary payload that did not achieve their full goals. 

AFAIK most that fail don't have insurance to begin with (or a PR department to express their annoyance) so they just have to go back to the lab and build another. 

BTW Did Orbcomm build a new one with the insurance money? they certainly talked like it was a total failure and they'd need to build another one to get the test data.

[HappyMartian]

That's exactly the argument made for the EELV designs. Which were not meant to be human rated

But smart engineers know 2 things. 1) Bigger engines have much worse combustion instability problems 2)All parts are not  equally prone to failure.

The math says the one big perfect engine is better than the 7 dwarfs.

Except the math does not take into account the consequence of failure.
1 engine fails on 1 engine LV --> Loss of Mission
1 engine fails on multiple copies of same engine --> Possible continuance of mission.

A lot is wrong here

a.  Not originally designed to be is not the same as meant to be.  There is nothing that says EELV's can't be human rated.

b.  Smart engineers know that bigger does not equate to large.  Large engines have combustion instability problems.  EELV engines are not large nor do they have combustion instability issues.

c.  But more of the same parts are have more chances to fail

d.  The multi engine scheme is just a marketing ploy.  More than a 2/3's of a Falcon 9 flight time has the same consequence from a engine failure as the EELV's.  It uses a single engine (which is also significantly different than the first stage engines) for second stage flight.   Find the time that a first stage engine on a US vehicle (other than SpaceX) cause a LOM.  It will be in the 90's and an Atlas with a bad set screw.  You have to go further to find an engine that let go.

With an engine failure on the first stage Loss of Mission may be a given, however one advantage of multiple engines on a first stage on a crewed mission might be that you don't have to immediately fire up the second stage or the high g abort rockets. Instead, you could decide to just 'go long' by continuing to burn the remaining first stage engines.

Again, to 'go even longer', you might also continue to refrain from using the abort system option and after the first stage burns through all of its propellant, you could decide to burn some or all of the propellant in the second stage. With enough delta-v capability, a flightpath diversion and more optimal ocean recovery site may be a possible option.

Obviously you might no longer need or want to accelerate or gain altitude, but by carefully deciding to 'go long' you might get a bit closer to a ship with a known location, an island, better recovery weather conditions, daylight, or a continent with significant ocean search and rescue capabilities.

Where the crew or a heavy robotic satellite or nuclear powered rover 'comes down' after a first stage engine failure may have different safety or political consequences. Being able to 'go long', despite a Loss of Mission, by continuing the first stage's powered flight with a reduced number of engines may sometimes be useful in launches from different sites around the world. 


Edited.

[watermod]

d.  The multi engine scheme is just a marketing ploy.  More than a 2/3's of a Falcon 9 flight time has the same consequence from a engine failure as the EELV's.

All this reminds me of the failure statistical analysis equations as they applied to electronics evolution.
It boiled down to an individual PHYSICAL parts count. 

A tube had more parts than a transistor.  The transistor was essentially one part on a single piece of  semiconductor so it's part count was pretty close to 1.  The tube was in single or low double digits  but that's already worse in the statistics.

Then industry went to the IC chip resulting in a many device design having a parts count of essentially 1 wafer die for the failure statistic (if one left out the manufacturing failures).   This resulted in the current state with hundreds of millions of devices on a die having a failure statistic of near 1. 

Now the multi-engine bit of the Falcon 9s is multiple devices so simple statistical failure analysis suggests a higher system failure rate - however - this ignores strategy.

The Falcon 9 has another strategy borrowed from the electronics and computer fields.  That is fault tolerance.   This strategy had to be used in the computer field in its early days when the mean time to failure of a tube or other part was on the order of five minutes.  Computers were slower than 4-banger calculators then so the average program could expect several hardware failures while running.
There were two solutions - 1 the 3 device  vote and the other constant saving of state to some medium.   
The latter has no analogue in rocketry but the former does.  Nine engines with sensors where it is unlikely that a problem with one will infect the others (the shielding between the engines) improves the system failure statistics provided that the system function doesn't require all the engines.  One would need to have access to the actual equations defining that system to see by how much it actually improves the system as a whole.   (where the system as a whole == the rocket mission)

As to the first strategy that is why I asked some questions in other discussions about rockets engines on a chip by the MIT guys and some of the electric/magnetic propulsion schemes.   From a failure standpoint a surface of rocket propulsion would be the equivalent of a dense IC in the computer world with the failure equation dropping to near 1.   But, that is not what SpaceX implemented.

[Lourens]

Lots of talk about what the statistics would say, but very little actual statistics in this thread. Let's fix that.

I can't do an entire F9 rocket, for lack of information, because it's way too much work, and because I'm not an actual rocket reliability statistician. But I think I know enough statistics to do some calculations on 1 engine vs. 9 engines, so that's what I'll do. I'll ignore any potential issues not related to the engines, and I'll ignore any information (on production processes, tests, etc.) except the engines' success rate in actual launches. We'll compare an F9 first stage with 9 Merlin 1D engines to a hypothetical Falcon 1e first stage with 1 Merlin 1D engine. A comparison with a single engine with equal thrust to 9 Merlin 1Ds would be interesting, but we don't have such an engine or reliability statistics on it, so we'll go with F9 vs. F1e.

Some assumptions:

* The Merlin 1C, 1D and the Merlin 1C and 1D Vacuum as used on the F9 are similar enough to be grouped together for the purpose of computing reliability statistics.
* The F9 can tolerate exactly 1 engine failure anywhere in flight
* The F9 engines are isolated enough that a failure of 1 won't affect the others

So far there have been 10 Falcon 9 flights, with 99 out of 100 Merlin 1D engines running successfully, and 1 failing in flight. That gives a (maximum likelihood) estimate of the success probability of a Merlin 1D of 99%. I'll get back to that figure later, but let's start with that.

The probability of success of the Falcon 1e first stage is thus 99%. For Falcon 9, success is if 9 engines work, or exactly 8 engines work. The probability of all 9 engines working is

0.99^9 = 0.91352

and that would be the success rate if Falcon 9 didn't have engine out capability. But it does, so we have to add the probability of exactly 8 engines working. This is

9 * 0.99^8 * 0.01 = 0.083047

We multiply by 9 because we don't specify which engine fails, so there are really 9 scenario's each with the same probability: engine 1 failing, engine 2 failing, and so on.

The total probability of success for the F9 first stage is the sum of those, which equals 0.99656 which is slightly better than the 0.99 of the Falcon 1e. So 9 engines wins!


That's not the whole story though, since that 99% is only a best guess given the data. The actual probability of success may be lower, and we may just have been lucky. Or the other way around. We have to take that into account, otherwise we would conclude for example that because the Merlin Vacuum ran successfully for 10 out of 10 launches, it cannot fail in the future, and that's of course silly. We can do this by calculating a confidence interval. From the 99 out of 100 recorded successes of the Merlin in general, we can say with 95% confidence that the actual probability of success is at least 0.96159 (single-tailed 95% Jeffreys interval, for those computing along).

So we can be 95% sure (not 100% because we have limited information) that the probability of a Falcon 1e first stage launching successfully is at least 96.159% (not 100% because the Merlin 1D is not perfect).

For the Falcon 9 with single engine out capability, we can be 95% sure that it has at least a probability of 95.562% of launching successfully.

This is actually a pretty interesting result, since we see now that the F9 has a lower success probability than the F1e, whereas at 99% single-engine success rate it had a higher success probability. Whether 1 or 9 engines is better depends on the reliability of those engines!

This means that SpaceX can have their cake and eat it, too, marketing-wise. For the original F1, with Merlin in the development stage and still relatively unreliable, a single engine has a bigger chance of success, even with engine out capability on the multi-engine vehicle. For the current F9 and the more developed Merlin 1D, based on the 99 out of 100 success count, we can be about 91% sure that the 9-engine configuration is more reliable than a single (Merlin 1D) engine one.


To get somewhat back on topic, I have a question. I've seen requirements for a small number of consecutive successes for Falcon 9 to be allowed to fly certain payloads. I don't remember the exact number, but it was too low to give any kind of confidence on its own. So I assume that the reliability assessment is mostly based on knowledge of the manufacturing and QA processes in production. Are these numbers combined with the flight data in a Bayesian manner? Or are they simply separate requirements with separate associated calculations? Does the stage have to be recertified if its manufacturing process doesn't change, but it has failed in flight after being certified?

[macpacheco]

Incremental changes on every commercial launch (within reason) then once a bunch of changes have been proved out do a "block upgrade" to the DoD version, all at once.

Who says the commercial customers (and their insurers) like being paying crash test dummies on every flight? Why wouldn't they opt for certified, stable and proven version too?

This would make sense if your payload is worth a billion dollars or so.
But if your payload costs just a few times the price of the launch, and SpaceX reputation is near spotless, customers are willing to accept the risk for the reduced launch costs.
I recall reading that a few of current/recent SpaceX customers state the launch price they got from SpaceX made the satellite operation a radically better deal than using other typical launch providers.
For instance, I think GPS satellite launches shouldn't require this certification, the payloads are cheap enough and built in production runs of 8 or more birds usually per model.

[Coastal Ron]

Incremental changes on every commercial launch (within reason) then once a bunch of changes have been proved out do a "block upgrade" to the DoD version, all at once.

Who says the commercial customers (and their insurers) like being paying crash test dummies on every flight? Why wouldn't they opt for certified, stable and proven version too?

This would make sense if your payload is worth a billion dollars or so.
But if your payload costs just a few times the price of the launch, and SpaceX reputation is near spotless, customers are willing to accept the risk for the reduced launch costs.

Launch customers have a vested interest in lowering launch costs, so risking a launch or two makes business sense.  That applies to both commercial and government.

I recall reading that a few of current/recent SpaceX customers state the launch price they got from SpaceX made the satellite operation a radically better deal than using other typical launch providers.

Most likely that's true for both Orbcomm and Iridium.