Abort options for Starship and Starship/SuperHeavy

[Lee Jay]

My point stands: 270 non-fatal launches does not equate to a failure probability of better than 1:270.

I think it does, just not by very much.

See here:  https://moleseyhill.com/2009-09-22-bayesian-probability-of-success.html

[InterestedEngineer]

My point stands: 270 non-fatal launches does not equate to a failure probability of better than 1:270.

I think it does, just not by very much.

See here:  https://moleseyhill.com/2009-09-22-bayesian-probability-of-success.html

Yes, a naive Bayesian model is more correct for non-learning situations.  270 successful flights gives a LOC of 1- 272/273 = .0037, or 1:272.

By this statistical model the probability of LOC for the dozen Dragon-2 flights is 1 - 12/14 =  14%.  Yes I'm counting the Dragon explosion on the test pad as a failure.

If you don't want to count the test pad failure then you also don't get to count the first dozen failures as Starship works out its problems.

If we count only since last failure, it's still 1 - 12/13  = 7.7% chance of failure for the next Dragon-2 launch.

Learning Bayesian models are fare less trivial but probably what needs to be used.

So why is someone claiming 1:270 LOC for Dragon-2 and why can't that analysis be applied to Starship?

[Robotbeat]

These estimates are all within the same order of magnitude and are different only based on what statistical assumptions and models are used.

I am skeptical of Probabilistic Risk Analysis alone, but I think it CAN be used in combination with flight history.

Like, maybe do 100 consecutive successful flights/landings in a row and PRA to analyze close calls, etc, show that 1:270 success probability can be exceeded. Maybe PRA is good for a half to one order of magnitude improvement in reliability by examining close calls and addressing them. But you shouldn’t rely on it for like 2 or 3 orders of magnitude. You should be relying mostly on empirical testing by actually flying the complete system.

[Hog]

My point stands: 270 non-fatal launches does not equate to a failure probability of better than 1:270.

I think it does, just not by very much.

See here:  https://moleseyhill.com/2009-09-22-bayesian-probability-of-success.html

Yes, a naive Bayesian model is more correct for non-learning situations.  270 flights gives a LOC of 1- 215/216 = .0046, or 1:216.

By this statistical model the probability of LOC for the dozen Dragon-2 flights is 1 - 12/14 =  14%.  Yes I'm counting the Dragon explosion on the test pad as a failure.

If you don't want to count the test pad failure then you also don't get to count the first dozen failures as Starship works out its problems.

If we count only since last failure, it's still 1 - 12/13  = 7.7% chance of failure for the next Dragon-2 launch.

So why is someone claiming 1:270 LOC for Dragon-2 and why can't that analysis be applied to Starship?

emphasis mine

Wasn't that a NASA safety requirement 1:270 LOC?

"The LOC requirement states that the odds of an accident killing or causing serious injury to a crewmember be no more than 1 in 270 flights for a 210-day mission at the International Space Station. That covers all aspects of the mission, including launch and re-entry."   Lisa Colloredo-NASA

"We have a very difficult LOC requirement to meet, and we knew that when we going in," Colloredo said. The 1-in-270 LOC requirement for commercial crew is more stringent than the 1-in-90 value at the end of the shuttle program. "I would say that we've made a lot of progress, and the providers have both done a lot of redesign work to improve their LOC numbers."

"The biggest challenge, ASAP reported, was meeting micrometeoroid and orbital debris protection requirements. NASA was working to improve the modeling of the risks posed to those spacecraft from micrometeoroids and orbital debris through experiments mounted on the station as well as on Dragon cargo spacecraft."

[Lee Jay]

I am skeptical of Probabilistic Risk Analysis alone,...

Me too, and I have a question.  As far as I can tell, the PRA just models all the known failure modes.  What do they do about the unknown unknowns?  Do they have some sort of fudge factor or safety factor they apply for those?

[DanClemmensen]

My point stands: 270 non-fatal launches does not equate to a failure probability of better than 1:270.

I think it does, just not by very much.

See here:  https://moleseyhill.com/2009-09-22-bayesian-probability-of-success.html

Sorry, I was unclear. If you have 270 successes and zero failures, your probability is better than 1:270, but you do not compute probability by a simple division: it does not "equate" in the mathematical sense. I was objecting to the computational method.

[InterestedEngineer]

I am skeptical of Probabilistic Risk Analysis alone,...

Me too, and I have a question.  As far as I can tell, the PRA just models all the known failure modes.  What do they do about the unknown unknowns?  Do they have some sort of fudge factor or safety factor they apply for those?

PRA models are naive horrible assumptions when the flight count is low.   They are massively over-optimistic.

When the flight count is high (e.g. at least 100), then you get the following huge advantages in your PRA model, with actual examples:

1.  Unknown unknowns or unknown knowns:  The cascading failures that nearly lead to the loss of Boeing's starliner (and its failure of mission).   AFAICT that fault tree didn't even exist.
2.  Badly calculated knowns:   Examples being the Dragon-2 fuel system and Shuttle SRBs and Shuttle tiles.   Tell us what the PRA models calculated before and after those incidents and that'll tell you how good PRA models are.  Feynman had some choice words to say on that matter.
3. Knowns get more known - e.g. heat shield erosion rates.  Models have mostly been wildly pessimistic but sometimes are too optimistic.   100 flights tell you almost precisely what the erosion rates are under a variety of conditions.

I note shuttle tiles took 113 flights to finally get a loss of craft (LOC) despite lots of earlier hints about failure.   So even more important is a process that recognizes failures and fixes them, and an architecture to support that kind of fixing.

What type of organization is good at recognizing failures and fixing them?  One that isn't subject to a sunk cost fallacy.

What type of organizations are most subject to sunk cost fallacies?  Bureaucratic orgs with tons of project managers, politics, and top-down design processes and low real world iteration counts.

What type of organizations are least subject to sunk cost fallacies?  Ones that move fast and break things and iterate quickly.  For example SpaceX.

What type of organizations make architectures that are fixable?  Ones led by a brilliant engineer and that emphasize "best part is no part" and actively tries to eliminate requirements.  Shuttle program is the antithesis of this.  SpaceX is the thesis.

Psychologically, most everyone has this exactly backwards in their heads.   Most people think the dotting of Is and crossing of Ts involved in PRA driven designs is the way to go for safety.  Usually they are the wrong Is and Ts and they forget to dot the Js.

Only by iterating do you find out you have to dot your Js,

TL;DR - PRA models are optimistic hokum and organizations that rely on them at low flight counts are fooling themselves and unlikely to fix real problems.   They are a useful tool at high flight counts because you can prove every aspect of their assumptions with real data.

[Lee Jay]

I note shuttle tiles took 113 flights to finally get a loss of craft (LOC) despite lots of earlier hints about failure.

FYI, that LOC/V was from a foam strike on the RCC, not the tiles.  A tile issue came close to causing another LOC/V (STS-27, I think?) but 107 was RCC.

[TheRadicalModerate]

Capsules with parachutes are inherently unable to achieve a high flight rate. And you absolutely do need a high flight rate to get safety to even get into the ballpark of motorcycle riding or general aviation (the general aviation is merely 2x safer than motorbike riding, BTW).

If you want to have a high flight rate with a capsule then you need a powered landing. At that point you have the same primary issue as the architecture you claim to be inherently unsafe.

I don't particularly like capsules, but these arguments are nonsense.  They were nice in the 60s.

Capsules by necessity have very low lift to drag and thus steep reentry. That means less total heat pulse but much worse peak heating. Which in turn means higher thermal stress of the heatshield. Which in turn means none of the TLR>3 reusable heatshields would hold. In effect capsule effectively means ablative heat shield which is incompatible with a high flight rate.

But even if currently low TRL solutions were developed, you still have a parachute which is needs at least a refurbishment:large parachutes, unlike those made for individual humans, are complex devices with pyros and delay charges, go through high stresses and are not quickly reusable.

We're not talking about reentry with capsules.  We're talking about escape with capsules, which hardly ever get deployed.  The thermal and mechanical stresses are extremely modest, because max-q is about as high/fast as they'd ever be used.  Whether or not you choose to use a parachute or a powered landing is pretty much a mid-level engineering decision--either will work.

Just so we're all on the same page on the various phases of flight and their aborts:

1) Pad abort:  Use the escape capsule.

2) Low-altitude abort, before you can get decent SS-SH staging separation: Use the capsule.

3) Mid-altitude, low-q abort:  Use the Starship.  If you can RTLS to the chopsticks, cool.  If you have to abort downrange with no chopsticks, do the belly flop, start to rotate higher than usual, and deploy the capsule.

4) Max-q abort:  Use the capsule.

5) Hypersonic ascent abort:  Use the Starship via an early staging.¹  If chopsticks unavailable for landing, use the capsule once you're at low altitude.

6) On-orbit abort:  Come home, standard conops.  If an off-target EDL is essential to get on the ground ASAP, use the escape capsule at low altitude.

7) Reentry abort:  No solution.¹

8) Trans- or sub-sonic belly flop abort:  Use the capsule.  It has plenty of altitude to orient itself, then land propulsively.

9) Flip/landing abort:  First, implement the flip maneuver higher, wasting some delta-v in the name of safety.  Then use the capsule, at whatever orientation you have to.  You'll be high enough that the capsule can orient itself.

10) Any off-target landing:  Use the capsule.  (You could argue that a smooth-surface landing with legs would allow Starship recovery, but I don't see anybody getting a 120t, 50m x 9m thing out of a cornfield in one piece.)

11) Post-landing abort (missed catch, engine shutdown explosion, rough-surface tip):  Use the capsule.

_____________
¹I'm omitting the cases where you have a significant Starship (not SuperHeavy) structural failure in hypersonic flight.  In some of those, the escape capsule would be handy.  But it makes it much more difficult to engineer, and most of those cases aren't survivable.

[Lee Jay]

Capsules with parachutes are inherently unable to achieve a high flight rate. And you absolutely do need a high flight rate to get safety to even get into the ballpark of motorcycle riding or general aviation (the general aviation is merely 2x safer than motorbike riding, BTW).

If you want to have a high flight rate with a capsule then you need a powered landing. At that point you have the same primary issue as the architecture you claim to be inherently unsafe.

I don't particularly like capsules, but these arguments are nonsense.  They were nice in the 60s.

Capsules by necessity have very low lift to drag and thus steep reentry. That means less total heat pulse but much worse peak heating. Which in turn means higher thermal stress of the heatshield. Which in turn means none of the TLR>3 reusable heatshields would hold. In effect capsule effectively means ablative heat shield which is incompatible with a high flight rate.

But even if currently low TRL solutions were developed, you still have a parachute which is needs at least a refurbishment:large parachutes, unlike those made for individual humans, are complex devices with pyros and delay charges, go through high stresses and are not quickly reusable.

We're not talking about reentry with capsules.  We're talking about escape with capsules, which hardly ever get deployed.  The thermal and mechanical stresses are extremely modest, because max-q is about as high/fast as they'd ever be used.  Whether or not you choose to use a parachute or a powered landing is pretty much a mid-level engineering decision--either will work.

Just so we're all on the same page on the various phases of flight and their aborts:

1) Pad abort:  Use the escape capsule.

2) Low-altitude abort, before you can get decent SS-SH staging separation: Use the capsule.

3) Mid-altitude, low-q abort:  Use the Starship.  If you can RTLS to the chopsticks, cool.  If you have to abort downrange with no chopsticks, do the belly flop, start to rotate higher than usual, and deploy the capsule.

4) Max-q abort:  Use the capsule.

5) Hypersonic ascent abort:  Use the Starship via an early staging.¹  If chopsticks unavailable for landing, use the capsule once you're at low altitude.

6) On-orbit abort:  Come home, standard conops.  If an off-target EDL is essential to get on the ground ASAP, use the escape capsule at low altitude.

7) Reentry abort:  No solution.¹

Trans- or sub-sonic belly flop abort:  Use the capsule.  It has plenty of altitude to orient itself, then land propulsively.

9) Flip/landing abort:  First, implement the flip maneuver higher, wasting some delta-v in the name of safety.  Then use the capsule, at whatever orientation you have to.  You'll be high enough that the capsule can orient itself.

10) Any off-target landing:  Use the capsule.  (You could argue that a smooth-surface landing with legs would allow Starship recovery, but I don't see anybody getting a 120t, 50m x 9m thing out of a cornfield in one piece.)

11) Post-landing abort (missed catch, engine shutdown explosion, rough-surface tip):  Use the capsule.

_____________
¹I'm omitting the cases where you have a significant Starship (not SuperHeavy) structural failure in hypersonic flight.  In some of those, the escape capsule would be handy.  But it makes it much more difficult to engineer, and most of those cases aren't survivable.

I agree with much of this, but if this thing is going to carry 12-100 people, that capsule gets pretty huge.

[TheRadicalModerate]

Wasn't that a NASA safety requirement 1:270 LOC?

A small quibble:  The current specs say:

1) 1:1000 launch
2) 1:1000 EDL
3) 1:270 for the extended on-orbit mission.

There are sizable problems with MMOD safety of Starship, which we really haven't dug into.  But for almost everything we've discussed up-thread, 1:1000 is probably the right number--if you're looking for NASA certification.

[TheRadicalModerate]

I agree with much of this, but if this thing is going to carry 12-100 people, that capsule gets pretty huge.

The capsule approach won't work for more than 8-10 people if it's D2-based.

But time is your friend.  SpaceX doesn't need a Starship that carries 100 people for at least ten years, and more likely 15-20.  There will be a Starship v2, v3, v4...  Plenty of time to come up with a better architecture if you really need to launch 100 people to LEO in one shot.  Note also that if you need to send 100 people to Mars and launch is incredibly cheap, launching them 10 at a time might be an acceptable solution.

[Robotbeat]

I am skeptical of Probabilistic Risk Analysis alone,...

Me too, and I have a question.  As far as I can tell, the PRA just models all the known failure modes.  What do they do about the unknown unknowns?  Do they have some sort of fudge factor or safety factor they apply for those?

I don’t know the specific answer to your question but I think part of the idea of the TRL scale is that you start eliminating these Unknown Unknowns by progressing technologies to higher in the TRL scale.

In other words, you use TRL as a tool to control/mitigate/prevent unknown unknowns.

[TheRadicalModerate]

I am skeptical of Probabilistic Risk Analysis alone,...

Me too, and I have a question.  As far as I can tell, the PRA just models all the known failure modes.  What do they do about the unknown unknowns?  Do they have some sort of fudge factor or safety factor they apply for those?

I don’t know the specific answer to your question but I think part of the idea of the TRL scale is that you start eliminating these Unknown Unknowns by professing technologies to higher in the TRL scale.

In other words, you use TRL as a tool to control/mitigate/prevent unknown unknowns.

If you're honest with TRL, it can help you with known unknowns, but not with unknown unknowns.

Think of the graph of all possible failures and their consequences.  The nodes in that graph are various components' reliability under various contingencies.  The components themselves are known.  Whether their reliability in each node is accurate is initially unknown, and then becomes progressively better known.

But the links in the graph may be true unknown unknowns.  They represent all the events you can think of, but none of the ones you can't.  Then, as you fly, you look at accidents or unanticipated failures, and add more links to the graph from the lessons learned.  That's really the only way to turn unknown unknowns into known unknowns, and eventually into reliable parts of the PRA.

[InterestedEngineer]

We're not talking about reentry with capsules.  We're talking about escape with capsules, which hardly ever get deployed. 

That's my fundamental disagreement with an encapsulated abort capsule.   hardly every deployed means the PRA is full of WAGs not data.

If financially you can figure out how to do this 50 times, then you can fill out the PRA module, find all the links on the graph that you helpfully pointed out (great post BTW), and then you've got something reliable.

I don't see how one can destroy 50 starships and have say 10 capsules reused to do his.   That's about 3-5 billion in equipment cost alone.

I can see how to do this with Starships by themselves though.   150 Starlink launches that pay for themselves.  Put 6-10 in the ocean to test that abort scenario (probably get that for free for real-world aborts).

[Lee Jay]

We're not talking about reentry with capsules.  We're talking about escape with capsules, which hardly ever get deployed. 

That's my fundamental disagreement with an encapsulated abort capsule.   hardly every deployed means the PRA is full of WAGs not data.

Abort systems have been tested a small number of times and then successfully deployed. Two were used successfully recently, one of which saved the lives of the crew.

The idea is that they are relatively simple systems and so a small amount of testing goes a long way.  Further, even if they are only 90% successful, that still multiplies your probability of survival by a factor of ten, say from 99 out of 100 to 999 out of 1000.

[InterestedEngineer]

We're not talking about reentry with capsules.  We're talking about escape with capsules, which hardly ever get deployed. 

That's my fundamental disagreement with an encapsulated abort capsule.   hardly every deployed means the PRA is full of WAGs not data.

Abort systems have been tested a small number of times and then successfully deployed. Two were used successfully recently, one of which saved the lives of the crew.

The idea is that they are relatively simple systems and so a small amount of testing goes a long way.  Further, even if they are only 90% successful, that still multiplies your probability of survival by a factor of ten, say from 99 out of 100 to 999 out of 1000.

A mechanism sitting on the top of its normally used staging system is reasonably simple (plus or minus hypergolic explosions).   They still fail.  And they fail when not needed as well as when needed.  The former increases probability of LOC.

One that has to eject out of a Starship, not so simple

[Lee Jay]

We're not talking about reentry with capsules.  We're talking about escape with capsules, which hardly ever get deployed. 

That's my fundamental disagreement with an encapsulated abort capsule.   hardly every deployed means the PRA is full of WAGs not data.

Abort systems have been tested a small number of times and then successfully deployed. Two were used successfully recently, one of which saved the lives of the crew.

The idea is that they are relatively simple systems and so a small amount of testing goes a long way.  Further, even if they are only 90% successful, that still multiplies your probability of survival by a factor of ten, say from 99 out of 100 to 999 out of 1000.

A mechanism sitting on the top of its normally used staging system is reasonably simple (plus or minus hypergolic explosions).   They still fail.  And they fail when not needed as well as when needed.

When was the last time that happened operationally?

One that has to eject out of a Starship, not so simple

Yeah - SS is a bad architecture for humans.

[Robotbeat]

We're not talking about reentry with capsules.  We're talking about escape with capsules, which hardly ever get deployed. 

That's my fundamental disagreement with an encapsulated abort capsule.   hardly every deployed means the PRA is full of WAGs not data.

Abort systems have been tested a small number of times and then successfully deployed. Two were used successfully recently, one of which saved the lives of the crew.

The idea is that they are relatively simple systems and so a small amount of testing goes a long way.  Further, even if they are only 90% successful, that still multiplies your probability of survival by a factor of ten, say from 99 out of 100 to 999 out of 1000.

A mechanism sitting on the top of its normally used staging system is reasonably simple (plus or minus hypergolic explosions).   They still fail.  And they fail when not needed as well as when needed.

When was the last time that happened operationally?

One that has to eject out of a Starship, not so simple

Yeah - SS is a bad architecture for humans.

Capsules are worse. Unaffordable except for the ultra-rich, and so it only flies a modest amount of times and is unsafe.

[Twark_Main]

A mechanism sitting on the top of its normally used staging system is reasonably simple (plus or minus hypergolic explosions).   They still fail.  And they fail when not needed as well as when needed.

When was the last time that happened operationally?

Probably https://spaceflightnow.com/2022/02/10/research-satellites-destroyed-in-astra-rocket-failure/

In the very early days SpaceX commissioned Futron to study causes of rocket failures (to help design Falcon). In their sample 28% of all launch failures were due to separation systems.