What are the chances that SLS fails its first launch?

[CorvusCorax]

Yes, I know this is a loaded topic, but I'm actually interested in a realistic assessment.

For context, NASA has an excellent track record with their launch vehicle designs. The Space Shuttle successfully launched and landed on its maiden flight (scary as it was). The Saturn V actually never had a launch failure (despite close calls). Some of the earlier Saturns and even earlier rockets had, but that was a head-on dive into the deep unknown and a hardware rich development program where people were expecting things to blow up to learn from it. However latestly since the STS program, the way NASA operates is to retire any substantial risk before the first launch, to avoid booms. NASA is in a better place in that regard now than ever before, having learned from STS mishaps and what lead to them, and although that modus operandi is one of the drivers for cost overruns and schedule delays SLS is encountering, at least, it should make it safe and successful. Right?

On the other hand however maiden launches of any vehicle expose a lot of unknown unknowns to the harsh test of reality. Many stresses and interactions, vibrations, loads, thermals, etc... so far have only been simulated based on models. NASA certainly has among the best models, it's their job, but no model is ever 100% accurate. Not a single engineer involved in the SLS program would deny that NASA is going to learn something new and unexpected on that maiden launch. And if the recent less than perfect wet dress rehearsal is any indication, there seems to be a lot of potential for unknown unknowns in a vehicle of this size and complexity level.
And as anyone knows, in a rocket launch basically everything has to go just right, or the launch goes wrong.

Now I don't want to go down the rabbit hole of speculating on the political fallout and consequences to the program should SLS actually go boom on its maiden launch. I think anyone can imagine that and it would only lead to a thread axed very quickly I'd rather wish the vehicle the best of luck to make it to orbit.

But how much luck does it need? I would like to discuss the actual technical risks that could potentially bring this vehicle down and how substantial are they.

SLS is using proven Shuttle technology, engines and propellants, which - although upping the scale, I think does retire a lot of risks compared to let's say yet unflown rockets using yet unflown oxygen rich staged combustion methane engines. (Note to self: Start a similar thread regarding Vulcans maiden launch when that gets closer to happening)

However STS had certain design "flaws" that are inherently risky which SLS inherits. The only catastrophic launch failure STS ever had was caused by these 2 elephants in the room. Granted, these were operated out of spec at the time despite knowing about the seals vulnerability in cold temperature, but nevertheless these giant solid boosters pose a substantial risk. One reason is because you can't turn them off once they are turned on (although this is more an issue regarding launch escape. Most launches on which the engines are turned off prematurely - past liftoff - typically are classified as a launch failure anyway, so that capability would usually not safe the mission) Another is that they haven't had too many tests in their new, thrust upgraded 5 segment configuration, and we saw some issues such as disintegrating nozzles during some of these tests too.
And then surely NASA wouldn't operate them out of specs again, like - let's say - wave their best before due date due to delays or something. Or would they?

But realistically,I only have superficial armchair-engineer insight into the program and am not a rocket scientist in the first place. So, can anyone here share some more substantial thoughts about

1. What flaw/mishap can you think of that could possibly bring down SLS on it's maiden launch?
2. How high is that risk as objectively and quantitatively as you can possibly tell.
3. What could be done or could have been done to avoid it?

[DanClemmensen]

I'm not a professional, so I cannot evaluate any particular technical risk as you asked. However, I feel that NASA should mitigate the programmatic risk of a failure by being more forthright about the possibility of a failure. The current message as understood by the public is "Artemis I will succeed".  It should be made more clear that Artemis I is a test and it might go boom. The public would accept that. In fact the public likes things that have a risk of going boom, like stock car races, as long as the risk is stated in advance.

One specific example is the SRB pull date. NASA should be explicit here, and state that the SRBs are past due. They will almost certainly work, but NASA has chosen to accept the risk because it's cheaper to lose the mission (and thereby gain valuable information) than it would be to scrub the mission and de-stack. I think the public will accept this if it is publicized before the mission.

[CorvusCorax]

I'm not a professional, so I cannot evaluate any particular technical risk as you asked. However, I feel that NASA should mitigate the programmatic risk of a failure by being more forthright about the possibility of a failure. The current message as understood by the public is "Artemis I will succeed".  It should be made more clear that Artemis I is a test and it might go boom. The public would accept that. In fact the public likes things that have a risk of going boom, like stock car races, as long as the risk is stated in advance.

One specific example is the SRB pull date. NASA should be explicit here, and state that the SRBs are past due. They will almost certainly work, but NASA has chosen to accept the risk because it's cheaper to lose the mission (and thereby gain valuable information) than it would be to scrub the mission and de-stack. I think the public will accept this if it is publicized before the mission.

I think it is too late for that. This would have needed to happen already, as the narrative of "infallible NASA" has long been set. Being public and up-front about potential failure is more a SpaceX thing - I remember Elon Musk publicly announcing a "~50% chance of success" for both Falcon Heavy and Starship - which I think is no coincidence. I assume SpaceX is optimizing for a 50% failure probability as a sweet spot to maximize the rate of advancement during a development program.

For reference. If you iterate conservatively, then your success probability might be 99% but you also have very little innovation. The progress speed will be low and as such reaching a certain amount of progress will be expensive.
If you iterate too aggressively then your success probability might be as low as 1%. But that means you are unlikely to get a lot of valuable data because you fail prematurely. In case of rockets, the vehicle will blow up before it even makes it off the pad. Reaching a certain amount of progress is then also slow and expensive (many many vehicles and lots of cleanup)
Obviously the fastest and cheapest way forward is somewhere in the middle, so SpaceX seems to have optimized for the 50% and is also publicizing the 50%. While NASA has gone the conservative route and seems to be expecting the 99%.

There's a catch though. The "fail fast" route where you accept a 50% failure rate only works if you are hardware rich and don't care if a vehicle goes boom because you have the next almost finished and 2 more in the construction pipeline, so no matter the outcome, the next launch - either fixing and repeating or stretching the envelope - will come a short time later. Something you can ONLY do if failure is an option and you can go with quick "will probably work, fingers crossed" engineering solutions. Aka be scrappy.

Based on SLS construction rates and prices, NASA can simply not afford to shrug off a "vehicle went boom, happens, we investigate the cause, fix, and try again in a few weeks" attitude. The construction/refurbishment pipeline only has so many vehicles for this decade, and they are already planned for operational missions. So the stakes are obviously so high that they are screwed if the 1 in 100 chance of a 99% success probability hits them square in the face.

But this is all politics and policy and a question of how you run a development program. In other words, things that were all set in stone long way back when Shuttle was shelved and Constellation and later SLS was born. NASA chose a "high probability of success program" - and of course that's what they also have communicated.

But the question is. How high is the probability of success really? IS it 99% or are those 99% an unrealistic illusion?

Obviously that was the goal, but now that the vehicle is done and on the pad and ... some corners had to be cut and some bumps entirely missed... what is it's actual success probability?

Sadly with a sample size of 1 we might never know for certain even after launch. Even if it's 90%+  you might get bad luck, or even if its 10% you might get lucky. But there are of course indicators such as problems in test campaigns and during assembly, some of which have been very public and open ( standing party thread joke is NASA makes a press conference for every screw they put in the vehicle)

Maybe we should list all tests that the vehicle has gone through, including the latest WDR, and list all the things that unexpectedly went wrong - and then extrapolate for the actual launch based on that.

Does anyone have a conclusive list of those? As in "everything that went wrong with SLS tests"?

I dimply remember there had been parts missing after an SRB test,  issues with fire on an engine blanket on a greenrun in Stennis, and of course the latest leaks and pressure issues on the WDR. If there is such a list, it has to be here on NSF, but google isn't really helping me right now.

[whitelancer64]

This is an off-the-cuff guess, but I'd put the chances at around 5% maybe a bit higher.

However, NASA is doing a lot of pre-flight testing to find any potential issues before they could be a problem in flight. I don't think there will be any major issues with the core stage or ICPS.

The long stack time of the SRBs is really the only glaring thing that seems to be an obvious potential problem right now, and I think that's attributable to NASA being willing to take more risks on an uncrewed test flight than they would with people on board.

The official cumulative LOC risk for Artemis II is 1 in 70 -- 1.4%.

[DanClemmensen]

The long stack time of the SRBs is really the only glaring thing that seems to be an obvious potential problem right now, and I think that's attributable to NASA being willing to take more risks on an uncrewed test flight than they would with people on board.

My uninformed guess agrees with your professional opinion. Why don't they say so?

[edkyle99]

I would say 50-50 for the inaugural flight, including all possible "launch vehicle failures" during ascent, including orbit shortfalls, etc. 

I'm thinking, for example, about the first NASA Titan 3E (TC-1), which was merely an upgraded Titan 3D with a proven Centaur upper stage.  Its February 11, 1974 "Proof Flight" with a Viking mass simulator and a SPHINX research satellite failed when Centaur failed to start.  Most everything on Titan 3E had previously flown.  SLS is using a brand new core stage of unprecedented size boosted by new-design SRBs, etc. 

Saturn 5 did fail once.  SA-502/Apollo 6 lost engines during ascent and, more critically, failed to restart its S-IVB stage upon reaching orbit, falling far short of the planned spacecraft insertion orbit.

 - Ed Kyle

[whitelancer64]

The long stack time of the SRBs is really the only glaring thing that seems to be an obvious potential problem right now, and I think that's attributable to NASA being willing to take more risks on an uncrewed test flight than they would with people on board.

My uninformed guess agrees with your professional opinion. Why don't they say so?

I'm not all that professional. I have no access to the data that Northrop Grumman engineers have of the measurements they took of the SRBs before and after stacking. They are the only ones who really know, and have the information available to make the professional assessment. NASA should be relying on them to make the call on whether or not the SRBs are OK to fly.

[butters]

Vibrations, electronics, solder joints and trace continuity. The Orion program has a history with cracked traces in vibration testing. The SLS #4 engine controller that stopped working was attributed to a solder joint. If I were to guess why Artemis-1 might go badly wrong during ascent, it would be something like that. It's a particularly violent ride on the SRBs for untold millions of tiny electrical conductors.

[Robotbeat]

Was going to say 10%, but after reading Ed Kyle’s post, 20%.

A partial failure (early shutdown, etc) is fairly likely, and they’ll still be able to claim some success.

[freddo411]

I hope SLS/Orion's very first flight manages to complete it's full flight.

I will guess that there is an >80% chance that there will be "anomalies" that may or may not affect the mission to a serious degree.   Thruster failures, leaks, sensor failures, software failures, separation events, boom deployments and simple human error mistakes are possibilities.

This is a test flight.   It would be shocking if 100% of things worked correctly.

* It's worth noting that there are things that are NOT being tested on this flight, and will fly for the very first time on Artemis II.   Life support tanks in the ESM, and a working launch escape system are two examples.

[leovinus]

I would give it a 1:20 or 5% of failing leading to Loss-of-mission (LOM). Why? LOM/LOC has been discussed at length for SLS and Shuttle. My recent summary attempt in the SLS thread includes various Shuttle and PRA attachments on LOM/LOC of 1:10 on the first Shuttle flight. The highest risks are consistently SRB and main engines. The SRB are new and improved compared to the first Shuttle mission. Twice as good as that mission is my throw on the dart board, 1:20.

[Jeff Lerner]

I feel it’s unreal to expect the entire flight to go perfectly. The question is what level of problem
Will necessitate a reflight of Artemis I.

My concern about this whole program has been there are so few vehicles being built they all have to work almost perfectly on each flight before the next one will be allowed to fly it’s mission.

That’s the beauty of the Starship program. SpaceX is churning vehicles out like sausages..if one has a problem the next one is in the queue ready to
Go.

[Eric Hedman]

My guess is less than a ten percent chance of failure.  Improvements in inspection of manufacturing processes have been made in most manufacturing industries.  I would like to think better inspection techniques have been applied to SLS than in first flights of previous generations of launchers.  I think the wet dress rehearsal had a good chance of catching the major remaining issues.

[CorvusCorax]

Vibrations, electronics, solder joints and trace continuity. The Orion program has a history with cracked traces in vibration testing. The SLS #4 engine controller that stopped working was attributed to a solder joint. If I were to guess why Artemis-1 might go badly wrong during ascent, it would be something like that. It's a particularly violent ride on the SRBs for untold millions of tiny electrical conductors.

That's what happens when you switch to lead-free solder....    Jokes aside, vibration testing should have taken care of that if the vibrations tested were worst case launch vibrations+margin on actual flight hardware. Now of course it's a maiden launch. No one truly knows what the *real* vibration environment will be until there's actual flight data, so if the model was off more than a factor of 1.5 ish ...

An SRB having unclean combustion due to aging propellant leading to excessive vibration which in turn causes electrical connections to fail which in turn triggers a failsafe procedure in the flight computer software that has been tested less than it should have (remember Boeing, the software specialists) which - through a chain of events triggers a guidance failure or loss of thrust with subsequent triggering of the explosive abort system...   I think that's the typical chain of individually harmless and small failures that when taken together you would count as a "bad day".

Now luckily all these flaws and failures need to trigger simultaneously to make it happen, while one alone would be harmless. But just because one alone would be harmless, it's likelihood is probably not all that low. We know the SRB propellant is old, we know the solder connections are error prone, and we know that some contractors make better avionic software than others. So if we give a rough guess 60% likelihood to each of these 4 independent issues/flaws to occur or exist independently - there's an overall 12% chance of simultaneous occurrence with big boom.

[Rocket Rancher]

And what chances would this group have given the first shuttle launch?

A high portability of more launch delays and a very low probability of a major failure. Whether you like NASA or not, you have to give them some credit for knowing what they are doing.

[dglow]

1:40 odds of catastrophe during launch
1:10 odds of failure whereby Orion does not reach the Moon | return to Earth | land safely
1:3 odds of mission success despite a significant component failure, requiring mitigation and delay to Artemis II

[Robotbeat]

Comparing Starship failure rates to SLS is like comparing two completely different systems. 

Starship may turn out to have the highest failure rate of any launch vehicle of all time.  Chances Starship fails on the next flight are very very high.  In fact, makes you wonder if Starship is just designed to fail - as if to say, if we couldn’t make our stuff work, then the other stuff surely wouldn’t work.  Seen this play out as a marketing trick in the industry during the 90s.

But SLS has tremendous heritage.  It will be highly successful.

Poe's Law strikes again!

[greybeardengineer]

I think the most likely failure cause for Artemis I is flight software. It is brand new and as I understand it Boeing was responsible for its definition, creation, and verification. Several years ago the development of SLS core stage flight software was reportedly in disarray and behind schedule. Of course Boeing's failings with CST-100 software is well known. I think the odds of LOV is about 10% and LOM is about 20%.

[woods170]

I would say 50-50 for the inaugural flight, including all possible "launch vehicle failures" during ascent, including orbit shortfalls, etc. 

This is IMO a good assessment of where SLS stands, with regards to chance of failure on the first launch.

Looking at orbital rockets introduced in the west, in the past 4 decades, the failure rate on first launch is approximately 45%. So, Ed's "50-50" odds is very close to that.

[CorvusCorax]

I think the most likely failure cause for Artemis I is flight software. It is brand new and as I understand it Boeing was responsible for its definition, creation, and verification. Several years ago the development of SLS core stage flight software was reportedly in disarray and behind schedule. Of course Boeing's failings with CST-100 software is well known. I think the odds of LOV is about 10% and LOM is about 20%.

One could be very naughty and say - since SLS flies all its hardware expendable, the LOV chances are 95% - minus for the return capsule.

95% of course because the vehilce only gets expended if SLS actually launches. There is a small but tangible programmatic risk that, in case of additional delays, cost overruns and possible launches of competing vehicles in combination with certain political factors, could lead to SLS being cancelled prior to maiden launch and the vehicle being put on display. I would put the chance of that as less than 5% or 1 in twenty, but this does obviously result in an increase in safety. A vehicle that does not fly usually does not blow up.