Are re-used boosters less finicky?

[LouScheffer]

Updated list:
Booster scrubs mission
1046.1       1 Bangabandhu
1047.1       0 Telstar 19
1048.1       0 Iridium/Grace
1046.2       0 Merah Putih
1049.1       0 Telstar 18
1048.2       0 SAOCOM
1047.2       0 Es'hail
1046.3       0 SSO-A
1050.1       0 CRS-16
1054.1       1 GPS-III
1049.2       0 Iridium
1048.2       0 Nusantara Satu
1051.1       0 SpX-Dm1
1052.1       0 FH
1053.1       0 FH
1055.1       0 FH
1056.1       0 CRS-17
1049.3       0 Starlink
1051.2       0 RADARSAT
1057.1       0 STP-2
1052.2       0 STP-2
1053.2       0 STP-2
1056.2       0 CRS-18
1047.3       1 AMOS-17 (extra static fire needed)
1048.4       0 Starlink 1
1059.1       0 CRS-19
1056.3       0 JCSat-18
1049.4       0 Starlink 2
1051.3       0 Starlink 3
1056.4       0 Starlink 4  #second stage valve issue, not counted here
1059.2       0 CRS-20
1048.5       1 Starlink 5  # Static fire anomaly; booster lost due to refurbishment issue
1051.4       0 Starlink 6
1058.1       0 DM-2
1049.5       0 Starlink 7
1059.3       0 Starlink 8
1060.1       0 GPS-III


Original post below:

Are the second and later launches of the same booster more or less likely to proceed through the countdown without encountering technical bugs causing a scrub?  There are reasons each direction might be possible - infant mortality vs wear-and-tear, for example:

these new boosters are always so finicky on their first launch...
[...]
I think we might begin to witness the positive effect of "flight proven" being turned around against all these "untested" boosters

there's ever only so much you can test in component tests, dress rehearsal or even a hold down hot fire.

Not true.  [...]
c.  Can't say that the grid fin failure was due to infant mortality, poor workmanship (which still applies to a refurb booster), effects from second stage engine.

But now we are emerging from the wilderness of opinions to the department of data, though we are not there yet.  There have been (or will be after GPS) 10 launches of Block 5.  The number of technical scrubs after the countdown has started are:

Booster scrubs mission
1046.1       1 Bangabandhu
1047.1       0 Telstar 19
1048.1       0 Iridium/Grace
1046.2       0 Merah Putih
1049.1       0 Telstar 18
1048.2       0 SAOCOM
1047.2       0 Es'hail
1046.3       0 SSO-A
1050.1       0 CRS-16
1054.1       1 GPS-III

So there have been 6 new and 4 used launches, and overall 2 of 10 (20%) had a technical scrub during countdown.  If two scrubs were evenly distributed among the 10 launches, we would naturally expect both on new boosters 15/45 (33%) of the time, both on used boosters 6/45 (13%) of the time, and one each 24/45 (53%) of the time.

Clearly the numbers are small, and not statistically significant, but the evidence points slightly in the direction of used boosters having fewer scrubs.  By the end of next year we may have better data on this.  (On the other hand, SpaceX is presumably trying to reduce the number of scrubs in general.  If they succeed, then the number of scrubs will remain low and the statistical power will be limited).

[Cheapchips]

As you say, theres not statistical significance at the moment.

It would be nice to have more details on both scrubs.  It might give insight into if they're unproven hardware issues or just early life failures.

[vaporcobra]

Add pre-Block 5 launches and this could be a bit more statistically interesting

[meekGee]

Are the second and later launches of the same booster more or less likely to proceed through the countdown without encountering technical bugs causing a scrub?  There are reasons each direction might be possible - infant mortality vs wear-and-tear, for example:

these new boosters are always so finicky on their first launch...
[...]
I think we might begin to witness the positive effect of "flight proven" being turned around against all these "untested" boosters

there's ever only so much you can test in component tests, dress rehearsal or even a hold down hot fire.

Not true.  [...]
c.  Can't say that the grid fin failure was due to infant mortality, poor workmanship (which still applies to a refurb booster), effects from second stage engine.

But now we are emerging from the wilderness of opinions to the department of data, though we are not there yet.  There have been (or will be after GPS) 10 launches of Block 5.  The number of technical scrubs after the countdown has started are:

Booster scrubs mission
1046.1       1 Bangabandhu
1047.1       0 Telstar 19
1048.1       0 Iridium/Grace
1046.2       0 Merah Putih
1049.1       0 Telstar 18
1048.2       0 SAOCOM
1047.2       0 Es'hail
1046.3       0 SSO-A
1050.1       0 CRS-16
1054.1       1 GPS-III

So there have been 6 new and 4 used launches, and overall 2 of 10 (20%) had a technical scrub during countdown.  If two scrubs were evenly distributed among the 10 launches, we would naturally expect both on new boosters 15/45 (33%) of the time, both on used boosters 6/45 (13%) of the time, and one each 24/45 (53%) of the time.

Clearly the numbers are small, and not statistically significant, but the evidence points slightly in the direction of used boosters having fewer scrubs.  By the end of next year we may have better data on this.  (On the other hand, SpaceX is presumably trying to reduce the number of scrubs in general.  If they succeed, then the number of scrubs will remain low and the statistical power will be limited).

We also had one in-flight anomaly, also on a new booster.

-----
ABCD: Always Be Counting Down

[Lar]

Bathtub curve tells us that IF component lifetimes are well over 2-3 launches, and IF infant mortality testing weeds out most infant mortality, the second launch OUGHT to be more reliable.

But we aren't really sure of either of the IFs yet (SpaceX almost certainly has more data than we do, and therefore may be more sure... ) I don't think. 

Fascinating topic. We had speculated that this might be the case, reflown boosters are "better"... I look forward to more data.

[groundbound]

There is another effect that may cause what amounts to reduced statistical significance for the next year or two.

The first time any booster sees it's n'th re-use, launch ops are likely to be more cautious just because they are expanding the re-use envelope, even if only by a little.

[deruch]

Are the second and later launches of the same booster more or less likely to proceed through the countdown without encountering technical bugs causing a scrub?  There are reasons each direction might be possible - infant mortality vs wear-and-tear, for example:

these new boosters are always so finicky on their first launch...
[...]
I think we might begin to witness the positive effect of "flight proven" being turned around against all these "untested" boosters

there's ever only so much you can test in component tests, dress rehearsal or even a hold down hot fire.

Not true.  [...]
c.  Can't say that the grid fin failure was due to infant mortality, poor workmanship (which still applies to a refurb booster), effects from second stage engine.

But now we are emerging from the wilderness of opinions to the department of data, though we are not there yet.  There have been (or will be after GPS) 10 launches of Block 5.  The number of technical scrubs after the countdown has started are:

Booster scrubs mission
1046.1       1 Bangabandhu
1047.1       0 Telstar 19
1048.1       0 Iridium/Grace
1046.2       0 Merah Putih
1049.1       0 Telstar 18
1048.2       0 SAOCOM
1047.2       0 Es'hail
1046.3       0 SSO-A
1050.1       0 CRS-16
1054.1       1 GPS-III

So there have been 6 new and 4 used launches, and overall 2 of 10 (20%) had a technical scrub during countdown.  If two scrubs were evenly distributed among the 10 launches, we would naturally expect both on new boosters 15/45 (33%) of the time, both on used boosters 6/45 (13%) of the time, and one each 24/45 (53%) of the time.

Clearly the numbers are small, and not statistically significant, but the evidence points slightly in the direction of used boosters having fewer scrubs.  By the end of next year we may have better data on this.  (On the other hand, SpaceX is presumably trying to reduce the number of scrubs in general.  If they succeed, then the number of scrubs will remain low and the statistical power will be limited).

The Bangabandhu scrub is listed in the log as being a ground system fault.  Seems like dubious reasoning to chalk that up to SpaceX using a new booster for that mission.  Should we also evaluate whether the likelihood of weather scrubs is higher/lower when using pre-flown boosters?

[rpapo]

The Bangabandhu scrub is listed in the log as being a ground system fault.  Seems like dubious reasoning to chalk that up to SpaceX using a new booster for that mission.  Should we also evaluate whether the likelihood of weather scrubs is higher/lower when using pre-flown boosters?

FWIW, being that the Bangabandhu launch was the very first Block 5 launch, there were changes to the GSE.  Any time you have a change, there are bugs to root out.  As said before, this has nothing to do with the difference between the first launch and a subsequent launch.

[meekGee]

The idea from the first post was to "emerge from the wilderness of opinions to the department of data".

If it's GSE, then it will statistically effect new and used booster in the same manner, and factor out.  However if some unforeseen effect causes there to be a difference, it'll show up in the data as well.

FWIW, same goes for weather...

[rpapo]

If it's GSE, then it will statistically effect new and used booster in the same manner, and factor out.  However if some unforeseen effect causes there to be a difference, it'll show up in the data as well.

Not entirely true.  Every time they have introduced a new "block", they have made changes to the GSE, and have had to work through new problems as a result.  These problems faded with more launches, and so the first launches were disproportionately affected.

[meekGee]

If it's GSE, then it will statistically effect new and used booster in the same manner, and factor out.  However if some unforeseen effect causes there to be a difference, it'll show up in the data as well.

Not entirely true.  Every time they have introduced a new "block", they have made changes to the GSE, and have had to work through new problems as a result.  These problems faded with more launches, and so the first launches were disproportionately affected.

.. and so unless there's a block 6, since we're already 20 launches onto it, we will soon have some statistical idea.

-----
ABCD: Always Be Counting Down

[mgeagon]

This is a great thread. Please update the opening post after every launch and we'll see significant statistics emerge before our eyes. It may be instructive to include pre-block 5 numbers, as a side comparison.

[FutureSpaceTourist]

Here’s an attempt to extend data back as far as first flight of first booster to be re-used. I’ll let others look at statistics - choices to be made about static fire scrubs vs launch scrubs, GSE vs vehicle issues, S1 vs S2 issues etc. I’ve ignored all fairing issues and have made a judgement about whether to include delays, not just scrubs, post static fire. Reason for each included delay noted below so you can choose to ignore if you wish. I have omitted all weather scrubs.

Irrespective of choices made, there are no known launch scrubs yet due to a flight proven booster. However, SES-11 had a launch delay due to a booster engine issue on flight proven 1031.2.

Booster scrubs mission
1021.1       0 CRS-8
1022         0 JCSAT-14
1023         1 Thaicom-8 (launch scrub S2 actuator issue)
1024?        0 Eutelsat 117W B & ABS 2A
1025?        0 CRS-9
1026?        0 JCSAT-16
1029.1       1 Iridium NEXT-1 (static fire scrub due to GSE)
1031.1       1 CRS-10 (launch scrub S2 actuator issue)
1030         1 Echostar 23 (static fire scrub unknown cause)
1021.2       0 SES-10
1032.1       1 NROL-76 (launch scrub S1 issue)
1034         0 Inmarsat 5 F4
1035.1       0 CRS-11
1029.2       0 Bulgariasat-1
1036.1       0 Iridium NEXT-2
1037         2 Intelsat 35e (launch scrubs: GNC & ground computer issues)
1039.1       0 CRS-12
1038.1       0 Formosat-5
1040         0 OTV-5
1041.1       0 Iridium NEXT-3
1031.2       1 SES-11 / Echostar 105 (launch delay, post static fire, due to S1 engine)
1042         0 Koreasat-5A
1035.2       1 CRS-13 (launch delay, post static fire, S2 fuel system particles)
1036.2       0 Iridium NEXT-4
1043.1       0 Zuma
1032.2       1 SES-16 (launch delay to replace S2 sensor)
1023.2   &
  1033   &   0 FH demo
  1025.2
1038.2       0 PAZ & Microsat 2a/2b
1044         0 Hispasat1F (30W-6)
1041.2       0 Iridium NEXT-5
1039.2       0 CRS-14
1045.1       1 TESS (launch scrub for more GNC analysis)
1046.1       1 Bangabandhu
1043.2       0 Iridium NEXT-6/GRACE-FO
1040.2       0 SES-12
1045.2       0 CRS-15
1047.1       0 Telstar 19
1048.1       0 Iridium/Grace
1046.2       0 Merah Putih
1049.1       0 Telstar 18
1048.2       0 SAOCOM
1047.2       0 Es'hail
1046.3       0 SSO-A
1050.1       0 CRS-16
1054.1       1 GPS-III

Edit to add: all data from the launch manifest and launch log threads, with many thanks to gongora & cartman respectively.

[RDMM2081]

Nice chart thanks!

Looking at the ~2017 era (model year?) boosters, I don't see any evidence that reused boosters were less finnicky.  1031.2, 1035.2 1032.2 all had various scrubs/delays, while 1031.1 and 1032.1 both had scrubs of their own, but 1035.1 actually had no scrubs on its "unproven" flight, but the same booster did scrub on its re-use attempt.  however 1029.1 did have a scrub, and 1029.2 did not. other boosters that were re-used successfully, 1036.1 and 1036.2 both launched scrub-free.  Hardly anything that demonstrates a pattern, yet.

What is more clear, is that 2018 overall had less scrubs.  In fact both 1045.1 and 1046.2 I personally attribute more to "overabundance of caution" type scrubs than actual problems.  1054.1 is really the first "issue" with block 5 boosters that has caused any launch delay (I am sure this is wrong, but speaking only to the scrub data in the chart).

[FutureSpaceTourist]

It’s important to look at the reasons for the scrubs:

[...] 1035.1 actually had no scrubs on its "unproven" flight, but the same booster did scrub on its re-use attempt. 

The 1035.2 scrub was a stage 2 issue so nothing to do with the flight proven booster. SES-11 is the only launch with a (post static fire) delay due to a booster issue on a flight proven stage.

[RDMM2081]

Good point of course, I missed that in my first glance through.

[CorvusCorax]

Would it be fair to note that 1033 had an inflight issue ( run outof ignition fluid ) on its first flight, while 1023.2 & 1025.2 both landed successfully.

Do we have data about delays/issues in static fire campaigns? Should that be included?

[FutureSpaceTourist]

I don’t think counting 1033 landing failure is fair. It was trying to do something never previously attempted and so was a test rather than a proven operational capability. For the same reason I personally wouldn’t have counted any failures associated with separation of the side boosters on that FH demo flight.

We do sometimes know when static fire dates have moved but we often don’t know why (and certainly not in publicly available info). It would be great to know how easy/difficult preparing each flight proven booster is (eg extent of any refurbishment work, components/engines replaced if any etc) but obviously that’s not public. So I think the best we can do is like at how reliable things are post static fire, when SpaceX think they are ready for flight.

[meekGee]

Cross threading at Gongora's suggestion.

I wrote an article about the recent Starlink-1 news and I'm also speculating that SpaceX might soon stop doing static fires before Starlink launches.

https://www.elonx.net/falcon-9-will-launch-dozens-of-starlink-satellites-and-there-could-be-up-to-7-such-launches-this-year/

That would be interesting.

My view on this from a few years back was that static fires could be eliminated for reused boosters once SpaceX feels comfortable that data collected through ascent and descent is understood well enough to be used in lieu of static fire data for the following flight.

Maybe we have arrived...

Several years ago ULA stopped doing WDRs for the Atlas V (unless for military, NASA, or customer request) because they got to the point where they almost never uncovered an issue before launch.

True.  And I like how you still use "almost" in there.

The thing is, a static fire still suffers from the risk of "but what if the static fire broke something".

That, plus that ability to do real time analysis during the regular hold-down, plus the fact the a major risk factor is solid motors that can't be tested anyway - all added up to a "why bother".

With F9, it's different.  The vehicle is designed for a very large number of flights, there are no solids, and so why not?

The path to removing static fires for SpaceX is different. Once a vehicle already flew, the chance of there being a pad-detectable fault that's not already detectable in the post flight data analysis is really low, basically limited to landing damage.

So as long as the landings are monitored, I think eventually that will be the path.

[Jim]

Cross threading at Gongora's suggestion.

I wrote an article about the recent Starlink-1 news and I'm also speculating that SpaceX might soon stop doing static fires before Starlink launches.

https://www.elonx.net/falcon-9-will-launch-dozens-of-starlink-satellites-and-there-could-be-up-to-7-such-launches-this-year/

That would be interesting.

My view on this from a few years back was that static fires could be eliminated for reused boosters once SpaceX feels comfortable that data collected through ascent and descent is understood well enough to be used in lieu of static fire data for the following flight.

Maybe we have arrived...

Several years ago ULA stopped doing WDRs for the Atlas V (unless for military, NASA, or customer request) because they got to the point where they almost never uncovered an issue before launch.

True.  And I like how you still use "almost" in there.

The thing is, a static fire still suffers from the risk of "but what if the static fire broke something".

That, plus that ability to do real time analysis during the regular hold-down, plus the fact the a major risk factor is solid motors that can't be tested anyway - all added up to a "why bother".

With F9, it's different.  The vehicle is designed for a very large number of flights, there are no solids, and so why not?

The path to removing static fires for SpaceX is different. Once a vehicle already flew, the chance of there being a pad-detectable fault that's not already detectable in the post flight data analysis is really low, basically limited to landing damage.

Not really.  It has nothing to do with solids.    It is also has nothing to do with flight data.  ULA only took on a schedule risk with the elimination of WDRs .  If there was a problem, they would find it on the day of launch,  scrub and fix it.

A static fire or WDR is not going to uncover flight structural or thermal issues or even flight control problems.  A static fire or WDR only looks at the plumbing and environments cause by the propellants and nothing more.  It doesn't test staging, deployment or recovery systems.   Avionics can be checked out by sim flights.

[meekGee]

Cross threading at Gongora's suggestion.

I wrote an article about the recent Starlink-1 news and I'm also speculating that SpaceX might soon stop doing static fires before Starlink launches.

https://www.elonx.net/falcon-9-will-launch-dozens-of-starlink-satellites-and-there-could-be-up-to-7-such-launches-this-year/

That would be interesting.

My view on this from a few years back was that static fires could be eliminated for reused boosters once SpaceX feels comfortable that data collected through ascent and descent is understood well enough to be used in lieu of static fire data for the following flight.

Maybe we have arrived...

Several years ago ULA stopped doing WDRs for the Atlas V (unless for military, NASA, or customer request) because they got to the point where they almost never uncovered an issue before launch.

True.  And I like how you still use "almost" in there.

The thing is, a static fire still suffers from the risk of "but what if the static fire broke something".

That, plus that ability to do real time analysis during the regular hold-down, plus the fact the a major risk factor is solid motors that can't be tested anyway - all added up to a "why bother".

With F9, it's different.  The vehicle is designed for a very large number of flights, there are no solids, and so why not?

The path to removing static fires for SpaceX is different. Once a vehicle already flew, the chance of there being a pad-detectable fault that's not already detectable in the post flight data analysis is really low, basically limited to landing damage.

Not really.  It has nothing to do with solids.    It is also has nothing to do with flight data.  ULA only took on a schedule risk with the elimination of WDRs .  If there was a problem, they would find it on the day of launch,  scrub and fix it.

A static fire or WDR is not going to uncover flight structural or thermal issues or even flight control problems.  A static fire or WDR only looks at the plumbing and environments cause by the propellants and nothing more.  It doesn't test staging, deployment or recovery systems.   Avionics can be checked out by sim flights.

Naw.

Flight data can show out-of-family or even variation-over-time for individual engines - pretty much what a static fire looks for, but over the full flight envelope.

Meanwhile  when you have solids, even if you could get full knowledge from a static fire of the main engines, you'd still be blind to defects in the solids or solid integration.

Schedule issues are just an added layer on top of this.  Reliability generally trumps schedule, unless the test doesn't help with reliability - which is what happened.

[LouScheffer]

Looking again at this, now with more data.   This just counts scrubs due to booster problems, not upper level winds, weather, etc.

Booster scrubs mission
1046.1       1 Bangabandhu
1047.1       0 Telstar 19
1048.1       0 Iridium/Grace
1046.2       0 Merah Putih
1049.1       0 Telstar 18
1048.2       0 SAOCOM
1047.2       0 Es'hail
1046.3       0 SSO-A
1050.1       0 CRS-16
1054.1       1 GPS-III
1049.2       0 Iridium
1048.2       0 Nusantara Satu
1051.1       0 SpX-Dm1
1052.1       0 FH
1053.1       0 FH
1055.1       0 FH
1056.1       0 CRS-17
1049.3       0 Starlink
1051.2       0 RADARSAT

So there have been 11 new and 8 used launches, and overall 2 of 19 (10.5%) had a technical scrub due to the booster(s) during countdown.  If two scrubs were evenly distributed among the 19 launches, we would naturally expect both on new boosters 55/171 (32%) of the time, both on used boosters 28/171 (16.3%) of the time, and one each 88/171 (51%) of the time.

Clearly the numbers are still small, and not statistically significant, but the evidence points slightly in the direction of used boosters having fewer scrubs.  On the other hand, the number of booster-related scrubs per mission seems to be decreasing in general, so it may not be possible to measure this with certainty.

[FutureSpaceTourist]

I think we can at least say that whatever amount of refurbishment is needed, SpaceX is doing enough to ensure launches proceed without issue post static fire. Only if/when times between reuses become small can we be confident that there is little refurbishment and thus be more confident that reused boosters likely are less finicky. I’m hoping that an increasing frequency of Starlink launches in the coming months/year may provide that evidence.

[aero]

At 60 satellites per launch, how many launches are required per phase?

When launching with the Starship, how many launches per plane? Or per phase if that is the metric.

[joek]

At 60 satellites per launch, how many launches are required per phase? ...

Depends on what you mean by "phase".  In any case, better asked (and likely already answered) in Starlink threads here and here (among others).

[Jim]

Flight data can show out-of-family or even variation-over-time for individual engines - pretty much what a static fire looks for, but over the full flight envelope.

Wrong again.  Flight envelope has no bearing on engine performance. 

Meanwhile  when you have solids, even if you could get full knowledge from a static fire of the main engines, you'd still be blind to defects in the solids or solid integration.

Meh.  Meaningless.  In over 1200 flights of monolithic composite cased SRMs on Delta II, IV and Atlas V, only one had a failure.  Meanwhile, Falcon 9 had an inflight failure of a Merlin, even after static firing as stage twice. 

Schedule issues are just an added layer on top of this.  Reliability generally trumps schedule, unless the test doesn't help with reliability - which is what happened.

And it has been shown that static test is also in this category.  And static test itself has lead to a mission loss.

[zodiacchris]

Shouldn’t you include the Shuttle solids and Challenger in there Jim?  Still makes it only one failure in 1300 flights, but what a failure it was...

[meekGee]

Flight data can show out-of-family or even variation-over-time for individual engines - pretty much what a static fire looks for, but over the full flight envelope.

Wrong again.  Flight envelope has no bearing on engine performance. 

Meanwhile  when you have solids, even if you could get full knowledge from a static fire of the main engines, you'd still be blind to defects in the solids or solid integration.

Meh.  Meaningless.  In over 1200 flights of monolithic composite cased SRMs on Delta II, IV and Atlas V, only one had a failure.  Meanwhile, Falcon 9 had an inflight failure of a Merlin, even after static firing as stage twice. 

Schedule issues are just an added layer on top of this.  Reliability generally trumps schedule, unless the test doesn't help with reliability - which is what happened.

And it has been shown that static test is also in this category.  And static test itself has lead to a mission loss.

Love it how you start with "wrong again" without even properly reading what's written.  It's like you have a message template that starts with that...

Things are changing Jim.  Instead of nit-picking, maybe you should address the core proposition here, which is that with a reusable vehicle, flight data from the previous launches gives them a previously unthinkable amount of data about the health of the system, which can be used as absolute metrics, for trend analysis, for in-family analysis - applied to the very specific system that's about to re-fly.

This is unprecedented, and completely different from "focus on process control during manufacturing and hope for the best" which is how expendables work.

Nobody gives a damn if there are a few failures while this transition is happening. There were far, far more failures during the development of expendable rockets, and the resultant ecosystem proved unable to innovate and move forward.

The future belongs to reusable vehicles with a high level of redundancy, not to expendables with a bunch of strap-ons and two main engines.

[LouScheffer]

Compared to the Delta-4, the  Falcon 9 has relatively few post-rollout delays.  From the beginning of 2016

Mission  scrubs
NROL-45    0
NROL-37    0
AFSPC      0
WGS-8      0
WGS-9      1
JPSS-1     1
NROL-47    1
PSP        1
NROL-71    3
WGS-10     1

This could potentially be due to lack of practice, hydrogen fuel (often thought to be finicky), or design.

It would be great to have the same figures for other launchers.   My intuitive guess would be that the shuttle was finicky, Atlas and Ariane are quite good, and Soyuz has the fewest technical delays.

[meekGee]

Compared to the Delta-4, the  Falcon 9 has relatively few post-rollout delays.  From the beginning of 2016

Mission  scrubs
NROL-45    0
NROL-37    0
AFSPC      0
WGS-8      0
WGS-9      1
JPSS-1     1
NROL-47    1
PSP        1
NROL-71    3
WGS-10     1

This could potentially be due to lack of practice, hydrogen fuel (often thought to be finicky), or design.

It would be great to have the same figures for other launchers.   My intuitive guess would be that the shuttle was finicky, Atlas and Ariane are quite good, and Soyuz has the fewest technical delays.

The interesting thing about this sequence is that the problems are definitely not clustered around the beginning of the timeline - they're in fact clustered around the end.

[tdperk]

Flight envelope has no bearing on engine performance. 

An objectively and obviously wrong claim.

Why do you think there are different Isp's for sea level and vacuum?

[spacenut]

Jim said monolithic solids, like the strap on one piece solids on Atlas V and Delta IV, and Delta II.

Shuttle solids were 4 individual casings bolted together.  Failure was between the casings on Challenger. 

Also, SpaceX having one engine failure with no loss of mission with 9 individual liquid engines. 

[FutureSpaceTourist]

The interesting thing about this sequence is that the problems are definitely not clustered around the beginning of the timeline - they're in fact clustered around the end.

IIRC a number of issues have been GSE related. I think this is a combination of ageing infrastructure and lack of/reduced investment in the infrastructure due to forthcoming replacement by Vulcan. So different issues to the focus of this thread.

[meekGee]

Flight envelope has no bearing on engine performance. 

An objectively and obviously wrong claim.

Why do you think there are different Isp's for sea level and vacuum?

Not only that, but back when grasshopper was proving reusability and was planning high altitude flights, Jim constantly argued that it will prove nothing because "the flight envelope is not the same".

Now that it is the same, it is suddenly nothing new or important because "flight envelope has no bearing".

Shrug.

[su27k]

This may have some bearing here, too bad nobody asked if this issue is related to reuse:

Jessica talking about Static Fire issues.

Sensors on Stage 1 detecting a small LOX leak. Inspection and repair occurred. Took a couple days.  That static fire occurred on Friday.

[intelati]

This may have some bearing here, too bad nobody asked if this issue is related to reuse:

Jessica talking about Static Fire issues.

Sensors on Stage 1 detecting a small LOX leak. Inspection and repair occurred. Took a couple days.  That static fire occurred on Friday.

There is some conversation in L2.

I don't think anything is official, but it certainly quacks like a duck...

But still, the reused boosters still have quite the record for reliability.

[speedevil]

It occurred to that the margin available for reuse of the booster might allow a complete recovery under nearly all circumstances from a single engine out.
If an engine fails, you end up not recovering the payload, not it missing its orbit.

[LouScheffer]

Looking again at this yet again.   This just counts scrubs due to booster problems, not upper level winds, weather, etc.

Booster scrubs mission
1046.1       1 Bangabandhu
1047.1       0 Telstar 19
1048.1       0 Iridium/Grace
1046.2       0 Merah Putih
1049.1       0 Telstar 18
1048.2       0 SAOCOM
1047.2       0 Es'hail
1046.3       0 SSO-A
1050.1       0 CRS-16
1054.1       1 GPS-III
1049.2       0 Iridium
1048.2       0 Nusantara Satu
1051.1       0 SpX-Dm1
1052.1       0 FH
1053.1       0 FH
1055.1       0 FH
1056.1       0 CRS-17
1049.3       0 Starlink
1051.2       0 RADARSAT
1057.1       0 STP-2
1052.2       0 STP-2
1053.2       0 STP-2
1056.2       0 CRS-18
1047.3       1 AMOS-18 (extra static fire needed)

So there have been 12 new and 12 used launches, and overall 3 of 24 (12.5%) had a technical scrub due to the booster(s) post rollout.  If three scrubs were evenly distributed among the 24 launches, we would naturally expect all on new boosters 220/2024 (11%) of the time, all on used boosters 220/2024 (11%) of the time, and two of one, one of the other each  792/2024 (39%) of the time.  So most likely, based on limited samples, there is little difference in finickyness between new and used boosters.

[FutureSpaceTourist]

This extract from the recent Discovery documentary talks about reuse generally but also includes a segment with Elon talking To Jim Bridenstine about insurance rates etc

[oiorionsbelt]

This comment doesn't fit here that well but I didn't see a better place and the picture illustrates well.
 I'm surprised that the returned F9's are both sooty and shiny.
 Do they "polish" them?
 If yes, is that more finicky?

Whew, on topic in the end.

[rockets4life97]

The decision not to static fire the booster for its third reuse on Starlink 9 (v1.0 launch would seem to indicate that the static fires are finding less issues making the static fire unnecessary.

[russianhalo117]

The decision not to static fire the booster for its third reuse on Starlink 9 (v1.0 launch would seem to indicate that the static fires are finding less issues making the static fire unnecessary.

It has been made an optional service.

[Nomadd]

The decision not to static fire the booster for its third reuse on Starlink 9 (v1.0 launch would seem to indicate that the static fires are finding less issues making the static fire unnecessary.

Maybe not so much that as finding all of the possible issues, so a computer can make the decision in 1/4 second that people were making in a day. I never really could figure out why the same standards carbon based lifeforms used couldn't be considered by software in the time between ignition and liftoff.

[wannamoonbase]

The decision not to static fire the booster for its third reuse on Starlink 9 (v1.0 launch would seem to indicate that the static fires are finding less issues making the static fire unnecessary.

Maybe not so much that as finding all of the possible issues, so a computer can make the decision in 1/4 second that people were making in a day. I never really could figure out why the same standards carbon based lifeforms used couldn't be considered by software in the time between ignition and liftoff.

When they were launching only once or twice a month they had the time.  So why not practice and collect data.  After 80 some launches and an increasing flight rate I completely agree with your assessment.

At some point as these Block 5 boosters increase the number of flights per booster we will start to see some kind of aging issues.  One would think that is has to be the engines, since they are the moving parts with pressure and thermal shocks.

Reflights or not, the F9 seems to have been pretty free of technical issues when launching.

[oldAtlas_Eguy]

I think it is more a factor of regular usage. Regular usage vs intermittent usage such as boosters sitting in storage or pads not seeing a launch for months. When atlas V's launch rate declined such that the pad usage went to about or less than 6 per year. The number of GSE issues have shot up. In launching from VAFB (Is it still AFB?), SpaceX will have the same issues with GSE and would be prudent in doing a SF to check the GSE not the booster. There are a lot of GSE that can fail when it is not in use regularly. Regular use means maintenance is more active in replacing fixing minor issues before they become major ones. If you are not doing full up tests or launches the issues rarely show up.

The same is true about boosters. The shorter the time between flights the less the need for that SF to check the booster for anything that could have gotten "sticky" while just sitting around.

It all comes down to more use means less need to test. Since time is the biggest factor in many failures.

[LouScheffer]

With the lastest GPS Scrub, we have now had 4 technical scrubs due to boosters, in 44 launches.  16 launches were new boosters, and 28 old.  3 scrubs were on new boosters.  What are the odds of that?

launches= 44 scrubs= 4 new boosters= 16 Used boosters= 28  total of 135751 possible cases
Assuming a scrub is equally likely, new or used, what are the odds of seeing N scrubs in new boosters?
Odds of  0 in new boosters: 1 * 20475 = 20475  cases,   odds= 0.150828
Odds of  1 in new boosters: 16 * 3276 = 52416  cases,   odds= 0.386119
Odds of  2 in new boosters: 120 * 378 = 45360  cases,   odds= 0.334141
Odds of  3 in new boosters: 560 * 28   = 15680  cases,   odds= 0.115506
Odds of  4 in new boosters: 1820 * 1   = 1820    cases,   odds= 0.0134069

So if new and used boosters were equally reliable, there is only a 13% chance the scrubs are as skewed as they are towards new boosters.  So reasonable evidence that used boosters are less finicky, but hardly conclusive.


Note: Only scrubs caused by boosters are counted.  Weather, range, second stage, GSE, etc. are not.
Booster scrubs mission
1046.1       1 Bangabandhu
1047.1       0 Telstar 19
1048.1       0 Iridium/Grace
1046.2       0 Merah Putih
1049.1       0 Telstar 18
1048.2       0 SAOCOM
1047.2       0 Es'hail
1046.3       0 SSO-A
1050.1       0 CRS-16
1054.1       1 GPS-III
--- 2019 ---
1049.2       0 Iridium
1048.2       0 Nusantara Satu
1051.1       0 SpX-Dm1
1052.1       0 FH
1053.1       0 FH
1055.1       0 FH
1056.1       0 CRS-17
1049.3       0 Starlink
1051.2       0 RADARSAT
1057.1       0 STP-2
1052.2       0 STP-2
1053.2       0 STP-2
1056.2       0 CRS-18
1047.3       1 AMOS-17 (extra static fire needed)
1048.4       0 Starlink
1059.1       0 CRS-19
1056.3       0 JCSat-18
--- 2020 ---
1049.4       0 Starlink
1046.4       0 Inflight abort
1051.3       0 Starlink
1056.4       0 Starlink
1059.2       0 CRS-20
1048.5       0 Starlink
1051.4       0 Starlink
1058.1       0 Crew Dragon
1049.5       0 Starlink (?)
1059.3       0 Starlink
1060.1       0 GPS-III-03
1058.2       0 Anasis-II
1051.5       0 Starlink
1049.6       0 Starlink
1059.4       0 SAOCOM 1B
1060.2       0 Starlink
1062.1       1 GPS-III

[WormPicker959]

With the lastest GPS Scrub, we have now had 4 technical scrubs due to boosters, in 44 launches.  16 launches were new boosters, and 28 old.  3 scrubs were on new boosters.  What are the odds of that?

launches= 44 scrubs= 4 new boosters= 16 Used boosters= 28  total of 135751 possible cases
Assuming a scrub is equally likely, new or used, what are the odds of seeing N scrubs in new boosters?
Odds of  0 in new boosters: 1 * 20475 = 20475  cases,   odds= 0.150828
Odds of  1 in new boosters: 16 * 3276 = 52416  cases,   odds= 0.386119
Odds of  2 in new boosters: 120 * 378 = 45360  cases,   odds= 0.334141
Odds of  3 in new boosters: 560 * 28   = 15680  cases,   odds= 0.115506
Odds of  4 in new boosters: 1820 * 1   = 1820    cases,   odds= 0.0134069

So if new and used boosters were equally reliable, there is only a 13% chance the scrubs are as skewed as they are towards new boosters.  So reasonable evidence that used boosters are less finicky, but hardly conclusive.


Note: Only scrubs caused by boosters are counted.  Weather, range, second stage, GSE, etc. are not.
Booster scrubs mission
1046.1       1 Bangabandhu
1047.1       0 Telstar 19
1048.1       0 Iridium/Grace
1046.2       0 Merah Putih
1049.1       0 Telstar 18
1048.2       0 SAOCOM
1047.2       0 Es'hail
1046.3       0 SSO-A
1050.1       0 CRS-16
1054.1       1 GPS-III
--- 2019 ---
1049.2       0 Iridium
1048.2       0 Nusantara Satu
1051.1       0 SpX-Dm1
1052.1       0 FH
1053.1       0 FH
1055.1       0 FH
1056.1       0 CRS-17
1049.3       0 Starlink
1051.2       0 RADARSAT
1057.1       0 STP-2
1052.2       0 STP-2
1053.2       0 STP-2
1056.2       0 CRS-18
1047.3       1 AMOS-17 (extra static fire needed)
1048.4       0 Starlink
1059.1       0 CRS-19
1056.3       0 JCSat-18
--- 2020 ---
1049.4       0 Starlink
1046.4       0 Inflight abort
1051.3       0 Starlink
1056.4       0 Starlink
1059.2       0 CRS-20
1048.5       0 Starlink
1051.4       0 Starlink
1058.1       0 Crew Dragon
1049.5       0 Starlink (?)
1059.3       0 Starlink
1060.1       0 GPS-III-03
1058.2       0 Anasis-II
1051.5       0 Starlink
1049.6       0 Starlink
1059.4       0 SAOCOM 1B
1060.2       0 Starlink
1062.1       1 GPS-III

The correct statistic to use here is the hypergeometric (black and white balls in urn, remember from your stats class?):

N=44
k=16
n=3
x=3



Cumulative probability of less than three new boosters having been randomly "picked" in such a sample is 0.9577, or in other words:

The odds of this happening "by chance" are 4.23%.

While this is a small sample size, it's just about statistically significant. If the next two booster scrubs are used boosters, it becomes 5.09%, just outside statistical significance.

[gparker]

The correct statistic to use here is the hypergeometric (black and white balls in urn, remember from your stats class?):

N=44
k=16
n=3
x=3

Hypergeometric n is 4 total scrubs, not 3. Using n=4 yields the same values as Lou's calculation: a 12.89% chance to get either 3 or 4 new-booster scrubs out of 4 total scrubs, assuming new and used boosters are equally likely to scrub.

[sdsds]

I'm having some difficulty understanding the value "44" in the above calculation. Shouldn't this value represent the number of times that there was an opportunity for a technical scrub to occur? So for a mission that scrubbed once and then flew, shouldn't that count as two opportunities for a technical scrub?

[gparker]

I'm having some difficulty understanding the value "44" in the above calculation. Shouldn't this value represent the number of times that there was an opportunity for a technical scrub to occur? So for a mission that scrubbed once and then flew, shouldn't that count as two opportunities for a technical scrub?

Perhaps it's more accurate to describe these calculations as counting "scrubbed launches" instead of "launch scrubs". 44 total launches; 4 of those launches each suffered some non-zero number of scrubs before flight.

[LouScheffer]

I'm having some difficulty understanding the value "44" in the above calculation. Shouldn't this value represent the number of times that there was an opportunity for a technical scrub to occur? So for a mission that scrubbed once and then flew, shouldn't that count as two opportunities for a technical scrub?

I think of this as 44 "launch campaigns", where a booster was considered ready, moved to the pad, and the countdown started.   Then in 4 cases something wrong in the booster was uncovered and caused a scrub.  Then the problem was fixed and the launch continued.  So "finicky" here means the odds that a given launch campaign will have at least one technical scrub due to the booster.

If boosters start having more than one technical problem in a launch campaign, a metric like "per attempt" rather than "per campaign" would make sense.  However the numbers are already so small (4 scrubs total) that distinctions like this are currently lost in the noise.

[sdsds]

Thanks, I appreciate the explanations provided by the terminology distinctions! And I agree with the assessment that numerically the distinctions are still "lost in the noise."

It does tie in with the question of why we as observers  — or more pertinently the operators of a launch system or of a range — care about a measure of how finicky a launch system is. For a casual observer watching a livestream a measure of "finickyness" would help us predict, once the count-down was in progress, the likelihood of a technical glitch preventing that particular count from reaching liftoff. I think for that the "per attempt" statistic would be more relevant than the "per campaign" statistic.

But for someone who is scheduling activity on a range? I bet for them the "per campaign" statistic is much more relevant!

[LouScheffer]

With the lastest GPS Scrub, we have now had 4 technical scrubs due to boosters, in 44 launches.  16 launches were new boosters, and 28 old.  3 scrubs were on new boosters.  What are the odds of that?

launches= 44 scrubs= 4 new boosters= 16 Used boosters= 28  total of 135751 possible cases
Assuming a scrub is equally likely, new or used, what are the odds of seeing N scrubs in new boosters?
Odds of  0 in new boosters: 1 * 20475 = 20475  cases,   odds= 0.150828
Odds of  1 in new boosters: 16 * 3276 = 52416  cases,   odds= 0.386119
Odds of  2 in new boosters: 120 * 378 = 45360  cases,   odds= 0.334141
Odds of  3 in new boosters: 560 * 28   = 15680  cases,   odds= 0.115506
Odds of  4 in new boosters: 1820 * 1   = 1820    cases,   odds= 0.0134069

So if new and used boosters were equally reliable, there is only a 13% chance the scrubs are as skewed as they are towards new boosters.  So reasonable evidence that used boosters are less finicky, but hardly conclusive.

The correct statistic to use here is the hypergeometric (black and white balls in urn, remember from your stats class?):

N=44
k=16
n=3
x=3

Cumulative probability of less than three new boosters having been randomly "picked" in such a sample is 0.9577, or in other words:

The odds of this happening "by chance" are 4.23%.

While this is a small sample size, it's just about statistically significant. If the next two booster scrubs are used boosters, it becomes 5.09%, just outside statistical significance.

I'm going to defend the good old practice of counting here.  There are at least 3 ways to solve this problem.  You can count instances, as I did, or use the hypergeometric distribution in two ways.  The great thing about math is that they all give exactly the same answer.  So all are correct.  But they are not equally easy to get right.

Counting:  for ascii math, say (n/m) is the "choose" function, n choose m, or choosing m things from n examples.  It's n!/(m!(n-m)!).  It's easy to remember, since n!/(n-m!) are the possible choices, but they can occur in m! possible orders.  In this case we need to know how many ways 44 rockets can have 4 failures, so it's 44*43*42*41/(1*2*3*4) = 135571 ways, all equally likely under the hypothesis.  Now how many ways can we have three failures among the new, and one among the old?  Three among the new is (16/3) or 16*15*14/(1*2*3) = 560 ways.  1 among the old is (28/1) = 28.  These are independent, so in total this can happen in 560*28  = 15680  ways,  giving odds of 15680/135571 = 0.115506 .

Alternatively, you can use the hypergeometric function in two ways.  What are the odds of having 44 rockets, 16 new, drawing 4 failures, and finding 3 are new?   That's a hypergeometric mass distribution function of pmf(3, 44, 16, 4) = 0.115506.

Alternatively, you can think of this as having 44 rockets, 4 of which are failures.  You draw 16 examples at random (the new ones).  What are the odds your sample contains 3 failures?  That's pmf (3, 44, 4, 16) = 0.115506.

All give the exact same result.  But I like the counting approach better, because you can SEE what's happening.  You can both do and check your work on a four function calculator (which I did).   With the hypergeometric distribution (as opposed to the hypergeometric series, related to but not what you want here), first you have to recall what it is and what it does (see your comment on remember your stats class).  Then you need to figure out what the arguments are (where the error was above).   Then you need to find a computer library that contains this function, and make sure you've got the right subfunction (cdf?  pmf?  etc).  Then to evaluate it you need to find the the order of parameters.  Finally you just get back a number, and it's hard to know if it's correct. That's why I like counting better.

Here is code that evaluates the hypergeometric pmf in both ways for our example.  All give the same answer.

from scipy.stats import hypergeom
for i in range(5):
  print(i, hypergeom.pmf(i, 44, 4, 16), hypergeom.pmf(i, 44, 16, 4))

[LouScheffer]

The current situation appears to the ultimate case of "used boosters are less finicky".   Any launches with new boosters are delayed, by weeks or months, while SpaceX looks at possible problems in a new batch of engines.  Meanwhile old boosters are still launching because they are known to work.

This reasoning is not completely airtight outside of SpaceX.  We *think* it's a problem with new engines since they said "investigate readings for engines" for the GPS flight.  And it's also possible that the risk occurs on the old engines too, but SpaceX is willing to take a chance on Starlink launches that it will not take for paying customers or astronauts.   However, by Occam's razor, the likely scenario is the new boosters have a potential bug, whereas old boosters are known to work.

[LouScheffer]

This is now confirmed.  From SpaceX's Koenigsmann:

Abort on the GPS III launch was caused by an early start on two of the nine first-stage engines.

[...] Engines were sent to McGregor, Texas for testing and they were able to reproduce the issue. The problem was traced to a blocked relief valve in the gas generator. There was leftover masking material from the production process.

[...]Problematic substance was sort of like nail polish. Only some of the recently produced engines have this problem.

So quite explicitly, re-used boosters were less finicky in this case.

[freddo411]

It is worth noting a couple of things about the SX merlin Gas Generator problem and solution

* There is a high level of instrumentation and a great deal of recorded historical data which is available to SX and to the Falcon9's avionics.   I suspect that this allows the avionics to have fairly tight limits, which when exceeded can allow for a engine shutdown.   

* SX has a deep and eager backlog of missions;   AF, NROL, NASA crew, private paying customers, and Starlink.   All of these have pressures to meet their schedules and expectations.     Note that SX did not hesitate to pause their cadence while they investigated this issue which in some respect can be considered a minor variation on normal behavior.   This behavior is laudable.

* The SX infrastructure and processes seems to be highly robust and adaptable.   The ability to quickly service a booster by pulling and replacing engines and the ability to test fire engines at McGregor is exemplary.   Others should follow this example.

* It's probably true that steep part of the learning curve on the F9 is long past, but that doesn't mean that there are not things still to be discovered and mastered -- just not as many and not as frequently.

I think all of these things can also be done on expendable boosters, but reused boosters, due to their lower costs, result in more flights, which makes all of the above a lot more likely to happen.

[Jansen]

New issues come as part of the continuous improvement process. More conservative approaches won’t see this as much, but are also much slower to innovate and improve.

One great thing about continuous vs block improvement is that it’s much easier to pinpoint the problem, rather than possibly being a dozen things that were recently changed.