Are re-used boosters less finicky?

[rockets4life97]

The decision not to static fire the booster for its third reuse on Starlink 9 (v1.0 launch would seem to indicate that the static fires are finding less issues making the static fire unnecessary.

[russianhalo117]

The decision not to static fire the booster for its third reuse on Starlink 9 (v1.0 launch would seem to indicate that the static fires are finding less issues making the static fire unnecessary.

It has been made an optional service.

[Nomadd]

The decision not to static fire the booster for its third reuse on Starlink 9 (v1.0 launch would seem to indicate that the static fires are finding less issues making the static fire unnecessary.

Maybe not so much that as finding all of the possible issues, so a computer can make the decision in 1/4 second that people were making in a day. I never really could figure out why the same standards carbon based lifeforms used couldn't be considered by software in the time between ignition and liftoff.

[wannamoonbase]

The decision not to static fire the booster for its third reuse on Starlink 9 (v1.0 launch would seem to indicate that the static fires are finding less issues making the static fire unnecessary.

Maybe not so much that as finding all of the possible issues, so a computer can make the decision in 1/4 second that people were making in a day. I never really could figure out why the same standards carbon based lifeforms used couldn't be considered by software in the time between ignition and liftoff.

When they were launching only once or twice a month they had the time.  So why not practice and collect data.  After 80 some launches and an increasing flight rate I completely agree with your assessment.

At some point as these Block 5 boosters increase the number of flights per booster we will start to see some kind of aging issues.  One would think that is has to be the engines, since they are the moving parts with pressure and thermal shocks.

Reflights or not, the F9 seems to have been pretty free of technical issues when launching.

[oldAtlas_Eguy]

I think it is more a factor of regular usage. Regular usage vs intermittent usage such as boosters sitting in storage or pads not seeing a launch for months. When atlas V's launch rate declined such that the pad usage went to about or less than 6 per year. The number of GSE issues have shot up. In launching from VAFB (Is it still AFB?), SpaceX will have the same issues with GSE and would be prudent in doing a SF to check the GSE not the booster. There are a lot of GSE that can fail when it is not in use regularly. Regular use means maintenance is more active in replacing fixing minor issues before they become major ones. If you are not doing full up tests or launches the issues rarely show up.

The same is true about boosters. The shorter the time between flights the less the need for that SF to check the booster for anything that could have gotten "sticky" while just sitting around.

It all comes down to more use means less need to test. Since time is the biggest factor in many failures.

[LouScheffer]

With the lastest GPS Scrub, we have now had 4 technical scrubs due to boosters, in 44 launches.  16 launches were new boosters, and 28 old.  3 scrubs were on new boosters.  What are the odds of that?

launches= 44 scrubs= 4 new boosters= 16 Used boosters= 28  total of 135751 possible cases
Assuming a scrub is equally likely, new or used, what are the odds of seeing N scrubs in new boosters?
Odds of  0 in new boosters: 1 * 20475 = 20475  cases,   odds= 0.150828
Odds of  1 in new boosters: 16 * 3276 = 52416  cases,   odds= 0.386119
Odds of  2 in new boosters: 120 * 378 = 45360  cases,   odds= 0.334141
Odds of  3 in new boosters: 560 * 28   = 15680  cases,   odds= 0.115506
Odds of  4 in new boosters: 1820 * 1   = 1820    cases,   odds= 0.0134069

So if new and used boosters were equally reliable, there is only a 13% chance the scrubs are as skewed as they are towards new boosters.  So reasonable evidence that used boosters are less finicky, but hardly conclusive.


Note: Only scrubs caused by boosters are counted.  Weather, range, second stage, GSE, etc. are not.
Booster scrubs mission
1046.1       1 Bangabandhu
1047.1       0 Telstar 19
1048.1       0 Iridium/Grace
1046.2       0 Merah Putih
1049.1       0 Telstar 18
1048.2       0 SAOCOM
1047.2       0 Es'hail
1046.3       0 SSO-A
1050.1       0 CRS-16
1054.1       1 GPS-III
--- 2019 ---
1049.2       0 Iridium
1048.2       0 Nusantara Satu
1051.1       0 SpX-Dm1
1052.1       0 FH
1053.1       0 FH
1055.1       0 FH
1056.1       0 CRS-17
1049.3       0 Starlink
1051.2       0 RADARSAT
1057.1       0 STP-2
1052.2       0 STP-2
1053.2       0 STP-2
1056.2       0 CRS-18
1047.3       1 AMOS-17 (extra static fire needed)
1048.4       0 Starlink
1059.1       0 CRS-19
1056.3       0 JCSat-18
--- 2020 ---
1049.4       0 Starlink
1046.4       0 Inflight abort
1051.3       0 Starlink
1056.4       0 Starlink
1059.2       0 CRS-20
1048.5       0 Starlink
1051.4       0 Starlink
1058.1       0 Crew Dragon
1049.5       0 Starlink (?)
1059.3       0 Starlink
1060.1       0 GPS-III-03
1058.2       0 Anasis-II
1051.5       0 Starlink
1049.6       0 Starlink
1059.4       0 SAOCOM 1B
1060.2       0 Starlink
1062.1       1 GPS-III

[WormPicker959]

With the lastest GPS Scrub, we have now had 4 technical scrubs due to boosters, in 44 launches.  16 launches were new boosters, and 28 old.  3 scrubs were on new boosters.  What are the odds of that?

launches= 44 scrubs= 4 new boosters= 16 Used boosters= 28  total of 135751 possible cases
Assuming a scrub is equally likely, new or used, what are the odds of seeing N scrubs in new boosters?
Odds of  0 in new boosters: 1 * 20475 = 20475  cases,   odds= 0.150828
Odds of  1 in new boosters: 16 * 3276 = 52416  cases,   odds= 0.386119
Odds of  2 in new boosters: 120 * 378 = 45360  cases,   odds= 0.334141
Odds of  3 in new boosters: 560 * 28   = 15680  cases,   odds= 0.115506
Odds of  4 in new boosters: 1820 * 1   = 1820    cases,   odds= 0.0134069

So if new and used boosters were equally reliable, there is only a 13% chance the scrubs are as skewed as they are towards new boosters.  So reasonable evidence that used boosters are less finicky, but hardly conclusive.


Note: Only scrubs caused by boosters are counted.  Weather, range, second stage, GSE, etc. are not.
Booster scrubs mission
1046.1       1 Bangabandhu
1047.1       0 Telstar 19
1048.1       0 Iridium/Grace
1046.2       0 Merah Putih
1049.1       0 Telstar 18
1048.2       0 SAOCOM
1047.2       0 Es'hail
1046.3       0 SSO-A
1050.1       0 CRS-16
1054.1       1 GPS-III
--- 2019 ---
1049.2       0 Iridium
1048.2       0 Nusantara Satu
1051.1       0 SpX-Dm1
1052.1       0 FH
1053.1       0 FH
1055.1       0 FH
1056.1       0 CRS-17
1049.3       0 Starlink
1051.2       0 RADARSAT
1057.1       0 STP-2
1052.2       0 STP-2
1053.2       0 STP-2
1056.2       0 CRS-18
1047.3       1 AMOS-17 (extra static fire needed)
1048.4       0 Starlink
1059.1       0 CRS-19
1056.3       0 JCSat-18
--- 2020 ---
1049.4       0 Starlink
1046.4       0 Inflight abort
1051.3       0 Starlink
1056.4       0 Starlink
1059.2       0 CRS-20
1048.5       0 Starlink
1051.4       0 Starlink
1058.1       0 Crew Dragon
1049.5       0 Starlink (?)
1059.3       0 Starlink
1060.1       0 GPS-III-03
1058.2       0 Anasis-II
1051.5       0 Starlink
1049.6       0 Starlink
1059.4       0 SAOCOM 1B
1060.2       0 Starlink
1062.1       1 GPS-III

The correct statistic to use here is the hypergeometric (black and white balls in urn, remember from your stats class?):

N=44
k=16
n=3
x=3



Cumulative probability of less than three new boosters having been randomly "picked" in such a sample is 0.9577, or in other words:

The odds of this happening "by chance" are 4.23%.

While this is a small sample size, it's just about statistically significant. If the next two booster scrubs are used boosters, it becomes 5.09%, just outside statistical significance.

[gparker]

The correct statistic to use here is the hypergeometric (black and white balls in urn, remember from your stats class?):

N=44
k=16
n=3
x=3

Hypergeometric n is 4 total scrubs, not 3. Using n=4 yields the same values as Lou's calculation: a 12.89% chance to get either 3 or 4 new-booster scrubs out of 4 total scrubs, assuming new and used boosters are equally likely to scrub.

[sdsds]

I'm having some difficulty understanding the value "44" in the above calculation. Shouldn't this value represent the number of times that there was an opportunity for a technical scrub to occur? So for a mission that scrubbed once and then flew, shouldn't that count as two opportunities for a technical scrub?

[gparker]

I'm having some difficulty understanding the value "44" in the above calculation. Shouldn't this value represent the number of times that there was an opportunity for a technical scrub to occur? So for a mission that scrubbed once and then flew, shouldn't that count as two opportunities for a technical scrub?

Perhaps it's more accurate to describe these calculations as counting "scrubbed launches" instead of "launch scrubs". 44 total launches; 4 of those launches each suffered some non-zero number of scrubs before flight.

[LouScheffer]

I'm having some difficulty understanding the value "44" in the above calculation. Shouldn't this value represent the number of times that there was an opportunity for a technical scrub to occur? So for a mission that scrubbed once and then flew, shouldn't that count as two opportunities for a technical scrub?

I think of this as 44 "launch campaigns", where a booster was considered ready, moved to the pad, and the countdown started.   Then in 4 cases something wrong in the booster was uncovered and caused a scrub.  Then the problem was fixed and the launch continued.  So "finicky" here means the odds that a given launch campaign will have at least one technical scrub due to the booster.

If boosters start having more than one technical problem in a launch campaign, a metric like "per attempt" rather than "per campaign" would make sense.  However the numbers are already so small (4 scrubs total) that distinctions like this are currently lost in the noise.

[sdsds]

Thanks, I appreciate the explanations provided by the terminology distinctions! And I agree with the assessment that numerically the distinctions are still "lost in the noise."

It does tie in with the question of why we as observers  — or more pertinently the operators of a launch system or of a range — care about a measure of how finicky a launch system is. For a casual observer watching a livestream a measure of "finickyness" would help us predict, once the count-down was in progress, the likelihood of a technical glitch preventing that particular count from reaching liftoff. I think for that the "per attempt" statistic would be more relevant than the "per campaign" statistic.

But for someone who is scheduling activity on a range? I bet for them the "per campaign" statistic is much more relevant!

[LouScheffer]

With the lastest GPS Scrub, we have now had 4 technical scrubs due to boosters, in 44 launches.  16 launches were new boosters, and 28 old.  3 scrubs were on new boosters.  What are the odds of that?

launches= 44 scrubs= 4 new boosters= 16 Used boosters= 28  total of 135751 possible cases
Assuming a scrub is equally likely, new or used, what are the odds of seeing N scrubs in new boosters?
Odds of  0 in new boosters: 1 * 20475 = 20475  cases,   odds= 0.150828
Odds of  1 in new boosters: 16 * 3276 = 52416  cases,   odds= 0.386119
Odds of  2 in new boosters: 120 * 378 = 45360  cases,   odds= 0.334141
Odds of  3 in new boosters: 560 * 28   = 15680  cases,   odds= 0.115506
Odds of  4 in new boosters: 1820 * 1   = 1820    cases,   odds= 0.0134069

So if new and used boosters were equally reliable, there is only a 13% chance the scrubs are as skewed as they are towards new boosters.  So reasonable evidence that used boosters are less finicky, but hardly conclusive.

The correct statistic to use here is the hypergeometric (black and white balls in urn, remember from your stats class?):

N=44
k=16
n=3
x=3

Cumulative probability of less than three new boosters having been randomly "picked" in such a sample is 0.9577, or in other words:

The odds of this happening "by chance" are 4.23%.

While this is a small sample size, it's just about statistically significant. If the next two booster scrubs are used boosters, it becomes 5.09%, just outside statistical significance.

I'm going to defend the good old practice of counting here.  There are at least 3 ways to solve this problem.  You can count instances, as I did, or use the hypergeometric distribution in two ways.  The great thing about math is that they all give exactly the same answer.  So all are correct.  But they are not equally easy to get right.

Counting:  for ascii math, say (n/m) is the "choose" function, n choose m, or choosing m things from n examples.  It's n!/(m!(n-m)!).  It's easy to remember, since n!/(n-m!) are the possible choices, but they can occur in m! possible orders.  In this case we need to know how many ways 44 rockets can have 4 failures, so it's 44*43*42*41/(1*2*3*4) = 135571 ways, all equally likely under the hypothesis.  Now how many ways can we have three failures among the new, and one among the old?  Three among the new is (16/3) or 16*15*14/(1*2*3) = 560 ways.  1 among the old is (28/1) = 28.  These are independent, so in total this can happen in 560*28  = 15680  ways,  giving odds of 15680/135571 = 0.115506 .

Alternatively, you can use the hypergeometric function in two ways.  What are the odds of having 44 rockets, 16 new, drawing 4 failures, and finding 3 are new?   That's a hypergeometric mass distribution function of pmf(3, 44, 16, 4) = 0.115506.

Alternatively, you can think of this as having 44 rockets, 4 of which are failures.  You draw 16 examples at random (the new ones).  What are the odds your sample contains 3 failures?  That's pmf (3, 44, 4, 16) = 0.115506.

All give the exact same result.  But I like the counting approach better, because you can SEE what's happening.  You can both do and check your work on a four function calculator (which I did).   With the hypergeometric distribution (as opposed to the hypergeometric series, related to but not what you want here), first you have to recall what it is and what it does (see your comment on remember your stats class).  Then you need to figure out what the arguments are (where the error was above).   Then you need to find a computer library that contains this function, and make sure you've got the right subfunction (cdf?  pmf?  etc).  Then to evaluate it you need to find the the order of parameters.  Finally you just get back a number, and it's hard to know if it's correct. That's why I like counting better.

Here is code that evaluates the hypergeometric pmf in both ways for our example.  All give the same answer.

from scipy.stats import hypergeom
for i in range(5):
  print(i, hypergeom.pmf(i, 44, 4, 16), hypergeom.pmf(i, 44, 16, 4))

[LouScheffer]

The current situation appears to the ultimate case of "used boosters are less finicky".   Any launches with new boosters are delayed, by weeks or months, while SpaceX looks at possible problems in a new batch of engines.  Meanwhile old boosters are still launching because they are known to work.

This reasoning is not completely airtight outside of SpaceX.  We *think* it's a problem with new engines since they said "investigate readings for engines" for the GPS flight.  And it's also possible that the risk occurs on the old engines too, but SpaceX is willing to take a chance on Starlink launches that it will not take for paying customers or astronauts.   However, by Occam's razor, the likely scenario is the new boosters have a potential bug, whereas old boosters are known to work.

[LouScheffer]

This is now confirmed.  From SpaceX's Koenigsmann:

Abort on the GPS III launch was caused by an early start on two of the nine first-stage engines.

[...] Engines were sent to McGregor, Texas for testing and they were able to reproduce the issue. The problem was traced to a blocked relief valve in the gas generator. There was leftover masking material from the production process.

[...]Problematic substance was sort of like nail polish. Only some of the recently produced engines have this problem.

So quite explicitly, re-used boosters were less finicky in this case.

[freddo411]

It is worth noting a couple of things about the SX merlin Gas Generator problem and solution

* There is a high level of instrumentation and a great deal of recorded historical data which is available to SX and to the Falcon9's avionics.   I suspect that this allows the avionics to have fairly tight limits, which when exceeded can allow for a engine shutdown.   

* SX has a deep and eager backlog of missions;   AF, NROL, NASA crew, private paying customers, and Starlink.   All of these have pressures to meet their schedules and expectations.     Note that SX did not hesitate to pause their cadence while they investigated this issue which in some respect can be considered a minor variation on normal behavior.   This behavior is laudable.

* The SX infrastructure and processes seems to be highly robust and adaptable.   The ability to quickly service a booster by pulling and replacing engines and the ability to test fire engines at McGregor is exemplary.   Others should follow this example.

* It's probably true that steep part of the learning curve on the F9 is long past, but that doesn't mean that there are not things still to be discovered and mastered -- just not as many and not as frequently.

I think all of these things can also be done on expendable boosters, but reused boosters, due to their lower costs, result in more flights, which makes all of the above a lot more likely to happen.

[Jansen]

New issues come as part of the continuous improvement process. More conservative approaches won’t see this as much, but are also much slower to innovate and improve.

One great thing about continuous vs block improvement is that it’s much easier to pinpoint the problem, rather than possibly being a dozen things that were recently changed.