Cat 3 is required to fly crew, so F9 was going to need this certification soon. Would like to know if this covers nuclear payloads, though.

Word is nuclear payloads need separate certification. Edit: Whitelancer beat me to it.

Just wondering. Where does the "14 consecutive successful missions" number come from? For 95% reliability wouldn't you need 19? I count 36 consecutive successful missions so far for SX.

Quote from: psherriffs on 11/09/2018 10:05 pmJust wondering. Where does the "14 consecutive successful missions" number come from? For 95% reliability wouldn't you need 19? I count 36 consecutive successful missions so far for SX.After 14 consecutive successes you have more than 50% confidence in 95% reliability. This is calculated as 0.95^14 = .487% This 48.7% is the average probability of getting at least one failure if the actual reliability is 95%. The confidence level is 100% less this number, or 51.3%.19 successful missions does not "prove" 95% reliability, it only establishes a 62.3% (100% - 0.95^19) confidence that the true reliability is at least 95%.

Quote from: envy887 on 11/09/2018 10:35 pmQuote from: psherriffs on 11/09/2018 10:05 pmJust wondering. Where does the "14 consecutive successful missions" number come from? For 95% reliability wouldn't you need 19? I count 36 consecutive successful missions so far for SX.After 14 consecutive successes you have more than 50% confidence in 95% reliability. This is calculated as 0.95^14 = .487% This 48.7% is the average probability of getting at least one failure if the actual reliability is 95%. The confidence level is 100% less this number, or 51.3%.19 successful missions does not "prove" 95% reliability, it only establishes a 62.3% (100% - 0.95^19) confidence that the true reliability is at least 95%.That's not how these probabilities work.To illustrate the point, lets take an extreme, unrealistic example. Lets assume that half of all launch vehicles are 95% reliable and all the other half are 90% reliable.You don't know if this particular vehicle is 90% reliable or 95% reliable. So lets look at how likely it is that it's 95% reliable versus 90% reliable.If it happens that the vehicle is 95% reliable and you launch 14 times, the odds all 14 are successful are .488, as calculated above. If it happens the vehicle is 90% reliable and you lauch 14 times, the odds all 14 are successful are .229 (.9^14).Since the 90% vehicles and 95% vehicles are equally represented in the population from which you are sampling, the odds you have a 95% reliable vehicle are .488 / (.488 + .229) = 0.680. So the odds you have a 95% reliable vehicle are 0.68.Now, suppose instead that 95%-reliable are one in a billion and that the others are all 90%-reliable. Now, if you see 14 launches in a row that are successful, the odds you have a 95%-reliable vehicle are .488 / (.488 + (.229 * (10^9 - 1)), which is approximately one in a billion.I hope this helps illustrate that everything depends on the prior probabilities. That is, it depends on the population of launch vehicles you are sampling from. If launch vehicles in general are likely to be reliable, you need far fewer launches to be confident in a new launch vehicle than if launch vehicles in general are unlikely to be reliable.It's not a very satisfying answer, because we don't really know much about the prior probabilities. So we have to assume something. But we should be aware that our calculations are based on these (very uncertain) prior probabilities and be appropriately cautious in our use of the probability figures we get from them.By the way, this is something that nearly everyone who isn't a professional statistician gets wrong. Probabilities are often non-intuitive. That's why people hire professional statisticians.

Quote from: ChrisWilson68 on 11/09/2018 11:32 pmQuote from: envy887 on 11/09/2018 10:35 pmQuote from: psherriffs on 11/09/2018 10:05 pmJust wondering. Where does the "14 consecutive successful missions" number come from? For 95% reliability wouldn't you need 19? I count 36 consecutive successful missions so far for SX.After 14 consecutive successes you have more than 50% confidence in 95% reliability. This is calculated as 0.95^14 = .487% This 48.7% is the average probability of getting at least one failure if the actual reliability is 95%. The confidence level is 100% less this number, or 51.3%.19 successful missions does not "prove" 95% reliability, it only establishes a 62.3% (100% - 0.95^19) confidence that the true reliability is at least 95%.That's not how these probabilities work.To illustrate the point, lets take an extreme, unrealistic example. Lets assume that half of all launch vehicles are 95% reliable and all the other half are 90% reliable.You don't know if this particular vehicle is 90% reliable or 95% reliable. So lets look at how likely it is that it's 95% reliable versus 90% reliable.If it happens that the vehicle is 95% reliable and you launch 14 times, the odds all 14 are successful are .488, as calculated above. If it happens the vehicle is 90% reliable and you lauch 14 times, the odds all 14 are successful are .229 (.9^14).Since the 90% vehicles and 95% vehicles are equally represented in the population from which you are sampling, the odds you have a 95% reliable vehicle are .488 / (.488 + .229) = 0.680. So the odds you have a 95% reliable vehicle are 0.68.Now, suppose instead that 95%-reliable are one in a billion and that the others are all 90%-reliable. Now, if you see 14 launches in a row that are successful, the odds you have a 95%-reliable vehicle are .488 / (.488 + (.229 * (10^9 - 1)), which is approximately one in a billion.I hope this helps illustrate that everything depends on the prior probabilities. That is, it depends on the population of launch vehicles you are sampling from. If launch vehicles in general are likely to be reliable, you need far fewer launches to be confident in a new launch vehicle than if launch vehicles in general are unlikely to be reliable.It's not a very satisfying answer, because we don't really know much about the prior probabilities. So we have to assume something. But we should be aware that our calculations are based on these (very uncertain) prior probabilities and be appropriately cautious in our use of the probability figures we get from them.By the way, this is something that nearly everyone who isn't a professional statistician gets wrong. Probabilities are often non-intuitive. That's why people hire professional statisticians.Ok, I meant "more than 50% confidence in 95% reliability OR BETTER". It's not a >50% confidence that the reliability is exactly 95%, just a >50% confidence that reliability is higher than 95%. You don't need to know the reliability population distribution of all launch vehicles to establish this after 14 consecutive successes of the same design.

This is all about comparing the odds that you've gotten this result because it's a reliable vehicle versus that you've gotten this result by chance even though the vehicle is not reliable.

Does the cat 3 cert include FH? If so it clears they way for FH as a candidate LV for Europa Clippper. Else FH would still need a cat 3 cert.

Quote from: oldAtlas_Eguy on 11/10/2018 02:18 pmDoes the cat 3 cert include FH? If so it clears they way for FH as a candidate LV for Europa Clippper. Else FH would still need a cat 3 cert.My understanding it does not. But FH could gain Cat 3 late next year (min 3 flights, plus Falcon 9 heritage).

3 consecutive successful flights of a common launch vehicle configuration from an evolved vehicle family developed by an LSC with a previously certified launch vehicle for Risk Category 3, instrumented to provide design verification & flight performance data

3 (minimum 2 consecutive) successful flights of a common launch vehicle configuration, instrumented to provide design verification & flight performance data

A.7 NASA IV&V is a comprehensive assessment of the common launch vehicle analyses conducted by the launch service contractor. It will be based on the complexities and heritage of the vehicle. The LSP will propose the appropriate level of technical assessment (LSP independent modeling, launch service contractor model evaluation, and/or launch service contractor analytical review) at a FPB used to assess the certification strategy for a specified common launch vehicle configuration

Did they actually use the 14 flight option, or the 6 flight option? There have been 6 flights of Block 5, and they have a good amount of insight into that vehicle from the Commercial Crew program.

Quote from: gongora on 11/10/2018 01:49 pmDid they actually use the 14 flight option, or the 6 flight option? There have been 6 flights of Block 5, and they have a good amount of insight into that vehicle from the Commercial Crew program.Yeah, I removed that part earlier just to be safe. I was under the impression that the other two options were for new variants of launch vehicles that already had Cat 3 certification. However, D4H got certification with less than 14, and I don't think D4M ever had it?

That's not how these probabilities work. [Examples deleted]I hope this helps illustrate that everything depends on the prior probabilities. That is, it depends on the population of launch vehicles you are sampling from. [...] It's not a very satisfying answer, because we don't really know much about the prior probabilities. So we have to assume something. But we should be aware that our calculations are based on these (very uncertain) prior probabilities and be appropriately cautious in our use of the probability figures we get from them.

this is something that nearly everyone who isn't a professional statistician gets wrong. Probabilities are often non-intuitive. That's why people hire professional statisticians.

Quote from: ChrisWilson68 on 11/10/2018 01:24 amThis is all about comparing the odds that you've gotten this result because it's a reliable vehicle versus that you've gotten this result by chance even though the vehicle is not reliable. Which is why the confidence is 50%. There is still an equal chance that the actual reliability is less than 95%, including due to weird population distributions.

I am not a reliability engineer, but I do know from working with reliability engineers that this is how they calculate confidence levels for reliability in populations with unknown distributions.

Quotethis is something that nearly everyone who isn't a professional statistician gets wrong. Probabilities are often non-intuitive. That's why people hire professional statisticians.A professional statistician won't be much help here. The problem is the assumptions behind the model (the priors) and not the mathematics of probability.

Quote from: envy887 on 11/10/2018 12:57 pmQuote from: ChrisWilson68 on 11/10/2018 01:24 amThis is all about comparing the odds that you've gotten this result because it's a reliable vehicle versus that you've gotten this result by chance even though the vehicle is not reliable. Which is why the confidence is 50%. There is still an equal chance that the actual reliability is less than 95%, including due to weird population distributions.Incorrect. See my previous posts where I worked out concrete examples where this is not true.

Quote from: envy887 on 11/10/2018 12:57 pmI am not a reliability engineer, but I do know from working with reliability engineers that this is how they calculate confidence levels for reliability in populations with unknown distributions.You're misunderstanding what they're doing.

They're not calculating it this way because these calculations give numbers that are true no matter what the prior distributions are. They're calculating it this way because, if they want to get numbers, they need to assume *something* about the priors. These calculations build in a particular set of assumptions about the priors.

A lot of people think what you are claiming here, that these formulas give results that are true no matter what the priors are.

But that's not true. The formulas assume something about the priors. They have to. They assume something about the priors that is a useful heuristic.There's nothing wrong with doing a calculation like that as long as you understand what it actually means and don't get overconfident about the numbers it gives.