Quote from: oldAtlas_Eguy on 09/01/2017 02:54 amThis whole item is the fallacy of statistics. The application of a general case that is a statistical event over a large population (the population here being many many differently designed complex systems not the multiple flights of a single system) to specify how a single system will behave in the future is the fallacy. An unknown-unknown is by definition not statistically definable as an analyzable risk. The unknown is that there are from the statistical population a very wide range to what is possible from no matter how many you try none fail to almost every one you try fail from something different.Well put!Quote from: Robotbeat on 09/01/2017 03:58 amWhatever. Ed's statistic are fine, although I quibble about how he defines failure.We use these crude methods because we don't /know/ anything with any kind of quantitative certainty except what the failures and successes tell us. How do you /quantify/ differences in culture, procedures, whether "demons" are being taken care of or not?You can't. Especially not us. So use statistics. Works crudely with a small sample, it has large error bars in such a case, but it still works....The major fallacy is thinking these crude methods can be accurate to a few significant digits. The second major fallacy is assuming away factors that are difficult to quantify:1. How likely is the fault to be repeated?2. How systemic is(are) the fault(s)?3. How much does culture adjust the statistics?4. How much does rapid development (or stasis) affect the numbers?5. How does deterioration in suppliers' quality system affect the likelihood of failure?6. How do near misses get counted? (As successes? As partial failures? As failures?)7. How does analyzing returned cores improve vehicle reliability?8. How much does a demoralized, overworked, down-sizing workforce affect reliability?9. Add your items here...These are tough-to-quantify factors, but assuming that they are not in play makes the crude methods even cruder. Think about all of the 'factors' that a full post-accident review board would list as contributing...Engineering is done with numbers is not equivalent to having a number is engineering.
This whole item is the fallacy of statistics. The application of a general case that is a statistical event over a large population (the population here being many many differently designed complex systems not the multiple flights of a single system) to specify how a single system will behave in the future is the fallacy. An unknown-unknown is by definition not statistically definable as an analyzable risk. The unknown is that there are from the statistical population a very wide range to what is possible from no matter how many you try none fail to almost every one you try fail from something different.
Whatever. Ed's statistic are fine, although I quibble about how he defines failure.We use these crude methods because we don't /know/ anything with any kind of quantitative certainty except what the failures and successes tell us. How do you /quantify/ differences in culture, procedures, whether "demons" are being taken care of or not?You can't. Especially not us. So use statistics. Works crudely with a small sample, it has large error bars in such a case, but it still works....
Problem is that you are adding together statistics from when Russia(USSR) had a vibrant space program and latter years where major corrosion of quality and funding have undermined that program. How do you quantify or weight the rate of increase of launch failures into success rate? Insurance companies charge 5x as much for Proton launches as they do for Falcon -- some here say their failure rates are nearly the same... Ariane 5 and Falcon 9 have nearly identical insurance rates. Are the insurance company actuarial less accurate than some armchair statisticians here?
At least eight of the nine cubesats sent by the Russian Soyuz 2.1a rocket into a 600-kilometer orbit July 14 alongside a larger spacecraft, the Kanopus-V-IK Russian Earth-imaging satellite, are not responding to commands from their operators.
The boundaries between failures are not crystal clear -- to place significant weight on interpretations of what constitutes a failure, while ignoring such things as listed above, highlights the fatal flaw in small number statistics. Especially so when 'impartial observer' objectivity is mandatory... and rare. I agree that insurance company calculations of loss on next/future flight might be best statistical treatment available.
Quote from: AncientU on 09/01/2017 03:58 pmThe boundaries between failures are not crystal clear -- to place significant weight on interpretations of what constitutes a failure, while ignoring such things as listed above, highlights the fatal flaw in small number statistics. Especially so when 'impartial observer' objectivity is mandatory... and rare. I agree that insurance company calculations of loss on next/future flight might be best statistical treatment available.In the shipping industry the definition of when each party is responsible is pretty well defined - from Wikipedia:"FOB, "Free On Board", is a term in international commercial law specifying at what point respective obligations, costs, and risk involved in the delivery of goods shift from the seller to the buyer under the Incoterms 2010 standard published by the International Chamber of Commerce."So if the payload is placed in the agreed upon orbit, and the payload is released without incident, then it's not the transportation company's responsibility for the function or non-function of the payload.
Or the failure rate actually was expected and accounted for.
SpaceX is now launching quickly, on pace for like 18 launches per year. By the end of next year, if they avoid failure, they'll have 40-50 consecutive successes. That's top of the class reliability, on par with Ariane 5, etc. So we don't have long for SpaceX to prove their reliability. And if they suffer failures? Well, then we know that's how things are. Either way, a bigger sample size and more insight.
Here are the 95% confidence intervals and point estimates for the reliability of several launch vehicles. Generally the tighter confidence intervals correspond to more launches. These statistics provide insight. It can be said, for example, that the Merlin 1D-powered Falcon 9 is currently likely less reliable than Atlas 5 or Ariane 5 ECA, but there is a chance (because the intervals overlap the point estimates) that it could end up being as reliable as those launchers. There is also a chance that it ends up in the Titan 4/Proton M/Briz M range. - Ed Kyle
Okay. Here I go with the reasoning why insurance companies like the F9. The F9 design has many elements of it that are designed to be fault/failure tolerant. These designed systems can handle both some probable failure modes as well as totally unknown ones. This gives insurance companies a great deal of confidence in the risk assessments on the F9. The Ariane has to primarily rely on it's QC program to get the same level of insurance company risks confidence levels. This is also the case for most other LV's. But this is also not to say that the F9 dose not have single point of failures for mission success. Those still exist but are fewer and are relegated to lower risk items (supposedly). You can only estimate that that is the case and rely on the QC program to manage them. The risk analysis is the evaluation usually of the rigorousness of the design and test as well as the effectiveness of the QC methodologies used. The insurance companies have usually the inside info to be able to make these evaluations but we do not.In many ways SpaceX decision to go the route of fault/failure tolerance actually lowered costs for a given payload size even though the increased weight penalties decreased the payload capability of the launcher. This was because it was much more cost effective to implement fault/failure tolerance than to make certain systems/parts very highly reliable by as much as a cost factor of two or more in a lot of the cases.How does this impact competition. SpaceX design philosophy was to make a highly reliable but very cheap booster not by using the very latest bleeding edge technology but by managing the cost and design trade-offs when multiple design philosophies are applied to the design of a single system. Until the other LV providers start to change their approaches to the design of the LVs from performance to cost as the most important item while still maintaining the same level of reliability they will forever be secondary when customers are deciding who will fly their p[ayload.
Unless I missed it another reliability enhancement from my viewpoint is SpaceX's elimination of pyrotechnics. Since they can't be tested, they appear less reliable than a mechanical action that can be flight pre-tested.Now add to this menu the unique capability to examine flown hardware for wear & deterioration.
Quote from: philw1776 on 09/02/2017 02:55 pmUnless I missed it another reliability enhancement from my viewpoint is SpaceX's elimination of pyrotechnics. Since they can't be tested, they appear less reliable than a mechanical action that can be flight pre-tested.Now add to this menu the unique capability to examine flown hardware for wear & deterioration.Nit: Pyrotechnics can be tested, and have been tested repeatedly for years. Whether their elimination contributes to higher reliability--or other factors such as the ability to examine flown hardware on a regular basis--should eventually be seen in the overall LV reliability. Certainly seems intuitive that such would increase reliability, but still a bit early to call as the data is limited.
