SpaceX Falcon 9 - AMOS-6 - (Pad Failure) - DISCUSSION THREAD (2)

[Stephen GW]

We must be getting close to an investigation round up report. Potentially about a month until "RTF" <---I guess we can call it that, although I hesitate as it wasn't a flight failure.

RTF is fine, it just means "return to fueling" in this case :-)

[baldusi]

I'd call it RTP - Return To Pad. 

I don't want to name the Pad Exploration Corporations, but it would be a Return To Pad(X).

[CorvusCorax]

Potentially about a month until "RTF" <---I guess we can call it that, although I hesitate as it wasn't a flight failure.

I think the critical point to justify the term "return to flight" is that the fleet is being grounded for whatever reason.

AFAIK that situation has happened in civil aviation too. Once they find a big enough flaw to ground a fleet, they need to fix it first, then they can return to flight. Don't need an in-flight disaster for that.

The thing is, in civil aviation, they often don't ground a fleet unless a crash investigation identifies a fundamental and fleet wide flaw in the vehicle design or operation.

With rockets I think its just a more pessimistic approach, if one blows up, you assume its a flaw in the design until proven otherwise and ground the fleet proactively, then try to find what really caused it.

One day maybe space flight will be so routine, that even when one blows up, they won't ground the fleet unless there's significant evidence that the entire fleet is indeed affected. After all with increasing reuse, causes like "maintenance error" and "material fatigue" will creep up in likelyhood until they will eventually end up above "design flaw"

[guckyfan]

Things may change with reuse. It may not always be necessary to ground the whole fleet after one failure.

[cscott]

In all the discussion there is an (hidden) unknown, which is the temperature of the He Gas entering the 2nd stage, or better the COPVīs. From our information exchange here we know only the He distribution to the pad.
Secondly, is there any thought why the use of supercooled LOX compromises the COPV in the 2nd stage, but not in the 1st?

Asked and answered above.
If the compressing helium drops below the solidification temperature of oxygen, causing the hypothesized failure mechanism, then it should be possible to raise its supply temperature so that it achieves a higher final temperature, closer to that of the sub-cooled oxygen and safely above the freezing temperature. 
(As said: if, should, safely, ... The devil is in the details.)
....

Understood, but one should not consider the compression alone. There is definitely expansion of the He gas coming from the He distribution system, which is at higher pressure than the COPF final pressure. This means in the end more expansion than compression. Knowing this, it is necessary to know also if the expanding He gas is cooled prior to entering into the 2nd stage, and if yes, to which temperature. Otherwise the logic of solidifying LOX might be wrong.

From the conversation earlier in this thread, it seems clear that you have *both* expansion and compression *simultaneously* at either end of the tank during the fill.  So whichever of those you need for your failure theory to make sense, it's present.

What I'm wondering about is whether this is an unexpectedly long time to close out the investigation.  We were told "almost done" a while ago, and as far as we know the RTF is still on for December.  So what gives?

WAG: one of the participants in the review wants to reproduce some of the claimed effects (although perhaps only in CFD code) for themselves before signing off.  So official sign-off is delayed for that third-party. If that third party is not directly involved in the RTF flight (say, NASA) and everyone else has signed off, they may be delaying the formal publication but not the actual RTF process.

Other theories?  Does the paperwork to finish up an investigation simply take a long time to collect all the required signatures?  Could the failure mechanism be so ITAR-restricted that we'll never get a formal report, SpaceX will just launch at some point?

[CameronD]

Other theories?  Does the paperwork to finish up an investigation simply take a long time to collect all the required signatures?  Could the failure mechanism be so ITAR-restricted that we'll never get a formal report, SpaceX will just launch at some point?

Here's my theory:  A vast amount of data, analysis and findings from several directions (including the 'we shot it and it blew up' one) have to be compiled into a report format that is both (a) meaningful and (b) understandable by those signing-off on it.  Given schedule pressure, so long as whichever parties are involved in the next launch are happy with the summary findings 'in principle', they might be quite okay for launch preparations to press ahead, with the detailed Report due out at some later date..

[Steven Pietrobon]

Here's a paper on the properties of Helium at various temperatures and pressures. An interesting curve is shown Figure 35 on page 31 of the pdf which I've added here. This is what is called the Joule-Thomson inversion curve. If you are like myself who are not familiar with what this curve means, let me explain.

On the vertical axis we have temperature in Kelvin, or degrees above absolute zero (-273.15 C). The horizontal axis is pressure, which goes from 0 to 40 atmospheres (4.053 MPa). The line is where the Joule-Thomson expansion coefficient (K/MPa) is zero. The expansion coefficient is the change in temperature with pressure when a gas is expanded through an ideal insulated nozzle with no work being done. The ideal insulated nozzle means that no heat should enter or exit the nozzle. No work means that the expanding gas does not push against something to move it, for example a balloon.

Inside the curve the coefficient is positive, meaning as the gas expands it cools. Outside the curve the coefficient is negative, which means as the gas expands it heats up. The highest point of the curve is called the inversion temperature. This is the maximum temperature in which the coefficient is positive. For Helium the temperature is about 43 K. We also have the inversion pressure which is the highest temperature where the coefficient is positive. This is at about 26 K and 38 atmospheres (3.85 MPa).

So, above 43 K or 3.85 MPa the coefficient is always negative. The loaded Helium tank pressure is about 38 MPa (way above the inversion pressure) and subcooled LOX is at 66.5 K, also above the inversion temperature. This means that supplying the tanks at this pressure and temperature will have the supercritical Helium expand into the tanks with a negative coefficient, causing the gas to warm and limiting the amount of gas that can be filled. This expansion is not the ideal case, since the nozzle is probably not well insulated allowing heat to either enter or escape and some work being done as the tank slightly expands under pressure.

To overcome this effect, the gas could be supplied at a lower temperature than 66.5 K. The freezing point of LOX at one atmosphere is 54.4 K, also above the Helium inversion temperature. So, even if the Helium was at 54.4 K, it should still heat up when it enters the Helium tanks and not freeze the LOX. If however the helium was below 54.4 K, as the source pressure would be above 3.85 MPa the gas should still heat up as it expands and enters the tank. However, if the starting temperature was very low, so as to reduce the time it takes to fill the tank, it will still heat up but maybe to the point where the expanded temperature was below the freezing point of LOX.

[hans_ober]

http://www.wacotrib.com/news/greater_waco/mcgregor/explosion-at-spacex-in-mcgregor-part-of-accident-investigation/article_a151e524-7787-5874-bc6b-b3062d09f608.html

[Jarnis]

http://www.wacotrib.com/news/greater_waco/mcgregor/explosion-at-spacex-in-mcgregor-part-of-accident-investigation/article_a151e524-7787-5874-bc6b-b3062d09f608.html

Open question; was this a planned / expected result of a test (yay, good news, it blew up!) OR an unexpected result of a test (huh, funny, it shouldn't blow up like that?)

Both would have implications regarding RTF...

[Jet Black]

To overcome this effect, the gas could be supplied at a lower temperature than 66.5 K. The freezing point of LOX at one atmosphere is 54.4 K, also above the Helium inversion temperature. So, even if the Helium was at 54.4 K, it should still heat up when it enters the Helium tanks and not freeze the LOX. If however the helium was below 54.4 K...

do you mean 44.4K? 54.4 is off the top of the graph well away from the curve. Also would you be likely to get any strange behaviours if there was a sharp jump over the line due to a drop/jump in pressure or temperature in the right region?

[spacekid]

http://www.wacotrib.com/news/greater_waco/mcgregor/explosion-at-spacex-in-mcgregor-part-of-accident-investigation/article_a151e524-7787-5874-bc6b-b3062d09f608.html

Open question; was this a planned / expected result of a test (yay, good news, it blew up!) OR an unexpected result of a test (huh, funny, it shouldn't blow up like that?)

Both would have implications regarding RTF...

It sounds like it was planned but I would have expected a public notification beforehand.

[Odysseus]

Here's a paper on the properties of Helium at various temperatures and pressures. An interesting curve is shown Figure 35 on page 31 of the pdf which I've added here. This is what is called the Joule-Thomson inversion curve. If you are like myself who are not familiar with what this curve means, let me explain.

On the vertical axis we have temperature in Kelvin, or degrees above absolute zero (-273.15 C). The horizontal axis is pressure, which goes from 0 to 40 atmospheres (4.053 MPa). The line is where the Joule-Thomson expansion coefficient (K/MPa) is zero. The expansion coefficient is the change in temperature with pressure when a gas is expanded through an ideal insulated nozzle with no work being done. The ideal insulated nozzle means that no heat should enter or exit the nozzle. No work means that the expanding gas does not push against something to move it, for example a balloon.

Inside the curve the coefficient is positive, meaning as the gas expands it cools. Outside the curve the coefficient is negative, which means as the gas expands it heats up. The highest point of the curve is called the inversion temperature. This is the maximum temperature in which the coefficient is positive. For Helium the temperature is about 43 K. We also have the inversion pressure which is the highest temperature where the coefficient is positive. This is at about 26 K and 38 atmospheres (3.85 MPa).

So, above 43 K or 3.85 MPa the coefficient is always negative. The loaded Helium tank pressure is about 38 MPa (way above the inversion pressure) and subcooled LOX is at 66.5 K, also above the inversion temperature. This means that supplying the tanks at this pressure and temperature will have the supercritical Helium expand into the tanks with a negative coefficient, causing the gas to warm and limiting the amount of gas that can be filled. This expansion is not the ideal case, since the nozzle is probably not well insulated allowing heat to either enter or escape and some work being done as the tank slightly expands under pressure.

To overcome this effect, the gas could be supplied at a lower temperature than 66.5 K. The freezing point of LOX at one atmosphere is 54.4 K, also above the Helium inversion temperature. So, even if the Helium was at 54.4 K, it should still heat up when it enters the Helium tanks and not freeze the LOX. If however the helium was below 54.4 K, as the source pressure would be above 3.85 MPa the gas should still heat up as it expands and enters the tank. However, if the starting temperature was very low, so as to reduce the time it takes to fill the tank, it will still heat up but maybe to the point where the expanded temperature was below the freezing point of LOX.

Exactly, therefore my curiosity regarding the He Gas inlet temperature.

[cscott]

do you mean 44.4K? 54.4 is off the top of the graph well away from the curve. Also would you be likely to get any strange behaviours if there was a sharp jump over the line due to a drop/jump in pressure or temperature in the right region?

My theory (expressed earlier in this thread, apologies for repeating) is that SpaceX reasoned exactly as Steven Pietrobon did above, and then were surprised when Nature unexpectedly supplied a transient below the inversion temperature.  As Jet Black hints, once you drop below the inversion temperature you get thermal runaway causing temperatures to drop still further.  Possibly leading to liquid He and almost certainly to solid LOX.

I don't know what surprise Nature had in store to generate the initial low-temperature transient.  Could be a rare natural effect, like the resonance-driven thermoacoustic heat engine mooted at the start of the thread; could be a rare human failure, like an inexperienced operator driving the system too hard; or could be a rare mechanical/electrical failure, like the chilling system working a bit better than expected (maybe someone left an access panel open or wrapped another layer of insulation around a pipe), a sensor failing to provide proper feedback to the chiller, or a failed counter-balancing or pressurization heater.  What we've heard from SpaceX so far seems to indicate "rare natural effect"... but they could be referring to the thermal runaway/helium inversion-point mechanism which was the result, not to the root cause which tipped the system into that state.  We've also heard "operational error" which could be referring to that root cause.

[Jet Black]

I don't know what surprise Nature had in store to generate the initial low-temperature transient.

I was wondering along the lines of something as simple as a bubble which could seed it (much like you seed freezing in a supercooled liquid)

[Herbie]

do you mean 44.4K? 54.4 is off the top of the graph well away from the curve. Also would you be likely to get any strange behaviours if there was a sharp jump over the line due to a drop/jump in pressure or temperature in the right region?

My theory (expressed earlier in this thread, apologies for repeating) is that SpaceX reasoned exactly as Steven Pietrobon did above, and then were surprised when Nature unexpectedly supplied a transient below the inversion temperature.  As Jet Black hints, once you drop below the inversion temperature you get thermal runaway causing temperatures to drop still further.  Possibly leading to liquid He and almost certainly to solid LOX.

I don't know what surprise Nature had in store to generate the initial low-temperature transient.  Could be a rare natural effect, like the resonance-driven thermoacoustic heat engine mooted at the start of the thread; could be a rare human failure, like an inexperienced operator driving the system too hard; or could be a rare mechanical/electrical failure, like the chilling system working a bit better than expected (maybe someone left an access panel open or wrapped another layer of insulation around a pipe), a sensor failing to provide proper feedback to the chiller, or a failed counter-balancing or pressurization heater.  What we've heard from SpaceX so far seems to indicate "rare natural effect"... but they could be referring to the thermal runaway/helium inversion-point mechanism which was the result, not to the root cause which tipped the system into that state.  We've also heard "operational error" which could be referring to that root cause.

I do not think it requires a particularly rare natural effect, just one outside SpaceX experience. Most of the filling of the COPV must take place well away from the temperature/pressure regime shown (final pressure quoted above is ~40 MPa) . In other words, the He is nearly always warming as it expands.

But the flip side of this may be that He cools when compressed (otherwise you might be able to design a perpetual motion machine with He as the working fluid). The in-flowing He may warm as it expands into the COPV, but the He already present in the COPV should cool as it is compressed. This compression-cooling (if it exists) may be significant enough to drop the temperature below the freezing point of the LOX.

We are missing quite a few important pieces of the puzzle. In particular, the mass, heat capacity, and other thermal properties of the metal making up the COPV liner, the pressure and temperature of the He in the feed line before the He enters the COPV, the pressure differential between the feed line and the COPV, the rate of fill of the COPV, the heat-capacity and heat flux through the overwrap, the He pressure and temperature in the COPV when it initially comes into contact with the LOX, the time it takes to fully immerse it in LOX, the path of the He feed line within the LOX tank, how the feed-line pressure changes over time, etc.

[rsdavis9]

Also I don't see any thermal runaway in any of these helium phenomenon. The linde liquefaction system is one type of "thermal runaway" Basically compress gas greatly. Allow to expand thru small hole. Make sure the incoming hot pipe exchanges heat with the outgoing cold pipe. Voila the area after the small orifice gets colder and colder until liquefaction occurs. I don't believe we have a system/interaction like this where high pressure exchanges heat with expanded cold pipe.

Not only that but the linde system only works with positive joule-thomson. That's why helium must be below the 30K(inside the graph) for linde to work.

[mn]

Other theories?  Does the paperwork to finish up an investigation simply take a long time to collect all the required signatures?  Could the failure mechanism be so ITAR-restricted that we'll never get a formal report, SpaceX will just launch at some point?

Here's my theory:  A vast amount of data, analysis and findings from several directions (including the 'we shot it and it blew up' one) have to be compiled into a report format that is both (a) meaningful and (b) understandable by those signing-off on it.  Given schedule pressure, so long as whichever parties are involved in the next launch are happy with the summary findings 'in principle', they might be quite okay for launch preparations to press ahead, with the detailed Report due out at some later date..

I suspect that the people signing off on the report are deeply involved in the investigation and could sign it without reading it.

The report is for others.

[mn]

http://www.wacotrib.com/news/greater_waco/mcgregor/explosion-at-spacex-in-mcgregor-part-of-accident-investigation/article_a151e524-7787-5874-bc6b-b3062d09f608.html

Open question; was this a planned / expected result of a test (yay, good news, it blew up!) OR an unexpected result of a test (huh, funny, it shouldn't blow up like that?)

Both would have implications regarding RTF...

It sounds like it was planned but I would have expected a public notification beforehand.

That's the part that has be wondering: it seems more like they were NOT expecting a kaboom (say they were testing a new improved loading procedure which is not supposed to fail).

We can't know much based on the little spacex says, but if you like to worry, this is good material.

[Stranger]

What again? 

[FinalFrontier]

Blew up another tank during WDR testing. More to follow hopefully they release some sort of statement about these test/failures/replicated incidents but I would not expect much until just prior to RTF.