SpaceX Falcon 9 - AMOS-6 - (Pad Failure) - DISCUSSION THREAD (2)

[Fred Bonyea]

In retrospect a really great place to have a temperature transducer would be inside the over-wrap of the helium bottle. Are helium COPVs ever manufactured with integrated temperature sensors?

Yes - almost.

We were winding bottles with embedded strain gauges in the early 2000s. Strain gauges are extremely sensitive to changes in temperature, so we were also developing software to take advantage of this trait to monitor temperature differentials as well as strain. We even developed and patented temperature profiling techniques that verified embedded sensor integrity.

The sensors were generally wound into the cases and the leads were gathered at one end, so the whole assembly looked something like model of an eye, with a band of nerves coming out of one end.

We were just starting to develop in-situ temperature and pressure profiles that the aerospace industry was showing interest in, and writing proposals to integrate these technologies into flight composites when funding for this type of research dried-up. Our composite development operations and our NDT development lab were both  dismantled - about the same time we started testing COPVs used by SpaceX. So we didn't do any instrumented testing with helium filled COPVs.

What is amazing how wildly differentiated the temperature measurements could be near the inside surface of a bottle. The temperature just inside the valve - where gases were expanding, could be -40 deg C, and the temperature at far end, were residual gas is compressed, could be over 200 C. (This is for a pressure vessel about two meters long and 30 cm wide.  Temperature differentials in small lines can be much more dramatic.)

[Comga]

In retrospect a really great place to have a temperature transducer would be inside the over-wrap of the helium bottle. Are helium COPVs ever manufactured with integrated temperature sensors?

Yes - almost.

We were winding bottles with embedded strain gauges in the early 2000s. Strain gauges are extremely sensitive to changes in temperature, so we were also developing software to take advantage of this trait to monitor temperature differentials as well as strain. We even developed and patented temperature profiling techniques that verified embedded sensor integrity.

The sensors were generally wound into the cases and the leads were gathered at one end, so the whole assembly looked something like model of an eye, with a band of nerves coming out of one end.

We were just starting to develop in-situ temperature and pressure profiles that the aerospace industry was showing interest in, and writing proposals to integrate these technologies into flight composites when funding for this type of research dried-up. Our composite development operations and our NDT development lab were both  dismantled - about the same time we started testing COPVs used by SpaceX. So we didn't do any instrumented testing with helium filled COPVs.

What is amazing how wildly differentiated the temperature measurements could be near the inside surface of a bottle. The temperature just inside the valve - where gases were expanding, could be -40 deg C, and the temperature at far end, were residual gas is compressed, could be over 200 C. (This is for a pressure vessel about two meters long and 30 cm wide.  Temperature differentials in small lines can be much more dramatic.)

Who are you referring to as "we"?
(Don't need the name if you don't wish to divulge it. Just the nature of the group would be interesting; established corporation, government lab, industrial supplier, university, start-up, etc.)
What gasses in what regimes were you working?

[Fred Bonyea]

...We were just starting to develop in-situ temperature and pressure profiles that the aerospace industry was showing interest in, and writing proposals to integrate these technologies into flight composites when funding for this type of research dried-up. Our composite development operations and our NDT development lab were both  dismantled - about the same time we started testing COPVs used by SpaceX. So we didn't do any instrumented testing with helium filled COPVs.

What is amazing how wildly differentiated the temperature measurements could be near the inside surface of a bottle. The temperature just inside the valve - where gases were expanding, could be -40 deg C, and the temperature at far end, were residual gas is compressed, could be over 200 C. (This is for a pressure vessel about two meters long and 30 cm wide.  Temperature differentials in small lines can be much more dramatic.)

Who are you referring to as "we"?
(Don't need the name if you don't wish to divulge it. Just the nature of the group would be interesting; established corporation, government lab, industrial supplier, university, start-up, etc.)
What gasses in what regimes were you working?

'We' was ATK - (which I should state because it can infer bias). I am retired.

With the composite in-situ testing, we were looking at feasibility with filament wound rocket motors. We also proposed using this technology over a much broader spectrum of composite system.

We measured high temperature differentials while filling and venting N2 gas  in a variety of bombs used in burn rate and stress/strain testing - usually in N2; baseline temperature regimes from -40 deg C to about 50 deg C.
In the burn rate testing, the plumbing is always dirty - sooty hydrocarbons - so you must be careful in the initial pressurizations because the gas hammer from incoming N2 can compress ambient O2 at the ends of the line.

I have also witnessed ARC (Accelerating Rate Calorimeter) bomb testing of explosives, oxidizers and plastics contaminated with oxidizers. When oxidizers are compressed, the rate of propagation of reactions accelerates exponentially.

It sounds like a micro version of this hammering might be occur in composite voids during LOX and H2 filling. The void geometry would be critical, and this could be very difficult to reproduce; especially so if frozen O2 is necessary to recreate the failure mode.

[Odysseus]

I tried to dig out how the Helium is provided to LC40. Is it still correct that it is provided on railcars, at a pressure of about 40 MPa and "ambient" temperature?
Cheers

No, it is stored in former rail cars.  It comes to the complex via pipeline.

I think with this information one can discard that the COPV sees "real" liquid He in the COPV.
The question is which minimum temperature the He reaches in the COPV. It depends on whether the He gas delivered from the pipeline is fed to a heat exchanger for cooling it down, and whether there is exchange of heat with the environment.
Applying hopefully not too simple thermodynamics, the He gas first expands at the delivery valve from the supply pressure (which means its temperature increases), and is next compressed  by the gas stream into the COPV  (which, means it gets cooler). The supply pressure should be about 40 MPa, the initial COPV pressure should be about 0,4 MPa (a little bit higher than in the LOX tank), and the final COPV pressure is about 30 MPa. Running the fill adiabatic and without external heat exchanger, the gas in the COPV should at the end have a higher temperature compared to the outlet of the pipeline valve. If there is a HX, the final temperature could be close to the HX outlet temperature, dependent on the pressure conditions.

[rsdavis9]

Sounds about right. I would guess they might have a heat exchanger which cools it to LN2 temp? Before it enters the copv.

[francesco nicoli]

Sounds about right. I would guess they might have a heat exchanger which cools it to LN2 temp? Before it enters the copv.

Ask REL, I'm sure they would be glad to provide their wonderful HEs to SpaceX

[cscott]

What is amazing how wildly differentiated the temperature measurements could be near the inside surface of a bottle. The temperature just inside the valve - where gases were expanding, could be -40 deg C, and the temperature at far end, were residual gas is compressed, could be over 200 C. (This is for a pressure vessel about two meters long and 30 cm wide.  Temperature differentials in small lines can be much more dramatic.)

This is the most interesting fact from my perspective.  Because of the Joule-Thompson transition at 50K, you could get both the "expansion" and "compression" sides of the COPV to pull heat the same way if the temperature gradient is correct.

At the "compression" end, it seems like the steady state temperature would be the inversion point, 50K.  If the temperature were above that point, compression would cool it; if the temperature fell below 50K then compression would warm it.[*]

But on the inlet side things are unstable: if the temperature starts above 50K, then expansion would warm it further.  Presumably this is the initial condition of the tank, and so some mechanism is in place to limit the temperature rise (such as immersing everything in LOX).

But if the temperature at the inlet is allowed to drop below 50K (and remember the "compression side" may have a feedback loop actively pushing it to exactly 50K), then further expansion will cause the temperature to drop still further, potentially all the way to liquidation (as Elon perhaps hinted).  That might have been unexpected (what's the initial source of the extra cooling?) and the thermal runaway might have been quite quick.

[*] The Joule-Thompson effect is valid only for expansion processes.  The discussion here has assumed that compression would have the opposite effect.  Is that actually so?  If not, then disregard talk about the behavior of the "compression end" of the tank---but the thermal runaway possible on the "expansion end" is still valid.

[Jcc]

I'm pretty sure compression always results in heating, while expansion may produce heating or cooling depending on the gas and the temperature and pressure.

[starhawk92]

I guess this is the right place for this question . . . .

I assume (at great personal risk, I know) this "solidification of LOX" issue has been a behavior of the system (and only a big problem under specific loading conditions).  Could another manifestation of this phenomena (du, due, de-du-du-du) this explain the occasional "sputtering" we've seen from the Merlins?  I believe at least one launch was halted due to an engine having this problem.

[rower2000]

Interesting allotropes of solid Oxygen:
(quote=Wikipedia)
The metastable molecule tetraoxygen (O
4) was discovered in 2001,[38][39] and was assumed to exist in one of the six phases of solid oxygen. It was proven in 2006 that this phase, created by pressurizing O
2 to 20 GPa, is in fact a rhombohedral O
8 cluster.[40] This cluster has the potential to be a much more powerful oxidizer than either O
2 or O
3 and may therefore be used in rocket fuel.[38][39] A metallic phase was discovered in 1990 when solid oxygen is subjected to a pressure of above 96 GPa[41] and it was shown in 1998 that at very low temperatures, this phase becomes superconducting.[42](/quote)
Can you increase the reactivity of O2 by freezing it? Superconductivity can throw your heat transfer models out-of-the window.

It appears that you are interpreting superconductivity as thermal superconductivity. I believe the context is more likely electrical superconductivity.

As a side remark, (electrical) superconductivity does have thermal consequences. When a potentially superconducting material is cooled below its phase transition temperature, its heat capacity instantly increases by a significant factor. That being said, I cannot imagine how this should have played a role here.

[Fred Bonyea]

Interesting allotropes of solid Oxygen:
(quote=Wikipedia)
The metastable molecule tetraoxygen (O
4) was discovered in 2001,[38][39] and was assumed to exist in one of the six phases of solid oxygen. It was proven in 2006 that this phase, created by pressurizing O
2 to 20 GPa, is in fact a rhombohedral O
8 cluster.[40] This cluster has the potential to be a much more powerful oxidizer than either O
2 or O
3 and may therefore be used in rocket fuel.[38][39] A metallic phase was discovered in 1990 when solid oxygen is subjected to a pressure of above 96 GPa[41] and it was shown in 1998 that at very low temperatures, this phase becomes superconducting.[42](/quote)
Can you increase the reactivity of O2 by freezing it? Superconductivity can throw your heat transfer models out-of-the window.

It appears that you are interpreting superconductivity as thermal superconductivity. I believe the context is more likely electrical superconductivity.

As a side remark, (electrical) superconductivity does have thermal consequences. When a potentially superconducting material is cooled below its phase transition temperature, its heat capacity instantly increases by a significant factor. That being said, I cannot imagine how this should have played a role here.

It would be useful if the source of ignition could be traced to an exotic condition that exists only when the fueling dips into a unique pressure/temperature phase sequence: If you modify the fueling routine, the problem goes away.

Returning to Unique Sources of Ignition I have encountered:

Another unanticipated problem occurred when a layer of urethane sealant was painted on a cement floor in a building where the floor was contaminated with powdered oxidizer. (oxidizer impregnated urethane is a very good rocket fuel.)  When forklifts started driving around on the newly sealed floor, the tyres of the forklift snap-crackled and popped like the floor was covered with ammonium tri-iodide.

This explosive situation was hard to characterize. The tyres were putting minimal loads on the urethane and oxidizer, and we could not duplicate the ignition in the laboratory using pressure gradients alone. We eventually concluded that the Urethane is such a good insulator, that a very high electrostatic potential developed, and static discharge ignited the urethane/oxidizer mixture.  (There may have been a frictional requirement too, so we were never certain of the ignition mechanism.)

In the SpaceX COPV, a carbon fiber-wound container has a powerful oxidizer soaking into its every pore. You also have swirling helium gas inside this case. If the case is insulative, there is an electrostatic potential between the LOX (which is also flowing) and the helium. Jim stated that there is no potential for an ESD event, but I see potential problem with an O2-soaked bottle  unless the resin used to coat the case windings is conductive. (Carbon fibers are naturally conductive, unless they are wetted with an nonconductive resin.)

Edited to add subject transition

[speedevil]

I assume (at great personal risk, I know) this "solidification of LOX" issue has been a behavior of the system (and only a big problem under specific loading conditions).  Could another manifestation of this phenomena (du, due, de-du-du-du) this explain the occasional "sputtering" we've seen from the Merlins?  I believe at least one launch was halted due to an engine having this problem.

This is only a problem at fill-time.
Once the tanks are filled, there is no question (even with subcooling) of there being any solid oxygen around.

[Fred Bonyea]

...
As a side remark, (electrical) superconductivity does have thermal consequences. When a potentially superconducting material is cooled below its phase transition temperature, its heat capacity instantly increases by a significant factor. That being said, I cannot imagine how this should have played a role here.

This could be critical if the LOX is entering a supercritical phase: I know the temperatures are very low, but if the caloric content of the O2 COPV is elevated while it is within the case walls, increasing the temperature of the system will release this latent heat, and if the (solid/liquid/gas) is trapped in a void volume, there will be thermal runaway.

Are the surface absorption properties of the COPV characterized, and do they vary from bottle to bottle? A COPV with a high void volume would be much more susceptible to gas permutation than a denser structure.

[spacekid]

In the SpaceX COPV, a carbon fiber-wound container has a powerful oxidizer soaking into its every pore. You also have swirling helium gas inside this case. If the case is insulative, there is an electrostatic potential between the LOX (which is also flowing) and the helium. Jim stated that there is no potential for an ESD event, but I see potential problem with an O2-soaked bottle  unless the resin used to coat the case windings is conductive. (Carbon fibers are naturally conductive, unless they are wetted with an nonconductive resin.)

I don't see where an ESD would come from unless the helium and LOX are static generators. The COPV metal container itself would be grounded to the tank. I would expect the helium, in constant contact with it's tank to be at the same potential as with the LOX in constant contact with it's tank and the carbon fibers.

[Odysseus]

In all the discussion there is an (hidden) unknown, which is the temperature of the He Gas entering the 2nd stage, or better the COPVīs. From our information exchange here we know only the He distribution to the pad.
Secondly, is there any thought why the use of supercooled LOX compromises the COPV in the 2nd stage, but not in the 1st?

[Comga]

In all the discussion there is an (hidden) unknown, which is the temperature of the He Gas entering the 2nd stage, or better the COPVīs. From our information exchange here we know only the He distribution to the pad.
Secondly, is there any thought why the use of supercooled LOX compromises the COPV in the 2nd stage, but not in the 1st?

Asked and answered above.
If the compressing helium drops below the solidification temperature of oxygen, causing the hypothesized failure mechanism, then it should be possible to raise its supply temperature so that it achieves a higher final temperature, closer to that of the sub-cooled oxygen and safely above the freezing temperature. 
(As said: if, should, safely, ... The devil is in the details.)

We don't know if the first stage COPVs are compromised or not.  All we know is that they have not failed in the eight or nine launch campaigns to date with sub-cooled oxygen.   It is undoubtedly under consideration by SpaceX.

[Fred Bonyea]

In the SpaceX COPV, a carbon fiber-wound container has a powerful oxidizer soaking into its every pore. You also have swirling helium gas inside this case. If the case is insulative, there is an electrostatic potential between the LOX (which is also flowing) and the helium. Jim stated that there is no potential for an ESD event, but I see potential problem with an O2-soaked bottle  unless the resin used to coat the case windings is conductive. (Carbon fibers are naturally conductive, unless they are wetted with an nonconductive resin.)

I don't see where an ESD would come from unless the helium and LOX are static generators. The COPV metal container itself would be grounded to the tank. I would expect the helium, in constant contact with it's tank to be at the same potential as with the LOX in constant contact with it's tank and the carbon fibers.

I don't see where an ESD would come from unless the helium and LOX are static generators. The COPV metal container itself would be grounded to the tank. I would expect the helium, in constant contact with it's tank to be at the same potential as with the LOX in constant contact with it's tank and the carbon fibers.

The composite may not be conductive, especially if the resin is urethane - a ridiculously good insulator.

If the case is conductive (as it should be), electrostatic discharge, as Jim stated, can be ruled out with high confidence. But if it is not, you not only have two flowing gases, but you also have a capacitor: kilometers of wound conductive fibers isolated by a thin insulator.

When incidents have been blamed on ESD in the past, I have joked that when someone is lying about what really happened, ESD gets the blame. But it happens.

[Odysseus]

In all the discussion there is an (hidden) unknown, which is the temperature of the He Gas entering the 2nd stage, or better the COPVīs. From our information exchange here we know only the He distribution to the pad.
Secondly, is there any thought why the use of supercooled LOX compromises the COPV in the 2nd stage, but not in the 1st?

Asked and answered above.
If the compressing helium drops below the solidification temperature of oxygen, causing the hypothesized failure mechanism, then it should be possible to raise its supply temperature so that it achieves a higher final temperature, closer to that of the sub-cooled oxygen and safely above the freezing temperature. 
(As said: if, should, safely, ... The devil is in the details.)
....

Understood, but one should not consider the compression alone. There is definitely expansion of the He gas coming from the He distribution system, which is at higher pressure than the COPF final pressure. This means in the end more expansion than compression. Knowing this, it is necessary to know also if the expanding He gas is cooled prior to entering into the 2nd stage, and if yes, to which temperature. Otherwise the logic of solidifying LOX might be wrong.

[Chris Bergin]

We must be getting close to an investigation round up report. Potentially about a month until "RTF" <---I guess we can call it that, although I hesitate as it wasn't a flight failure.

[Archibald]

I'd call it RTP - Return To Pad.