SpaceX Falcon 9 - AMOS-6 - (Pad Failure) - DISCUSSION THREAD (2)

[jaufgang]

At any rate, if they were indeed experimenting with untested variations of the fuelling procedure on the launch pad with the payload on top, that will undoubtedly be pointed out as a serious lapse in the failure investigation report  It will be interesting to see if that is the case.

It is equally possible that the approved and tested method has an unexpected and extremely low probability failure mode.

Well yeah, that is more or less what I was saying (less concisely) in my first paragraph above.

[Robotbeat]

So you change a timing/procedure on such complex matter, while preparing for a launch, keeping the payload aboard ?
And NOW they are learning a lot ? And people find lots of previous knowledge about this being at risk?
This is not unfortunate, or a business process failure.
SpaceX often talks about Information Technology and its way of doing.
When your software drives something complex and critical you never ever do something out of what you tested. You never do something first-time in production environment, expecially if you have people on an airplane under control, or a patient under a medical device, or a payload of somebody else.
Something very wrong happened, it sounds.
And it's worse than a bad COPV.
I feel a little disillusioned.
Please SpaceX keep going, but you did a big mess.

presumptuous
(of a person or their behavior) failing to observe the limits of what is permitted or appropriate

IMHO, the only presumptuous thing was putting the dang payload on the rocket.

I get what Jim is saying about the lack of a need for static fires, but I really do think it can help catch problems like this without destroying the payload. Without destroying the payload because you're not supposed to put the payload on the vehicle during static fires!. The static fire test, then, wouldn't be operational but would be a kind of last-minute test.

Gah, I just wish SpaceX hadn't put the payload on.

[Robotbeat]

I wonder, do they pre-cool the helium first to very cold temperatures (i.e. below freezing point of oxygen) in order to allow them to fill much faster?

I hope they find ways to belt-and-suspenders this kind of failure mode, given there's STILL a possibility it's related to past failures. Keeping oxygen from getting into the matrix, etc.

[CameronD]

I get what Jim is saying about the lack of a need for static fires, but I really do think it can help catch problems like this without destroying the payload. Without destroying the payload because you're not supposed to put the payload on the vehicle during static fires!. The static fire test, then, wouldn't be operational but would be a kind of last-minute test.

Gah, I just wish SpaceX hadn't put the payload on.

..but if they don't put the payload on, isn't that kinda missing the whole point of the Static Fire?

Tell me I'm wrong but AIUI, the idea of static fire is to ensure all the pad stuff works with all of the rocket stuff and the engines and tankage and fuelling was all supposed to be sorted out long before the rocket got to the Cape.  Meaning that if their procedures were all sorted, this particular incident should have occurred back at the proving ground, not on the pad.

Assuming Stage 2 was tested and certified okay long before mating to the rocket, is it possible something to do with the COPV overwrap came adrift during transport to the Cape?

[CyndyC]

Do the COPVs heat up as they're filled?  Can anyone please confirm that?

So heres a question I got asked, and maybe someone knows or can come up with a good answer.   

So we know that SpaceX was just in the process of but not finished with the O2 loading phase of the second stage, and the tanks were not pressurized for firing.  This per their statements.  That being the case, why was there such a high pressure of helium being supplied at this time?   I

To load the COPV's.  As the temperature drops in the COPV's, more He can be loaded.

[Blackstar]

National Reconnaissance Office?

The thread referred to a recording. Since when does the NRO allow recordings of its events?

[gongora]

National Reconnaissance Office?

The thread referred to a recording. Since when does the NRO allow recordings of its events?

The since redacted Reddit thread had a comment that said it was a non-classified recording available internally to their employees.

[woods170]

Actually, that is very likely not a design flaw. It's practically impossible to prevent oxygen molecules from penetrating the overwrap, given a COPV submerged in LOX. This is something Air Liquide found out in the 1970's. It's the very reason why Ariane 5 has it's COPV's outside the tanks.
...

Hence, a design flaw (?!)

Based on the results from testing by Air Liquide (and others) ESA decided not to venture into the (then) unknown and go for tried-and-tested. As in: COPV's on the outside of the launcher. That was in the 1980's.
SpaceX, two decades later, decided to submerge the COPV's in LOX, which, at that time, was no longer an unknown. But that was "regular" LOX and with the right precautions and procedures it can be safely done.

This basic set-up was not changed when SpaceX went to sub-cooled LOX and, given from what I hear from industry experts, a change is not needed as long as certain aspects and properties of the sub-cooled LOX are kept in mind.
So, was there a design-flaw in the rocket? My hunch, based on industry-input, is no. Was there a flaw in handling the properties of sub-cooled LOX? My hunch, again based on industry-input, is yes.

Going into speculation from this point forward but I have a feeling that on Amos-6 SpaceX has discovered an unexpected result of using sub-cooled LOX.

Isn't that a flaw in the design?

No, IMO it isn't. Remember the first flight of Delta IV Heavy? Both the boosters and the central core stage shut-down early from a poorly understood cavitation effect in LOX feedlines. It wasn't a design flaw, but a lack of insight and understanding, despite the fact that LOX had been used on launch vehicles for 3+ decades by then. Designs of vehicles, particularly launch vehicles, are optimized to match the known properties of everything related to launching into space. The key word here is "known". It is actually pretty hard to design against unknowns. SpaceX might just as well have been reminded of this, in a rather spectacular way.

[JamesH65]

Actually, that is very likely not a design flaw. It's practically impossible to prevent oxygen molecules from penetrating the overwrap, given a COPV submerged in LOX. This is something Air Liquide found out in the 1970's. It's the very reason why Ariane 5 has it's COPV's outside the tanks.
...

Hence, a design flaw (?!)

Based on the results from testing by Air Liquide (and others) ESA decided not to venture into the (then) unknown and go for tried-and-tested. As in: COPV's on the outside of the launcher. That was in the 1980's.
SpaceX, two decades later, decided to submerge the COPV's in LOX, which, at that time, was no longer an unknown. But that was "regular" LOX and with the right precautions and procedures it can be safely done.

This basic set-up was not changed when SpaceX went to sub-cooled LOX and, given from what I hear from industry experts, a change is not needed as long as certain aspects and properties of the sub-cooled LOX are kept in mind.
So, was there a design-flaw in the rocket? My hunch, based on industry-input, is no. Was there a flaw in handling the properties of sub-cooled LOX? My hunch, again based on industry-input, is yes.

Going into speculation from this point forward but I have a feeling that on Amos-6 SpaceX has discovered an unexpected result of using sub-cooled LOX.

Isn't that a flaw in the design?

No, IMO it isn't. Remember the first flight of Delta IV Heavy? Both the boosters and the central core stage shut-down early from a poorly understood cavitation effect in LOX feedlines. It wasn't a design flaw, but a lack of insight, despite the fact that LOX had been used on launch vehicles for 3+ decades by then. Designs of vehicles, particularly launch vehicles, are optimized to match the known properties of everything related to launching into space. The key word here is "known". It is actually pretty hard to design against unknowns. SpaceX might just as well have been reminded of this, in a rather spectacular way.

The world is full of unexpected, even in areas that are presumed fully explained. The delay to Starliner because of unexpected aerodynamic effects for example - and yet aerodynamics has been known about for decades.....They designed it, now they are having to have another go. It's simply part of the development cycle.

[jgoldader]

I'm struggling to get my mind around oxygen ice crystals=explosion.  Could somebody who understands cryos/etc. correct me on these two ideas?

1. Would building up ice crystals result in effective local delamination of the overwrap from the metal?  I could then imagine the metal pressure vessel failing as loading continued.

2. Otherwise, I'm left with some sort of peizoelectric thing going on with oxygen ice crystals in the presence of carbon-rich fuel: Spark+fuel+oxygen.

Thanks for helping with my ignorance.

[guckyfan]

My understanding. LOX ice crystals form, then the COPV expands when filled to capacity. That exerts pressure on ice crystals in intimate contact with the carbon fiber. LOX could escape the way it came in, ice crystals can not.

Oxygen plus carbon plus pressure = boom.

[Silmfeanor]

How sure are we that this actually has been said by Elon? How credible is the source?
Sorry if this already has been established. If so feel free to remove this post.

The marksman comment in the thread came from a known SpaceX employee.

Also, there's photos of the shredded remains of the mock-up out there.

Where could one find these photos? Or is it L2?

[rsdavis9]

L2 mcgregor shows the shredded copv.

[AndyX]

How sure are we that this actually has been said by Elon? How credible is the source?
Sorry if this already has been established. If so feel free to remove this post.

The marksman comment in the thread came from a known SpaceX employee.

Also, there's photos of the shredded remains of the mock-up out there.

Where could one find these photos? Or is it L2?

L2 McGregor, one of my favorite sections of all time, with other rigging and testing found in the panoramic super hi res photos. L2 link: https://forum.nasaspaceflight.com/index.php?topic=34957.940

Farewell, Mock Up and we thank you.

[LouScheffer]

quoted from  https://www.reddit.com/r/spacex/comments/57balr/details_from_elons_speech_at_the_nro/

Details from Elon's speech at the NRO (self.spacex)
The other thing we discovered is that we can exactly replicate what happened on the launch pad if someone shoots the rocket.

Speculation:  Shooting the rocket is one branch of the fault tree.  I suspect they wanted to rule it out by showing it would result in different signatures than observed.  This would be very hard to do analytically, so they tried it in Texas.  To their surprise, it gave exactly the same signatures as what they saw.  (Again, this is speculation...)

[spacekid]

So, the business decision to compress time from rollout to launch causes a more aggressive prop loading sequence which breaks the temperature regime acceptable for that construction of the COPV. Specifically, causing a temperature drop inside the COPV that cooled the outer shell of the vessel to below the freezing temperature of Oxygen, then stressing it (pressurising the COPV toward its 6000psi working pressure). The LOX, that is normally allowed to permeate the carbon overwrap until it is displaced out of the fiber by the expanding of the COPV, froze between the fibers and the crystals cut the fibers, since the fibers are strong in tension, but weak in shear which the crystals applied.

Am I parsing this correctly?


Edit. ... yes the combustion part. Ignited instead of shear failed?

I'm wondering if the solid Oxygen cut the fibers, then the COPV was weakened causing it to burst piercing the fuel tank and setting off the explosion, not the carbon fibers burning.

[nicp]

Does oxygen expand when it freezes? Most substances do not. Water is unusual.

[jgoldader]

Oxygen plus carbon plus pressure = boom.

But this would just be a spontaneous thing?  It's sitting there, then boom?

[spacekid]

You never do something first-time in production environment, expecially if you have people on an airplane under control, or a patient under a medical device, or a payload of somebody else.

First time changes are made all the time to production lines, it's just when it goes wrong that you hear about it.

For products that don't cost much, if you detect it in time, no problem. If it gets out, then you can have a major recall (maybe Galaxy Note 7?).

To your point, for expensive and life critical applications, you need to limit your exposure or risk. For SpaceX, it would be to do it in McGregor or without a payload on top.

[spacekid]

Speculation:  Shooting the rocket is one branch of the fault tree.  I suspect they wanted to rule it out by showing it would result in different signatures than observed.  This would be very hard to do analytically, so they tried it in Texas.  To their surprise, it gave exactly the same signatures as what they saw.  (Again, this is speculation...)

Could be that the shooting exercise could provide information on how rockets can fail. If just a bullet piercing the LOX tank could cause it, it seems that a piece of COPV exiting the tank would be similar. Depends if the bullet hit the fuel tank too.