SpaceX Falcon 9 - AMOS-6 - (Pad Failure) - DISCUSSION THREAD (2)

[mulp]

If anyone here understands legal/congress speak could they be so kind to translate what on earth that letter from the 25 congressmen/congresswomen is all about?

 To me it all seemed like empty-speak.

It struck me as "keep up the good work, we've got your back." This letter is undoubtedly in response to the one a few days ago from Rep. Mike Crossman (R-ULA):

http://www.businessinsider.com/spacex-accident-coffman-congressional-letter-2016-9

It asks for increased scrutiny of SpaceX's investigation practices, given its plans with NASA to launch astronauts to the International Space Station. It also lobs pointed questions at the Federal Aviation Administration (FAA), NASA, and the US Air Force (USAF) about the certification process of SpaceX hardware, pricing schemes, risk assessment, and more.

Shall we use NASA as the benchmark?  The shuttle... Apollo...

The shuttle flew with known safety risks multiple times, up until the final flights. Yet some in Congress seem to blame Obama for ending the shuttle program in favor of commercial crew which SpaceX is most likely to win prime US supplier.

The shuttle had known safety issues by design as cost cutting solutions to Congress not wanting to pay more. SpaceX is knocking out the traditional Congress favored corporate cronies by coming in much lower in cost/price.

The unsettling thing at this point is I'm guessing this last failure will not be clearly identified as to cause, so after review, possible flaws in fabrication will be listed as possibles, so more rigorous testing and inspection, possibly beefing up joints, etc., and then increased monitoring, and time to return to flight hoping for signals of a problem without a total failure. If nothing bad happens in a dozen flights it will be the possible landmine that gets triggered in the future when it's forgotten by most.

Not every failure can be traced to its root cause.

And unlike, for example, the shuttle where people knew where to start looking immediately, I don't think anyone was sitting around thinking "is this part of the design really safe enough?"

And we have had in recent memory train accidents and death where more money should have been spent sooner on known safety problems like weak tank cars, poorly maintained track and equipment, and the need for year 2000 train control systems in year 2015. Congress is biased to risking life and safety to cut spending.

[Jim]

"SpaceX is knocking out the traditional Congress favored corporate cronies by coming in much lower in cost/price."

But it looks like there is additional risk

[cscott]

Possibly.  Proton has been successful with the lower cost/higher risk trade-off.  Perhaps NASA/Air Firce won't want their flagship missions on Falcon... or else they'll slowly come around to faster/cheaper/smaller.  Flagship missions aren't SpaceX's bread and butter, though.  The comm sats are.

[cscott]

Back to the helium system. An exploding COPV can ignite itself in the LOx tank, or rupture the common bulkhead and ignite the RP-1, or both. What else in the helium system may cause an explosion in the second stage near the common bulkhead without obvious signs in the telemetry? How is helium piped into the tank? Do the hoses go through ports on the tank wall? What would happen if the hose tore into the seal on a port? Would there be a slow or fast leak?

Jim outlined a long list a few pages back.

[baldusi]

From today interview to Mrs Shotwell:

On Sept. 1 it wasn’t clear whether the cause lay in the ground support equipment, or inside the rocket. You have made the determination that it was inside the rocket and not some procedure during preparation for the static test?

We believe that the composite over wrapped pressure vessel [the helium bottle], known as a COPv, let go in the tank. What caused it, the exact reason it let go, we’re still investigating. I don’t believe it was a ground system cause, but we’re still looking at the data.

Was this public knowledge before?

[Dante80]

From today interview to Mrs Shotwell:

On Sept. 1 it wasn’t clear whether the cause lay in the ground support equipment, or inside the rocket. You have made the determination that it was inside the rocket and not some procedure during preparation for the static test?

We believe that the composite over wrapped pressure vessel [the helium bottle], known as a COPv, let go in the tank. What caused it, the exact reason it let go, we’re still investigating. I don’t believe it was a ground system cause, but we’re still looking at the data.

Was this public knowledge before?

No. The last feedback we had was more generic (a failure in the helium system).

[jbbo78]

Also from that interview:

Q?:What’s the possibility that there’s a design issue with that helium bottle?

Gwynne: I don’t think it’s a design issue with the bottle. I think it probably is more focused on the operations, which is one of the reasons we believe we can get back to flight so quickly.

But we have to finish the investigation. We’re not going to fly until we’re ready to fly.

Q?: When you say it’s more focused on operations, you mean filling of the helium tank, or the filling of the LOX tank, or what?

Gwynne: All of it. We’re going to look at all of it.

"Operations" here means some procedural thing, like fill rate or timing?  Sounds like it's not one of the typical COPV failure modes that have been brought up many times.

Another interesting nugget - they weren't watching the LOX tank camera for this one.

[guckyfan]

Incredibly detailed info. More in the update thread.

[HMXHMX]

So perhaps now we can drop all the other speculation and focus on the "why did the bottle fail" question.  CONOPS or design or fabrication flaw?

I'm going to vote design, even though she said she didn't think so.  Actually, I'll amend that to design and quality control, since I believe the issue may be both, i.e., the design choices mean that a quality control or fabrication failure has a higher chance of happening.

But normal caveat applies: I don't have access to insider information, only various public and private sources.

[rsdavis9]

I'm betting on fill of helium and lox causing undo thermal stress. Possibly because of aluminum liner.
They will just have to change the speed/sequence of helium and lox.

[jpo234]

I'm betting on fill of helium and lox causing undo thermal stress. Possibly because of aluminum liner.
They will just have to change the speed/sequence of helium and lox.

The funny thing is, we learn from the same source, that the JCSat-14 S1 has gone through 11 hot fires:
1x pre-launch testing in McGregor
1x static fire on the pad
1x launch
8x post launch tests in McGregor
and it is still in one piece. My understanding is, that the pressure system of S1 and S2 is very similar.

[rsdavis9]

I'm betting on fill of helium and lox causing undo thermal stress. Possibly because of aluminum liner.
They will just have to change the speed/sequence of helium and lox.

The funny thing is, we learn from the same source, that the JCSat-14 S1 has gone through 11 hot fires:
1x pre-launch testing in McGregor
1x static fire on the pad
1x launch
8x post launch tests in McGregor
and it is still in one piece. My understanding is, that the pressure system of S1 and S2 is very similar.

Hopefully they will find some sequence/rate that produces the problem at the mcgregor tests.

[spacekid]

Also from that interview:

Q?:What’s the possibility that there’s a design issue with that helium bottle?

Gwynne: I don’t think it’s a design issue with the bottle. I think it probably is more focused on the operations, which is one of the reasons we believe we can get back to flight so quickly.

But we have to finish the investigation. We’re not going to fly until we’re ready to fly.

Q?: When you say it’s more focused on operations, you mean filling of the helium tank, or the filling of the LOX tank, or what?

Gwynne: All of it. We’re going to look at all of it.

"Operations" here means some procedural thing, like fill rate or timing?  Sounds like it's not one of the typical COPV failure modes that have been brought up many times.

Another interesting nugget - they weren't watching the LOX tank camera for this one.

I would expect SpaceX to be setting up a fill experiment. It'll be interesting to see if they set up a stage which would indicate they're not sure of the cause or a subset like putting a COPV in LOX and try filling it different ways.

Once they determine the root cause, as I've mentioned before it will be interesting to see if this could have caused the CRS-7 failure. For instance, if it's a tank leak through the over wrap, what if the leak on CRV-7 wasn't as severe but enough to over-stressed the strut causing it to fail?

[rsdavis9]

So if they only fill with lox and helium and no rp1. How bad of an explosion and subsequent destruction would we have?

I suppose it would be as bad as the initial blast we can see on the video before the rp1 and lox go up together.

[MP99]

Those gators are part of Cape security but seriously there is a dedicated thread for all the recent comments that are non-technical about the LV or pad infrastructure...
http://forum.nasaspaceflight.com/index.php?topic=41119.0

ISTR that Jim signs himself off as the night gator. :-)

Cheers, Martin

[woods170]

From today interview to Mrs Shotwell:

On Sept. 1 it wasn’t clear whether the cause lay in the ground support equipment, or inside the rocket. You have made the determination that it was inside the rocket and not some procedure during preparation for the static test?

We believe that the composite over wrapped pressure vessel [the helium bottle], known as a COPv, let go in the tank. What caused it, the exact reason it let go, we’re still investigating. I don’t believe it was a ground system cause, but we’re still looking at the data.

Was this public knowledge before?

"Massive breach in the cryogenic helium system" had already been released via the SpaceX anomaly updates. Some folks here (me included) figured that THAT piece of information, along with the hi-speed event (93 milliseconds) could mean only one thing: catastrophic COPV failure.

This is now confirmed by Gwynne. And it also pretty much shuts-up the folks pointing to other parts of the cryogenic helium system.

[kaiser]

From today interview to Mrs Shotwell:

On Sept. 1 it wasn’t clear whether the cause lay in the ground support equipment, or inside the rocket. You have made the determination that it was inside the rocket and not some procedure during preparation for the static test?

We believe that the composite over wrapped pressure vessel [the helium bottle], known as a COPv, let go in the tank. What caused it, the exact reason it let go, we’re still investigating. I don’t believe it was a ground system cause, but we’re still looking at the data.

Was this public knowledge before?

"Massive breach in the cryogenic helium system" had already been released via the SpaceX anomaly updates. Some folks here (me included) figured that THAT piece of information, along with the hi-speed event (93 milliseconds) could mean only one thing: catastrophic COPV failure.

This is now confirmed by Gwynne. And it also pretty much shuts-up the folks pointing to other parts of the cryogenic helium system.

Eh, if a fitting or hose let go downstream from the COPV, the quick release of pressure could have caused the COPV to break free/torque/bust pretty immediately.

Edit:  Still likely COPV, just saying that the above statement doesn't prove it was a COPV.

[woods170]

From today interview to Mrs Shotwell:

On Sept. 1 it wasn’t clear whether the cause lay in the ground support equipment, or inside the rocket. You have made the determination that it was inside the rocket and not some procedure during preparation for the static test?

We believe that the composite over wrapped pressure vessel [the helium bottle], known as a COPv, let go in the tank. What caused it, the exact reason it let go, we’re still investigating. I don’t believe it was a ground system cause, but we’re still looking at the data.

Was this public knowledge before?

"Massive breach in the cryogenic helium system" had already been released via the SpaceX anomaly updates. Some folks here (me included) figured that THAT piece of information, along with the hi-speed event (93 milliseconds) could mean only one thing: catastrophic COPV failure.

This is now confirmed by Gwynne. And it also pretty much shuts-up the folks pointing to other parts of the cryogenic helium system.

Eh, if a fitting or hose let go downstream from the COPV, the quick release of pressure could have caused the COPV to break free/torque/bust pretty immediately.

Edit:  Still likely COPV, just saying that the above statement doesn't prove it was a COPV.

I disagree. "Letting go" is an industry term for catastrophic failure of a pressure vessel or burst disk. The term goes back to at least the 1970's. I've seen "letting go" being used in this context in multiple failure investigation reports with the oldest one dating back to 1971. Gwynne using the same term "letting go" in direct relation to a COPV is IMO all-telling.

[Kabloona]

From today interview to Mrs Shotwell:

On Sept. 1 it wasn’t clear whether the cause lay in the ground support equipment, or inside the rocket. You have made the determination that it was inside the rocket and not some procedure during preparation for the static test?

We believe that the composite over wrapped pressure vessel [the helium bottle], known as a COPv, let go in the tank. What caused it, the exact reason it let go, we’re still investigating. I don’t believe it was a ground system cause, but we’re still looking at the data.

Was this public knowledge before?

"Massive breach in the cryogenic helium system" had already been released via the SpaceX anomaly updates. Some folks here (me included) figured that THAT piece of information, along with the hi-speed event (93 milliseconds) could mean only one thing: catastrophic COPV failure.

This is now confirmed by Gwynne. And it also pretty much shuts-up the folks pointing to other parts of the cryogenic helium system.

Eh, if a fitting or hose let go downstream from the COPV, the quick release of pressure could have caused the COPV to break free/torque/bust pretty immediately.

Edit:  Still likely COPV, just saying that the above statement doesn't prove it was a COPV.

I disagree. "Letting go" is an industry term for catastrophic failure of a pressure vessel or burst disk. The term goes back to at least the 1970's. I've seen "letting go" being used in this context in multiple failure investigation reports with the oldest one dating back to 1971. Gwynne using the same term "letting go" in direct relation to a COPV is IMO all-telling.

Yes, she would have mentioned another component as a potential root cause if they thought it was anything other than a COPV "letting go."

They maybe haven't "proved" it to themselves yet, but clearly they suspect a COPV itself.

[kaiser]

From today interview to Mrs Shotwell:

On Sept. 1 it wasn’t clear whether the cause lay in the ground support equipment, or inside the rocket. You have made the determination that it was inside the rocket and not some procedure during preparation for the static test?

We believe that the composite over wrapped pressure vessel [the helium bottle], known as a COPv, let go in the tank. What caused it, the exact reason it let go, we’re still investigating. I don’t believe it was a ground system cause, but we’re still looking at the data.

Was this public knowledge before?

"Massive breach in the cryogenic helium system" had already been released via the SpaceX anomaly updates. Some folks here (me included) figured that THAT piece of information, along with the hi-speed event (93 milliseconds) could mean only one thing: catastrophic COPV failure.

This is now confirmed by Gwynne. And it also pretty much shuts-up the folks pointing to other parts of the cryogenic helium system.

Eh, if a fitting or hose let go downstream from the COPV, the quick release of pressure could have caused the COPV to break free/torque/bust pretty immediately.

Edit:  Still likely COPV, just saying that the above statement doesn't prove it was a COPV.

I disagree. "Letting go" is an industry term for catastrophic failure of a pressure vessel or burst disk. The term goes back to at least the 1970's. I've seen "letting go" being used in this context in multiple failure investigation reports with the oldest one dating back to 1971. Gwynne using the same term "letting go" in direct relation to a COPV is IMO all-telling.

And I've used "letting go" in other ways throughout my career, and I've seen senior aerospace engineers use it in other ways also.  It's not a technical term.

The first thing that I thought of when they said "let go" was that it broke free from mounts (but maybe not via a strut failing) and shot off like a projectile.  On a pressurized vessel, that's what I think of when people say "let go".  If it catastrophically failed and just exploded, I've also seen that described as "let go" also though.

It could mean different things to different people, and again -- it very likely is a COPV issue, but the statement by Shotwell does not specifically say that it has to be the COPV.  Point is that it's not "all-telling" because it's not a precise technical term for a root cause.    The COPV could have catastrophically failed, and the root cause of that failure be somewhere other than the COPV.