SpaceX Falcon 9 - AMOS-6 - (Pad Failure) - DISCUSSION THREAD (2)

[gadgetmind]

As fascinating as these discussions regards "business process" are, the way I read the Shotwell interview she used this phrase regarding the CRS-7 failure and for the Amos-6 one said "operations".

[Stranger]

As fascinating as these discussions regards "business process" are, the way I read the Shotwell interview she used this phrase regarding the CRS-7 failure and for the Amos-6 one said "operations".

if she had said the same, then the causes of accidents and the same))

[Fred Bonyea]

We had a couple of bonding failures that were traced back to the cleaning process. In one case, hydraulic oil was found in the bonding region. The source of the oil turned out to be that a new grit blast hose had been installed just prior to cleaning the object. The hose was contaminated when a vendor's leak testing equipment had a bad seal in the pressurizing component, allowing hydraulic oil to spray into the hose being tested for integrity.

In another case, a similar hose used for cleaning had been left uncapped in a corrosive environment and the fittings corroded internally. These types of problems are very difficult to identify when developing a manufacturing process for the first time. There will be failures.

     

[Rocket Science]

As far as I can recall crewed test flights are supposed to be in 2017.... anyone else?

Yeah, prior to the AMOS-6 mishap they were essentially targeting December 31, 2017 so they would still make it under the wire (though, IIRC, the last date I remember actually being bandied about was the ~20th).  Which I read as: We're still officially targeting a 2017 launch but in reality everyone who's paying attention should expect us to launch in 2018 because there will inevitably be some slip as we get closer to crunch time.

Thanks, I like to be an "equal opportunity doubter" so I will say the same thing I said about a Boeing December launch date means in the "upcoming new year"...

[john smith 19]

We're still close to 2 years from a crewed launch on F9.

2? Wasn't it supposed to be around this time next year?

My mistake but the fact remains commercial Crew will be fitted with an LES.  Obviously the question is would it have picked up and reacted to this situation fast enough but that's a different issue.

[D_Dom]

I thought this process generates high-purity gas on its own, kind of how distillation works.

Everything that happens after the distillate is collected must be controlled to very high standards. Storage, transfer, chillin' etc.

[Shredder56]

Back to the business process.  The example of the part from a new vendor jogged a memory.  Wasn't there an issue with a valve in the first Dragon that came from a vendor other than the one that was qualified?  That sounds like the type of process foul we are looking for.

[LouScheffer]

Another example of a "business process error" was the error in the Hubble mirror.   In particular, the commission criticized Perkin Elmer for not involving the original designers in the fabrication of the mirror.   This became important when the simple null corrector and the precise null corrector gave different results (by about a half wavelength).  The technicians were not suspicious, thinking that's why they have a coarse and fine corrector.  But if they talked to the optics designer, they would have found the coarse corrector should be good to something like lambda/20.  So there is no way the two correctors should be that different, and at least one of them must be wrong.  Any investigation into this would have quickly found the assembly error in the fine corrector.

Apparently the optic designers *were* involved in the prototype, which worked perfectly.   This lead management to treat the production mirror as an over-the-wall exercise, presumably so the optics designers could then work on other projects.

[jg]

Another example of a "business process error" was the error in the Hubble mirror.   In particular, the commission criticized Perkin Elmer for not involving the original designers in the fabrication of the mirror.   This became important when the simple null corrector and the precise null corrector gave different results (by about a half wavelength).  The technicians were not suspicious, thinking that's why they have a coarse and fine corrector.  But if they talked to the optics designer, they would have found the coarse corrector should be good to something like lambda/20.  So there is no way the two correctors should be that different, and at least one of them must be wrong.  Any investigation into this would have quickly found the assembly error in the fine corrector.

Apparently the optic designers *were* involved in the prototype, which worked perfectly.   This lead management to treat the production mirror as an over-the-wall exercise, presumably so the optics designers could then work on other projects.

In this case, not having independent checking on the mirror was also a clear business process error.  Of course, then some unclean, oh, I mean, uncleared people would have had to have had access to the lab where they made the mirror, at the wrong end of the building that optics for the KH11's were being built.

[LouScheffer]

Another example of a "business process error" was the error in the Hubble mirror.   In particular, the commission criticized Perkin Elmer for not involving the original designers in the fabrication of the mirror.   

In this case, not having independent checking on the mirror was also a clear business process error.  Of course, then some unclean, oh, I mean, uncleared people would have had to have had access to the lab where they made the mirror, at the wrong end of the building that optics for the KH11's were being built.

It's common to blame this on security clearances, but there were other ways to check the mirror that were skipped for pure business reasons.  Eastman Kodak, as a sub-contractor, built a backup mirror (now in the Smithsonian) and hence had precisely the test equipment needed, with no security issues.  But PE was sure that their mirror was OK, and were under cost and schedule pressure, and this test would take time and money.  So they skipped it.  Furthermore, Kodak had put in a bid as the prime mirror fabricator.  This also had a sub-contracted backup, as NASA mandated, and Kodak's bid included swapping the mirrors for testing at the end.  But NASA rejected Kodak's bid as more expensive, in hindsight another business process error.

[CyndyC]

It's common to blame this on security clearances, but there were other ways to check the mirror that were skipped for pure business reasons.

Wow, those were a lot of interesting inside details on the Hubble mirror error, Lou. The tv documentary was now obviously more focused on the repair than on the reasons behind needing it. The project manager was shown with a tear in his eye, that was about it

[Jet Black]

I know it's O/T to the strict topic of SpaceX, but it's really good to see the great examples of business process errors and how they were the root cause to fabrication errors. Thanks to the various posters for your knowledge.

[jaufgang]

quoted from  https://www.reddit.com/r/spacex/comments/57balr/details_from_elons_speech_at_the_nro/

Details from Elon's speech at the NRO (self.spacex)
submitted 23 minutes ago * by Ravenous117

Addressing the anomaly:

“We are close to figuring it out. It might have been formation of solid oxygen in the carbon over-wrap of one of the bottles in the upper stage tanks. If it was liquid it would have been squeezed out but under pressure it could have ignited with the carbon. This is the leading theory right now, but it is subject to confirmation.  The other thing we discovered is that we can exactly replicate what happened on the launch pad if someone shoots the rocket. We don’t think that is likely this time around, but we are definitely going to have to take precautions against that in the future. We looked at who would want to blow up a SpaceX rocket. That turned out to be a long list. I think it is unlikely this time, but it is something we need to recognize as a real possibility in the future.”

Addressing return to flight:

“The plan is to get back to launch in early December and that will be from pad 39A at the Cape and we will be launching around the same time from Vandenberg as well. Pad 40 will probably be back in action around March or April next year. Probably around May or so is when we will launch Falcon Heavy. We are going to re-fly the first returned core December or January.  We have test fired one of the returned cores 8 times and it looks good.  That is promising for testing re-flight.”

Other interesting points:

3D printing works fine for super-dracos, but too much work is needed to make it feasible on Raptors.

Elon envisions Mars as a direct democracy, not a representative democracy.

Still wants communication satellite constellation to provide revenue for Mars.

In talking about the IAC, “Crazy people are a lot faster to the mic than scientists.”

These are my personal accounts of what I heard from Elon live and the rough transcript is from a recording of the event. I do not know much other than what I heard but I wanted to share with you guys. Enjoy.

[Stranger]

Oxygen ignited by carbon?

[Welsh Dragon]

Oxygen ignited by carbon?

Ignited with carbon.

[Jim]

Addressing the anomaly:

“We are close to figuring it out. It might have been formation of solid oxygen in the carbon over-wrap of one of the bottles in the upper stage tanks. If it was liquid it would have been squeezed out but under pressure it could have ignited with the carbon. This is the leading theory right now, but it is subject to confirmation. 

How is that not a flaw in the vehicle and a "business process error"?

[Jim]

  The other thing we discovered is that we can exactly replicate what happened on the launch pad if someone shoots the rocket. We don’t think that is likely this time around, but we are definitely going to have to take precautions against that in the future. We looked at who would want to blow up a SpaceX rocket. That turned out to be a long list. I think it is unlikely this time, but it is something we need to recognize as a real possibility in the future.”

That is plain paranoia.  There always has been that threat.

[Stranger]

Ignited with carbon.

Sorry. My English is weak

[savuporo]

  The other thing we discovered is that we can exactly replicate what happened on the launch pad if someone shoots the rocket. We don’t think that is likely this time around, but we are definitely going to have to take precautions against that in the future. We looked at who would want to blow up a SpaceX rocket. That turned out to be a long list. I think it is unlikely this time, but it is something we need to recognize as a real possibility in the future.”

That is plain paranoia.  There always has been that threat.

Echoes of this

[jpo234]

Addressing the anomaly:

“We are close to figuring it out. It might have been formation of solid oxygen in the carbon over-wrap of one of the bottles in the upper stage tanks. If it was liquid it would have been squeezed out but under pressure it could have ignited with the carbon. This is the leading theory right now, but it is subject to confirmation. 

How is that not a flaw in the vehicle and a "business process error"?

I think one can prevent this by modifying the tanking procedure. Prefill the COPVs until they reach nominal pressure, then add the LOX and keep filling the COPVs while they get colder. No actual changes in the vehicle, just procedural changes.