SpaceX Falcon 9 - AMOS-6 - (Pad Failure) - DISCUSSION THREAD (2)

[Coastal Ron]

An aerospace exec pinning a launch vehicle failure on a "business process" falls somewhere between "rarely happens" and "never happens," so it's understandable we're left scratching our heads.

I don't know that it's that rare.  The Challenger shuttle accident was a business process failure.  And an argument could be made that not recognizing that the Columbia shuttle was damaged before it returned to Earth was a business process failure.

[su27k]

I thought that SpaceX originally stayed away from the risky high performance launch vehicles in favor of the tried and true. Now, to squeeze extra performance out of their launch vehicles, they are perhaps going to extra risky lengths.

They started with tried and true since they didn't have money to do anything fancy. High performance doesn't mean risky, RD-180 and Centaur have better performance and are reliable.

[CameronD]

for us visual folks, here's what a business process looks like mapped out

https://en.wikipedia.org/wiki/Business_process_mapping#Example

as you can see someone chose the wrong process for this scenario

Input: Falcon 9  ->  Make Breakfast -> Output: Scrambled Eggs

Is it only me that doesn't understand that diagram??   

I see it more like this:

[cscott]

I wonder if "business process" has anything to do with Spacecom?  Perhaps there wasn't clear communication between Spacecom and SpaceX, and that communications gap ultimately contributed to the failure?

The most obvious way this could happen is if Spacecom were exerting schedule pressure and that caused SpaceX to be more bold than usual.  But there are also all the pound/kilogram conversion stories in the aerospace literature.  I wonder if it is possible to abuse the payload enough during static fire for it to lead to a bad day?

[woods170]

As has been said before on this thread at least once, payload on top or not during static fire is the customer's decision - not SpaceX's

no, it is a Spacex decision with customer's concurrence.  The customer didn't ask to be put on top.

Yes, and if the customer disagrees, SpaceX will remove the payload before doing the static fire. It's all in the contract. In this case, the customer did not disagree with having the payload on top. So, the customer shares the responsibility, given that it's their payload.

[Stranger]

and all of these changes in the "business processes" - the preparation of the rocket operations SpaseX how it coordinates with NASA?
Or they do what they want with their rocket?

[Jet Black]

An aerospace exec pinning a launch vehicle failure on a "business process" falls somewhere between "rarely happens" and "never happens," so it's understandable we're left scratching our heads.

I don't know that it's that rare.  The Challenger shuttle accident was a business process failure.  And an argument could be made that not recognizing that the Columbia shuttle was damaged before it returned to Earth was a business process failure.

The problem there is that almost anything can be reduced to a business process failure. The CRS-7 can be reduced to that since their process did not include internal testing of the struts, for example.

[envy887]

An aerospace exec pinning a launch vehicle failure on a "business process" falls somewhere between "rarely happens" and "never happens," so it's understandable we're left scratching our heads.

I don't know that it's that rare.  The Challenger shuttle accident was a business process failure.  And an argument could be made that not recognizing that the Columbia shuttle was damaged before it returned to Earth was a business process failure.

The problem there is that almost anything can be reduced to a business process failure. The CRS-7 can be reduced to that since their process did not include internal testing of the struts, for example.

Challenger and Columbia had design compromises that were compounded by process failures; CRS-7 had a manufacturing flaw compounded by process failures.

SpaceX is indicating that there were no design or manufacturing flaws in the AMOS-6 LV, and that a process failure was the root cause of the loss of a properly designed and built vehicle. That's rather unusual.

[.Scott]

By her saying it was a business issue and not a vehicle issue, that implies we aren't talking a quality of product or supply chain problem.

Taking her literally (which may prove to be ill-advised), she also contrasted it to an operations problem.  Obviously, this "business issue" eventually manifested itself in more than misplaced money and paragraphs.

The business decision placing the payload isn't likely the same one that caused the explosion.

So what about the RP and LOX?  Where do they come from?  Is the LOX manufactured on site?  How are vendors involved?  Have there been problems with these before?

[Jim]

So what about the RP and LOX?  Where do they come from?  Is the LOX manufactured on site?  How are vendors involved?  Have there been problems with these before?

LOX comes from a Praxair plant in Mims, FL that is used by all launch vehicles at the Cape.

[Kabloona]

So what about the RP and LOX?  Where do they come from?  Is the LOX manufactured on site?  How are vendors involved?  Have there been problems with these before?

LOX comes from a Praxair plant in Mims, FL that is used by all launch vehicles at the Cape.

Praxair has a 5-year contract to supply LOX for KSC/Cape.

https://www.nasa.gov/centers/kennedy/news/releases/2014/release-20140616d.html#.V_zhDn-9KSM

[Rocket Science]

IMHO, the only thing standing between SpaceX and their Mars ambitions, long-term, is getting the Falcon rocket family to work reliably.

They need to get their workhorse tweaked. They need to address whatever business issues that lead to failures.

And I think abandoning the reusable upper stage may help this because it means they're separating their cutting-edge development side from their operations side, which needs to be highly reliable.

Anyway, that's what I hope for, anyway. If they can manage to learn from this in that way, to perhaps change part of themselves to something they've never had before (consistent, professional, reliable operations), then I think they'll succeed on putting people on Mars.

You raised my brows this morning Chris with this statement... Bridging those two points from Earth to Mars requires building quite a solid span. They disappointed me in what happened, there someone said it... When I see them fly without incident for the next couple of years we'll see... They are going to have to re-earn my trust in them... On the other hand, I will continue to be entertained by Elon's dream factory...

[Dante80]

From the Update Thread:

Full quote from Jeff Foust's summary article on SpaceNews (Shotwell says SpaceX “homing in” on cause of Falcon 9 pad explosion):

“We’re homing in on what happened,” she said. “I think it’s going to point not to a vehicle issue or an engineering design issue but more of a business process issue.”

Note: emphasis mine.

Meaning what? Trying to save a little time in the countdown to shorten the launch campaign?  And what was the risk vs reward?  As for not being a vehicle or an design issue, it is something worse, it is a something worst, it is a cultural issue.  What other "short cuts" are being done without rigorous engineering review.

Jim brings up a really good point - I don't want it to get lost if there's discussion over in the Update thread.

Wayne Hale wrote a few days ago about asking "why" seven times as part of a root cause investigation. Identifying the proximate cause is good, but if the underlying defects that allowed it to happen without being caught are fixed, then you're just waiting for another failure to happen.

Not stating that this was the case, just curious however about what sort of cultural / procedural actions can cause a company to miss something like we discussed above (thermo-acoustic resonance coupling to mechanical resonance)?

This is indeed a good point. What Hale explains in his - superlative imo - post is that there is a big and apparent difference between finding what happened and finding why it happened. You cannot stop at the first part.

To quote his example:

Rule #5 for accident investigators:  ask ‘why’ seven times. It is much too easy to come to a first level conclusion and leave the investigation.  That is guaranteed to result in future accidents.I have no idea why the train crash occurred, but let’s take an imaginary trip through the kind of questions that an accident investigator should ask later in the investigation when a proximate cause is identified.  Here is that strictly hypothetical example:

Q1: Why did the train not stop? A1:  The brakes failed to apply when commanded by the operator. 

Q2: Why did the brakes fail?  A2:  part X in the braking system failed. 

Q3: Why did part X in the braking system fail?  A3 It was installed improperly at the last maintenance period.

Q4:  Why was part X installed improperly? A4:  The maintenance installation procedure was incorrect. 

Q5:  Why was the maintenance procedure incorrect? A5:  The procedure was not updated when a new part manufacturer was selected to build part X. 

Q6: Why was the procedure not updated?  A6:  The process for updating maintenance procedures did not allow for a change in part manufacturer. 

Q7:  Why did the process not allow for a new manufacturer:  A7:  It was not foreseen that a new part manufacturer would make a part that needed new installation procedures. 

Following this hypothetical case – and just note that I know nothing about the train crash, I am just making this up as a teaching tool – an accident investigator would find that the proximate cause of the accident was a braking failure, but the root cause was an inadequate process to account for new part manufacturers and the corrective action is to update the maintenance procedure change process to ensure that when a new part is introduced, the maintenance procedures are updated properly.

[Lee Jay]

"Oh, and follow Gibbs rule #13.  Look it up."

Gibbs rule #13:  Never, ever involve lawyers.

Wayne Hale is always good for good advice.

[BrightLight]

Full quote from Jeff Foust's summary article on SpaceNews (Shotwell says SpaceX “homing in” on cause of Falcon 9 pad explosion):

“We’re homing in on what happened,” she said. “I think it’s going to point not to a vehicle issue or an engineering design issue but more of a business process issue.”

Note: emphasis mine.

OK - Since i'm not a VP of a big corporation nor a rocket engineer, what does  "business process issue" - is it pad flow, process sequence - inquiring  minds want to know?

[Mike_1179]

Full quote from Jeff Foust's summary article on SpaceNews (Shotwell says SpaceX “homing in” on cause of Falcon 9 pad explosion):

“We’re homing in on what happened,” she said. “I think it’s going to point not to a vehicle issue or an engineering design issue but more of a business process issue.”

Note: emphasis mine.

OK - Since i'm not a VP of a big corporation nor a rocket engineer, what does  "business process issue" - is it pad flow, process sequence - inquiring  minds want to know?

Not a formal definition, but based on what we do in my (non-aerospace) industry: A choice in how you operate that is driven not by what is needed to meet the customer's end goal (get payload to orbit) but to meet some internal goal of the company (get to orbit for less than $XXX, total time from delivery of stage to static fire of less than XXX days, etc).

[Rocket Science]

"Oh, and follow Gibbs rule #13.  Look it up."

Gibbs rule #13:  Never, ever involve lawyers.

Wayne Hale is always good for good advice.

So is William Shakespeare in his opinion of them...

[starsilk]

So what about the RP and LOX?  Where do they come from?  Is the LOX manufactured on site?  How are vendors involved?  Have there been problems with these before?

LOX comes from a Praxair plant in Mims, FL that is used by all launch vehicles at the Cape.

impurities in the LOX that aren't an issue until supercooled? would not affect other vehicles... or same with RP but this issue seems to be LOX related.

hard to see how that would be a 'business process issue' unless they elected not to pay for the 'extra super pure' LOX.

edit: or of course the Helium. much easier to see how impurities from buying 'cheaper' could cause a problem there.

[Mike_1179]

So what about the RP and LOX?  Where do they come from?  Is the LOX manufactured on site?  How are vendors involved?  Have there been problems with these before?

LOX comes from a Praxair plant in Mims, FL that is used by all launch vehicles at the Cape.

impurities in the LOX that aren't an issue until supercooled? would not affect other vehicles... or same with RP but this issue seems to be LOX related.

hard to see how that would be a 'business process issue' unless they elected not to pay for the 'extra super pure' LOX.

edit: or of course the Helium. much easier to see how impurities from buying 'cheaper' could cause a problem there.

LOX is made by cryogenic distillation of air. Cool down air enough and various gasses start condensing out of it based on the vaporization temperature. I thought this process generates high-purity gas on its own, kind of how distillation works.

[gongora]

SpaceNews: Satellite CTOs dream of five-year-duration spacecraft (cost-cuts included)

> Would you put your satellite on a rocket for static-fire test to save a day?

Antonio Abad [hispasat]

I was in fact presented that choice [by SpaceX] and we decided not to put it on the rocket for the static fire. Remember what I said earlier: This event should remind us to be very humble in this industry. So no, I did not authorized SpaceX to put our satellite on the vehicle for the static fire.

Yohann Leroy [eutelsat]

We also considered it was not worth it for the two launches we had.