SpaceX’s Crew Dragon spacecraft static fire anomaly - THREAD 3

[birdman]

Additionally, the SuperDraco thrusters recovered from the test site remained intact, underscoring their reliability.

...so nobody's gonna point out how ridiculous this line is?

I wonder what they mean by "intact". Like, "parts are kind of identifiable" or "plug some fuel in and fire away"

[OxCartMark]

SpaceX release:


Initial data reviews indicated that the anomaly occurred approximately 100 milliseconds prior to ignition of Crew Dragon’s eight SuperDraco thrusters and during pressurization of the vehicle’s propulsion systems. Evidence shows that a leaking component allowed liquid oxidizer – nitrogen tetroxide (NTO) – to enter high-pressure helium tubes during ground processing. A slug of this NTO was driven through a helium check valve at high speed during rapid initialization of the launch escape system, resulting in structural failure within the check valve. The failure of the titanium component in a high-pressure NTO environment was sufficient to cause ignition of the check valve and led to an explosion.

My thought when I read "a leaking component" is that the writer went a bit out of the way to not say it was the check valve referred to in the next sentence, the one that went boom.  While not certain it leads me to believe there may be (an)other component(s) in the system that leaked here to there.  Not sure what it may be (differential pressure sensor???) but it seems that writing was passed through an intentionally vague filter.

edit: or it could be that they don't even know which of a plurality of components leaked, still investigating, though admittedly its a simple system and the number of suspect leak paths / components would be very low.

[dglow]

This: it's in a DTIC memo from the 60s (page 9).  I don't know why it's "unexpected" for titanium to ignite when NTO is rammed into it at high pressure.

It's unexpected because SpaceX apparently didn't review the literature.

If you want to go down that route...NASA didn't either as NASA let DM-1 use this config.  Lets not go down that path in this thread please. It will just muddy it all up.

I am glad however that we learned in ~4 months what happened, that it seems easily fixable, and that lessons were learned and no lives lost to learn them.  Good day!

Theory: the “unexpected” was meant to refer to this event at high pressure, and the PR person re-ordered it to make it “clearer.” Money on the Dragon propulsion designers (at least some of them) understanding the materials risk but thinking it didn’t apply since the NTO should be on the other side of this valve located between the helium and NTO systems....just not whoever reviewed/approved the statement.

Possible, and a simple explanation. To review the exact lines from SpaceX's press release:

It is worth noting that the reaction between titanium and NTO at high pressure was not expected. Titanium has been used safely over many decades and on many spacecraft from all around the world.

There is room for ambiguity here. But the choice of the word 'reaction' (rather than, say, 'interaction') followed by a sentence extolling the safe use of titanium... well, it's possible to read that as an M&P oversight. I certainly hope it wasn't.

And for the record, I've no desire to cast aspersions on SpaceX; merely suggesting that SpaceX may have missed something is not in and of itself nefarious.  My original comment, in response to SWGlassPit, was simply that if SpaceX didn't know about it (the potential for a reaction) then of course it would be "unexpected" to them.

[jgoldader]

This reminds me of the Mars Observer, which was lost (most likely) due to a failed check valve.  That case might’ve been discussed upthread.

[Kabloona]

This reminds me of the Mars Observer, which was lost (most likely) due to a failed check valve.  That case might’ve been discussed upthread.

Yes, in fact my first thought after the Dragon failure was the Mars Observer check valve failure mode. We discussed it in Thread 2 some time ago, and also in L2.

https://forum.nasaspaceflight.com/index.php?topic=48047.msg1951990#msg1951990

[LtWigglesworth]

This reminds me of the Mars Observer, which was lost (most likely) due to a failed check valve.  That case might’ve been discussed upthread.

In a survey of accidents in the oil and chemical industries, around 3.5% of incidents were caused by leaking check valves.

> Duguid, I M. 1998. "Analysis of past incidents in the oil, chemical and petrochemical industries." United Kingdom.

[PM3]

It should, Titanium is used everywhere for NTO. The fact it can burn it I don't think is well known. Titanium is generally pretty non-reactive.

Interesting tweets on this topic by Ben Brockert:

https://twitter.com/wikkit/status/1150855184924336128?s=20

SpaceX update on the disappearing Dragon. (link: https://www.spacex.com/news/2019/07/15/update-flight-abort-static-fire-anomaly-investigation) spacex.com/news/2019/07/1… “It is worth noting that the reaction between titanium and NTO at high pressure was not expected” is a frightening sentence; titanium is well known to have poor oxidizer compatibility and be easy to set on fire. “resistant to N2O4 except under impact... found that Ti impacts sporadically under reasonably well controlled test conditions; the ignition freq. is increased markedly by Ti filings or glass particles on the impact surface” Compatibility of Materials w/ Rocket Prop and Oxidizers

This: it's in a DTIC memo from the 60s (page 9).  I don't know why it's "unexpected" for titanium to ignite when NTO is rammed into it at high pressure.

This may be a misinterpretation of that 60's source.

Here is another, more detailed 60's source on the same issue: http://contrails.iit.edu/files/original/WADDTR61-175.pdf

It says that if Titanium has been soaked with NTO for 24 hours, and then it is impacted by a 0.5 inch diameter "flat striker" having an energy of 200 ft-lbs, it has a 50% chance of igniting.

Also note that there are different Titanium alloys, which may behave differently. SpaceX report may say short "Titanium" for some special alloy.

[lonestriker]

Is this the first instance of potentially designing for reuse (check valves) that caused issues vs. using old school expendable tech (burst disks)?  It's analogous to using pneumatics/pushers vs. pyros for stage separation and faring deployment in terms of being able to test repeatably.  I would assume if this issue were on just SpaceX's craft without NASA as a customer, they might consider redesigning the system to be more robust and retain the check valves.  But for speed of return-to-flight, the simplest solution is probably sufficient if they can make the replacement of the burst disks a reasonably easy post-mission maintenance task.

I assume they'll be able to go through this entire process for the static firing test of the updated D2 capsule, replacing the disks, and then flying the IFA.

[clarkeo]

As the issues was only on the high pressure side, I think we can assume that reusability is now a distant concern since they gave up on propulsive landings, as the only use of the super-dracos is for abort. Or do they pre-pressurise the system just in case?

[cleonard]

Is this the first instance of potentially designing for reuse (check valves) that caused issues vs. using old school expendable tech (burst disks)?  It's analogous to using pneumatics/pushers vs. pyros for stage separation and faring deployment in terms of being able to test repeatably.  I would assume if this issue were on just SpaceX's craft without NASA as a customer, they might consider redesigning the system to be more robust and retain the check valves.  But for speed of return-to-flight, the simplest solution is probably sufficient if they can make the replacement of the burst disks a reasonably easy post-mission maintenance task.

I assume they'll be able to go through this entire process for the static firing test of the updated D2 capsule, replacing the disks, and then flying the IFA.

I believe that this scenario only occurred due to firing the abort engine.  No abort, no breaking the disks.  An abort is not a normal reuse situation.  I would not be surprised that after an abort, the Dragon would be retired.

Sent from my LEX727 using Tapatalk

[envy887]

Additionally, the SuperDraco thrusters recovered from the test site remained intact, underscoring their reliability.

...so nobody's gonna point out how ridiculous this line is?

Not at all. It rules out the SDs as a potential root cause.

[Vettedrmr]

Bully on SpaceX for holding this conference and providing the release above. In one swift move they've come quite clean, sharing an abundance of what they currently know and how they've learned it, despite the investigation only being 4/5 complete.

This is a welcome balm in light of the discussion upthread and our hand-wringing around the administrator's recent comments. If this is a reflection of Bridenstine's new 'communication policy' well good on him too.

After reading what I can, I think Bridenstine may deserve more credit than at initial glance.  What if:

1. SpaceX, after getting criticized in the past for releasing info too quickly, chooses to let NASA control all public releases
2. NASA doesn't see the need to release anything to the public, for whatever reason. 
3. Bridenstine gets impatient with no info being released, and is feeling criticism for no info coming out of his organization, and let's loose with his own press release on the lack of information. 

This essentially authorizes SpaceX to make a release on their own, which they did today.  If NASA was feeling any pressure about no info coming out, SpaceX was feeling probably 10 times as much.

If I'm right, hat tip to Bridenstine and SpaceX.  If I'm not, hat tip to SpaceX anyway.

Have a good one,
Mike

[dglow]

Additionally, the SuperDraco thrusters recovered from the test site remained intact, underscoring their reliability.

...so nobody's gonna point out how ridiculous this line is?

Not at all. It rules out the SDs as a potential root cause.

Agree, this is an attempt to emphasize the fact that the SDs were not involved in the mishap.

But it comes off ham-fisted. Pointing out the engines are intact when the rest of the capsule was destroyed? It sounds like something Elon might say at a press conference. (theory: he learned the SDs were found intact and demanded this detail be included)

Good intention, sub-optimal communication.

[dglow]

Bully on SpaceX for holding this conference and providing the release above. In one swift move they've come quite clean, sharing an abundance of what they currently know and how they've learned it, despite the investigation only being 4/5 complete.

This is a welcome balm in light of the discussion upthread and our hand-wringing around the administrator's recent comments. If this is a reflection of Bridenstine's new 'communication policy' well good on him too.

After reading what I can, I think Bridenstine may deserve more credit than at initial glance.  What if:

1. SpaceX, after getting criticized in the past for releasing info too quickly, chooses to let NASA control all public releases
2. NASA doesn't see the need to release anything to the public, for whatever reason. 
3. Bridenstine gets impatient with no info being released, and is feeling criticism for no info coming out of his organization, and let's loose with his own press release on the lack of information. 

This essentially authorizes SpaceX to make a release on their own, which they did today.  If NASA was feeling any pressure about no info coming out, SpaceX was feeling probably 10 times as much.

If I'm right, hat tip to Bridenstine and SpaceX.  If I'm not, hat tip to SpaceX anyway.

Have a good one,
Mike

Yeah, but NASA (Kathy Leuders) and SpaceX (Hans Koenigsmann) were both part of the teleconference, and each answered questions from the press. It was a coordinated communications effort, which is great.

[Kabloona]

Is this the first instance of potentially designing for reuse (check valves) that caused issues vs. using old school expendable tech (burst disks)?  It's analogous to using pneumatics/pushers vs. pyros for stage separation and faring deployment in terms of being able to test repeatably.

No, it's common practice to use check valves in pressurization plumbing of monoprop and biprop systems. Mars Observer (probably) had a similar failure when NTO migrated slowly over time through a check valve, then got blown into the MMH side of the biprop system upon pressurization.

Burst disks are a pain because they must be replaced every time you test or operate the system, and also because burst disks can fragment and blow small bits of metal downstream into your highly-sensitive propulsion components.

[OxCartMark]

So in my mind we have ~100cc of N2O4 liquid which rams up against some solid titanium with enough energy to probably hammer burst the valve or line at that point and to ignite the surface of the valve metal and maybe some wire insulation and paint or whatever the flame and N2O spray up against but at that point its surface combustion with a small volume of N2O4 which within milliseconds will be followed by a flood of fire extinguishing Helium.  So far that sounds like about as much energy as crashing a 125cc dirt bike into the side of a barn at 25 mph (to use scientific units) with possibly continuing combustion of the titanium liberating more energy over the next few seconds.  That to me doesn't seem nearly energetic enough to blow the entire thing (less the SDs which were miraculously preserved) to smithereens.  So something happened subsequently and I assume exceedingly quickly subsequently.  Anyone care to elaborate on that?  More quickly than say a camp fire acting on the outside of the propellant tanks which would take a minute or so to burst.  Perhaps the only scenario that I can envision is that on the tank side of the check valve the inside of the metal N2O4 line, the surface interface between metal and N2O4 detonated all the way back into the N2O4 tank thus pressurizing it (it wasn't to that point pressurized(?)) and bursting it which was the dramatically energetic event.

What do we suppose these propellant lines and tanks are made from?  Stainless steel?  Titanium?  Any info on flame front speed in these conditions?

[Kabloona]

What do we suppose these propellant lines and tanks are made from?  Stainless steel?  Titanium?  Any info on flame front speed in these conditions?

Usual practice is stainless steel lines and titanium tanks, but I have no knowledge of Dragon particulars.

[OxCartMark]

Slightly off topic but getting back to topic...

The SR-71, A-12, YF-11 aircraft used TEB to ignite fuel in the afterburners.  IIRC, all of that plumbing including storage tanks was either made from gold or all contact surfaces were covered with gold.  I came away with the impression that was what was necessary with TEB.  But then as I fell into the SpaceX vortex I found out that F9 uses something more mundane in its TEB&A plumbing, stainless steel IIRC.  Why?  Is gold extreme overkill or is stainless maybe a bit risky or is it a matter of not knowing as much about compatibility back in the 60s and now we know stainless steel is compatible with TEB or does the mixture of TEB&A somehow reduce the reactivity so that stainless is safe?  This has long been a question in my mind but I'm now asking it here because one of the potential answers is that SpaceX is willing to sail closer to the material incompatibility borderline than may be advisable. Or not. I realize this seems as if I'm suggesting this to be the case and I'm not, just asking for opinions from knowledgeable folks.

[lonestriker]

Is this the first instance of potentially designing for reuse (check valves) that caused issues vs. using old school expendable tech (burst disks)?  It's analogous to using pneumatics/pushers vs. pyros for stage separation and faring deployment in terms of being able to test repeatably.  I would assume if this issue were on just SpaceX's craft without NASA as a customer, they might consider redesigning the system to be more robust and retain the check valves.  But for speed of return-to-flight, the simplest solution is probably sufficient if they can make the replacement of the burst disks a reasonably easy post-mission maintenance task.

I assume they'll be able to go through this entire process for the static firing test of the updated D2 capsule, replacing the disks, and then flying the IFA.

I believe that this scenario only occurred due to firing the abort engine.  No abort, no breaking the disks.  An abort is not a normal reuse situation.  I would not be surprised that after an abort, the Dragon would be retired.

Sent from my LEX727 using Tapatalk

To repeat the (previously fatal) D2 static fire test, they'll need to activate the Super Dracos to show that there is no "kaboom".  After this static fire test, they'll need to replace the disks ahead of the IFA test.

What I've read here and elsewhere is that even if you don't use the system, you may want to replace the disks anyways between uses since you don't want them to be out of spec and accidentally opening at inopportune times.  I wonder if D2 Cargo flights may just opt to fly without SD thrusters entirely to trade complete safety for the chance of recovering cargo in the event of an abort scenario off the pad.

So it's going to be a bit of extra work when they refurbish the D2 to convert from Crew to Cargo.

[LouScheffer]

It should, Titanium is used everywhere for NTO. The fact it can burn it I don't think is well known. Titanium is generally pretty non-reactive.

Interesting tweets on this topic by Ben Brockert:

https://twitter.com/wikkit/status/1150855184924336128?s=20

SpaceX update on the disappearing Dragon. (link: https://www.spacex.com/news/2019/07/15/update-flight-abort-static-fire-anomaly-investigation) spacex.com/news/2019/07/1… “It is worth noting that the reaction between titanium and NTO at high pressure was not expected” is a frightening sentence; titanium is well known to have poor oxidizer compatibility and be easy to set on fire. “resistant to N2O4 except under impact... found that Ti impacts sporadically under reasonably well controlled test conditions; the ignition freq. is increased markedly by Ti filings or glass particles on the impact surface” Compatibility of Materials w/ Rocket Prop and Oxidizers

Selective quoting perhaps.   The very next sentence (just after the quote) says "The ignitions do not spread beyond the impact area."

This is reinforced by a more detailed report about titanium reacting with N2O4, up to and including rifle bullet strikes: "Propagation of the reaction does not occur even though sufficient N2O4 is present to allow complete oxidation of the metal", and  "In no cases, however, has ignition been observed to cover more than a small fraction of the impact area as a surface fusion only". 

Most of these reports were not done under high pressure, but you'd think the 30-06 bullet impact (about 3000 joules) would create some extremely high pressure, at least temporarily.  Even in this case, "no propagation of the ignition was observed, and neither container was damaged to a greater extent than the empty impacted container."

So at least to me, nothing in this literature jumps out at me as a "Duh!  They should have seen this" moment.