Author Topic: NASA CRS-7 report released  (Read 23136 times)

Offline envy887

  • Senior Member
  • *****
  • Posts: 7992
  • Liked: 6532
  • Likes Given: 2927
« Last Edit: 03/13/2018 01:41 pm by Chris Bergin »

Offline Craig_VG

  • Full Member
  • **
  • Posts: 216
  • Liked: 706
  • Likes Given: 486
Re: NASA CRS-7 report released
« Reply #1 on: 03/12/2018 10:27 pm »
NASA concluded that it was not the manufacturer's fault for faulty strut, but instead SpaceX's fault for not following manufacturer guidelines.

Key Parts:

Quote
The use of an industrial grade 17-4 PH SS (precipitation-hardening stainless
steel) casting in a critical load path under cryogenic conditions and flight environments,
without additional part screening, and without regard to manufacturer recommendations for a
4:1 factor of safety, represents a design error – directly related to the F9-020 CRS-7 launch
failure as a “credible” cause

Quote
Lastly, the key technical finding by the
IRT with regard to this failure was that it was due to a design error: SpaceX chose to use an
industrial grade (as opposed to aerospace grade) 17-4 PH SS (precipitation-hardening stainless
steel) cast part (the “Rod End”) in a critical load path under cryogenic conditions and strenuous
flight environments.
« Last Edit: 03/12/2018 10:28 pm by Craig_VG »

Offline Coastal Ron

  • Senior Member
  • *****
  • Posts: 8115
  • I live... along the coast
  • Liked: 9403
  • Likes Given: 11150
Re: NASA CRS-7 report released
« Reply #2 on: 03/12/2018 10:47 pm »
NASA concluded that it was not the manufacturer's fault for faulty strut, but instead SpaceX's fault for not following manufacturer guidelines.

Key Parts:

Quote
The use of an industrial grade 17-4 PH SS (precipitation-hardening stainless
steel) casting in a critical load path under cryogenic conditions and flight environments,
without additional part screening, and without regard to manufacturer recommendations for a
4:1 factor of safety, represents a design error – directly related to the F9-020 CRS-7 launch
failure as a “credible” cause

Quote
Lastly, the key technical finding by the
IRT with regard to this failure was that it was due to a design error: SpaceX chose to use an
industrial grade (as opposed to aerospace grade) 17-4 PH SS (precipitation-hardening stainless
steel) cast part (the “Rod End”) in a critical load path under cryogenic conditions and strenuous
flight environments.

I think everyone should read the report, since selective quoting can sometimes be misleading. For instance, my own selective quoting:

Quote
SpaceX in their AIT report identifies “material defect” as the “most probable” cause for the rod end breaking. However, the IRT’s view is that while “rod end breakage due to material defect” is credible, the IRT does not denote it a “most probable” since the IRT also views “rod end manufacturing damage”, “rod end strut mis-installation”, “rod end collateral damage” or some other part of the axial strut breaking as equally credible causes to have liberated the COPV.

I'll also note that I didn't see any mention of SpaceX requiring certification from the supplier, and that the supplier had not been consistently doing the certification process (or doing it at all). And while it can be argued that "as designed" the parts were not of sufficient strength for the application they were used in, SpaceX knew this, which was why they required additional testing to weed out any non-compliant parts - and they paid for that service.

Maybe my perspective is colored by my manufacturing background, and that the processes SpaceX paid for would have produced acceptable parts.

Regardless, history has shown that SpaceX has addressed the problem, and I don't really find anything too damning in this report - though that may not stop SpaceX detractors from damning them anyways... ;)
If we don't continuously lower the cost to access space, how are we ever going to afford to expand humanity out into space?

Offline Craig_VG

  • Full Member
  • **
  • Posts: 216
  • Liked: 706
  • Likes Given: 486
Re: NASA CRS-7 report released
« Reply #3 on: 03/12/2018 11:35 pm »
Sorry if my quoting came off as pushing a particular viewpoint. That was not my intention.

I thought it best described the 'new info' released in this document.

Offline Nomadd

  • Senior Member
  • *****
  • Posts: 8576
  • Highway Whatever
  • Liked: 58421
  • Likes Given: 1143
Re: NASA CRS-7 report released
« Reply #4 on: 03/13/2018 02:54 am »
I thought they had a 4 to 1 margin. And NASA did not conclude it wasn't the manufacturer's fault. They concluded that SpaceX should have used higher qualification standards that are there to guard against manufacturing faults. I don't really see anything in that report that contradicts what SpaceX came up with.
« Last Edit: 03/13/2018 03:05 am by Nomadd »
Those who danced were thought to be quite insane by those who couldn't hear the music.

Offline S.Paulissen

  • Full Member
  • ****
  • Posts: 441
  • Boston
  • Liked: 332
  • Likes Given: 509
Re: NASA CRS-7 report released
« Reply #5 on: 03/13/2018 03:11 am »
I thought they had a 4 to 1 margin. And NASA did not conclude it wasn't the manufacturer's fault. They concluded that SpaceX should have used higher qualification standards that are there to guard against manufacturing faults. I don't really see anything in that report that contradicts what SpaceX came up with.

YMMV with source.

"He said that the strut was designed to handle 10,000Ibs of force but failed at 2,000Ibs of force."

https://www.siliconrepublic.com/discovery/spacex-rocket-exploded-because-of-broken-strut-explains-ceo-elon-musk
"An expert is a person who has found out by his own painful experience all the mistakes that one can make in a very narrow field." -Niels Bohr
Poster previously known as Exclavion going by his real name now.

Offline Coastal Ron

  • Senior Member
  • *****
  • Posts: 8115
  • I live... along the coast
  • Liked: 9403
  • Likes Given: 11150
Re: NASA CRS-7 report released
« Reply #6 on: 03/13/2018 03:35 am »
Sorry if my quoting came off as pushing a particular viewpoint. That was not my intention.

I thought it best described the 'new info' released in this document.

No, I did not want to imply that. And of course any time anyone uses a partial amount of an original text it's "selective quoting" - as I noted my selective quote was.

My apologies for the confusion...  :o
If we don't continuously lower the cost to access space, how are we ever going to afford to expand humanity out into space?

Online FutureSpaceTourist

  • Global Moderator
  • Senior Member
  • *****
  • Posts: 33521
  • UK
    • Plan 28
  • Liked: 58912
  • Likes Given: 26334
Re: NASA CRS-7 report released
« Reply #7 on: 03/13/2018 08:07 am »
Couple of key quotes:

Quote
It is important to note that the IRT’s conclusions regarding the direct, and immediate causes are consistent with the determination made by the SpaceX AIT investigation findings. Where the IRT differs with SpaceX is in regards to the initiating cause. SpaceX in their AIT report identifies “material defect” as the “most probable” cause for the rod end breaking. However, the IRT’s view is that while “rod end breakage due to material defect” is credible, the IRT does not denote it a “most probable” since the IRT also views “rod end manufacturing damage”, “rod end strut mis-installation”, “rod end collateral damage” or some other part of the axial strut breaking as equally credible causes to have liberated the COPV. Lastly, the key technical finding by the IRT with regard to this failure was that it was due to a design error: SpaceX chose to use an industrial grade (as opposed to aerospace grade) 17-4 PH SS (precipitation-hardening stainless steel) cast part (the “Rod End”) in a critical load path under cryogenic conditions and strenuous flight environments. The implementation was done without adequate screening or testing of the industrial grade part, without regard to the manufacturer’s recommendations for a 4:1 factor of safety when using their industrial grade part in an application, and without proper modeling or adequate load testing of the part under predicted flight conditions. This design error is directly related to the Falcon 9 CRS-7 launch failure as a “credible” cause.

Quote
In summary, the IRT determined that subject to the normal technical review of SpaceX’s corrective actions implementation, including correction of their design error, the F9-020 CRS-7 flight anomaly is resolved with “credible”, direct, intermediate, and initiating causes. That the initiating causes include more than just rod end “material defect”, and the initiating causes are rated by the IRT as “probable”.

P.S. I’ve put this in SpaceX General as I think discussion of the summary report’s contents naturally goes rather beyond the one mission.
« Last Edit: 03/13/2018 01:25 pm by Chris Bergin »

Offline woods170

  • IRAS fan
  • Senior Member
  • *****
  • Posts: 11483
  • IRAS fan
  • The Netherlands
  • Liked: 15619
  • Likes Given: 9804
Re: NASA CRS-7 report released
« Reply #8 on: 03/13/2018 08:39 am »
Just about the only new thing from this report is that the stage 2 LOX tank failed due to the liberated COPV shooting upwards and colliding with - and thus damaging - the upper dome of the LOX tank.

Everything else has already been reported, and discussed, ad nauseam.

Offline CorvusCorax

  • Full Member
  • ****
  • Posts: 1598
  • Germany
  • Liked: 3182
  • Likes Given: 2357
Re: NASA CRS-7 report released
« Reply #9 on: 03/13/2018 09:29 am »
I think the main lesson from this report to be taken is - 4 eyes see more than 2.

It was generally in the best interest of both NASA AND SpaceX, that an independent review was undertaken in close cooperation with SpaceX to verify the internal investigation, procedures and to investigate alternate causes.

Most importantly the document concludes with:

Quote
*The IRT notes that all credible causes and technical findings identified by the IRT were corrected and/or mitigated by SpaceX and LSP
for the Falcon 9 Jason-3 mission.  That flight, known as “F9-19”, was the last flight of the Falcon 9 version 1.1 launch vehicle, and flew successfully on 17 January 2016.

The review also looked into some issues other than the causes of the mishap, such as the mentioned low nitrogen purge rates or telemetry protocols - all of which led to vehicle improvements.

Competitors, and political speakers opposed to SpaceX might want to use this report as propaganda against SpaceX and their methods.
I think likely everyone at SpaceX knows better. They will be thankful to NASA for both vindicating and improving their own procedures as well as the vehicle.

The public sometimes sees investigations like this as "looking who is to blame" -- but this is a technical investigation. The do not search for a scapegoat, but for things that can be improved for the benefit of everyone.  Same as with NTSB investigations after aircrashes and the like. They are not to find some poor fool to be liable, but to improve the design and procedures, learn, and prevent bad things from happening again or ever in the future.

As such, the more issues such an investigation can identify for improvement - whether related to the mishap or not - is a good thing (tm)

Offline Chris Bergin

Re: NASA CRS-7 report released
« Reply #10 on: 03/13/2018 01:41 pm »
Support NSF via L2 -- Help improve NSF -- Site Rules/Feedback/Updates
**Not a L2 member? Whitelist this forum in your adblocker to support the site and ensure full functionality.**

Offline sanman

  • Senior Member
  • *****
  • Posts: 5616
  • Liked: 1140
  • Likes Given: 8

Offline Michael Baylor

  • NSF Reporter
  • Full Member
  • ****
  • Posts: 901
  • Liked: 4867
  • Likes Given: 865
Re: NASA CRS-7 report released
« Reply #12 on: 03/13/2018 03:19 pm »
I thought they had a 4 to 1 margin. And NASA did not conclude it wasn't the manufacturer's fault. They concluded that SpaceX should have used higher qualification standards that are there to guard against manufacturing faults. I don't really see anything in that report that contradicts what SpaceX came up with.

They did not abide by the 4 to 1 margin.

Quote from the IRT report.

Quote
The implementation was done without adequate screening or testing of the
industrial grade part, without regard to the manufacturer’s recommendations for a 4:1 factor of
safety when using their industrial grade part in an application, and without proper modeling or
adequate load testing of the part under predicted flight conditions.
« Last Edit: 03/13/2018 03:21 pm by Michael Baylor »

Offline jg

  • Full Member
  • **
  • Posts: 294
  • Liked: 162
  • Likes Given: 7
Re: NASA CRS-7 report released
« Reply #13 on: 03/13/2018 04:22 pm »
TF-4 (technical finding 4)  is interesting: it shows SpaceX's telemetry system is suffering from bufferbloat. Bufferbloat made their analysis much harder (since they lost a lot of their telemetry data as a result).  As gamers say: "Lag kills".

I'd love to be able to help Spacex out with this problem.  Note that I coined the word "bufferbloat" 6 or so years ago; we
didn't have a succinct term for the problem of excess buffering before that, and have been helping the Internet community face up to the problem.  If you know a good person I could contact who would pay attention, PM me.

BTW: most people suffer from bufferbloat routinely in their home networks.  See my blog for more information:

https://gettys.wordpress.com

There are beginning to be commercial products that dramatically help the problem for your home network.

But solutions depend on the circumstances; there is no magic wand here for SpaceX.

Offline envy887

  • Senior Member
  • *****
  • Posts: 7992
  • Liked: 6532
  • Likes Given: 2927
Re: NASA CRS-7 report released
« Reply #14 on: 03/13/2018 04:56 pm »
TF-4 (technical finding 4)  is interesting: it shows SpaceX's telemetry system is suffering from bufferbloat. Bufferbloat made their analysis much harder (since they lost a lot of their telemetry data as a result).  As gamers say: "Lag kills".

I'd love to be able to help Spacex out with this problem.  Note that I coined the word "bufferbloat" 6 or so years ago; we
didn't have a succinct term for the problem of excess buffering before that, and have been helping the Internet community face up to the problem.  If you know a good person I could contact who would pay attention, PM me.

BTW: most people suffer from bufferbloat routinely in their home networks.  See my blog for more information:

https://gettys.wordpress.com

There are beginning to be commercial products that dramatically help the problem for your home network.

But solutions depend on the circumstances; there is no magic wand here for SpaceX.

I'd be wary of assuming anything in the report is still current. 2015 was a LONG time ago in the SpaceX dev cadence. SpaceX has flow a new major version and 4 new sub-versions of Falcon 9 since then, with a 5th sub-version to fly within a month or so.

The last page notes that all the technical findings were already resolved or mitigated before Jason-3 flew in 2015.

Offline jg

  • Full Member
  • **
  • Posts: 294
  • Liked: 162
  • Likes Given: 7
Re: NASA CRS-7 report released
« Reply #15 on: 03/13/2018 05:17 pm »
TF-4 (technical finding 4)  is interesting: it shows SpaceX's telemetry system is suffering from bufferbloat. Bufferbloat made their analysis much harder (since they lost a lot of their telemetry data as a result).  As gamers say: "Lag kills".

I'd love to be able to help Spacex out with this problem.  Note that I coined the word "bufferbloat" 6 or so years ago; we
didn't have a succinct term for the problem of excess buffering before that, and have been helping the Internet community face up to the problem.  If you know a good person I could contact who would pay attention, PM me.

BTW: most people suffer from bufferbloat routinely in their home networks.  See my blog for more information:

https://gettys.wordpress.com

There are beginning to be commercial products that dramatically help the problem for your home network.

But solutions depend on the circumstances; there is no magic wand here for SpaceX.

I'd be wary of assuming anything in the report is still current. 2015 was a LONG time ago in the SpaceX dev cadence. SpaceX has flow a new major version and 4 new sub-versions of Falcon 9 since then, with a 5th sub-version to fly within a month or so.

The last page notes that all the technical findings were already resolved or mitigated before Jason-3 flew in 2015.

I hope so.  But my experience is that bufferbloat is not well understood, and it rears its ugly head again and again. It's nature is you fix one place, and find it rears its ugly head elsewhere all too soon.

It's a mistake we've made all over the industry.  We didn't even have a decent word for bufferbloat when I started. Most engineers don't think in the right terms.  And as to general solutions, while we had fixes for ethernet pretty quickly, other technologies are far lagging.  We didn't have *any* running code for WiFi until 18 months ago, and there is at best
one commercial product on the market at this date that has that.

Jim

Offline Andy Bandy

  • Full Member
  • *
  • Posts: 171
  • California
  • Liked: 110
  • Likes Given: 5
Re: NASA CRS-7 report released
« Reply #16 on: 03/13/2018 06:20 pm »
Less technical but more details on why it took this long a summary to be made public. NASA said it would, then it said it wouldn't, and it seems to have taken an effort by Congress to get the full story out there. There are some links in the post to more detailed stories on the whole thing.

http://www.parabolicarc.com/2018/03/13/nasa-falcon-9-failure-2015-caused-design-error/

Offline Lar

  • Fan boy at large
  • Global Moderator
  • Senior Member
  • *****
  • Posts: 13438
  • Saw Gemini live on TV
  • A large LEGO storage facility ... in Michigan
  • Liked: 11808
  • Likes Given: 11015
Re: NASA CRS-7 report released
« Reply #17 on: 03/13/2018 06:30 pm »
I thought they had a 4 to 1 margin. And NASA did not conclude it wasn't the manufacturer's fault. They concluded that SpaceX should have used higher qualification standards that are there to guard against manufacturing faults. I don't really see anything in that report that contradicts what SpaceX came up with.

They did not abide by the 4 to 1 margin.

Quote from the IRT report.

Quote
The implementation was done without adequate screening or testing of the
industrial grade part, without regard to the manufacturer’s recommendations for a 4:1 factor of
safety when using their industrial grade part in an application, and without proper modeling or
adequate load testing of the part under predicted flight conditions.

I thought a key SpaceX finding from their own internal investigation contradicted the first part of this... that SpaceX had contracted for, and was relying on, screening/testing by the part vendor, had reason to beleive that was "adequate" but determined that they were misled and added an additional layer of testing after part receipt.
"I think it would be great to be born on Earth and to die on Mars. Just hopefully not at the point of impact." -Elon Musk
"We're a little bit like the dog who caught the bus" - Musk after CRS-8 S1 successfully landed on ASDS OCISLY

Offline kaiser

  • Member
  • Posts: 61
  • Liked: 79
  • Likes Given: 24
Re: NASA CRS-7 report released
« Reply #18 on: 03/13/2018 06:34 pm »
Interesting to see the note about the telemetry latency.  I'm finding a lot of newer systems aggregating and streaming in lots of ways that result in lots of extra latency, and how that latency can ripple through so many other aspects of the system. 

Good to see NASA point out that low latency transmission is key to having as much data as possible for FRB, and that SpaceX implemented it. Kudos to them.

Offline LouScheffer

  • Senior Member
  • *****
  • Posts: 3077
  • Liked: 5407
  • Likes Given: 684
Re: NASA CRS-7 report released
« Reply #19 on: 03/13/2018 06:51 pm »
Quote from the IRT report.

Quote
The implementation was done without adequate screening or testing of the industrial grade part, without regard to the manufacturer’s recommendations for a 4:1 factor of safety when using their industrial grade part in an application, and without proper modeling or adequate load testing of the part under predicted flight conditions.

This, in a nutshell, is a large part of what makes designing and operating rockets and airplanes so expensive.  For commercial stuff like building and bridges, you over-design by a factor or 4 or 6.   This margin helps in two ways - you can use very simple (often visual) inspection, and you are protected against errors in calculating loads.   For most civil construction, you could have a hidden flaw that makes each part only half as strong as it should be, AND a calculation error that makes the loads twice what you calculated, and your building will still stand.  Of course the building is 6 times as heavy as it really needs to be, but that's OK.

But if mass concerns drive you to reduce your margins to 50%, now you need both fancy and detailed inspections, to insure no hidden flaws, and much more accurate load calculations, to make sure you really found the worst case and calculated it correctly.  From the quote it appears SpaceX made errors in both areas.

This line of reasoning leads some to believe that space stuff could be much cheaper if we had cheaper transport, since then you could use the higher mass margins to allow for cheaper design and construction.   So far this has not happened, perhaps since the needed margins are so big, or because other considerations (such as the difficulty of repair once in operation) also require extreme design care and inspections in any event.

Tags:
 

Advertisement NovaTech
Advertisement SkyTale Software GmbH
Advertisement Northrop Grumman
Advertisement
Advertisement Brady Kenniston
Advertisement NextSpaceflight
Advertisement Nathan Barker Photography
0