Discussion of NASA data review delaying SpaceX launches

[Barley]

the launch vehicles are not static.  These are not the same design as a year ago.  Every upperstage is new.

The solution to this is version control.  You should not assume that chronological order of the launch implies version number of the hardware.

For example if Psyche is using upper stage serial number 1002 you can launch serial numbers 1001 and 1003 and use those as your baseline.  (Assuming 1001-1003 are identically configured, if they are not rearrange things so they are.)

Similarly for boosters and launch pad procedures.

Coming up with a solution isn't the issue, tbh; recognising that there's a problem is.

The people supporting the delay of 6-22 have recognized the problem and come up with a non-solution.  So clearly recognizing the problem is not the hard part.

I am in complete agreement with following procedures as written and not making ad-hoc changes.  Changes should be made in regular order.  I have some sympathy for the argument that the cost of following the procedure is low enough that making a change is not yet warranted.  I have no brief for the argument that everything is for the best in the best of all possible worlds and no change should ever be made.

This episode has disclosed a bug.  It should be tracked until it is fixed.

[RedLineTrain]

I think the larger question begged is whether LSP is earning its keep.  Why does NASA need to review data when SpaceX has already done so, on a rocket that has been exceedingly well characterized?  The value of a re-review sure seems to be diminishing quickly.

how do you know that SpaceX already has?  It isn't for "characterization"

No, I don't know that SpaceX already has.  But why wouldn't they, at least in some automated fashion?  They want to provide a good and reliable service.  More money is riding on the launch for SpaceX than it is for NASA.

[alugobi]

It's not about money for NASA.

[DanClemmensen]

It's not about money for NASA.

That's because NASA is not a for-profit company. SpaceX is. If this rule has a probability of costing SpaceX money, then SpaceX should add that prpbably cost into the price of the launch. If NASA chooses to pay this premium, then SpaceX does not lose money. If NASA chooses to change the rules on order to  save the premium, then SpaceX does not lose money.

[RedLineTrain]

It's not about money for NASA.

On the contrary, these data reviews are because NASA self-insures.

[alugobi]

So, it either is or it isn't.  I don't care.  My point is that, from appearances, SX and NASA have different priorities.  Make money and get to Mars vs. don't screw the pooch again.

[steveleach]

So, it either is or it isn't.  I don't care.  My point is that, from appearances, SX and NASA have different priorities.  Make money and get to Mars vs. don't screw the pooch again.

I don't think anyone would be arguing over "don't screw the pooch again".  The concern is around whether the priority has become "make sure your backside is covered when the pooch gets screwed again".

[dondar]

what a bizarre thread.
NASA reviews involve SpaceX engineers. SpaceX doesn't have Falcon engineers dedicated to NASA.  NASA and Air Force flights demand more preparation and more time from SpaceX engineers than any other flights. Hence the "dead zone" before any NASA/Air Force flight.
This time/availability of SpaceX engineering force is contracted by NASA/Air Force.
The BS about "data" is BS.

[steveleach]

what a bizarre thread.
NASA reviews involve SpaceX engineers. SpaceX doesn't have Falcon engineers dedicated to NASA.  NASA and Air Force flights demand more preparation and more time from SpaceX engineers than any other flights. Hence the "dead zone" before any NASA/Air Force flight.
This time/availability of SpaceX engineering force is contracted by NASA/Air Force.
The BS about "data" is BS.

It's only bizarre if you haven't read it properly ;-)

[dondar]

what a bizarre thread.
NASA reviews involve SpaceX engineers. SpaceX doesn't have Falcon engineers dedicated to NASA.  NASA and Air Force flights demand more preparation and more time from SpaceX engineers than any other flights. Hence the "dead zone" before any NASA/Air Force flight.
This time/availability of SpaceX engineering force is contracted by NASA/Air Force.
The BS about "data" is BS.

It's only bizarre if you haven't read it properly ;-)

The only practical reason for NASA review delay (i.e. extra work) is some out of family event during (one of) previous Falcon launch, which SpaceX team has to process and to wrap some "satisfactory story" for NASA review committee.
Check it out this news about identical episode but from different era (pre Starship/superheavy push).
https://spacenews.com/falcon-9-investigation-ongoing-as-spacex-continues-starlink-launches/
See the difference? the difference is the number of available engineers currently busy with Falcon hardware.

[Jim]

The only practical reason for NASA review delay (i.e. extra work) is some out of family event during (one of) previous Falcon launch, which SpaceX team has to process and to wrap some "satisfactory story" for NASA review committee.

No, NASA does the data review.

[LouScheffer]

It would be interesting to know what these reviews uncover.  In theory another set of eyes on the data can uncover things the developers may have missed.

One could imagine that the initial few reviews of a new rocket type would be the most valuable.  NASA of course has experience with a wide variety of rockets that have collectively developed many different issues over the years.  These issues could easily be potential problems the manufacturer has not yet thought of or encountered.

But once a rocket has launched many times, and the manufacturer has addressed the potential problems found in prior NASA reviews, it would seem the chance of a NASA review catching something the manufacturer missed would be slight.

[matthewkantar]

The reviewers get to keep their jobs, a prime goal in government.

[edzieba]

But once a rocket has launched many times, and the manufacturer has addressed the potential problems found in prior NASA reviews, it would seem the chance of a NASA review catching something the manufacturer missed would be slight.

That assumes the vehicle and its operations are static, which is not the case with Falcon 9.

[steveleach]

It would be interesting to know what these reviews uncover.  In theory another set of eyes on the data can uncover things the developers may have missed.

One could imagine that the initial few reviews of a new rocket type would be the most valuable.  NASA of course has experience with a wide variety of rockets that have collectively developed many different issues over the years.  These issues could easily be potential problems the manufacturer has not yet thought of or encountered.

But once a rocket has launched many times, and the manufacturer has addressed the potential problems found in prior NASA reviews, it would seem the chance of a NASA review catching something the manufacturer missed would be slight.

I'm not sure it matters how slight the risk is - the owner of the risk (in this case NASA) is going to want to assess any new data themselves. Or at least, review any assessment by the launch provider. Even if just to see that there isn't any significant new data. I would if I were them.

But that's not really what this thread is about.

[woods170]

I find it unfortunate that they have to analyse every prior Falcon launch. Of course I understand why (if there was an indicator of a possible issue that wasn’t analysed and anything ever happened there’d be hell to pay).

But they have way more data on Falcon than any other launch vehicle NASA uses. They have also analysed all prior FH launches. It saddens me that data from 71 prior launches this year wouldn’t be enough and it would have to be 72 if Starlink 6-22 launches.

It has nothing to do with the amount of data.  NASA wants the data of the last mission before theirs.

Delaying that "last mission before theirs" has happened before, in which case NASA does not get the data of the last planned mission before theirs, but of the last actually flown mission before theirs. Which are two different things. This entire discussion going on in this thread revolves around the fallacy of believing that you might miss an issue that crops up conveniently "only" in the last planned mission before an important NASA mission:

https://www.space.com/spacex-delays-starlink-launch-crew-5-astronaut-mission

The fact that NASA has now allowed two Starlink missions, which were originally scheduled to fly BEFORE important NASA missions, to be delayed until AFTER those important NASA missions, goes to show that NASA is fine with using older data to scrutinize for potential issues. If NASA believed in the fallacy mentioned above, they would not have allowed SpaceX to postpone those two Starlink missions beyond the NASA missions.

[LouScheffer]

But once a rocket has launched many times, and the manufacturer has addressed the potential problems found in prior NASA reviews, it would seem the chance of a NASA review catching something the manufacturer missed would be slight.

I'm not sure it matters how slight the risk is - the owner of the risk (in this case NASA) is going to want to assess any new data themselves. Or at least, review any assessment by the launch provider. Even if just to see that there isn't any significant new data. I would if I were them.

I agree.  But at some point it, the best way to improve safety (at least for a given employee count) would likely be to work on improvements to known small risks, as opposed to redundantly scouring the data for newly developing risks.  After all, the manufacturer already has high incentives to spot problems that may be developing.

However, we are far from this point.  The FAA, for example, carefully looks at the data from the first N flights, then lets the manufacturer and user handle the data review after that.  However N is huge - "Altogether, Boeing’s fleet of 787 test planes has logged more than 4,800 hours in more than 1,700 flights."  Furthermore, these tests are designed to explore the corners of the flight envelope, whereas rockets always attempt to launch in the most favorable conditions they can find.

So I don't think NASA will give up reviews any time soon.  At most they will do what folks here have suggested - only doing a quick pass review of previous flights that are too close in time to do a full review.

[RedLineTrain]

It would be interesting to know what these reviews uncover.  In theory another set of eyes on the data can uncover things the developers may have missed.

One could imagine that the initial few reviews of a new rocket type would be the most valuable.  NASA of course has experience with a wide variety of rockets that have collectively developed many different issues over the years.  These issues could easily be potential problems the manufacturer has not yet thought of or encountered.

But once a rocket has launched many times, and the manufacturer has addressed the potential problems found in prior NASA reviews, it would seem the chance of a NASA review catching something the manufacturer missed would be slight.

I'm not sure it matters how slight the risk is - the owner of the risk (in this case NASA) is going to want to assess any new data themselves. Or at least, review any assessment by the launch provider. Even if just to see that there isn't any significant new data. I would if I were them.

But that's not really what this thread is about.

I think that it's really what this thread is about.

1.  SpaceX also owns the risk here and it was arguably higher than NASA's on this flight.
2.  I doubt that the insurers reviewed the data for Viasat's recent Falcon Heavy launch.
3.  I doubt that a NASA data review at this point increases the chance of success or NASA's understanding of the chance of success.  Maybe a couple of years ago it did.

[meekGee]

But once a rocket has launched many times, and the manufacturer has addressed the potential problems found in prior NASA reviews, it would seem the chance of a NASA review catching something the manufacturer missed would be slight.

That assumes the vehicle and its operations are static, which is not the case with Falcon 9.

That doesn't change the fact that delaying the "less important" launch till after the high value launch only reduces the amount of data the high value launch has to work with.

This is purely and entirely a CYA exercise.

[steveleach]

It would be interesting to know what these reviews uncover.  In theory another set of eyes on the data can uncover things the developers may have missed.

One could imagine that the initial few reviews of a new rocket type would be the most valuable.  NASA of course has experience with a wide variety of rockets that have collectively developed many different issues over the years.  These issues could easily be potential problems the manufacturer has not yet thought of or encountered.

But once a rocket has launched many times, and the manufacturer has addressed the potential problems found in prior NASA reviews, it would seem the chance of a NASA review catching something the manufacturer missed would be slight.

I'm not sure it matters how slight the risk is - the owner of the risk (in this case NASA) is going to want to assess any new data themselves. Or at least, review any assessment by the launch provider. Even if just to see that there isn't any significant new data. I would if I were them.

But that's not really what this thread is about.

I think that it's really what this thread is about.

1.  SpaceX also owns the risk here and it was arguably higher than NASA's on this flight.
2.  I doubt that the insurers reviewed the data for Viasat's recent Falcon Heavy launch.
3.  I doubt that a NASA data review at this point increases the chance of success or NASA's understanding of the chance of success.  Maybe a couple of years ago it did.

The thread is about delaying a Starlink launch until after a NASA launch so that you can say the data is not available, rather than not having the time to review it. You don't use the data either way.

It is not about whether it is worth reviewing it if it is available and you have the time.