My Unsolicited Take on Starship Flight Testing

[Jim]

 SpaceX doesn't have the budget like NASA did in the 60's.  This has led to some choices like flight testing emphasis over ground testing.  Ground testing requires expensive infrastructure.  SpaceX choose to be hardware rich and spend money and time gaining experience and building the vehicle and refining its design. With modern avionics, SpaceX can get more data from more test points on a vehicle in flight than NASA did during ground tests.   NASA had 4 non-flight test articles built for each stage for Saturn V (static fire, structural loads, facilities, and ground dynamics) testing. So isn't it until the fifth (or 7th to include Apollo 4 & 6) Starship launch before a comparison to the Saturn V can be made?

[ulm_atms]

Yes.....all of it.   

[vaporcobra]

Do you mean comparing Starship to Saturn V is unwise in the sense that "Starship isn't that impressive because Saturn V reached orbit on its first launch" ignores the different ways they were developed?

[abaddon]

I don't really see the need to ask questions, the statement is very clear and concise.

[RedLineTrain]

With the size of Starship (ultimately at least ~3x the thrust of Saturn V), it seems like extensive ground testing is nigh on impossible.  Even NASA's Apollo-era budgets, willingness to use governmental powers, and lesser focus on environmental impact might not cover it.

Because of this, I hesitate to compare the Saturn V and Starship.  They're just in different vehicle classes (Moon-class and Mars-class, respectively) and are each a product of their times.  And then Saturn V doesn't have a long enough flight history to know for sure about its ultimate reliability.

[Metalskin]

I've seen people scorn and praise SpaceX's approach and I've never known how to assess when people raise previous programmes.

Thank you Jim, your take really helps me a lot. =Obviously I'm just an anonymous armchair rocket fan, but what you state makes sense.

The only thing I don't know how to factor in is how computer modeling and simulation has impacted the ability to test designs. I imagine that is a huge advantage for SpaceX, but not sure how that would impact testing methodology.

[Brigantine]

SN8 - SN11 and SN15 had no RVacs and no heat shield, so lets consider them as mere component testing. Discount S24 as a 2nd stage test since stage separation never happened, so S25 was the 1st complete stage test of stage 2.

The more I get into the details, the more I agree with OP.
It's bizarre but it seems stage 2 ID#s 1-19, 21-24, 26, 27 all don't even count as equivalent to Saturn V ground test articles.

[Jim]

Do you mean comparing Starship to Saturn V is unwise in the sense that "Starship isn't that impressive because Saturn V reached orbit on its first launch" ignores the different ways they were developed?

Exactly

[kevin-rf]

Somewhere someone added up all the Raptor flight time and in two test flights it has exceeded all Saturn V F1 flight time....

[TrueBlueWitt]

Somewhere someone added up all the Raptor flight time and in two test flights it has exceeded all Saturn V F1 flight time....

Don't forget, Saturn V got very lucky it Survived POGO and probably some other issues on early flights. Even with all the testing.
And it wasn't trying to be re-usable. 

https://www.nasa.gov/history/50-years-ago-solving-the-pogo-effect/

If this was Saturn V, no one would have cared at all if the booster blew up AFTER stage sep. Correct?

[meekGee]

One point of comparison is that (to my knowledge) the several Saturn V development articles represented a design that wasn't evolving as much as SS is.

Meaning: NASA won't typically build a multi stage rocket without really knowing how staging will work.  Or try launching it from a stage 0 that's "probably going to break" just to see if maybe they can get away with it.

The fact that BFR/ITS/SS evolved so much (material of choice, diameter, EDL method) is not unprecedented, because that was mostly on paper.   But the amount that the 9 m stainless SS is evolving (e.g. heat shield method, staging method) - that pretty much is, and warrants the "failures" during the development flights.

[edzieba]

One point of comparison is that (to my knowledge) the several Saturn V development articles represented a design that wasn't evolving as much as SS is.

Meaning: NASA won't typically build a multi stage rocket without really knowing how staging will work.  Or try launching it from a stage 0 that's "probably going to break" just to see if maybe they can get away with it.

The Saturn programme swapped out engines and stages as it progressed. Saturn-I and Saturn-V were entirely different vehicles, despite the progression from one to the other being clear and direct.
As for radical mid-programme changes; at the time Saturn-V was being drawn up, EOR was still the proposed CONOPS. The entire Apollo system architecture changed during vehicle development.

[kevin-rf]

Don't forget as Saturn was evolving into what it became they had concepts for first stage reuse and even went so far as to dunk an engine into the Gulf of Mexico after test firing it.

[meekGee]

One point of comparison is that (to my knowledge) the several Saturn V development articles represented a design that wasn't evolving as much as SS is.

Meaning: NASA won't typically build a multi stage rocket without really knowing how staging will work.  Or try launching it from a stage 0 that's "probably going to break" just to see if maybe they can get away with it.

The Saturn programme swapped out engines and stages as it progressed. Saturn-I and Saturn-V were entirely different vehicles, despite the progression from one to the other being clear and direct.
As for radical mid-programme changes; at the time Saturn-V was being drawn up, EOR was still the proposed CONOPS. The entire Apollo system architecture changed during vehicle development.

Well the assertion being countered was that "Saturn V" flew the first time, so I ignored Saturn 1.

Also for the same reason didn't talk about Starship refiling and the extended mission profile, just basic rocket functionality (which in this case includes EDL)

[Asteroza]

Would Saturn 1 flights count, in the same way Hoppy and early starship hops could count?

[envy887]

SpaceX did build a lot of ground test boosters and ships, and tested all of ground handling, flight dynamics, static firing, and structures with them.

That said, the difference in resources driving different ground test thoroughness and fidelity is an excellent point. NASA spent almost 50 billion on Saturn V (inflation adjusted) before its first successful flight.

[Barley]

SpaceX did build a lot of ground test boosters and ships, and tested all of ground handling, flight dynamics, static firing, and structures with them.

That said, the difference in resources driving different ground test thoroughness and fidelity is an excellent point. NASA spent almost 50 billion on Saturn V (inflation adjusted) before its first successful flight.

Jim did mention modern avionics and telemetry, but I think people are ignoring how much this affects the ability to do inflight testing, and hence how you do testing.

Restricted telemetry requires more ground testing because you can't get as much data from a test flight.  Compare the Saturn V camera pods with sticking on another GoPro, on board image processing to get critical frames and adding that to the 100Mbps of telemetry.  Remember Apollo was closer in time to a Sopwith Camel than a StarShip.

[Yggdrasill]

I completely agree they are products of different times.

At it's peak, the Apollo program had 400,000 people working on it. SpaceX has 12,000 employees, and most of them aren't working on Starship. Though they do have suppliers that increase the number as well. So, maybe something like 10,000? 2.5% of Apollo.

With modern technology, fewer people can do more, but you still have fewer eyes on everything. Some of the mistakes SpaceX has made would probably have been caught in the Apollo program.

[spacenut]

SpaceX cannot do a full thrust test on the ground of all 33 booster engines.  It would break away from the hold down clamps.  Saturn V did not try to save the booster either after staging.  Sure they can test each engine separately, but not all at one time. 

SpaceX is building this rocket on the cheap compared to NASA.  As said 400,000 vs 12,000 employees. 

[jpo234]

SpaceX choose to be hardware rich and spend money and time gaining experience and building the vehicle and refining its design.

NASA never planned to mass produce the Saturn V. SpaceX on the other hand wants to build hundreds or even thousands of Starships and boosters. Being hardware rich comes naturally from this goal.