Human vs. Autonomous Testing of Commercial Spacecraft

[Rocket Science]

In light of the fatal accident of SpaceShip 2, I suggested on that perhaps we should consider autonomous or even remote testing of such new vehicles. Setting aside the X-15 program, all of NASA’s vehicles were tested unmanned until the first flight of Columbia. Will another fatal accident cause irreparable damage to the commercialization of spaceflight? Does your opinion change for sub-orbital compared to orbital flights?

[john smith 19]

Not sure if this would be better in the SS2 thread but here's one professional's take on the risk

Newsgroups: rec.aviation.military
From: [email protected] (Mary Shafer)
Subject: Re: QUOTES- Aviation related
Date: Tue, 16 Jan 1996 22:03:05 GMT

On Thu, 11 Jan 1996 15:53:28 -0800, Brian varine <[email protected]> said:

Brian> I remember ROTFL on a post a while back where Mary Shafer said
Brian> something like

Brian> "Absolute safety is for those people without the balls to
Brian> accept reality" or something to that effect.

Brian> How about correcting me on this one?

I wrote, "Insisting on absolute safety is for people who don't have
the balls to live in the real world."

It appeared on sci.space or sci.space.shuttle in 1989 or 1990 during
one of the cyclical "why did NASA blow up the shuttle" threads or on
rec.military during a "how dare those pilots crash the taxpayers'
airplanes" thread, also a cyclical thread.  I had gotten to a point of
complete exasperation when I wrote this.

Here's the whole thing:

   But, no matter what you do, it will never be perfectly, 100% risk-free
   to fly.  Or to drive, or to walk, or to do anything.

   One of our pilots here died when he waited too long to eject from a
   spinning aircraft.  He was wrong; he should have jumped out earlier.
   He failed in his duty, IMO.

   One of our engineers was walking his dog when a car driven by a kid
   jumped the curb and hit him.  Only his leg was broken.  But he walks
   his dog again, now.  Who know better than him the danger?

   There's no way to make life perfectly safe; you can't get out of it alive.

   You can't even predict every danger.  How can you guard against a hazard
   you can't even conceive of?

   I agree that the days of "kick the tires and light the fires" are gone,
   but insisting on perfect safety is the single most reliable way of
   killing an aerospace project.

   I've been on both sides of the FRR (Flight Readiness Review) process
   for a number of aeronautical projects.  Experienced engineers try to
   think of everything that can go wrong.  But airplanes can still
   surprise the best team.

   I've had to sign a form, certifying that to the best of my knowledge
   everything that we're going to do on a flight is safe.  I've never
   seriously asked myself "What will I say to the AIB (Accident
   Investigation Board)" because once one starts on that, the form will
   never be signed, the flight will never be flown, and the research will
   never be done.

   But I have asked myself "Have I told everybody exactly what we're
   going to do and what the _known_ risks are and are we agreed that
   these risks are acceptable" and when I can answer that "yes" I sign
   the form.  That also answers the question of what I'd say to the AIB.

   I'm not talking about abstract theories here, I'm talking about test
   pilots that I've known for decades.  Believe me, I _know_ exactly what
   the consequences of a mistake on my part could mean.  The reminders
   are all around me: Edwards AFB—killed in the XB-49, Lilly
   Ave—first NASA pilot killed at what's now Dryden, Love Rd—I _saw_
   Mike's burning F-4 auger into the lakebed, with him in it.  But once
   I've done my best, like everybody else on the team, it's time to go
   fly the airplane.

   Insisting on perfect safety is for people who don't have the balls to
   live in the real world.

—
Mary Shafer               NASA Dryden Flight Research Center, Edwards, CA
SR-71 Flying Qualities Lead Engineer     Of course I don't speak for NASA
[email protected]                               DoD #362 KotFR 
URL http://www.dfrc.nasa.gov/People/Shafer/mary.html.

For us this is an academic exercise. For her these were people with faces, families and lives that had ended. BTW in another post she mentioned that the pilot loss rate at Dryden was about 1 pilot a year throughout the decade.  The question is always had everything possible been done to minimize the risk?

Test flying has become safer, it will never (by its nature) be safe.

I'll note that XCOR has to start flying before it can worry about having a crash, and I hope that will start happening soon  I'll wish Jeff, Doug and Oleta good luck with that. 

[QuantumG]

Or to say the same thing another way: there's always more you could have done to prevent an accident, but at some point you have to fly.

[pagheca]

I imagined the topic to be debate in this thread is: pro and cons of human vs. autonomous/remote testing of commercial spacecraft. It's not which is the acceptable safety limit of human testing.

[Rocket Science]

For example... It is generally accepted that you risk the least amount of personnel...

[QuantumG]

For example... It is generally accepted that you risk the least amount of personnel...

I really wish you'd speak in complete sentences. You risk the least amount of personnel to do what? You can risk no personnel by doing nothing - just send 'em all home. If your goal is to build a fighter plane before the war is over you're likely to be willing to accept more risks, and find more volunteers willing to take those risks, than if you're trying to build a new board game or whatever. Nothing ventured, nothing gained.

[Avron]

Or to say the same thing another way: there's always more you could have done to prevent an accident, but at some point you have to fly.

The best way is to learn.. fly, there is always a branch of the fault tree that will be unexpected..  however, that fly does not mean it cannot be remote or automated..  we have had autopilots for some time now..

[sdsds]

We are approaching the point where purely autonomous or at least remotely-operated vehicles might be developed and tested at a cost not significantly more than the cost of developing human-operated vehicles. Obviously SpaceX took this approach with Dragon.

I believe this approach could be taken for sub-orbital flight with a vehicle design and test plan similar to that of the Xcor Lynx. But the cost trades are at best unclear.

And then there's the perception issue. How long will the "Spam in a can" comment continue to haunt us?

[Rocket Science]

For example... It is generally accepted that you risk the least amount of personnel...

I really wish you'd speak in complete sentences. You risk the least amount of personnel to do what? You can risk no personnel by doing nothing - just send 'em all home. If your goal is to build a fighter plane before the war is over you're likely to be willing to accept more risks, and find more volunteers willing to take those risks, than if you're trying to build a new board game or whatever. Nothing ventured, nothing gained.

During flight test and aircraft analogies don't apply here...This is for spacecraft.

[QuantumG]

We are approaching the point where purely autonomous or at least remotely-operated vehicles might be developed and tested at a cost not significantly more than the cost of developing human-operated vehicles. Obviously SpaceX took this approach with Dragon.

SpaceX has spent millions developing Dragon and they'll spend over a billion before anyone flies on it. How is that "not significantly more"?

[Space Ghost 1962]

I think as everything is FBW now, the cost/risk of NOT doing autonomous first for manned SC, then phasing in manned tests,  then qualified manned flight with fallback to autonomous in the case of incident is moot.

Please also consider the cost and degree of testing (to destruction) that autonomous first allows. You can also get started with it far sooner. Sooner is cheaper/safer.

[sdsds]

How is that "not significantly more"?

It's a subtle point. I don't think you are suggesting that if the cargo on Dragon Flight 1 had included a human instead of a wheel of cheese then the costs SpaceX incurred to reach that point would have been less. Are you suggesting there's something a human pilot of Dragon Flight 1 could have provided which would have allowed them to fly the mission at a lower cost? I believe Vostok also flew first without a pilot. So is Mercury the comparison? It needed a pilot, and eliminating that need would have been a hit to development both in terms of cost and schedule....

I think as everything is FBW now

Not everything reaching space is fly-by-wire now. At least it wasn't ten years ago when SpaceShipOne flew.

[QuantumG]

It's a subtle point. I don't think you are suggesting that if the cargo on Dragon Flight 1 had included a human instead of a wheel of cheese then the costs SpaceX incurred to reach that point would have been less.

I don't think your first test flight would be like that flight if you were going with a crew vehicle from the start. You'd do something like the Orion EFT-1 flight to test the heat shield - a simple suborbital slam into the atmosphere. Then you'd fly the vehicle with crew all the way to the station.

[sdsds]

You'd do something like the Orion EFT-1 flight to test the heat shield - a simple suborbital slam into the atmosphere. Then you'd fly the vehicle with crew all the way to the station.

Ah thanks, now I understand! Of course you are right: having a human pilot in the loop might have greatly decreased the cost of development compared with autonomous operation ... for the portions of the flight related to ISS combined operations!

[Rocket Science]

How is that "not significantly more"?

It's a subtle point. I don't think you are suggesting that if the cargo on Dragon Flight 1 had included a human instead of a wheel of cheese then the costs SpaceX incurred to reach that point would have been less. Are you suggesting there's something a human pilot of Dragon Flight 1 could have provided which would have allowed them to fly the mission at a lower cost? I believe Vostok also flew first without a pilot. So is Mercury the comparison? It needed a pilot, and eliminating that need would have been a hit to development both in terms of cost and schedule....

I think as everything is FBW now

Not everything reaching space is fly-by-wire now. At least it wasn't ten years ago when SpaceShipOne flew.

Mercury flew unmanned until Ham because he had "The Right Stuff"...

[QuantumG]

Mercury flew unmanned until Ham because he had "The Right Stuff"...

Chimps were flown because the effects of spaceflight on humans wasn't well understood. They didn't actually control anything.

[Rocket Science]

Mercury flew unmanned until Ham because he had "The Right Stuff"...

Chimps were flown because the effects of spaceflight on humans wasn't well understood. They didn't actually control anything.

No, really?

[sdsds]

No, really?

Hah, hah!

You're humorously pointing out that Mercury too flew without pilot control before its first piloted flight. Thanks for correcting my misconception so gently!

I think the point here is the belief held by some that having a pilot in control on early flights shortens the time to initial operating capability, reduces the complexity of the avionics, and thus decreases the costs incurred to reach IOC.

Personally I believe there is another factor which discourages development of autonomous rocket powered vehicles: they look too much like missiles.

[QuantumG]

You're humorously pointing out that Mercury too flew without pilot control before its first piloted flight.

It "flew" the way a stone flies if you throw it, yes.

[yg1968]

The only thing that could have gone wrong was pilot error, and this was apparently a rare case of a well-trained, experienced test pilot doing one thing wrong at the worst possible time, something that can never be fully designed out of a system.

A computer would fully eliminate that possibility.

He discusses why that option was rejected here:

-Have automation question pilot decision.

You know that I am usually on your side with respect to automation capabilities. In this case, the design philosophy of the aircraft (given that it would NOT be able to have hundreds of very incremental test flights during which the automation systems would be wrung out, as these systems WILL have bugs/errors in them) was to have everything possible be manual, non-boosted and non-automated. We made concessions in certain areas where it was not possible to manually control things (pitch control while supersonic, for instance), but in general, the philosophy was to rely on the pilots and intensive simulation of every failure mode we could think of given the very small number of flights that could be flown, given the cost of flying a glide flight, much less a powered flight.

“Are you sure you want to destroy the plane by deploying at this time?” We have multiple sensors that all agree he’s making big mistake...

Again, if it would be possible, as in the development of a Gulfstream bizjet, to have zillions of incremental test flights, we might have gone that route. But our judgement was that we would be more successful relying on pilots and extremely intensive training, as Scaled had been successful over its 30 year history without a fatality in a test flight.

https://groups.google.com/forum/m/#!msg/cozy_builders/YOnxrd26tXc/3OmYdxvqF_AJ

The post above is right on topic for this thread. My own view is that VG may have to partially automate the unlocking mechanism in order for people to have confidence in their spacecraft in the future. If they suggest that no fix is necessary, people will be worried about the same accident happening again.

[mheney]

They need to do tests in order to get their insurance rates down, if for nothing else.

They also need the tests to get customers.  I've got a ticket on XCOR's vehicle, but I sure as heck don't
want to be on one of the first dozen flights off the assembly line.  My level of adventurousness is
high, but it doesn't go all the way to "test pilot".

You do test flights to characterize a vehicle before that class of vehicle enters operational service.
Saying "there really is too much testing" sounds like uninformed speculation.  You CAN have too
much testing - but we've come nowhere near that situation.

[wjbarnett]

"....requirements are to standards/laws/regulations"

Requirements describe the FUNCTIONS desired by the person or entity who is paying/sponsoring the development project. Complying with standards/laws/regulations is only a PART of that. Requirements is how engineering is performed. Without requirements, you're just playing around to see if something works. Which is absolutely fine, but don't expect someone else to pay for that...

[Vultur]

The only thing that could have gone wrong was pilot error, and this was apparently a rare case of a well-trained, experienced test pilot doing one thing wrong at the worst possible time, something that can never be fully designed out of a system.

A computer would fully eliminate that possibility.

He discusses why that option was rejected here:

-Have automation question pilot decision.

You know that I am usually on your side with respect to automation capabilities. In this case, the design philosophy of the aircraft (given that it would NOT be able to have hundreds of very incremental test flights during which the automation systems would be wrung out, as these systems WILL have bugs/errors in them) was to have everything possible be manual, non-boosted and non-automated. We made concessions in certain areas where it was not possible to manually control things (pitch control while supersonic, for instance), but in general, the philosophy was to rely on the pilots and intensive simulation of every failure mode we could think of given the very small number of flights that could be flown, given the cost of flying a glide flight, much less a powered flight.

“Are you sure you want to destroy the plane by deploying at this time?” We have multiple sensors that all agree he’s making big mistake...

Again, if it would be possible, as in the development of a Gulfstream bizjet, to have zillions of incremental test flights, we might have gone that route. But our judgement was that we would be more successful relying on pilots and extremely intensive training, as Scaled had been successful over its 30 year history without a fatality in a test flight.

https://groups.google.com/forum/m/#!msg/cozy_builders/YOnxrd26tXc/3OmYdxvqF_AJ

The post above is right on topic for this thread. My own view is that VG may have to partially automate the unlocking mechanism in order for people to have confidence in their spacecraft in the future. If they suggest that no fix is necessary, people will be worried about the same accident happening again.

Maybe. But realistically, what are the chances someone would make the same mistake again (assuming it WAS a mistake and not something like, say, the airspeed indicator saying Mach 1.4 instead of Mach 1)? Surely all SpaceShipTwo pilots will be very aware of this accident...