Author Topic: NASA Releases COTS Final Report: Commercial Orbital Transportation Services, A N  (Read 16169 times)

Offline Tea Party Space Czar

  • President, Tea Party in Space
  • Full Member
  • ***
  • Posts: 349
  • TEA Party in Space Czar
  • Washington DC
  • Liked: 214
  • Likes Given: 257
NASA has released the final report on COTS:  Commercial Orbital Transportation Services, A New Era in Spaceflight

Download it here:  http://www.nasa.gov/sites/default/files/files/SP-2014-617.pdf

It is an amazing read.  I will go on the record and say this is the best report issued by NASA since the Commercial Market Assessment for Crew and Cargo Systems released in March, 2011.  The only thing that mattered in that 40 page report was the last page and Appendix B. 

However, this new report is so damning of the FAR and traditional contracting methodology.  Even better is that it is all sourced.  Its an amazing document.  Chapter 4 is required reading for all policy wonks and people who really give a damn about how you want to run a program. 

Every staff member on the Science Committee in the House and Commerce, Justice, and Science in the Senate, as well as the appropriators, need to read this report from bow to stern... with particular emphasis on Chapter 4.

Read Chapter 4 - Mandatory stuff.  Wow... Like really powerful stuff.

Respectfully,
Andrew Gasser

Quote from: Commercial Orbital Transportation Services, A New Era in Spaceflight
The COTS program aimed to maximize the use of government funds in order to stimulate the U.S. commercial space transportation sector. Of the $500 million originally allocated in 2006, C3PO designated only 3 percent for program management, leaving 97 percent, or about $485 million, to give directly to the commercial partners.

Quote from: Commercial Orbital Transportation Services, A New Era in Spaceflight
...C3PO increased the percentage of the budget withheld for program management from 3 percent to about 5 percent to accommodate the additional development time needed for NASA’s new partner. C3PO Manager Lindenmoyer reported that a traditionally run NASA Program may dedicate 10 to 15 percent of its budget to overhead, so even at the increased 5 percent level, COTS still represented a significant cost savings when compared to traditional NASA programs with markedly higher management costs, and budgets in the range of tens of billions of dollars.

Love Space Act Agreements.  They work every time they are tried.

The appendix showing milestones is just amazing (Page 112).  AMAZING. 

Offline Helodriver

  • Full Member
  • ****
  • Posts: 984
  • Liked: 5445
  • Likes Given: 577
Thanks for the link, will be studying this one closely.

Offline john smith 19

  • Senior Member
  • *****
  • Posts: 7330
  • Everyplaceelse
  • Liked: 1073
  • Likes Given: 7197
NASA has released the final report on COTS:  Commercial Orbital Transportation Services, A New Era in Spaceflight

Download it here:  http://www.nasa.gov/sites/default/files/files/SP-2014-617.pdf
AIUI the case against SAA's is that NASA does not have the right to compel changes to hardware in the way that FAR gives them.

What I've never understood is that it always seems to be set up as either SAA or cost plus (or one of its variants).

But the USG decides what kinds of contract it will use and what their scope is.

I don't get why it cannot devise something that's a sort of "SAA+"  :(

But this won't be the first time the complexities of your government behavior baffle me.
BFS. The worlds first Methane fueled FFORSC engined CFRP structured A380 sized aerospaceplane tail sitter capable of flying in Earth and Mars atmospheres. BFR. The worlds biggest Methane fueled FFORSC engined CFRP structured booster for BFS. First flight to Mars by end of 2022. Forward looking statements. T&C apply. Believe no one. Run your own numbers. So, you are going to Mars to start a better life? Picture it in your mind. Now say what it is out loud.

Offline QuantumG

  • Senior Member
  • *****
  • Posts: 8485
  • Australia
  • Liked: 3405
  • Likes Given: 799
AIUI the case against SAA's is that NASA does not have the right to compel changes to hardware in the way that FAR gives them.

.. and there's no need to. The customer-supplier relationship does that.
Jeff Bezos has billions to spend on rockets and can go at whatever pace he likes! Wow! What pace is he going at? Well... have you heard of Zeno's paradox?

Offline john smith 19

  • Senior Member
  • *****
  • Posts: 7330
  • Everyplaceelse
  • Liked: 1073
  • Likes Given: 7197
AIUI the case against SAA's is that NASA does not have the right to compel changes to hardware in the way that FAR gives them.

.. and there's no need to. The customer-supplier relationship does that.
I'd agree, but I can (sort of) see NASA's PoV.  :(

While no one wants to do a bad job the worst case scenario is that they and the supplier disagree on a point and the NASA guy is right.. Under FAR25 they have the last word and the suppliers change costs are met by NASA. The fear (I think) is under an SAA NASA (reluctantly) flies the vehicle and things go badly wrong.  :(

However this ignores a few small details about NASA's ability to control the situation.

1)NASA is both the customer and the certifying authority that the vehicle is suitable to take their astronauts (IIRC the FAA covers non NASA "spaceflight participants" and non NASA astronauts).

2) NASA could simply refuse to certify the vehicle safe to fly. AFAIK that could not stop non NASA crews flying, but I think it would make private (paying) customers very wary of getting on board.

The problem with acquiring such a big stick (the line engineers in design and production may not be that worried about such matters but their management will definitely pay attention) is that it only really works if there is a serious alternative that can fly instead.

And of course there is the substantially lower cost of SAA "program management" over FAR25 rules.

So paradoxically a situation where NASA cannot compel changes (but does have more than one vehicle  design to choose from) can result in a faster, safer  programme  than one where all the money goes to one supplier.

What I'm not sure is the extent to which (under SAA) the developers can ask the NASA staff on site "How do you prefer this done?" or "What don't you like about us doing it this way?"

BFS. The worlds first Methane fueled FFORSC engined CFRP structured A380 sized aerospaceplane tail sitter capable of flying in Earth and Mars atmospheres. BFR. The worlds biggest Methane fueled FFORSC engined CFRP structured booster for BFS. First flight to Mars by end of 2022. Forward looking statements. T&C apply. Believe no one. Run your own numbers. So, you are going to Mars to start a better life? Picture it in your mind. Now say what it is out loud.

Online ChrisWilson68

  • Senior Member
  • *****
  • Posts: 3625
  • Sunnyvale, CA
  • Liked: 2204
  • Likes Given: 2804
While no one wants to do a bad job the worst case scenario is that they and the supplier disagree on a point and the NASA guy is right.. Under FAR25 they have the last word and the suppliers change costs are met by NASA.

And under FAR the worst case scenario is that NASA and the supplier disagree, the supplier is right, but NASA forces their choice and people die because of it.

Both NASA and the supplier want the same thing -- safety and low cost.  Why assume a government employee is more likely to be right than an employee of a private company?

Offline QuantumG

  • Senior Member
  • *****
  • Posts: 8485
  • Australia
  • Liked: 3405
  • Likes Given: 799
No-one is going to say "gee NASA, although you're our biggest customer, we just refuse to do it your way. Take it or leave it." If NASA chooses to leave it, they're out of business.

On the other hand, if NASA isn't the biggest customer, why require things be done their way?

It's just bureaucratic boohooey.
Jeff Bezos has billions to spend on rockets and can go at whatever pace he likes! Wow! What pace is he going at? Well... have you heard of Zeno's paradox?

Offline john smith 19

  • Senior Member
  • *****
  • Posts: 7330
  • Everyplaceelse
  • Liked: 1073
  • Likes Given: 7197
No-one is going to say "gee NASA, although you're our biggest customer, we just refuse to do it your way. Take it or leave it." If NASA chooses to leave it, they're out of business.

On the other hand, if NASA isn't the biggest customer, why require things be done their way?

It's just bureaucratic boohooey.
That's the fear of the SAA. Under an SAA a supplier can do that and if (as the Legislature seem very keen on doing) its a down select to one supplier NASA has no reply to that. IMHO that's one good reason for retaining 2 suppliers and (ideally) certifying all designs (if acceptable) to fly to the ISS.

IRL I think the relationship between NASA's on site team and the supplier is likely to be fairly cordial and not adversarial, and most of the time disputes will be resolved quite readily.

Sadly one thing this report does not seem to cover is what happened on CCiCAP, which was under FAR, and how its results compared with COTS.  :(
« Last Edit: 06/04/2014 08:24 AM by john smith 19 »
BFS. The worlds first Methane fueled FFORSC engined CFRP structured A380 sized aerospaceplane tail sitter capable of flying in Earth and Mars atmospheres. BFR. The worlds biggest Methane fueled FFORSC engined CFRP structured booster for BFS. First flight to Mars by end of 2022. Forward looking statements. T&C apply. Believe no one. Run your own numbers. So, you are going to Mars to start a better life? Picture it in your mind. Now say what it is out loud.

Online ChrisWilson68

  • Senior Member
  • *****
  • Posts: 3625
  • Sunnyvale, CA
  • Liked: 2204
  • Likes Given: 2804
Sadly one thing this report does not seem to cover is what happened on CCiCAP, which was under FAR, and how its results compared with COTS.  :(

CCiCap used Space Act Agreements, not FAR.

Offline woods170

  • IRAS fan
  • Senior Member
  • *****
  • Posts: 8126
  • IRAS fan
  • The Netherlands
  • Liked: 4226
  • Likes Given: 1291
Sadly one thing this report does not seem to cover is what happened on CCiCAP, which was under FAR, and how its results compared with COTS.  :(
So far all phases of Commercial Crew applied SAA. That will not be the case for the next phase: CCtCap. That one will use FAR.

Offline pippin

  • Regular
  • Senior Member
  • *****
  • Posts: 2555
  • Liked: 289
  • Likes Given: 39
I don't get why it cannot devise something that's a sort of "SAA+"  :(

I think you are somewhat missing the point of the methodology.
That "+" is what causes the overhead cost. The program management overhead is low for a reason with SAAs and the reason is that the customer doesn't need to understand the exact details of the implementation.

If you want to mandate a change as the customer you need to be sure you understand what you are doing because otherwise you are adding risk to the mission. And this "understanding" doesn't work just by the infinite wisdom of NASA's engineers and managers, it requires documentation, staff to read and write the documentation, communication about the underlying concepts and so on.
Even though a lot of this is probably being done right now, too, you need a different level if you really want to move decisions somewhere else.

And then comes the responsibility layer and in the end the lawyers and so on....

No, you can't have less bureaucracy with the same influence, the means of cost reduction is exactly that you put all the responsibility on the supplier and let them do their thing and if they screw up it's their problem.


That said... there is one big caveat with this kind of approach and that is that it only works if you can really get rid of the responsibility. It's quite easy to do if the value of the stuff you are sending up is moderate or low in comparison to the transportation cost.
But there are situation where NASA will not be able to rid itself of the responsibility. As soon as NASA is not just the customer but might also be a supplier (as in science mission where NASA has to meet objectives for the science community), if a lot of money gets sunk if a mission fails or if the life of NASA's astronauts is at stake the situation is a different one.
In such a case NASA WILL need more oversight and the overhead cost will increase, no matter how the business model looks like.
It's not enough to say "I demand that this flight be safe", if it isn't, you've got a problem, you can't get out of that responsibility as a government customer.

There is no golden bullet that solves all problems in the world just by clever contracting.
« Last Edit: 06/04/2014 12:41 PM by pippin »

Offline Jim

  • Night Gator
  • Senior Member
  • *****
  • Posts: 32197
  • Cape Canaveral Spaceport
  • Liked: 10849
  • Likes Given: 321

What I've never understood is that it always seems to be set up as either SAA or cost plus (or one of its variants).


It isn't.  NASA uses a lot of firm fixed price contracts, for spacecraft, launch services, processing supports, etc.

CRS is firm fixed price. 

I don't understand why everybody thinks NASA uses cost plus and why SAA's are the only answer.
« Last Edit: 06/04/2014 12:44 PM by Jim »

Offline Go4TLI

  • Full Member
  • ****
  • Posts: 816
  • Liked: 94
  • Likes Given: 0
While no one wants to do a bad job the worst case scenario is that they and the supplier disagree on a point and the NASA guy is right.. Under FAR25 they have the last word and the suppliers change costs are met by NASA.

And under FAR the worst case scenario is that NASA and the supplier disagree, the supplier is right, but NASA forces their choice and people die because of it.

Both NASA and the supplier want the same thing -- safety and low cost.  Why assume a government employee is more likely to be right than an employee of a private company?

I think you are spinning this way too much in favor of a point of you prefer and doing a disservice to everything else as a consequence. 

Nobody suggested "NASA always knows best".  Your answer is a worse case possibility in any customer/supplier relationship and almost under any contracting model.   

Offline Go4TLI

  • Full Member
  • ****
  • Posts: 816
  • Liked: 94
  • Likes Given: 0

What I've never understood is that it always seems to be set up as either SAA or cost plus (or one of its variants).


It isn't.  NASA uses a lot of firm fixed price contracts, for spacecraft, launch services, processing supports, etc.

CRS is firm fixed price. 

I don't understand why everybody thinks NASA uses cost plus and why SAA's are the only answer.

I tend to agree.  It is frustrating to see the ignorance repeated time and time again around here. Various contracting mechanisms are appropriate and depend on the situation at hand.  Ultimately it is the customers choice on the type of contract they want depending on how much influence they want and how likely the scope of the contract is expected to be tweaked, etc.

These types of contract mechanisms are not unique to the government and are used throughout industry as well.

Offline woods170

  • IRAS fan
  • Senior Member
  • *****
  • Posts: 8126
  • IRAS fan
  • The Netherlands
  • Liked: 4226
  • Likes Given: 1291
I don't understand why everybody thinks NASA uses cost plus and why SAA's are the only answer.

Well, I wouldn't say everybody but I'll admit there is a large bunch of ill-informed folks around. Even on this forum lot's of people seem to think that FAR equates to cost plus (which it doesn't btw).

Offline Sesquipedalian

  • Whee!
  • Full Member
  • ****
  • Posts: 631
  • Liked: 178
  • Likes Given: 534
While no one wants to do a bad job the worst case scenario is that they and the supplier disagree on a point and the NASA guy is right.. Under FAR25 they have the last word and the suppliers change costs are met by NASA.

And under FAR the worst case scenario is that NASA and the supplier disagree, the supplier is right, but NASA forces their choice and people die because of it.

Both NASA and the supplier want the same thing -- safety and low cost.  Why assume a government employee is more likely to be right than an employee of a private company?

I think you are spinning this way too much in favor of a point of you prefer and doing a disservice to everything else as a consequence. 

Nobody suggested "NASA always knows best".  Your answer is a worse case possibility in any customer/supplier relationship and almost under any contracting model.

You utterly failed to grasp the point of that post.  ChrisWilson68 was responding to a poster who was trying to argue on the basis of worst case scenario.  ChrisWilson68's point was that you could just as easily argue the opposite worst case scenario, and so the distinction was meaningless.

Offline yg1968

  • Senior Member
  • *****
  • Posts: 9550
  • Liked: 1321
  • Likes Given: 842
I don't understand why everybody thinks NASA uses cost plus and why SAA's are the only answer.

Well, I wouldn't say everybody but I'll admit there is a large bunch of ill-informed folks around. Even on this forum lot's of people seem to think that FAR equates to cost plus (which it doesn't btw).

SAAs may not be the only answer. But they are the best answer for public-private partnerships. Going to FAR for CCtCap is a mistake and is only being done because of pressure from Congress. FAR contracts allow NASA to have much more oversight which can lead to additional costs because of back and forth changes. It's obvious that the lessons from COTS haven't been learned.

See this post, for a specific example of this issue:
http://forum.nasaspaceflight.com/index.php?topic=28699.msg1196645#msg1196645
« Last Edit: 06/04/2014 02:46 PM by yg1968 »

Offline john smith 19

  • Senior Member
  • *****
  • Posts: 7330
  • Everyplaceelse
  • Liked: 1073
  • Likes Given: 7197
CCiCap used Space Act Agreements, not FAR.
I did not know this. It was my distinct impression that CCiCAP (IE producing the certification documentation for the various systems) was going to be FAR.

I think you are somewhat missing the point of the methodology.
That "+" is what causes the overhead cost. The program management overhead is low for a reason with SAAs and the reason is that the customer doesn't need to understand the exact details of the implementation.
You're confusing my notation with intent.

AIUI SAA's constrain what NASA staff can and cannot tell contractor staff to stop NASA "designing" the product.

It seemed to me that something where a NASA monitor could answer questions and comment on the contractor's approach (when asked) might avoid situations where the contractor thought their approach was compliant but was floored.

I just called it SAA+ as being something more than SAA but rather less than the full FAR25 process.
BFS. The worlds first Methane fueled FFORSC engined CFRP structured A380 sized aerospaceplane tail sitter capable of flying in Earth and Mars atmospheres. BFR. The worlds biggest Methane fueled FFORSC engined CFRP structured booster for BFS. First flight to Mars by end of 2022. Forward looking statements. T&C apply. Believe no one. Run your own numbers. So, you are going to Mars to start a better life? Picture it in your mind. Now say what it is out loud.

Offline pippin

  • Regular
  • Senior Member
  • *****
  • Posts: 2555
  • Liked: 289
  • Likes Given: 39
Well, I haven't worked with NASA but it would be quite unusual if NASA and the contractor could not _talk_ about what they are doing/expecting when they see the need to.
Yes, I've seen cases like that with contractors but that's usually relationships where you do one contract and then you part ways...

Offline yg1968

  • Senior Member
  • *****
  • Posts: 9550
  • Liked: 1321
  • Likes Given: 842
CCiCap used Space Act Agreements, not FAR.
I did not know this. It was my distinct impression that CCiCAP (IE producing the certification documentation for the various systems) was going to be FAR.

CCiCap is under SAAs. CPC (part I of certification) is under FAR and is a program that is parallel to CCiCap. Under CPC, companies must produce a plan for certification which will occur under CCtCap (phase 2 of certification plus test flights).
« Last Edit: 06/04/2014 06:46 PM by yg1968 »