NASA Releases COTS Final Report: Commercial Orbital Transportation Services, A N

[Tea Party Space Czar]

NASA has released the final report on COTS:  Commercial Orbital Transportation Services, A New Era in Spaceflight

Download it here:  http://www.nasa.gov/sites/default/files/files/SP-2014-617.pdf

It is an amazing read.  I will go on the record and say this is the best report issued by NASA since the Commercial Market Assessment for Crew and Cargo Systems released in March, 2011.  The only thing that mattered in that 40 page report was the last page and Appendix B. 

However, this new report is so damning of the FAR and traditional contracting methodology.  Even better is that it is all sourced.  Its an amazing document.  Chapter 4 is required reading for all policy wonks and people who really give a damn about how you want to run a program. 

Every staff member on the Science Committee in the House and Commerce, Justice, and Science in the Senate, as well as the appropriators, need to read this report from bow to stern... with particular emphasis on Chapter 4.

Read Chapter 4 - Mandatory stuff.  Wow... Like really powerful stuff.

Respectfully,
Andrew Gasser

The COTS program aimed to maximize the use of government funds in order to stimulate the U.S. commercial space transportation sector. Of the $500 million originally allocated in 2006, C3PO designated only 3 percent for program management, leaving 97 percent, or about $485 million, to give directly to the commercial partners.

...C3PO increased the percentage of the budget withheld for program management from 3 percent to about 5 percent to accommodate the additional development time needed for NASA’s new partner. C3PO Manager Lindenmoyer reported that a traditionally run NASA Program may dedicate 10 to 15 percent of its budget to overhead, so even at the increased 5 percent level, COTS still represented a significant cost savings when compared to traditional NASA programs with markedly higher management costs, and budgets in the range of tens of billions of dollars.

Love Space Act Agreements.  They work every time they are tried.

The appendix showing milestones is just amazing (Page 112).  AMAZING. 

[Helodriver]

Thanks for the link, will be studying this one closely.

[john smith 19]

NASA has released the final report on COTS:  Commercial Orbital Transportation Services, A New Era in Spaceflight

Download it here:  http://www.nasa.gov/sites/default/files/files/SP-2014-617.pdf

AIUI the case against SAA's is that NASA does not have the right to compel changes to hardware in the way that FAR gives them.

What I've never understood is that it always seems to be set up as either SAA or cost plus (or one of its variants).

But the USG decides what kinds of contract it will use and what their scope is.

I don't get why it cannot devise something that's a sort of "SAA+" 

But this won't be the first time the complexities of your government behavior baffle me.

[QuantumG]

AIUI the case against SAA's is that NASA does not have the right to compel changes to hardware in the way that FAR gives them.

.. and there's no need to. The customer-supplier relationship does that.

[john smith 19]

AIUI the case against SAA's is that NASA does not have the right to compel changes to hardware in the way that FAR gives them.

.. and there's no need to. The customer-supplier relationship does that.

I'd agree, but I can (sort of) see NASA's PoV. 

While no one wants to do a bad job the worst case scenario is that they and the supplier disagree on a point and the NASA guy is right.. Under FAR25 they have the last word and the suppliers change costs are met by NASA. The fear (I think) is under an SAA NASA (reluctantly) flies the vehicle and things go badly wrong. 

However this ignores a few small details about NASA's ability to control the situation.

1)NASA is both the customer and the certifying authority that the vehicle is suitable to take their astronauts (IIRC the FAA covers non NASA "spaceflight participants" and non NASA astronauts).

2) NASA could simply refuse to certify the vehicle safe to fly. AFAIK that could not stop non NASA crews flying, but I think it would make private (paying) customers very wary of getting on board.

The problem with acquiring such a big stick (the line engineers in design and production may not be that worried about such matters but their management will definitely pay attention) is that it only really works if there is a serious alternative that can fly instead.

And of course there is the substantially lower cost of SAA "program management" over FAR25 rules.

So paradoxically a situation where NASA cannot compel changes (but does have more than one vehicle  design to choose from) can result in a faster, safer  programme  than one where all the money goes to one supplier.

What I'm not sure is the extent to which (under SAA) the developers can ask the NASA staff on site "How do you prefer this done?" or "What don't you like about us doing it this way?"

[ChrisWilson68]

While no one wants to do a bad job the worst case scenario is that they and the supplier disagree on a point and the NASA guy is right.. Under FAR25 they have the last word and the suppliers change costs are met by NASA.

And under FAR the worst case scenario is that NASA and the supplier disagree, the supplier is right, but NASA forces their choice and people die because of it.

Both NASA and the supplier want the same thing -- safety and low cost.  Why assume a government employee is more likely to be right than an employee of a private company?

[QuantumG]

No-one is going to say "gee NASA, although you're our biggest customer, we just refuse to do it your way. Take it or leave it." If NASA chooses to leave it, they're out of business.

On the other hand, if NASA isn't the biggest customer, why require things be done their way?

It's just bureaucratic boohooey.

[john smith 19]

No-one is going to say "gee NASA, although you're our biggest customer, we just refuse to do it your way. Take it or leave it." If NASA chooses to leave it, they're out of business.

On the other hand, if NASA isn't the biggest customer, why require things be done their way?

It's just bureaucratic boohooey.

That's the fear of the SAA. Under an SAA a supplier can do that and if (as the Legislature seem very keen on doing) its a down select to one supplier NASA has no reply to that. IMHO that's one good reason for retaining 2 suppliers and (ideally) certifying all designs (if acceptable) to fly to the ISS.

IRL I think the relationship between NASA's on site team and the supplier is likely to be fairly cordial and not adversarial, and most of the time disputes will be resolved quite readily.

Sadly one thing this report does not seem to cover is what happened on CCiCAP, which was under FAR, and how its results compared with COTS. 

[ChrisWilson68]

Sadly one thing this report does not seem to cover is what happened on CCiCAP, which was under FAR, and how its results compared with COTS.  :(

CCiCap used Space Act Agreements, not FAR.

[woods170]

Sadly one thing this report does not seem to cover is what happened on CCiCAP, which was under FAR, and how its results compared with COTS. 

So far all phases of Commercial Crew applied SAA. That will not be the case for the next phase: CCtCap. That one will use FAR.

[pippin]

I don't get why it cannot devise something that's a sort of "SAA+" 

I think you are somewhat missing the point of the methodology.
That "+" is what causes the overhead cost. The program management overhead is low for a reason with SAAs and the reason is that the customer doesn't need to understand the exact details of the implementation.

If you want to mandate a change as the customer you need to be sure you understand what you are doing because otherwise you are adding risk to the mission. And this "understanding" doesn't work just by the infinite wisdom of NASA's engineers and managers, it requires documentation, staff to read and write the documentation, communication about the underlying concepts and so on.
Even though a lot of this is probably being done right now, too, you need a different level if you really want to move decisions somewhere else.

And then comes the responsibility layer and in the end the lawyers and so on....

No, you can't have less bureaucracy with the same influence, the means of cost reduction is exactly that you put all the responsibility on the supplier and let them do their thing and if they screw up it's their problem.


That said... there is one big caveat with this kind of approach and that is that it only works if you can really get rid of the responsibility. It's quite easy to do if the value of the stuff you are sending up is moderate or low in comparison to the transportation cost.
But there are situation where NASA will not be able to rid itself of the responsibility. As soon as NASA is not just the customer but might also be a supplier (as in science mission where NASA has to meet objectives for the science community), if a lot of money gets sunk if a mission fails or if the life of NASA's astronauts is at stake the situation is a different one.
In such a case NASA WILL need more oversight and the overhead cost will increase, no matter how the business model looks like.
It's not enough to say "I demand that this flight be safe", if it isn't, you've got a problem, you can't get out of that responsibility as a government customer.

There is no golden bullet that solves all problems in the world just by clever contracting.

[Jim]

What I've never understood is that it always seems to be set up as either SAA or cost plus (or one of its variants).

It isn't.  NASA uses a lot of firm fixed price contracts, for spacecraft, launch services, processing supports, etc.

CRS is firm fixed price. 

I don't understand why everybody thinks NASA uses cost plus and why SAA's are the only answer.

[Go4TLI]

While no one wants to do a bad job the worst case scenario is that they and the supplier disagree on a point and the NASA guy is right.. Under FAR25 they have the last word and the suppliers change costs are met by NASA.

And under FAR the worst case scenario is that NASA and the supplier disagree, the supplier is right, but NASA forces their choice and people die because of it.

Both NASA and the supplier want the same thing -- safety and low cost.  Why assume a government employee is more likely to be right than an employee of a private company?

I think you are spinning this way too much in favor of a point of you prefer and doing a disservice to everything else as a consequence. 

Nobody suggested "NASA always knows best".  Your answer is a worse case possibility in any customer/supplier relationship and almost under any contracting model.   

[Go4TLI]

What I've never understood is that it always seems to be set up as either SAA or cost plus (or one of its variants).

It isn't.  NASA uses a lot of firm fixed price contracts, for spacecraft, launch services, processing supports, etc.

CRS is firm fixed price. 

I don't understand why everybody thinks NASA uses cost plus and why SAA's are the only answer.

I tend to agree.  It is frustrating to see the ignorance repeated time and time again around here. Various contracting mechanisms are appropriate and depend on the situation at hand.  Ultimately it is the customers choice on the type of contract they want depending on how much influence they want and how likely the scope of the contract is expected to be tweaked, etc.

These types of contract mechanisms are not unique to the government and are used throughout industry as well.

[woods170]

I don't understand why everybody thinks NASA uses cost plus and why SAA's are the only answer.

Well, I wouldn't say everybody but I'll admit there is a large bunch of ill-informed folks around. Even on this forum lot's of people seem to think that FAR equates to cost plus (which it doesn't btw).

[Sesquipedalian]

While no one wants to do a bad job the worst case scenario is that they and the supplier disagree on a point and the NASA guy is right.. Under FAR25 they have the last word and the suppliers change costs are met by NASA.

And under FAR the worst case scenario is that NASA and the supplier disagree, the supplier is right, but NASA forces their choice and people die because of it.

Both NASA and the supplier want the same thing -- safety and low cost.  Why assume a government employee is more likely to be right than an employee of a private company?

I think you are spinning this way too much in favor of a point of you prefer and doing a disservice to everything else as a consequence. 

Nobody suggested "NASA always knows best".  Your answer is a worse case possibility in any customer/supplier relationship and almost under any contracting model.

You utterly failed to grasp the point of that post.  ChrisWilson68 was responding to a poster who was trying to argue on the basis of worst case scenario.  ChrisWilson68's point was that you could just as easily argue the opposite worst case scenario, and so the distinction was meaningless.

[yg1968]

I don't understand why everybody thinks NASA uses cost plus and why SAA's are the only answer.

Well, I wouldn't say everybody but I'll admit there is a large bunch of ill-informed folks around. Even on this forum lot's of people seem to think that FAR equates to cost plus (which it doesn't btw).

SAAs may not be the only answer. But they are the best answer for public-private partnerships. Going to FAR for CCtCap is a mistake and is only being done because of pressure from Congress. FAR contracts allow NASA to have much more oversight which can lead to additional costs because of back and forth changes. It's obvious that the lessons from COTS haven't been learned.

See this post, for a specific example of this issue:
http://forum.nasaspaceflight.com/index.php?topic=28699.msg1196645#msg1196645

[john smith 19]

CCiCap used Space Act Agreements, not FAR.

I did not know this. It was my distinct impression that CCiCAP (IE producing the certification documentation for the various systems) was going to be FAR.

I think you are somewhat missing the point of the methodology.
That "+" is what causes the overhead cost. The program management overhead is low for a reason with SAAs and the reason is that the customer doesn't need to understand the exact details of the implementation.

You're confusing my notation with intent.

AIUI SAA's constrain what NASA staff can and cannot tell contractor staff to stop NASA "designing" the product.

It seemed to me that something where a NASA monitor could answer questions and comment on the contractor's approach (when asked) might avoid situations where the contractor thought their approach was compliant but was floored.

I just called it SAA+ as being something more than SAA but rather less than the full FAR25 process.

[pippin]

Well, I haven't worked with NASA but it would be quite unusual if NASA and the contractor could not _talk_ about what they are doing/expecting when they see the need to.
Yes, I've seen cases like that with contractors but that's usually relationships where you do one contract and then you part ways...

[yg1968]

CCiCap used Space Act Agreements, not FAR.

I did not know this. It was my distinct impression that CCiCAP (IE producing the certification documentation for the various systems) was going to be FAR.

CCiCap is under SAAs. CPC (part I of certification) is under FAR and is a program that is parallel to CCiCap. Under CPC, companies must produce a plan for certification which will occur under CCtCap (phase 2 of certification plus test flights).