NASA Releases COTS Final Report: Commercial Orbital Transportation Services, A N

[Tea Party Space Czar]

NASA has released the final report on COTS:  Commercial Orbital Transportation Services, A New Era in Spaceflight

Download it here:  http://www.nasa.gov/sites/default/files/files/SP-2014-617.pdf

It is an amazing read.  I will go on the record and say this is the best report issued by NASA since the Commercial Market Assessment for Crew and Cargo Systems released in March, 2011.  The only thing that mattered in that 40 page report was the last page and Appendix B. 

However, this new report is so damning of the FAR and traditional contracting methodology.  Even better is that it is all sourced.  Its an amazing document.  Chapter 4 is required reading for all policy wonks and people who really give a damn about how you want to run a program. 

Every staff member on the Science Committee in the House and Commerce, Justice, and Science in the Senate, as well as the appropriators, need to read this report from bow to stern... with particular emphasis on Chapter 4.

Read Chapter 4 - Mandatory stuff.  Wow... Like really powerful stuff.

Respectfully,
Andrew Gasser

The COTS program aimed to maximize the use of government funds in order to stimulate the U.S. commercial space transportation sector. Of the $500 million originally allocated in 2006, C3PO designated only 3 percent for program management, leaving 97 percent, or about $485 million, to give directly to the commercial partners.

...C3PO increased the percentage of the budget withheld for program management from 3 percent to about 5 percent to accommodate the additional development time needed for NASA’s new partner. C3PO Manager Lindenmoyer reported that a traditionally run NASA Program may dedicate 10 to 15 percent of its budget to overhead, so even at the increased 5 percent level, COTS still represented a significant cost savings when compared to traditional NASA programs with markedly higher management costs, and budgets in the range of tens of billions of dollars.

Love Space Act Agreements.  They work every time they are tried.

The appendix showing milestones is just amazing (Page 112).  AMAZING. 

[Helodriver]

Thanks for the link, will be studying this one closely.

[john smith 19]

NASA has released the final report on COTS:  Commercial Orbital Transportation Services, A New Era in Spaceflight

Download it here:  http://www.nasa.gov/sites/default/files/files/SP-2014-617.pdf

AIUI the case against SAA's is that NASA does not have the right to compel changes to hardware in the way that FAR gives them.

What I've never understood is that it always seems to be set up as either SAA or cost plus (or one of its variants).

But the USG decides what kinds of contract it will use and what their scope is.

I don't get why it cannot devise something that's a sort of "SAA+" 

But this won't be the first time the complexities of your government behavior baffle me.

[QuantumG]

AIUI the case against SAA's is that NASA does not have the right to compel changes to hardware in the way that FAR gives them.

.. and there's no need to. The customer-supplier relationship does that.

[john smith 19]

AIUI the case against SAA's is that NASA does not have the right to compel changes to hardware in the way that FAR gives them.

.. and there's no need to. The customer-supplier relationship does that.

I'd agree, but I can (sort of) see NASA's PoV. 

While no one wants to do a bad job the worst case scenario is that they and the supplier disagree on a point and the NASA guy is right.. Under FAR25 they have the last word and the suppliers change costs are met by NASA. The fear (I think) is under an SAA NASA (reluctantly) flies the vehicle and things go badly wrong. 

However this ignores a few small details about NASA's ability to control the situation.

1)NASA is both the customer and the certifying authority that the vehicle is suitable to take their astronauts (IIRC the FAA covers non NASA "spaceflight participants" and non NASA astronauts).

2) NASA could simply refuse to certify the vehicle safe to fly. AFAIK that could not stop non NASA crews flying, but I think it would make private (paying) customers very wary of getting on board.

The problem with acquiring such a big stick (the line engineers in design and production may not be that worried about such matters but their management will definitely pay attention) is that it only really works if there is a serious alternative that can fly instead.

And of course there is the substantially lower cost of SAA "program management" over FAR25 rules.

So paradoxically a situation where NASA cannot compel changes (but does have more than one vehicle  design to choose from) can result in a faster, safer  programme  than one where all the money goes to one supplier.

What I'm not sure is the extent to which (under SAA) the developers can ask the NASA staff on site "How do you prefer this done?" or "What don't you like about us doing it this way?"

[ChrisWilson68]

While no one wants to do a bad job the worst case scenario is that they and the supplier disagree on a point and the NASA guy is right.. Under FAR25 they have the last word and the suppliers change costs are met by NASA.

And under FAR the worst case scenario is that NASA and the supplier disagree, the supplier is right, but NASA forces their choice and people die because of it.

Both NASA and the supplier want the same thing -- safety and low cost.  Why assume a government employee is more likely to be right than an employee of a private company?

[QuantumG]

No-one is going to say "gee NASA, although you're our biggest customer, we just refuse to do it your way. Take it or leave it." If NASA chooses to leave it, they're out of business.

On the other hand, if NASA isn't the biggest customer, why require things be done their way?

It's just bureaucratic boohooey.

[john smith 19]

No-one is going to say "gee NASA, although you're our biggest customer, we just refuse to do it your way. Take it or leave it." If NASA chooses to leave it, they're out of business.

On the other hand, if NASA isn't the biggest customer, why require things be done their way?

It's just bureaucratic boohooey.

That's the fear of the SAA. Under an SAA a supplier can do that and if (as the Legislature seem very keen on doing) its a down select to one supplier NASA has no reply to that. IMHO that's one good reason for retaining 2 suppliers and (ideally) certifying all designs (if acceptable) to fly to the ISS.

IRL I think the relationship between NASA's on site team and the supplier is likely to be fairly cordial and not adversarial, and most of the time disputes will be resolved quite readily.

Sadly one thing this report does not seem to cover is what happened on CCiCAP, which was under FAR, and how its results compared with COTS. 

[ChrisWilson68]

Sadly one thing this report does not seem to cover is what happened on CCiCAP, which was under FAR, and how its results compared with COTS.  :(

CCiCap used Space Act Agreements, not FAR.

[woods170]

Sadly one thing this report does not seem to cover is what happened on CCiCAP, which was under FAR, and how its results compared with COTS. 

So far all phases of Commercial Crew applied SAA. That will not be the case for the next phase: CCtCap. That one will use FAR.

[pippin]

I don't get why it cannot devise something that's a sort of "SAA+" 

I think you are somewhat missing the point of the methodology.
That "+" is what causes the overhead cost. The program management overhead is low for a reason with SAAs and the reason is that the customer doesn't need to understand the exact details of the implementation.

If you want to mandate a change as the customer you need to be sure you understand what you are doing because otherwise you are adding risk to the mission. And this "understanding" doesn't work just by the infinite wisdom of NASA's engineers and managers, it requires documentation, staff to read and write the documentation, communication about the underlying concepts and so on.
Even though a lot of this is probably being done right now, too, you need a different level if you really want to move decisions somewhere else.

And then comes the responsibility layer and in the end the lawyers and so on....

No, you can't have less bureaucracy with the same influence, the means of cost reduction is exactly that you put all the responsibility on the supplier and let them do their thing and if they screw up it's their problem.


That said... there is one big caveat with this kind of approach and that is that it only works if you can really get rid of the responsibility. It's quite easy to do if the value of the stuff you are sending up is moderate or low in comparison to the transportation cost.
But there are situation where NASA will not be able to rid itself of the responsibility. As soon as NASA is not just the customer but might also be a supplier (as in science mission where NASA has to meet objectives for the science community), if a lot of money gets sunk if a mission fails or if the life of NASA's astronauts is at stake the situation is a different one.
In such a case NASA WILL need more oversight and the overhead cost will increase, no matter how the business model looks like.
It's not enough to say "I demand that this flight be safe", if it isn't, you've got a problem, you can't get out of that responsibility as a government customer.

There is no golden bullet that solves all problems in the world just by clever contracting.

[Jim]

What I've never understood is that it always seems to be set up as either SAA or cost plus (or one of its variants).

It isn't.  NASA uses a lot of firm fixed price contracts, for spacecraft, launch services, processing supports, etc.

CRS is firm fixed price. 

I don't understand why everybody thinks NASA uses cost plus and why SAA's are the only answer.

[Go4TLI]

While no one wants to do a bad job the worst case scenario is that they and the supplier disagree on a point and the NASA guy is right.. Under FAR25 they have the last word and the suppliers change costs are met by NASA.

And under FAR the worst case scenario is that NASA and the supplier disagree, the supplier is right, but NASA forces their choice and people die because of it.

Both NASA and the supplier want the same thing -- safety and low cost.  Why assume a government employee is more likely to be right than an employee of a private company?

I think you are spinning this way too much in favor of a point of you prefer and doing a disservice to everything else as a consequence. 

Nobody suggested "NASA always knows best".  Your answer is a worse case possibility in any customer/supplier relationship and almost under any contracting model.   

[Go4TLI]

What I've never understood is that it always seems to be set up as either SAA or cost plus (or one of its variants).

It isn't.  NASA uses a lot of firm fixed price contracts, for spacecraft, launch services, processing supports, etc.

CRS is firm fixed price. 

I don't understand why everybody thinks NASA uses cost plus and why SAA's are the only answer.

I tend to agree.  It is frustrating to see the ignorance repeated time and time again around here. Various contracting mechanisms are appropriate and depend on the situation at hand.  Ultimately it is the customers choice on the type of contract they want depending on how much influence they want and how likely the scope of the contract is expected to be tweaked, etc.

These types of contract mechanisms are not unique to the government and are used throughout industry as well.

[woods170]

I don't understand why everybody thinks NASA uses cost plus and why SAA's are the only answer.

Well, I wouldn't say everybody but I'll admit there is a large bunch of ill-informed folks around. Even on this forum lot's of people seem to think that FAR equates to cost plus (which it doesn't btw).

[Sesquipedalian]

While no one wants to do a bad job the worst case scenario is that they and the supplier disagree on a point and the NASA guy is right.. Under FAR25 they have the last word and the suppliers change costs are met by NASA.

And under FAR the worst case scenario is that NASA and the supplier disagree, the supplier is right, but NASA forces their choice and people die because of it.

Both NASA and the supplier want the same thing -- safety and low cost.  Why assume a government employee is more likely to be right than an employee of a private company?

I think you are spinning this way too much in favor of a point of you prefer and doing a disservice to everything else as a consequence. 

Nobody suggested "NASA always knows best".  Your answer is a worse case possibility in any customer/supplier relationship and almost under any contracting model.

You utterly failed to grasp the point of that post.  ChrisWilson68 was responding to a poster who was trying to argue on the basis of worst case scenario.  ChrisWilson68's point was that you could just as easily argue the opposite worst case scenario, and so the distinction was meaningless.

[yg1968]

I don't understand why everybody thinks NASA uses cost plus and why SAA's are the only answer.

Well, I wouldn't say everybody but I'll admit there is a large bunch of ill-informed folks around. Even on this forum lot's of people seem to think that FAR equates to cost plus (which it doesn't btw).

SAAs may not be the only answer. But they are the best answer for public-private partnerships. Going to FAR for CCtCap is a mistake and is only being done because of pressure from Congress. FAR contracts allow NASA to have much more oversight which can lead to additional costs because of back and forth changes. It's obvious that the lessons from COTS haven't been learned.

See this post, for a specific example of this issue:
http://forum.nasaspaceflight.com/index.php?topic=28699.msg1196645#msg1196645

[john smith 19]

CCiCap used Space Act Agreements, not FAR.

I did not know this. It was my distinct impression that CCiCAP (IE producing the certification documentation for the various systems) was going to be FAR.

I think you are somewhat missing the point of the methodology.
That "+" is what causes the overhead cost. The program management overhead is low for a reason with SAAs and the reason is that the customer doesn't need to understand the exact details of the implementation.

You're confusing my notation with intent.

AIUI SAA's constrain what NASA staff can and cannot tell contractor staff to stop NASA "designing" the product.

It seemed to me that something where a NASA monitor could answer questions and comment on the contractor's approach (when asked) might avoid situations where the contractor thought their approach was compliant but was floored.

I just called it SAA+ as being something more than SAA but rather less than the full FAR25 process.

[pippin]

Well, I haven't worked with NASA but it would be quite unusual if NASA and the contractor could not _talk_ about what they are doing/expecting when they see the need to.
Yes, I've seen cases like that with contractors but that's usually relationships where you do one contract and then you part ways...

[yg1968]

CCiCap used Space Act Agreements, not FAR.

I did not know this. It was my distinct impression that CCiCAP (IE producing the certification documentation for the various systems) was going to be FAR.

CCiCap is under SAAs. CPC (part I of certification) is under FAR and is a program that is parallel to CCiCap. Under CPC, companies must produce a plan for certification which will occur under CCtCap (phase 2 of certification plus test flights).

[yg1968]

Well, I haven't worked with NASA but it would be quite unusual if NASA and the contractor could not _talk_ about what they are doing/expecting when they see the need to.
Yes, I've seen cases like that with contractors but that's usually relationships where you do one contract and then you part ways...

There was ways around it. Brent Jett mentionned that MOUs were used under CCDev2 to get around that issue.

[pippin]

That's what I meant. If you have a somewhat decent relationship and you want to do something then you'll find a way...

[ChrisWilson68]

It seemed to me that something where a NASA monitor could answer questions and comment on the contractor's approach (when asked) might avoid situations where the contractor thought their approach was compliant but was floored.

I just called it SAA+ as being something more than SAA but rather less than the full FAR25 process.

My impression was that SAAs already work like that.  Just look at the milestones for COTS and CRS -- many of them are review milestones -- PDRs, CDRs, safety reviews, etc.  Those reviews are events where the contractor gives numerous presentation to dozens of NASA engineers about the details of their plans, and the NASA people can comment and ask any questions they want.  The contractor has to answer all questions to the satisfaction of the NASA folks before the review is considered complete and before they get paid for it.

And in addition to the formal review events, I got the impression that the contractors and the NASA people continue talking.

I think a big part of the reason these SAA programs have been so successful is precisely because the contractors get so much advice from NASA people.

[jongoff]

What I've never understood is that it always seems to be set up as either SAA or cost plus (or one of its variants).

It isn't.  NASA uses a lot of firm fixed price contracts, for spacecraft, launch services, processing supports, etc.

CRS is firm fixed price. 

I don't understand why everybody thinks NASA uses cost plus and why SAA's are the only answer.

Admittedly I used to be in the camp that didn't know you could have FAR firm fixed price contracts. Jim and GO4TLI are right that there's nothing inherent in FAR that requires cost-plus.

On a related note, I've been seeing some rumors flying around the internet that Senator Shelby is inserting additional cost-accounting requirements in with the funding bill for Commercial Crew. The rumors were implying a move to cost-plus, but were sufficiently vague that I'm not confident in that interpretation.

Anyone here that understand gov't contracting (Jim, Go4TLI, others) have any insights into what Senator Shelby added to the bill, and whether it makes any sense or is at all justified?

~Jon

[QuantumG]

I'm pretty sure they want SpaceX employees to punch clocks.. it's another way to slow them down so Boeing can "compete".

[yg1968]

I'm pretty sure they want SpaceX employees to punch clocks.. it's another way to slow them down so Boeing can "compete".

Shelby is doing this to sabotage commercial crew like he always does.

[john smith 19]

My impression was that SAAs already work like that.  Just look at the milestones for COTS and CRS -- many of them are review milestones -- PDRs, CDRs, safety reviews, etc.  Those reviews are events where the contractor gives numerous presentation to dozens of NASA engineers about the details of their plans, and the NASA people can comment and ask any questions they want.  The contractor has to answer all questions to the satisfaction of the NASA folks before the review is considered complete and before they get paid for it.

And in addition to the formal review events, I got the impression that the contractors and the NASA people continue talking.

I think a big part of the reason these SAA programs have been so successful is precisely because the contractors get so much advice from NASA people.

It was more I got the feeling that you could not ask those questions during construction, when in principal any changes would be much cheaper to implement, rather than after the fact in review. 

[peter-b]

As I understand it, the design reviews happen *before* construction. It makes sure that the thing that's going to be constructed meets requirements. Look at the COTS milestones chart for example.

[john smith 19]

Sorry to interrupt but has anyone seen this?

http://www.space-access.org/updates/sasalert060414.html

Shelby is asking for full FAR25 compliant cost reporting, despite CRS being a firm fixed price contract.

I'm not sure if it's too late to do anything about this or if I'm just out of the loop, but it is COTS related.

[yg1968]

Sorry to interrupt but has anyone seen this?

http://www.space-access.org/updates/sasalert060414.html

Shelby is asking for full FAR25 compliant cost reporting, despite CRS being a firm fixed price contract.

I'm not sure if it's too late to do anything about this or if I'm just out of the loop, but it is COTS related.

Yes, I mentionned it in this thread:
http://forum.nasaspaceflight.com/index.php?topic=34827.msg1209848#msg1209848

But you are right, it targets the next rounds of commercial crew (CCtCap) and commercial cargo (CRS2).

[vulture4]

Sorry to interrupt but has anyone seen this?

http://www.space-access.org/updates/sasalert060414.html

Shelby is asking for full FAR25 compliant cost reporting, despite CRS being a firm fixed price contract.

I'm not sure if it's too late to do anything about this or if I'm just out of the loop, but it is COTS related.

Yes, I mentionned it in this thread:
http://forum.nasaspaceflight.com/index.php?topic=34827.msg1209848#msg1209848

But it you are right, it targets the next rounds of commercial crew (CCtCap) and commercial cargo (CRS2).

I think it's reasonably safe to say that Shelby is just using this as a way to attack Commercial Crew by saddling it with higher costs. I've won a few small NASA contracts and the stupidity of it is that today in most cases NASA wants invoices and cost data as though the contract were cost plus even though in reality the total you can ask for is firmly capped. An woe betide the project that doesn't spend all the money on schedule, it makes the contracting officer look bad. But the FAR says that under firm fixed price you get your price and you deliver your product, and your costs are irrelevant.

[john smith 19]

I think it's reasonably safe to say that Shelby is just using this as a way to attack Commercial Crew by saddling it with higher costs. I've won a few small NASA contracts and the stupidity of it is that today in most cases NASA wants invoices and cost data as though the contract were cost plus even though in reality the total you can ask for is firmly capped. An woe betide the project that doesn't spend all the money on schedule, it makes the contracting officer look bad. But the FAR says that under firm fixed price you get your price and you deliver your product, and your costs are irrelevant.

What I find surprising is that (AFAIK) CRS is firm fixed price. No cost plus. so he's wanting to add all that reporting retroactively 

[mlindner]

I think it's reasonably safe to say that Shelby is just using this as a way to attack Commercial Crew by saddling it with higher costs. I've won a few small NASA contracts and the stupidity of it is that today in most cases NASA wants invoices and cost data as though the contract were cost plus even though in reality the total you can ask for is firmly capped. An woe betide the project that doesn't spend all the money on schedule, it makes the contracting officer look bad. But the FAR says that under firm fixed price you get your price and you deliver your product, and your costs are irrelevant.

What I find surprising is that (AFAIK) CRS is firm fixed price. No cost plus. so he's wanting to add all that reporting retroactively 

If that's actually the case (which would be surprising to me) it would probably violate the existing contract if the government tried it. They'd be breaking the law and promptly get a lawsuit.

[Jim]

I've won a few small NASA contracts and the stupidity of it is that today in most cases NASA wants invoices and cost data as though the contract were cost plus even though in reality the total you can ask for is firmly capped.

How can you surmise from a "few" contracts that you have done and apply it to "in most case"?   You have insufficient data to base your claim on, especially on the larger contracts.

[beancounter]

I've won a few small NASA contracts and the stupidity of it is that today in most cases NASA wants invoices and cost data as though the contract were cost plus even though in reality the total you can ask for is firmly capped.

How can you surmise from a "few" contracts that you have done and apply it to "in most case"?   You have insufficient data to base your claim on, especially on the larger contracts.

Apparently Vulture4 has done some.  Fair question, have you or more generally, anyone else on the blog who can add further information?
Thanks

[JohnFornaro]

I've won a few small NASA contracts and the stupidity of it is that today in most cases NASA wants invoices and cost data as though the contract were cost plus even though in reality the total you can ask for is firmly capped.

How can you surmise from a "few" contracts that you have done and apply it to "in most case"?   You have insufficient data to base your claim on, especially on the larger contracts.

Apparently Vulture4 has done some.  Fair question, have you or more generally, anyone else on the blog who can add further information?
Thanks

He probably meant "in most cases" pertaining to the "few small contracts" that he had been awarded.  At least, if English grammar is assumed to be appropriate tool for charitably parsing the meaning of his statement.

However, he did say "today in most cases", which would imply knowledge about other cases than his own.

A clarification from Vulture4 would be helpful.

[JohnFornaro]

What I've never understood is that it always seems to be set up as either SAA or cost plus (or one of its variants).

It isn't.  NASA uses a lot of firm fixed price contracts, for spacecraft, launch services, processing supports, etc.

CRS is firm fixed price. 

I don't understand why everybody thinks NASA uses cost plus and why SAA's are the only answer.

I tend to agree.  It is frustrating to see the ignorance repeated time and time again around here. Various contracting mechanisms are appropriate and depend on the situation at hand.  Ultimately it is the customers choice on the type of contract they want depending on how much influence they want and how likely the scope of the contract is expected to be tweaked, etc.

These types of contract mechanisms are not unique to the government and are used throughout industry as well.

Certainly it's generally true about the "types of contract mechanisms" not being particularly unique in principle.  However, it's also generally true that when the government gets hold of a non-unique contracting mechanism, the cost of the widgets contracted for tends to skyrocket as reams of bureatcratic requirements become placed before the satisfaction of the contract terms.

And there's the continuing deliberate characterization of "ignorance" to those people who see the contractural complexities as unecessary. 

For example, the whole of FAR applies to SLS.  As it turned out, the whole of FAR applied to a $150K NASA contract that I applied for some few years ago.  The several people in this thread who have anecdotal contractural stories about their NASA experiences may be ignorant about all of the intricacies of FAR, but who isn't?

The common underlying thread to the anecdotes is the complexity issue; contractural complexity without cause.  Call it CCC.  Sheesh.

The SpaceAccess alert cannot be said to be spin, by my reading:

Sorry to interrupt but has anyone seen this?

http://www.space-access.org/updates/sasalert060414.html

Shelby is asking for full FAR25 compliant cost reporting, despite CRS being a firm fixed price contract. ...

Yes, I mentionned it in this thread:
http://forum.nasaspaceflight.com/index.php?topic=34827.msg1209848#msg1209848

...

I think it's reasonably safe to say that Shelby is just using this as a way to attack Commercial Crew by saddling it with higher costs.

Senator Shelby's assertion that this is about ensuring "that taxpayers get the best value for their dollar" is patent nonsense - by NASA's own study, one Commercial Cargo launcher development came in at a tenth or less NASA's likely in-house costs, and Commercial Crew looks set to follow in saving massive amounts of taxpayer dollars...

There is no incompetence involved in these new developments.  This is an intelligently designed effort to maintain the status quo, where the United States government will have no assured ability to launch its own astros, and now, maybe even cargo, to as mundane a location as low Earth orbit.

[Sean Lynch]

There is no incompetence involved in these new developments.  This is an intelligently designed effort to maintain the status quo, where the United States government will have no assured ability to launch its own astros, and now, maybe even cargo, to as mundane a location as low Earth orbit.

The impact to commercial providers to NASA, from page 117-119 is pretty clear, from FAR contracts to requirements to conduct testing at NASA facilities potentially operated by competitors.

The Committee encourages
NASA to develop plans to fully utilize NASA-owned rocket testing
infrastructure for commercially developed launch vehicles to ensure
that these vehicles are not only tested in the same manner as Government-
developed launch vehicles but at the same facilities to ensure
consistency in testing across all potential vehicles.

Perhaps I'm wrong but with the new requirements placed on NASA, every aspect of a company's private proprietary information could be laid bare for the competition to feast upon under these rules.
To protect competition, we don't require disclosure and testing of proprietary fracking fluid formulation in EPA labs.
I don't know what to say.

[Jim]

Perhaps I'm wrong but with the new requirements placed on NASA, every aspect of a company's private proprietary information could be laid bare for the competition to feast upon under these rules.

no, it still would be protected as it is always

[Sean Lynch]

Perhaps I'm wrong but with the new requirements placed on NASA, every aspect of a company's private proprietary information could be laid bare for the competition to feast upon under these rules.

no, it still would be protected as it is always

Putting all the compartmentalized data in one basket, briefcase or thumbdrive is a pretty tempting nest egg some might reach for.  There is precedent in the industry.

[Jim]

Putting all the compartmentalized data in one basket, briefcase or thumbdrive is a pretty tempting nest egg some might reach for.  There is precedent in the industry.

That has been a risk since day 1.  Right now, my laptop contains proprietary and ITAR information on all US launchers and some foreign ones as well.   But it is protected and I have a duty to keep it protected and not share the information with unauthorized individuals.