Author Topic: Commercial Crew - Discussion Thread 1  (Read 640858 times)

Offline joek

  • Senior Member
  • *****
  • Posts: 4869
  • Liked: 2782
  • Likes Given: 1096
Re: Commercial Crew (CCtCAP) - Discussion Thread
« Reply #140 on: 10/11/2014 10:46 pm »
NASA itself stated that price would be given double value over all the other factors.

NASA never stated or implied that price was double the value of all other factors (and I have never claimed otherwise).

As far as I can tell, the "price above all" meme has grown from an inaccurate reading of the evaluation factors and conflation of the total Mission Suitability score with the Mission Suitability: Technical, Crew Safety and Mission Assurance subfactor score (or something like that).

In short, one could reasonably assert that Price is approximately twice the weight of the Mission Suitability: Technical, Crew Safety and Mission Assurance subfactor score.  However, asserting that Price is approximately twice the weight of the Mission Suitability total score is incorrect.

Quote from: CCtCap RFP
M.1   SOURCE SELECTION AND EVALUATION FACTORS—GENERAL
...
(e)   Relative Order of Importance of Evaluation Factors: Mission Suitability and Past Performance, when combined, are approximately equal to Price.  The Price factor is more important than Mission Suitability, which is more important than Past Performance.
... or that Price is is approximately equal to ...
Quote
M.2   MISSION SUITABILITY FACTOR
...
Mission Suitability Subfactors (Scored Elements)Weight (Points)
Subfactor 1: Technical, Crew Safety and Mission Assurance525
Subfactor 2: Management Approach400
Subfactor 3: Small Business Utilization75
Total1000
... plus ...
Quote
M.4   PAST PERFORMANCE FACTOR
...  The Past Performance evaluation is an assessment of the Government’s confidence in the Offeror’s ability to perform the solicitation requirements by assigning a confidence rating to the overall Past Performance factor.
« Last Edit: 10/11/2014 11:06 pm by joek »

Offline NovaSilisko

  • Full Member
  • ****
  • Posts: 1828
  • Liked: 1440
  • Likes Given: 1301
Re: Commercial Crew (CCtCAP) - Discussion Thread
« Reply #141 on: 10/11/2014 11:22 pm »
Quote
M.4   PAST PERFORMANCE FACTOR
...  The Past Performance evaluation is an assessment of the Government’s confidence in the Offeror’s ability to perform the solicitation requirements by assigning a confidence rating to the overall Past Performance factor.

Seems to me this really is one of the more important points for the outcome we got, and the selection makes a lot more sense when you take it into account.
« Last Edit: 10/11/2014 11:30 pm by NovaSilisko »

Offline Rocket Science

  • Senior Member
  • *****
  • Posts: 10586
  • NASA Educator Astronaut Candidate Applicant 2002
  • Liked: 4548
  • Likes Given: 13523
Re: Commercial Crew (CCtCAP) - Discussion Thread
« Reply #142 on: 10/11/2014 11:44 pm »
Unfortunately past performance skews the results against any new participants in the industry... So what was the point of a competition? Assign a contract and get on with it...
"The laws of physics are unforgiving"
~Rob: Physics instructor, Aviator

Offline sdsds

  • Senior Member
  • *****
  • Posts: 7201
  • “With peace and hope for all mankind.”
  • Seattle
  • Liked: 2050
  • Likes Given: 1962
Re: Commercial Crew (CCtCAP) - Discussion Thread
« Reply #143 on: 10/12/2014 12:03 am »
past performance skews the results against any new participants in the industry... So what was the point of a competition?

The apparent point of the competition was to apply some pricing pressure on the established market participants. Personally I assume that had SNC not been competing, Boeing's offer would have been different and its price would have been even higher than what they bid. So maybe the competition worked just as intended?
« Last Edit: 10/12/2014 12:04 am by sdsds »
— 𝐬𝐝𝐒𝐝𝐬 —

Offline joek

  • Senior Member
  • *****
  • Posts: 4869
  • Liked: 2782
  • Likes Given: 1096
Re: Commercial Crew (CCtCAP) - Discussion Thread
« Reply #144 on: 10/12/2014 12:08 am »
Unfortunately past performance skews the results against any new participants in the industry... So what was the point of a competition? Assign a contract and get on with it...

In cases with no history the evaluation must be neutral; per FAR 15.305(a)(2)(iv); incorporated by reference in the CCtCap RFP:
Quote

In the case of an offeror without a record of relevant past performance or for whom information on past performance is not available, the offeror may not be evaluated favorably or unfavorably on past performance.

Offline TomH

  • Senior Member
  • *****
  • Posts: 2938
  • Vancouver, WA
  • Liked: 1868
  • Likes Given: 909
Re: Commercial Crew (CCtCAP) - Discussion Thread
« Reply #145 on: 10/12/2014 12:24 am »
Unfortunately past performance skews the results against any new participants in the industry... So what was the point of a competition? Assign a contract and get on with it...

In cases with no history the evaluation must be neutral; per FAR 15.305(a)(2)(iv); incorporated by reference in the CCtCap RFP:
Quote

In the case of an offeror without a record of relevant past performance or for whom information on past performance is not available, the offeror may not be evaluated favorably or unfavorably on past performance.

How does that work? Does a favorable prior performer get positive points, an unfavorable prior performer negative points, and a new offeror zero points? If another metric, how is it set up?

Offline Rocket Science

  • Senior Member
  • *****
  • Posts: 10586
  • NASA Educator Astronaut Candidate Applicant 2002
  • Liked: 4548
  • Likes Given: 13523
Re: Commercial Crew (CCtCAP) - Discussion Thread
« Reply #146 on: 10/12/2014 12:31 am »
past performance skews the results against any new participants in the industry... So what was the point of a competition?

The apparent point of the competition was to apply some pricing pressure on the established market participants. Personally I assume that had SNC not been competing, Boeing's offer would have been different and its price would have been even higher than what they bid. So maybe the competition worked just as intended?
I'd of given Boeing 8B five years ago and would be flying today...
"The laws of physics are unforgiving"
~Rob: Physics instructor, Aviator

Offline joek

  • Senior Member
  • *****
  • Posts: 4869
  • Liked: 2782
  • Likes Given: 1096
Re: Commercial Crew (CCtCAP) - Discussion Thread
« Reply #147 on: 10/12/2014 01:00 am »
How does that work? Does a favorable prior performer get positive points, an unfavorable prior performer negative points, and a new offeror zero points? If another metric, how is it set up?

Yes, more-or-less. Past performance is not numerically scored, but is expressed as a confidence level.  If past performance information is available and relevant, it would typically be expressed on a scale of, e.g., "low" (negative) to "high" (positive).  Those without a history would typically be graded "unknown", or "neutral".

A negative confidence level works against you; a positive confidence level works for you.  All other things equal, the competitor with the highest confidence level wins, with "unknown" or "neutral" being the equivalent of zero.  However, an "unknown" or "neutral" confidence level cannot be used as the sole basis for acceptance or rejection.

However, past performance and confidence level cannot be divorced from other evaluation factors.  A proposal which has a very high price and a very high confidence level does not necessarily get a pass vs. a proposal with a lower price and a lower confidence level.  That is where fuzzy interpretations of timeliness, need, and "value to the government" comes into play, and where disputes due to such fuzzy interpretations arise.

Offline vt_hokie

  • Senior Member
  • *****
  • Posts: 3054
  • Hazlet, NJ
  • Liked: 118
  • Likes Given: 436
Re: Commercial Crew (CCtCAP) - Discussion Thread
« Reply #148 on: 10/12/2014 01:03 am »
Unfortunately past performance skews the results against any new participants in the industry... So what was the point of a competition? Assign a contract and get on with it...

Exactly.  The outcome basically just reinforced my cynical suspicion all along that this "commercial" program was little more than a charade that would inevitably lead to more or less what we could have had by awarding a cost plus contract to the entrenched establishment without wasting all this time pretending to want to spur innovation.

Offline joek

  • Senior Member
  • *****
  • Posts: 4869
  • Liked: 2782
  • Likes Given: 1096
Re: Commercial Crew (CCtCAP) - Discussion Thread
« Reply #149 on: 10/12/2014 01:25 am »
Exactly.  The outcome basically just reinforced my cynical suspicion all along that this "commercial" program was little more than a charade that would inevitably lead to more or less what we could have had by awarding a cost plus contract to the entrenched establishment without wasting all this time pretending to want to spur innovation.

Forget for a moment the term "commercial", and instead consider replacing it with the term "competitive", as in the "Competitive Crew Program".  Does that make you happier?  And on what basis do you assert that a cost+ (presumably sole-source?) contract would have lead to a better result?

Offline Rocket Science

  • Senior Member
  • *****
  • Posts: 10586
  • NASA Educator Astronaut Candidate Applicant 2002
  • Liked: 4548
  • Likes Given: 13523
Re: Commercial Crew (CCtCAP) - Discussion Thread
« Reply #150 on: 10/12/2014 01:26 am »
Unfortunately past performance skews the results against any new participants in the industry... So what was the point of a competition? Assign a contract and get on with it...

Exactly.  The outcome basically just reinforced my cynical suspicion all along that this "commercial" program was little more than a charade that would inevitably lead to more or less what we could have had by awarding a cost plus contract to the entrenched establishment without wasting all this time pretending to want to spur innovation.
It is endemic with the ever shifting priorities that come with Presidential election cycles and the changing of the NASA Administrator. Throw in the congressional meddling and we have a disjointed space program with the goal posts being frequently moved resulting in wasted time and money. Russia seems to always be consistent whether is under the U.S.S.R or the current regime... It might not be innovative but is rarely has gaps in space access...
"The laws of physics are unforgiving"
~Rob: Physics instructor, Aviator

Offline clongton

  • Expert
  • Senior Member
  • *****
  • Posts: 12053
  • Connecticut
    • Direct Launcher
  • Liked: 7347
  • Likes Given: 3749
Re: Commercial Crew (CCtCAP) - Discussion Thread
« Reply #151 on: 10/12/2014 10:14 am »
However, asserting that Price is approximately twice the weight of the Mission Suitability total score is incorrect.

The last sentence in my post explicitly stated:
Quote
Mind you that does not say that price is primary - there is no primary - only that price is 2x more important than any other "single" factor.
Chuck - DIRECT co-founder
I started my career on the Saturn-V F-1A engine

Offline clongton

  • Expert
  • Senior Member
  • *****
  • Posts: 12053
  • Connecticut
    • Direct Launcher
  • Liked: 7347
  • Likes Given: 3749
Re: Commercial Crew (CCtCAP) - Discussion Thread
« Reply #152 on: 10/12/2014 10:20 am »
I'd of given Boeing 8B five years ago and would be flying today...

No they wouldn't. Atlas still isn't human rated yet, in spite of the relatively easy technical requirements to do so. Without SNC and SpaceX in the mix Boeing and ULA would have had absolutely no pressure to get it done. It would have been business as usual, drawn out as long as possible and get as much taxpayer money for it as possible.
Chuck - DIRECT co-founder
I started my career on the Saturn-V F-1A engine

Offline Rocket Science

  • Senior Member
  • *****
  • Posts: 10586
  • NASA Educator Astronaut Candidate Applicant 2002
  • Liked: 4548
  • Likes Given: 13523
Re: Commercial Crew (CCtCAP) - Discussion Thread
« Reply #153 on: 10/12/2014 11:40 am »
I'd of given Boeing 8B five years ago and would be flying today...

No they wouldn't. Atlas still isn't human rated yet, in spite of the relatively easy technical requirements to do so. Without SNC and SpaceX in the mix Boeing and ULA would have had absolutely no pressure to get it done. It would have been business as usual, drawn out as long as possible and get as much taxpayer money for it as possible.
Then that fly’s in the face of NASA selection due to “past performance” which they appear to give them high marks... Or is that just based on the “legacy companies” they swallowed up over the years which then should have no bearing on selection...
"The laws of physics are unforgiving"
~Rob: Physics instructor, Aviator

Offline Jim

  • Night Gator
  • Senior Member
  • *****
  • Posts: 37440
  • Cape Canaveral Spaceport
  • Liked: 21450
  • Likes Given: 428
Re: Commercial Crew (CCtCAP) - Discussion Thread
« Reply #154 on: 10/12/2014 12:17 pm »
Unfortunately past performance skews the results against any new participants in the industry... So what was the point of a competition? Assign a contract and get on with it...

SNC could and did bring in experienced subcontractors, that helps getting around this.

Past performance is always a consideration.
« Last Edit: 10/12/2014 12:22 pm by Jim »

Offline Jim

  • Night Gator
  • Senior Member
  • *****
  • Posts: 37440
  • Cape Canaveral Spaceport
  • Liked: 21450
  • Likes Given: 428
Re: Commercial Crew (CCtCAP) - Discussion Thread
« Reply #155 on: 10/12/2014 12:20 pm »

Exactly.  The outcome basically just reinforced my cynical suspicion all along that this "commercial" program was little more than a charade that would inevitably lead to more or less what we could have had by awarding a cost plus contract to the entrenched establishment without wasting all this time pretending to want to spur innovation.

Wings do not equate to innovation. 
Also, SNC's "innovation" bit it in the arse, they had to re-engine the vehicle.

Offline Rocket Science

  • Senior Member
  • *****
  • Posts: 10586
  • NASA Educator Astronaut Candidate Applicant 2002
  • Liked: 4548
  • Likes Given: 13523
Re: Commercial Crew (CCtCAP) - Discussion Thread
« Reply #156 on: 10/12/2014 12:36 pm »
Unfortunately past performance skews the results against any new participants in the industry... So what was the point of a competition? Assign a contract and get on with it...

SNC could and did bring in experienced subcontractors, that helps getting around this.

Past performance is always a consideration.
I agree with you Jim but should it not include "on time and within budget"?
"The laws of physics are unforgiving"
~Rob: Physics instructor, Aviator

Offline Jim

  • Night Gator
  • Senior Member
  • *****
  • Posts: 37440
  • Cape Canaveral Spaceport
  • Liked: 21450
  • Likes Given: 428
Re: Commercial Crew (CCtCAP) - Discussion Thread
« Reply #157 on: 10/12/2014 12:49 pm »

I agree with you Jim but should it not include "on time and within budget"?

I think you mean "should include"

And SNC past experience for other projects always came in on time and within budget?




Offline Rocket Science

  • Senior Member
  • *****
  • Posts: 10586
  • NASA Educator Astronaut Candidate Applicant 2002
  • Liked: 4548
  • Likes Given: 13523
Re: Commercial Crew (CCtCAP) - Discussion Thread
« Reply #158 on: 10/12/2014 01:54 pm »

I agree with you Jim but should it not include "on time and within budget"?

I think you mean "should include"

And SNC past experience for other projects always came in on time and within budget?
Fair point Jim, unfortunately the sample size for human rated spacecraft is rather small.
"The laws of physics are unforgiving"
~Rob: Physics instructor, Aviator

Offline Prober

  • Senior Member
  • *****
  • Posts: 10348
  • Save the spin....I'm keeping you honest!
  • Nevada
  • Liked: 721
  • Likes Given: 729
Re: Commercial Crew (CCtCAP) - Discussion Thread
« Reply #159 on: 10/12/2014 04:47 pm »
I'd of given Boeing 8B five years ago and would be flying today...

No they wouldn't. Atlas still isn't human rated yet, in spite of the relatively easy technical requirements to do so. Without SNC and SpaceX in the mix Boeing and ULA would have had absolutely no pressure to get it done. It would have been business as usual, drawn out as long as possible and get as much taxpayer money for it as possible.

don't you think the politics of the RD-180 hurt that process?

When it comes to use of taxpayer money this program is no different.  Lot's of unneeded bells, and whistles on some of the designs.

2017 - Everything Old is New Again.
"I fear all we have done is to awaken a sleeping giant..." --Isoroku Yamamoto

Tags:
 

Advertisement NovaTech
Advertisement Northrop Grumman
Advertisement
Advertisement Margaritaville Beach Resort South Padre Island
Advertisement Brady Kenniston
Advertisement NextSpaceflight
Advertisement Nathan Barker Photography
1