Author Topic: GAO Feb 2017 report on commercial crew  (Read 10941 times)

Online FutureSpaceTourist

  • Senior Member
  • *****
  • Posts: 3092
  • UK
    • Plan 28
  • Liked: 2054
  • Likes Given: 696
GAO Feb 2017 report on commercial crew
« on: 02/16/2017 07:04 PM »
Title is:

Quote
Schedule Pressure Increases as Contractors Delay Key Events

Edit: from http://www.gao.gov/assets/690/682859.pdf
« Last Edit: 02/16/2017 07:07 PM by FutureSpaceTourist »

Offline envy887

  • Senior Member
  • *****
  • Posts: 2152
  • Liked: 941
  • Likes Given: 609
Re: GAO Feb 2017 report on commercial crew
« Reply #1 on: 02/16/2017 07:38 PM »
Nothing about the aerodynamic issues with Starliner on Atlas V? That's interesting. Was that already considered resolved?

Offline Folgers25

  • Member
  • Posts: 28
  • Space Coast, Fla.
  • Liked: 4
  • Likes Given: 19
Re: GAO Feb 2017 report on commercial crew
« Reply #2 on: 02/16/2017 08:45 PM »
On page 20 there is an example saying Crew Dragon's TPS has been identified as a high risk hazard. Wonder why that is?
Shake 'n bake

Offline Toast

Re: GAO Feb 2017 report on commercial crew
« Reply #3 on: 02/16/2017 09:49 PM »
On page 20 there is an example saying Crew Dragon's TPS has been identified as a high risk hazard. Wonder why that is?

It's high-risk because failure of TPS would have a very high chance of causing a crew fatality. In context, they're using it as an example to show how the risk is being mitigated. Hence the next line in the report:

Quote
As a result, the program’s quality assurance team witnessed SpaceX engineers installing spherical tiles on their crew capsule’s heat shield to verify the required process was followed.

Offline gongora

  • Global Moderator
  • Full Member
  • *****
  • Posts: 1342
  • US
  • Liked: 871
  • Likes Given: 715
Re: GAO Feb 2017 report on commercial crew
« Reply #4 on: 02/17/2017 06:21 AM »
Some random snips from the document.  If you're interested in the Commercial Crew Program you should probably just read the document, it doesn't take that long.

Quote
NASA has also made changes to the contracts that have increased their value. While the contracts are fixed-price, their values can increase if NASA adds to the scope of the work or otherwise changes requirements. As of October 2016, NASA had increased the value of contract line item 001 for Boeing by $47 million for hardware and software requirement changes, and contract line item 001 for SpaceX by $91 million for a hardware requirement change and the addition of cargo during an ISS test flight.

In addition to its budget, NASA submits quarterly reports to the appropriations committees on the Commercial Crew Program’s progress that include cost and schedule updates from each contractor and information on contractor specific-risks, including procurement-sensitive material that is not publically releasable.

SpaceX officials told us they completed integrated testing and qualification of several major spacecraft components, including its environmental control and life support system—which controls air quality and temperature, among other functions, to ensure crew survivability—and its spacesuit in November 2016.

As of October 2016, the program’s schedule risk analysis indicated that both contractors’ certification dates would likely slip into early 2019.

In 2015, the United States modified its contract with Roscosmos to provide crew transportation to the ISS for six astronauts through 2018 with rescue and return through late spring 2019. The contract extension was valued at $491 million or approximately $82 million per seat. NASA’s contract with Roscosmos permits it to delay the use of the final seat by up to 6 months to late spring 2019, with a return flight approximately 6 months later.

The ISS program office stated that NASA could construct another “year in space” mission, similar to the mission undertaken by NASA astronaut Scott Kelly. According to the ISS program office, NASA and Roscosmos are discussing this option; however, there is no agreement in place. Under this scenario, an astronaut could begin a mission near the end of calendar year 2018 using one of the final Soyuz seats and would return to Earth via one of the Commercial Crew Program contractors’ crew transportation systems approximately one year later in 2019.

In March 2016, Boeing modified its previously approved parachute test plan by replacing six drop tests, which simulate select forces—for example, mass—on the parachute system for one full-scale test event, which simulates all aspects of a parachute system. Through discussions with the program, Boeing has increased the number of full-scale test events to five, with an option for two additional tests if deemed necessary.

Finally, both the program and a NASA advisory group consider SpaceX’s plan to fuel the launch vehicle after the astronauts are on board the spacecraft to be a potential safety risk. SpaceX’s perspective is that this operation may be a lower risk to the crew; NASA and SpaceX’s risk evaluation is ongoing.

SpaceX requested, and the program approved, proposals to split its critical design review into three reviews because portions of its design had not been ready at previous reviews. ... SpaceX’s final planned design review was held in August 2016; however, the program reported that a number of outstanding areas, primarily related to ground systems, still needed to be reviewed. SpaceX officials told us these areas were reviewed in November 2016. Further, according to SpaceX, these separate reviews were in order to perform review of designs that were completed earlier than anticipated, to allow SpaceX and NASA teams to focus in greater detail on certain systems, and to accommodate design updates driven in part by changes to NASA requirements.

Under each contract, the current loss of crew requirement is 1 in 270, meaning that the contractors’ systems must carry no more than a 1 in 270 probability of incurring loss of crew. Near the end of the Space Shuttle program, the probability of loss of crew was approximately 1 in 90. Program officials told us that Commercial Crew is the first NASA program that the agency will evaluate against a probabilistic loss of crew requirement. They said that if the contractors cannot meet the agency loss of crew requirement at 1 in 270, NASA could still certify their systems by employing operational alternatives. This would entail a potentially increased level of risk or uncertainty related to the level of risk for the crew.

The Commercial Crew Program gained visibility into the contractors’ parachute qualification testing plans through technical forums. As a result, the program has proposed six additional tests for each contractor to increase the reliability of their parachute systems.

The Commercial Crew Program has developed productive working relationships with both contractors, but the level of visibility that the program has required thus far has also taken more time than the program or contractors anticipated. The program’s standing review board has stated that the contract is structured to allow NASA unprecedented levels of visibility, but that it was intended to be used primarily for high-risk areas. However, the standing review board found, and both contractors told us, that the program has requested high levels of visibility on most items and there are signs that the contractors’ patience is waning. Both contractors expressed concerns that the program requests more interaction and data than they originally anticipated at the time of the contract award. For example, Boeing and SpaceX officials told us that the program often requests additional in-person engagement with their engineers, such as repeat presentations to multiple boards on the same technical issue, and has also asked for the same data in multiple formats or from multiple stakeholders.

A verification closure notice is required for each of the 332 ISS requirements, which are applicable to anyone who docks with the station, and the 280 Commercial Crew Program requirements.

In order to ensure that the United States has continued access to the ISS if the Commercial Crew Program’s contractors experience additional schedule delays, we recommend the NASA Administrator develop a contingency plan for maintaining a presence on the ISS beyond 2018, including options to purchase additional Russian Soyuz seats, and report to Congress on the results.

I'd be really surprised if NASA doesn't buy the additional Soyuz seats currently held by Boeing.

Offline woods170

  • IRAS fan
  • Senior Member
  • *****
  • Posts: 6945
  • IRAS fan
  • The Netherlands
  • Liked: 2438
  • Likes Given: 738
Re: GAO Feb 2017 report on commercial crew
« Reply #5 on: 02/17/2017 07:40 AM »
What that WP article fails to identify is that the NASA-side of the program has caused, and is expected to continue to cause, it's fair share of the delays.
From the GAO Report:

Quote from: GAO
Sustaining Program’s Level of Visibility Might Be Difficult as Schedule Pressure Builds

The Commercial Crew Program has developed productive working relationships with both contractors, but the level of visibility that the program has required thus far has also taken more time than the program or contractors anticipated. The program’s standing review board has stated that the contract is structured to allow NASA unprecedented levels of visibility, but that it was intended to be used primarily for high-risk areas. However, the standing review board found, and both contractors told us, that the program has requested high levels of visibility on most items and there are signs that the contractors’ patience is waning. Both contractors expressed concerns that the program requests more interaction and data than they originally anticipated at the time of the contract award. For example, Boeing and SpaceX officials told us that the program often requests additional in-person engagement with their engineers, such as repeat presentations to multiple boards on the same
technical issue, and has also asked for the same data in multiple formats or from multiple stakeholders. Program officials told us that they are
constantly working to find a balance between obtaining the visibility they need to be able to eventually certify the crew transportation systems for human spaceflight while giving the contractors room to independently
work through issues for their systems.

In short: NASA is excercising much more insight than was previously intended in the contract. And that's putting a burden on the contractors.

Quote from: GAO
Program Office Workload Is an Emerging Schedule Risk

Program officials told us that one of their greatest upcoming challenges will be to keep pace with the contractors’ schedules so that the program does not delay certification. Specifically, they told us they are concerned about an upcoming “bow wave” of work because the program must complete two oversight activities—phased safety reviews and verification closure notices—concurrently in order to support the contractors’ ISS design certification reviews, uncrewed and crewed flight test missions, and final certification.

Sounds like the Program Office underestimated the amount of work to be done on the NASA-side of CCP.

Quote from: GAO
The Commercial Crew Program’s review and approval of the contractors’ hazard reports have taken longer than planned. The program originally planned to complete phase two in early 2016 but currently does not expect to complete this phase until June 2017. The Commercial Crew Program has a goal of reviewing hazard reports within 8 weeks of receiving them, but a recent report by the NASA Office of Inspector General found that the reviews are taking longer than anticipated and a backlog has developed. In response to the Inspector General’s report, NASA officials noted that, while the timeliness of the hazard review process is important, what is more important is having thorough, detailed hazard reports in order to understand safety risks and ensuring the safety of each system. As of October 2016, the Commercial Crew Program had approved 117 of the anticipated 195 hazard reports and planned to approve approximately half of the remaining reports for both contractors by the end of 2016. Program officials told us that the hazard reports that are still open are related to items that they would not expect to be closed because they involve some of the more complicated design work that the contractors have not yet finalized. Program officials also pointed out other
ways that the contractors have contributed to phase two delays, including receiving incomplete hazard reports that required several iterations before the program could begin its formal review.

That is a 15-month delay in reviewing hazard reports. Guess what: certification of both contractors is delayed with (near) exactly that same amount of time (14 months for Boeing, 15 months for SpaceX). That's no coincidence IMO. It's bad performance by NASA and called-out by GAO in it's previous CCP report.

From the conclusion of the report:
Quote from: GAO
Gaining this level of visibility has taken more time than the program or contractors anticipated, but the early upfront investment in time should ultimately make the certification process go smoother. In addition, while the Commercial Crew Program should be mindful about placing undue burdens on its contractors, it ultimately has the responsibility for ensuring the safety of U.S. astronauts, and its contracts with Boeing and SpaceX give it deference to determine the level of insight required to do so.

So yeah, the contractors are having technical trouble and had overly optimistic (aggresive) schedules. But the NASA part of CCP is not exactly up-to-speed either.
« Last Edit: 02/17/2017 09:15 PM by Chris Bergin »

Offline Robotbeat

  • Senior Member
  • *****
  • Posts: 26262
  • Minnesota
  • Liked: 6222
  • Likes Given: 4569
Re: GAO Feb 2017 report on commercial crew
« Reply #6 on: 02/17/2017 12:51 PM »
I'm glad for this report. It keeps all parties accountable and provides a far clearer view of the program's status than could be obtained by just listening to one party or the other. Good on GAO.
Chris  Whoever loves correction loves knowledge, but he who hates reproof is stupid.

To the maximum extent practicable, the Federal Government shall plan missions to accommodate the space transportation services capabilities of United States commercial providers. US law http://goo.gl/YZYNt0

Offline yg1968

  • Senior Member
  • *****
  • Posts: 8939
  • Liked: 1078
  • Likes Given: 717
Re: GAO Feb 2017 report on commercial crew
« Reply #7 on: 02/17/2017 03:18 PM »
Quote from: page 34 of the PDF of the GAO Report
Recommendation 1: In order to ensure that the United States has continued access to the ISS if the Commercial Crew Program’s contractors experience additional schedule delays, we recommend the NASA Administrator develop a contingency plan for maintaining a presence on the ISS beyond 2018, including options to purchase additional Russian Soyuz seats, and report to Congress on the results.

Here is what NASA answered in regards to this recommendation (see page 34 of the PDF):

Quote from: Gerst
Management’s Response: NASA concurs with the recommendation. NASA will develop a contingency plan for maintaining a presence on the ISS beyond 2018 if the Commercial Crew Program’s partners experience additional schedule delays. NASA anticipates the contiengency plan to be ready in approximately six weeks.

Estimated Completion Date: March 13, 2017


« Last Edit: 02/17/2017 03:26 PM by yg1968 »

Offline SWGlassPit

  • I break space hardware
  • Full Member
  • ****
  • Posts: 421
  • Liked: 245
  • Likes Given: 33
Re: GAO Feb 2017 report on commercial crew
« Reply #8 on: 02/17/2017 03:25 PM »
On page 20 there is an example saying Crew Dragon's TPS has been identified as a high risk hazard. Wonder why that is?

It's high-risk because failure of TPS would have a very high chance of causing a crew fatality. In context, they're using it as an example to show how the risk is being mitigated. Hence the next line in the report:

Quote
As a result, the program’s quality assurance team witnessed SpaceX engineers installing spherical tiles on their crew capsule’s heat shield to verify the required process was followed.

You're conflating risk with criticality.  Criticality explores the expected outcome with the assumption that a given component has failed.  Risk includes the likelihood of the component failing.

Offline rockets4life97

  • Full Member
  • ***
  • Posts: 325
  • Liked: 157
  • Likes Given: 83
Re: GAO Feb 2017 report on commercial crew
« Reply #9 on: 02/17/2017 03:34 PM »
The most disappointing part of the report in my view is that NASA can't keep its hands-off. A major goal of the commercial cargo and crew programs was to re-orientate NASA's relationship with the launch providers. I hope NASA takes the GAO concerns seriously and considers how they can move towards a lighter touch, while still getting to a high-standard for certification.

Offline yg1968

  • Senior Member
  • *****
  • Posts: 8939
  • Liked: 1078
  • Likes Given: 717

Offline Jim

  • Night Gator
  • Senior Member
  • *****
  • Posts: 30805
  • Cape Canaveral Spaceport
  • Liked: 9048
  • Likes Given: 292
Re: GAO Feb 2017 report on commercial crew
« Reply #11 on: 02/17/2017 03:38 PM »
The most disappointing part of the report in my view is that NASA can't keep its hands-off. A major goal of the commercial cargo and crew programs was to re-orientate NASA's relationship with the launch providers.

No, NASA already has re-oriented its relationship with the launch providers.  The goal of the crew program was to use similar processes.

Offline Toast

Re: GAO Feb 2017 report on commercial crew
« Reply #12 on: 02/17/2017 03:40 PM »
You're conflating risk with criticality.  Criticality explores the expected outcome with the assumption that a given component has failed.  Risk includes the likelihood of the component failing.

I won't argue with your definition, but I was simply using the language from the report.

Offline rockets4life97

  • Full Member
  • ***
  • Posts: 325
  • Liked: 157
  • Likes Given: 83
Re: GAO Feb 2017 report on commercial crew
« Reply #13 on: 02/17/2017 03:47 PM »
The most disappointing part of the report in my view is that NASA can't keep its hands-off. A major goal of the commercial cargo and crew programs was to re-orientate NASA's relationship with the launch providers.

No, NASA already has re-oriented its relationship with the launch providers.  The goal of the crew program was to use similar processes.

Okay. To be sure I understand, you mean:
Commercial cargo: re-orientate relationship with launch providers (accomplished)
Commercial crew: continue re-orientation using similar processes (which GAO is saying maybe isn't quite happening)

Offline SWGlassPit

  • I break space hardware
  • Full Member
  • ****
  • Posts: 421
  • Liked: 245
  • Likes Given: 33
Re: GAO Feb 2017 report on commercial crew
« Reply #14 on: 02/17/2017 04:02 PM »
You're conflating risk with criticality.  Criticality explores the expected outcome with the assumption that a given component has failed.  Risk includes the likelihood of the component failing.

I won't argue with your definition, but I was simply using the language from the report.

Assuming they aren't misusing the term, that means there is (was) a concern about the likelihood of failure of the TPS.  High criticality (well, Crit 1 -- low numbers are more critical) items are not necessarily high risk.

Offline Jim

  • Night Gator
  • Senior Member
  • *****
  • Posts: 30805
  • Cape Canaveral Spaceport
  • Liked: 9048
  • Likes Given: 292
Re: GAO Feb 2017 report on commercial crew
« Reply #15 on: 02/17/2017 04:02 PM »

Commercial cargo: re-orientate relationship with launch providers (accomplished)

No, existing launch services for science spacecraft has re-orientated relationship with launch providers since the 90's.  This was needs to be applied to crew.
Commercial cargo is an outlier.  It is for low risk logistics.


Offline yg1968

  • Senior Member
  • *****
  • Posts: 8939
  • Liked: 1078
  • Likes Given: 717
Re: GAO Feb 2017 report on commercial crew
« Reply #16 on: 02/17/2017 06:51 PM »

Commercial cargo: re-orientate relationship with launch providers (accomplished)

No, existing launch services for science spacecraft has re-orientated relationship with launch providers since the 90's.  This was needs to be applied to crew.
Commercial cargo is an outlier.  It is for low risk logistics.

Hopefully, this will happen once the commercial crew providers are certified.

Offline Chris Bergin

Re: GAO Feb 2017 report on commercial crew
« Reply #17 on: 02/17/2017 09:16 PM »
If you're going to post links, guys, make sure they aren't 3842375823423 characters long. Short URL them or don't post, because it breaks the internet. ;)

Offline deruch

  • Full Member
  • ****
  • Posts: 1377
  • California
  • Liked: 1069
  • Likes Given: 1348
Re: GAO Feb 2017 report on commercial crew
« Reply #18 on: 02/18/2017 03:58 AM »
The most disappointing part of the report in my view is that NASA can't keep its hands-off. A major goal of the commercial cargo and crew programs was to re-orientate NASA's relationship with the launch providers. I hope NASA takes the GAO concerns seriously and considers how they can move towards a lighter touch, while still getting to a high-standard for certification.
That "realignment" was only for the earlier "SAA" stages of the commercial crew/cargo program.  When they finished CCiCap and switched to a traditional contract for CCtCap that was specifically because NASA determined that they weren't willing to hazard crews on transport systems in which they had levels of insight similar to the COTS/CRS/CCiCap programs.  This pivot back to a traditional insight/oversight system for the final stage of the Crew Program was 100% intentional.  And it explains why they switched to contracts as opposed to Space Act Agreements because with contracts they can dictate changes and set requirements in ways that they couldn't otherwise with SAAs.
Shouldn't reality posts be in "Advanced concepts"?  --Nomadd

Online QuantumG

  • Senior Member
  • *****
  • Posts: 7972
  • Australia
  • Liked: 2672
  • Likes Given: 651
Re: GAO Feb 2017 report on commercial crew
« Reply #19 on: 02/18/2017 05:29 AM »
This pivot back to a traditional insight/oversight system for the final stage of the Crew Program was 100% intentional.

That must be why it was such a surprise to everyone, including the people who no longer work in the commercial crew office after they fought hard to stop it.

Jeff Bezos has billions to spend on rockets and can go at whatever pace he likes! Wow! What pace is he going at? The slowest possible.

Tags: