-
#1220
by
dglow
on 26 Feb, 2020 19:31
-
At this point Boeing had best fly another unmanned mission regardless of what NASA requests. It will be a bad look if they don't, and bad PR isn't something the organization can afford.
-
#1221
by
mme
on 26 Feb, 2020 19:35
-
Man, this is just egg on NASA'S face. How in the world did they sign off from a Flight Readiness Review (or whatever it's called for a launch) when apparently ZERO system integration testing was done with the rocket hardware? Yeah, Boeing should have done it, but how in the world did NASA sign off on it?
This is gonna be a long, tough, slog to get through. But a necessary one.
I'm not ready to blame NASA until we know what the test plans and results were that Boeing presented to NASA as compared to what they did. So the important thing to me is in this quote:
Boeing said it followed all of the testing procedures NASA required of it prior to the Starliner test flight. The agency gave Boeing the green light to proceed after a flight readiness review prior to the mission.
What were the NASA requirements for testing and did Boeing really meet those requirements? Did NASA require integrated testing? (I hope so.) If so, did Boeing apply an "interesting" interpretation of what that entailed?
What level of detail is required in the reporting from a commercial vendor (I don't know.) There has to be some level of trust or it just becomes a NASA managed project. I'd love a review comparing the oversight of the two vendors, both voluntary (where the vendor asks for input or clarification) and involuntary (where NASA insists on drilling into the details that the vendor claims are "fine.")
-
#1222
by
ZachF
on 26 Feb, 2020 19:44
-
What were the NASA requirements for testing and did Boeing really meet those requirements? Did NASA require integrated testing? (I hope so.) If so, did Boeing apply an "interesting" interpretation of what that entailed?
From a standpoint of very basic common sense how do you skip a full integration test, even if it's not a NASA requirement? Skipping it is just so boneheaded that I can't believe it even occurred. It's like not looking before you cross the street levels of boneheadedness.
-
#1223
by
FutureSpaceTourist
on 26 Feb, 2020 19:58
-
I keep reading this statement from the Orlando Sentinel article:
Boeing said it followed all of the testing procedures NASA required of it prior to the Starliner test flight. The agency gave Boeing the green light to proceed after a flight readiness review prior to the mission.
I have to agree with others’ comments. Boeing should have done this sort of testing without NASA asking, but for NASA to approve flying without it?! Presumably the (lack of) requirement was the same for SpaceX? Would be interesting to know what testing SpaceX did.
As to the way forward from here, if NASA ask for such testing now then that’s a change in requirements ...and I guess may add significantly more delay (especially if Boeing don’t already have an appropriate rig they can do such testing on). What a mess.
Given that OFT has happened, I wouldn’t automatically now require the ground based integration testing that wasn’t done. Starting from where they are I think it has to be a matter of working out what gaps in assurance remain (including the significant gaps due to process failures) and filling those. Ground based integration may we’ll be a part of that but not just for the sake of doing it (if inflight evidence is sufficient).
Of course the more significant any changes/fixes that need to be made the less relevant OFT evidence becomes and the more likely ground based assurance is needed first.
-
#1224
by
dglow
on 26 Feb, 2020 20:02
-
Can somebody with industry knowledge describe more precisely what would/should have been involved in "full integration testing" between the Starliner and Atlas?
Coming from a software perspective, I cannot fathom why any interaction crossing such disparate hardware boundaries wouldn't have been tested with full production code running on full production hardware.
So please, disabuse – or abuse – me of my misunderstanding: does this report imply that Boeing did not bother to park an Altas V nav computer next to the Starliner and run through mission simulations? And barring that could they not have done so after stacking? (late IMO, but nonetheless an opportunity for 'fully integrated' testing)
-
#1225
by
mme
on 26 Feb, 2020 20:07
-
What were the NASA requirements for testing and did Boeing really meet those requirements? Did NASA require integrated testing? (I hope so.) If so, did Boeing apply an "interesting" interpretation of what that entailed?
From a standpoint of very basic common sense how do you skip a full integration test, even if it's not a NASA requirement? Skipping it is just so boneheaded that I can't believe it even occurred. It's like not looking before you cross the street levels of boneheadedness.
I agree 100% that skipping the full up integration testing is a bone headed move. What I'm saying is that we (at least I) don't know how the requirements where written nor how Boeing interpreted, implemented and reported on those requirements.
Boeing might as well be arguing "NASA didn't say we had to put pants on before going out in public."
-
#1226
by
SteveU
on 26 Feb, 2020 20:12
-
From the article:
Software issues are also plaguing another arm of Boeing, which is dealing with the fall out of problems with its 737 Max airplanes that led to the deaths of 346 people and has grounded the planes.
“[NASA’s advisory panel] would never tie those two together, it’s a completely different arm of the organization," said Saindon, who was a first officer with JetBlue Airways and a former director of aviation safety programs at the U.S. Naval Safety Center.
Still, he added, “that doesn’t mean it’s not a business right? And they’re trying to do things efficiently and cost-effectively.”
Something many of us have said - just surprised some one from the advisory panel had the guts to say it!
Actually may be a positive going forward. Now Boeing is going to have to really step up and prove they know what they are doing.
-
#1227
by
thirtyone
on 26 Feb, 2020 20:37
-
Anyone know if this was from a NASA conference, or if it was just early inside information reported by the Sentinel? I know there's a call-in next week where they were supposed to release the results of the report.
-
#1228
by
woods170
on 26 Feb, 2020 21:02
-
I really don't see how at this point anyone at NASA could justify allowing for Boeing to go ahead and proceed with the crew mission. This is supposed to be a relatively finalized vehicle and it has failed in many aspects. This is clearly not isolated failures, but rather a system-wide lack of care. Boeing should have to prove they can meet all objectives before crew flies on the vehicle, and they can only prove that by re-flying the OFT mission under serious scrutiny the entire way, with a very watchful NASA eye. We are now at failure number 4 on this iteration of the vehicle and it seems like Boeing doesn't care and that is unacceptable.
Apparently, the watchful eye of NASA failed as well in this case.
-
#1229
by
Captain Crutch
on 26 Feb, 2020 21:05
-
I really don't see how at this point anyone at NASA could justify allowing for Boeing to go ahead and proceed with the crew mission. This is supposed to be a relatively finalized vehicle and it has failed in many aspects. This is clearly not isolated failures, but rather a system-wide lack of care. Boeing should have to prove they can meet all objectives before crew flies on the vehicle, and they can only prove that by re-flying the OFT mission under serious scrutiny the entire way, with a very watchful NASA eye. We are now at failure number 4 on this iteration of the vehicle and it seems like Boeing doesn't care and that is unacceptable.
Apparently, the watchful eye of NASA failed as well in this case.
Evidently NASA didn’t have a watchful eye... having one would solve a few issues...
-
#1230
by
woods170
on 26 Feb, 2020 21:09
-
I keep reading this statement from the Orlando Sentinel article:
Boeing said it followed all of the testing procedures NASA required of it prior to the Starliner test flight. The agency gave Boeing the green light to proceed after a flight readiness review prior to the mission.
I have to agree with others’ comments. Boeing should have done this sort of testing without NASA asking, but for NASA to approve flying without it?! Presumably the (lack of) requirement was the same for SpaceX? Would be interesting to know what testing SpaceX did.
As to the way forward from here, if NASA ask for such testing now then that’s a change in requirements ...and I guess may add significantly more delay (especially if Boeing don’t already have an appropriate rig they can do such testing on). What a mess.
Given that OFT has happened, I wouldn’t automatically now require the ground based integration testing that wasn’t done. Starting from where they are I think it has to be a matter of working out what gaps in assurance remain (including the significant gaps due to process failures) and filling those. Ground based integration may we’ll be a part of that but not just for the sake of doing it (if inflight evidence is sufficient).
Of course the more significant any changes/fixes that need to be made the less relevant OFT evidence becomes and the more likely ground based assurance is needed first.
SpaceX did full-up integrated testing. Courtesy of building both the launcher and the spacecraft. As well as doing a for-real IFA, which requires the two talking to each other.
-
#1231
by
Arb
on 26 Feb, 2020 21:24
-
-
#1232
by
Mondagun
on 26 Feb, 2020 21:24
-
Can somebody with industry knowledge describe more precisely what would/should have been involved in "full integration testing" between the Starliner and Atlas?
Coming from a software perspective, I cannot fathom why any interaction crossing such disparate hardware boundaries wouldn't have been tested with full production code running on full production hardware.
So please, disabuse – or abuse – me of my misunderstanding: does this report imply that Boeing did not bother to park an Altas V nav computer next to the Starliner and run through mission simulations? And barring that could they not have done so after stacking? (late IMO, but nonetheless an opportunity for 'fully integrated' testing)
I would be very interested in an answer to this question as well. Furthermore, does anyone have an idea what exactly the launch provider would provide for "full integration testing"? Would the launch provider give a complete set of hardware+software that simulates the rocket's on-board computer? Or only software? Or just interface control documents?
-
#1233
by
Vettedrmr
on 26 Feb, 2020 21:36
-
I can't speak to what is done in the space industry, but I would expect it to be at least as rigorous as flight control system testing. For formal qualification testing, we use(d) flightworthy hardware with the actual flight software. Plus any subsystems we could test with (electrical power, hydraulics, cooling, etc.), and simulating with actual ICD-driven interfaces any subsystems (engines, for example) that we couldn't.
But, the next step above us was integration on the actual vehicle, bringing in those systems that require the vehicle, like the engines and brakes. And so on, and so on.
It just baffles me what Boeing did/didn't do and what NASA allowed Boeing to (I hate to use this phrase but nothing else is coming to mind) "get away with."
But, kudos to ASAP in this debacle; they're proving their worth.
Have a good one,
Mike
-
#1234
by
cebri
on 26 Feb, 2020 21:49
-
But, kudos to ASAP in this debacle; they're proving their worth.
Exactly, this is what they were created for. Not to try to convince everyone that Ares I was the safest vehicle to ever see the light of day.
Anyway, IMO this doesn't make Boeing look worse. Based on what we already knew, it is clear they lack the right processes (and dare I say, the culture) to develop software that is safe and reliable. However, the focus here has shifted to NASA. How on earth did they allow this? This is like basic 101 software development. It's was not a hidden bug difficult to detect that caused all the issues during OFT, there were software errors that could have been detected if they had performed some basic tests.
Maybe they will give us an explanation this friday.
-
#1235
by
Captain Crutch
on 26 Feb, 2020 21:59
-
Maybe they will give us an explanation this friday.
Given their track record so far with this vehicle... I wouldn’t hold your breath. It seems like their goal is to hide as much as possible and down play everything that’s gone wrong thus far. Boeing appears to care more about their image than about those that have next to no choice but to trust their lives in the hand of Boeing for these missions.
-
#1236
by
Comga
on 27 Feb, 2020 02:19
-
What were the NASA requirements for testing and did Boeing really meet those requirements? Did NASA require integrated testing? (I hope so.) If so, did Boeing apply an "interesting" interpretation of what that entailed?
NASA didn’t really have requirements for certifying a crew vehicle over which they didn’t have full engineering control and responsibility. They had never done that before. They made this stuff up as they went along.
Not to make excuses for NASA, because it is their job to do them right, but big program reviews are enormous events. Dozens of people. Many days to more than a week of presentations. Spiral bound printouts of PowerPoint presentations running to hundreds of pages. Footnotes refer to thousands of pages of technical analysis.
(note: PowerPoint obfuscates. See
Edward Tufte)
It’s REALLY hard to figure out what’s NOT there.
All presenters, including myself when I present, believe we have done a good job and incorporated feedback to fill in the holes. If engineers were told to achieve a goal, like Atlas-Starliner data communications, without a specific tool, like a hardware-in-loop simulations, they work at alternatives until they are convinced, right or wrong, that the alternatives are adequate. Then they project their confidence to the reviewers.
It is the job of extraordinary classical system engineers to keep these efforts in perspective. Good management values these people. Shabby management chafes at their cost.
Sharp eyed customers who stay focused throughout these ordeals keep checklist, mental or written, of what they need to see. They are also rare.
edit: SpaceX seems to be run by one of the greatest classical systems engineers. First principles on the left. Mars colony on the right. Whatever it takes to connect them in between.
Are there any systems engineers in Boeing’s C suite?
-
#1237
by
Rocket Science
on 27 Feb, 2020 06:33
-
NASA apparently caught drunk on Boeing's Kool-Aid...
-
#1238
by
Svetoslav
on 27 Feb, 2020 07:18
-
There's one thing I wonder...
Where did Chris Ferguson disappear? Boeing showed him everywhere - during launch, landing, recovery ops, press conferences... Haven't seen him since the investigation started.
It would be interesting to hear what he would say about this...
-
#1239
by
Vettedrmr
on 27 Feb, 2020 11:24
-
NASA didn’t really have requirements for certifying a crew vehicle over which they didn’t have full engineering control and responsibility. They had never done that before. They made this stuff up as they went along.
If they did that's just absurd. FAA has safety development standards (DO-178), Navy has various NAVAIR documents, Air Force uses Mil-Std documents and DO-178, etc. For NASA to say "well, we always had control before, and that was OK, but now we don't have control so we're not going to impose anything", well, first off, they probably DID impose some safety standard, and second, if they didn't, that's so ridiculous as to be borderline criminal negligence.
Whatever balls were dropped, both NASA and Boeing have got a real mess, and have had one for some time. Pad abort test and OFT just brought it into the light (as was probably inevitable). Awesome news is no one was hurt.
Have a good one,
Mike