-
#1340
by
Rocket Science
on 06 Mar, 2020 17:00
-
Lovarro: Two specific things: one thing in IRT we had a routine where there might have been four logical conditions, four ways the software could have run. Didn't check all four conditions, was a designer/tester choice. Not unusual you don't test all logical condition, do need to recognize we do need to test all logical conditions now. Process side, delegated too much authority to software board to approve changes and actions as it applies to software, software/hardware integration change should have been brought up all the way to engineering review board.
Wow. These are not subtle mistakes/omissions, but really basic ones. I cannot understand how a mature organisation could make such mistakes. This is what I might expect for an organisation with little experience that doesn’t understand the significance of what it is doing, in terms of criticality, and/or the importance of hardware & software integration.
The company is mature, the Boeing team are rookies (and are behaving as such) when it comes to HSF. The last time a HR spacecraft was was designed was by their grandparents an under the banner or North American/Rockwell..
-
#1341
by
FutureSpaceTourist
on 06 Mar, 2020 17:04
-
-
#1342
by
thirtyone
on 06 Mar, 2020 17:08
-
I'd like to start out by saying if Boeing and NASA do what they said they'd do on the call, I'd be pretty satisfied. The software process issues are indeed extensive, as quite a few people suspected very early on when the MET bug was revealed. However, the seriousness with which they are taking the problem seems to match the seriousness of their process issues. I hope Boeing comes out of this a better organization, and I hope we have a safe and successful Starliner when all of the problems are thoroughly solved, because redundant access is the smart thing to do IMO.
The reluctance of anyone to want to say "we have to fly OFT" is a little disappointing, but I think anyone who's been following this shouldn't be too surprised. At the same time, I genuinely think the process issues are many times more important than flying another OFT. The core issue behind making the decision is exactly as Doug mentioned - there may be ways to satisfy the remaining untested parts of Starliner without actually going to orbit. Of course, as-is, without any of these organizational changes, I don't trust that Boeing is actually capable of doing that, so I want them to do that if possible.
Also, I wanted to generally mention - when an independent NASA review board comes out of a fairly serious incident ("high visibility close call") and prescribes a list of recommended procedural changes, I'd be careful about questioning those recommendations. I realize Boeing was probably trying to diffuse the blame somewhat by saying that other companies should also learn from these recommendations, but it's still true - if you are working on safety critical software, it'd probably be a good idea to look at these recommendations yourself and see if they apply for your project. There are lessons for everyone. It is my feeling that the other commercial crew vendor is stronger with their test processes, but it would be nice for them to come out and verify that they are following the process recommendations of the review board when those are made available (as Boeing claims).
-
#1343
by
MattMason
on 06 Mar, 2020 17:35
-
In case anyone has forgotten, Boeing Defense, Space & Security Division is the modern amalgamation of these past companies--all of them involved in NASA's most significant triumphs and failures.
* North American Aviation
* Rockwell Corporation
* McDonnell Douglas
Despite the rather embarrassing OFT, we've watched Boeing or its past incarnations:
* Put men on the Moon (Boeing S-I-C stage, S-II stage, Apollo CSM)
* Construct the ISS US modules
* Build the X-37B military spaceplane
* Create the Surveyor and Lunar Orbiter probes
* blah, blah, blah
I can go on and on, but I think people have have enough of my lists.
What I strongly suspect today's Boeing has suffered is s dramatic brain drain of people who knew the special needs of human spacecraft. I can't defend how remarkably, fundamentally strange, if not stupid, the technical fallout has been.
It's as if Boeing's decisions have resurrected the many ghosts of their acquisitions all over. Those who forget history...
-
#1344
by
MattMason
on 06 Mar, 2020 17:42
-
Lovarro: Two specific things: one thing in IRT we had a routine where there might have been four logical conditions, four ways the software could have run. Didn't check all four conditions, was a designer/tester choice. Not unusual you don't test all logical condition, do need to recognize we do need to test all logical conditions now. Process side, delegated too much authority to software board to approve changes and actions as it applies to software, software/hardware integration change should have been brought up all the way to engineering review board.
Wow. These are not subtle mistakes/omissions, but really basic ones. I cannot understand how a mature organisation could make such mistakes. This is what I might expect for an organisation with little experience that doesn’t understand the significance of what it is doing, in terms of criticality, and/or the importance of hardware & software integration.
If you're suggesting it's "basic" to do 100% code coverage in a hardware integration lab, I'm not sure I buy that. That's nowhere near as cheap or easy as people in this thread are making it out to be. You'd be talking millions of test cases at the absolute least.
Yet SpaceX manages to do it. This is human spaceflight. If you can’t commit to 100% code coverage, get out of the business. If you think it’s too expensive or difficult to do so, get out of the business. This isn’t a joke. If they can’t do what needs to be done they have no business attempting to send humans to space.
[citation needed]
I'm not sure even shuttle flight software got 100% code coverage, though they got close, and that was enormously expensive.
NASA used the SAIL OV-095 full avionics and electronics simulator for that, the only "Orbiter" that flew every mission before the actual mission.
And that was run by Rockwell International until
Boeing picked it up in 1996.
-
#1345
by
freddo411
on 06 Mar, 2020 18:06
-
What is going on with this call?
Doug and Kathy from NASA spend multiple minutes describing why a decision on an OFT re-fly cannot be made yet, and will not be made without a thorough review of Boeing's forthcoming proposal. They fall over themselves explaining to the press why a decision is not yet warranted.
Following this, Jim Chilton of Boeing jumps in with a curt, "We're ready to re-fly, just need a decision from NASA."
Something about this contrast didn't feel right. Like Boeing either isn't playing ball, or is trying to pass the buck.
It's obvious that NASA is on the hook to pay for a large part if not all of an OFT refly.
I'm not seeing that. I think the issues are more complicated.
Obviously, you don't want to just say "go refly" before you've taken a close look at everything. Keep in mind they are also experiencing some schedule pressure to stop depending on Soyuz for crew transport, and they have to make sure they can maintain crewed status, while also protecting for the possibility that something goes wrong with one of the other crewed vehicles. Simply saying "it's obvious they're going to pay" is massively oversimplifying, if not wrong altogether.
My bold.
This is an absolutely, 100%, prohibited attitude. It's NOT ok to let scheduling issues drive technical and/or safety and/or QA processes.
It's a very, very, very bright line. Do not go there.
Also, why would we even think this way? We've got Dragon2, we've got soyuz. It's been good enough for 10 years (and yes, I've not been enitrely happy with soyuz) so it's tenable until Starliner get's built PROPERLY.
It makes absolutely zero sense to rush Starliner. Write a check to the Russians if needed, we've done a whole lot of that over the last 10 years without much fuss.
-
#1346
by
abaddon
on 06 Mar, 2020 18:26
-
I'd like to start out by saying if Boeing and NASA do what they said they'd do on the call, I'd be pretty satisfied
Agreed. The tonal shift from the OFT post-flight presser to now is encouraging I feel more confident both Boeing and NASA are very serious about looking under all the rocks, and fixing root causes of what allowed the OFT issues to make it all the way through to flight.
The reluctance of anyone to want to say "we have to fly OFT" is a little disappointing
It was very interesting watching the careful dance around the giant elephant in the room. Lovarro basically indicating that Boeing would propose next steps to certify for the crewed flight test which could result in an OFT re-fly, or could be done differently, and Boeing saying that they are prepared to re-fly but would do whatever NASA wants, in what seemed a little like game of hot potato. If left me with the impression that whoever is going to make that decision maybe wasn't present on the call (which would mean Bridenstine, presumably). That is, of course, an inference, and it could just be nobody wants to talk about the decision when it seems clear it won't be decided for some time.
-
#1347
by
SWGlassPit
on 06 Mar, 2020 18:38
-
What is going on with this call?
Doug and Kathy from NASA spend multiple minutes describing why a decision on an OFT re-fly cannot be made yet, and will not be made without a thorough review of Boeing's forthcoming proposal. They fall over themselves explaining to the press why a decision is not yet warranted.
Following this, Jim Chilton of Boeing jumps in with a curt, "We're ready to re-fly, just need a decision from NASA."
Something about this contrast didn't feel right. Like Boeing either isn't playing ball, or is trying to pass the buck.
It's obvious that NASA is on the hook to pay for a large part if not all of an OFT refly.
I'm not seeing that. I think the issues are more complicated.
Obviously, you don't want to just say "go refly" before you've taken a close look at everything. Keep in mind they are also experiencing some schedule pressure to stop depending on Soyuz for crew transport, and they have to make sure they can maintain crewed status, while also protecting for the possibility that something goes wrong with one of the other crewed vehicles. Simply saying "it's obvious they're going to pay" is massively oversimplifying, if not wrong altogether.
My bold.
This is an absolutely, 100%, prohibited attitude. It's NOT ok to let scheduling issues drive technical and/or safety and/or QA processes.
It's a very, very, very bright line. Do not go there.
Also, why would we even think this way? We've got Dragon2, we've got soyuz. It's been good enough for 10 years (and yes, I've not been enitrely happy with soyuz) so it's tenable until Starliner get's built PROPERLY.
It makes absolutely zero sense to rush Starliner. Write a check to the Russians if needed, we've done a whole lot of that over the last 10 years without much fuss.
Agree. And yet... (see, e.g., Moon 2024, but that's not on topic here). Schedule pressure will always exist.
Saying we've got Dragon and Soyuz is fine until you look at that period of time when we had three cargo vehicles fail in the span of eight months: Orb-3 failed on October 28, 2014, 59P failed on April 28, 2015, and SpX-7 failed on June 28, 2015 (aside: what in the world is up with the 28th of the month??).
Don't for one moment think that can't happen again. We were fine for the loss of cargo vehicles, because of the stockpile of supplies on ISS. Not so with crew.
-
#1348
by
Rocket Science
on 06 Mar, 2020 18:48
-
I'd like to start out by saying if Boeing and NASA do what they said they'd do on the call, I'd be pretty satisfied
Agreed. The tonal shift from the OFT post-flight presser to now is encouraging I feel more confident both Boeing and NASA are very serious about looking under all the rocks, and fixing root causes of what allowed the OFT issues to make it all the way through to flight.The reluctance of anyone to want to say "we have to fly OFT" is a little disappointing
It was very interesting watching the careful dance around the giant elephant in the room. Lovarro basically indicating that Boeing would propose next steps to certify for the crewed flight test which could result in an OFT re-fly, or could be done differently, and Boeing saying that they are prepared to re-fly but would do whatever NASA wants, in what seemed a little like game of hot potato. If left me with the impression that whoever is going to make that decision maybe wasn't present on the call (which would mean Bridenstine, presumably). That is, of course, an inference, and it could just be nobody wants to talk about the decision when it seems clear it won't be decided for some time.
I'll just take a wait and see on this one...
-
#1349
by
Metalskin
on 06 Mar, 2020 19:37
-
I'm not seeing that. I think the issues are more complicated.
Obviously, you don't want to just say "go refly" before you've taken a close look at everything. Keep in mind they are also experiencing some schedule pressure to stop depending on Soyuz for crew transport, and they have to make sure they can maintain crewed status, while also protecting for the possibility that something goes wrong with one of the other crewed vehicles. Simply saying "it's obvious they're going to pay" is massively oversimplifying, if not wrong altogether.
My bold.
This is an absolutely, 100%, prohibited attitude. It's NOT ok to let scheduling issues drive technical and/or safety and/or QA processes.
It's a very, very, very bright line. Do not go there.
Also, why would we even think this way? We've got Dragon2, we've got soyuz. It's been good enough for 10 years (and yes, I've not been enitrely happy with soyuz) so it's tenable until Starliner get's built PROPERLY.
It makes absolutely zero sense to rush Starliner. Write a check to the Russians if needed, we've done a whole lot of that over the last 10 years without much fuss.
Ironically two key pressures on code coverage for testing is time and cost. If you have deadlines, code coverage for testing tends to get dropped, because it often/normally takes longer to write the tests than it does to write the code. And that leads into costs. A company trying to reduce costs will also cut back on code coverage due to the cost.
Personally, my view is that approaching 100% code coverage is quite possible. In commercial/retail applications, it's often not desirable as the cost outweighs the benefit, but I would have thought that mission critical systems would change this equation.
-
#1350
by
leovinus
on 06 Mar, 2020 19:39
-
Lovarro: Two specific things: one thing in IRT we had a routine where there might have been four logical conditions, four ways the software could have run. Didn't check all four conditions, was a designer/tester choice. Not unusual you don't test all logical condition, do need to recognize we do need to test all logical conditions now. Process side, delegated too much authority to software board to approve changes and actions as it applies to software, software/hardware integration change should have been brought up all the way to engineering review board.
Wow. These are not subtle mistakes/omissions, but really basic ones. I cannot understand how a mature organisation could make such mistakes. This is what I might expect for an organisation with little experience that doesn’t understand the significance of what it is doing, in terms of criticality, and/or the importance of hardware & software integration.
If you're suggesting it's "basic" to do 100% code coverage in a hardware integration lab, I'm not sure I buy that. That's nowhere near as cheap or easy as people in this thread are making it out to be. You'd be talking millions of test cases at the absolute least.
Yet SpaceX manages to do it. This is human spaceflight. If you can’t commit to 100% code coverage, get out of the business. If you think it’s too expensive or difficult to do so, get out of the business. This isn’t a joke. If they can’t do what needs to be done they have no business attempting to send humans to space.
[citation needed]
I'm not sure even shuttle flight software got 100% code coverage, though they got close, and that was enormously expensive.
NASA used the SAIL OV-095 full avionics and electronics simulator for that, the only "Orbiter" that flew every mission before the actual mission.
And that was run by Rockwell International until Boeing picked it up in 1996.
Thank you for that pointer to Software Avionics Integration Laboratory (SAIL). Out of interest, I went to the Nasa Technical Reports Server (NTRS) and had a look for details. The best hit IMHO was "The Legacy of Space Shuttle Flight Software", report
20110014946 on Shuttle avionics software. The abstract explains:
This paper will present an overview of the evolution of the Primary Avionics Software System (PASS) project and processes over thirty years, an argument for strong statistical control of software processes with examples, an overview of the success story for identifying and driving out errors before flight, a case study of the few significant software issues and how they were either identified before flight or slipped through the process onto a flight vehicle, and identification of the valuable lessons learned over the life of the project.
which sounds very relevant for the discussion here on Starliner software. I was floored reading the "Lessons Learned" Section IX, p33-35, and then comparing to what just played out since Starliner OFT. Here are some striking quotes:
b) Operational Scenarios
All possible operational scenarios, both nominal and off-nominal, which the software could be exposed to should be identified, addressed in requirements, accommodated via design/code, and tested.
Note the mention of "off-nominal" scenarios which reminded me of the out-of-control thrusters when the MET was wrong.
c) Interface Testing
Software Interface Control Document requirements should be explicitly verified in an end-to-end manner. Two PASS in-flight software problems were due to the failure to completely verify the PASS Systems Management (SM) to Spacelab ICD. Both required in-flight patches due to mission objective impacts for specific payloads.
End-to-end testing recommendation. Explicit. Whether or not end-to-end testing was discussed for Starliner, it is recommended right there in the Nasa report on Shuttle avionics under "lessons learned".
h) Hardware / Software Integration Testing
It is extremely valuable to utilize flight or flight-like hardware in end-to-end hardware/software integration testing. The shuttle flight software team followed the principle of “test like you fly” by utilizing flight-like GPCs and other hardware in integrated testing. Getting as close to actual hardware performance and timing is essential.
End-to-end testing again.
i) Manual Processes
Many processes such as late updates to flight reconfiguration are initially done manually. Three in-flight software problems in the 1983-1985 period were introduced this way. Manual processes require continuous management oversight to insure rigorous and correct execution.
What Can Be Learned from PASS’ Most Critical Error?
Historically, the moment of greatest risk to a crew from a PASS fault occurred on June 26, 1984 when the STS-41D launch countdown was aborted at T – 6 seconds when PASS detected an anomaly in orbiter's main engine number three. Without the fortuitous SSME anomaly, STS-41D would have launched with about a 1 in 6 chance of being unable to separate the Solid Rocket Booster (SRB) or the External Tank (ET) which could have resulted in the loss of crew and vehicle.
While it is great that Boeing and Nasa are turning a corner with the recommendations of the IRT, I still feel a lot of the OFT issues could have been avoided based on known Nasa "lessons learned" from previous spaceflight vehicles such as the Shuttle.
-
#1351
by
ThePonjaX
on 06 Mar, 2020 19:57
-
In case anyone has forgotten, Boeing Defense, Space & Security Division is the modern amalgamation of these past companies--all of them involved in NASA's most significant triumphs and failures.
* North American Aviation
* Rockwell Corporation
* McDonnell Douglas
Despite the rather embarrassing OFT, we've watched Boeing or its past incarnations:
* Put men on the Moon (Boeing S-I-C stage, S-II stage, Apollo CSM)
* Construct the ISS US modules
* Build the X-37B military spaceplane
* Create the Surveyor and Lunar Orbiter probes
* blah, blah, blah
I can go on and on, but I think people have have enough of my lists.
What I strongly suspect today's Boeing has suffered is s dramatic brain drain of people who knew the special needs of human spacecraft. I can't defend how remarkably, fundamentally strange, if not stupid, the technical fallout has been.
It's as if Boeing's decisions have resurrected the many ghosts of their acquisitions all over. Those who forget history...
Do you ever worked in a company which is taken over by other one? I did, twice. After 5 fivers one company is in control, the methods and processes are from the winner, after 10 years the other company is forgotten.
The other factor is simpler: time. People ages, changes, retires. The company changes over time.
So really the Boeing achievements are now the past, nothing on that list helps Boeing to make a better spacecraft. Boeing seems to be evolved in a company which has serious problems to manage and implement modern projects where software and hardware has the same importance.
Can Boeing do it better? yes, of course but we don't know if they are willing to implement the necessary measures because some causes seems to be in the core of company. I hope so because could be great for the Starliner and Max issues.
BTW I'm not naive I think this happens to all the companies, for example in the future we'll see how Spacex evolves, now they're the old Boeing: a young, vibrant engineering company with a visionary leader. In the future? who knows.
-
#1352
by
gaballard
on 06 Mar, 2020 20:04
-
Lovarro: Two specific things: one thing in IRT we had a routine where there might have been four logical conditions, four ways the software could have run. Didn't check all four conditions, was a designer/tester choice. Not unusual you don't test all logical condition, do need to recognize we do need to test all logical conditions now. Process side, delegated too much authority to software board to approve changes and actions as it applies to software, software/hardware integration change should have been brought up all the way to engineering review board.
Wow. These are not subtle mistakes/omissions, but really basic ones. I cannot understand how a mature organisation could make such mistakes. This is what I might expect for an organisation with little experience that doesn’t understand the significance of what it is doing, in terms of criticality, and/or the importance of hardware & software integration.
If you're suggesting it's "basic" to do 100% code coverage in a hardware integration lab, I'm not sure I buy that. That's nowhere near as cheap or easy as people in this thread are making it out to be. You'd be talking millions of test cases at the absolute least.
Yet SpaceX manages to do it. This is human spaceflight. If you can’t commit to 100% code coverage, get out of the business. If you think it’s too expensive or difficult to do so, get out of the business. This isn’t a joke. If they can’t do what needs to be done they have no business attempting to send humans to space.
[citation needed]
I'm not sure even shuttle flight software got 100% code coverage, though they got close, and that was enormously expensive.
SpaceX: Lessons Learned, from the 2013 Embedded Linux Conference:
Automation is important, and continuous integration is "very valuable", Rose said. He suggested building for every platform all of the time, even for "things you don't use any more". SpaceX does that and has found interesting problems when building unused code. Unit tests are run from the continuous integration system any time the code changes. "Everyone here has 100% unit test coverage", he joked, but running whatever tests are available, and creating new ones is useful. When he worked on video games, they had a test to just "warp" the character to random locations in a level and had it look in the four directions, which regularly found problems.
That whole article is very eye-opening in how different the testing culture is at SpaceX versus Boeing.
Some other quotes:
Rose said "software is invisible". To make software more visible, you need to know what it is doing, he said, which means creating "metrics on everything you can think of". With a rocket, you can't just connect via JTAG and "fire up gdb", so the software needs to keep track of what it is doing. Those metrics should cover areas like performance, network utilization, CPU load, and so on.
The metrics gathered, whether from testing or real-world use, should be stored as it is "incredibly valuable" to be able to go back through them, he said. For his systems, telemetry data is stored with the program metrics, as is the version of all of the code running so that everything can be reproduced if needed.
SpaceX has programs to parse the metrics data and raise an alarm when "something goes bad" . . . The same programs run on the data whether it is generated from a developer's test, from a run on the spacecraft, or from a mission. Any failures should be seen as an opportunity to add new metrics
. . . He likes to "geek out on error reporting" . . .
When the build fails, it should "fail loudly" with a "monitor that starts flashing red" and email to everyone on the team. When that happens, you should "respond immediately" to fix the problem
I'd be very interested to know how many of these processes Boeing has in place.
-
#1353
by
dglow
on 06 Mar, 2020 20:20
-
SpaceX: Lessons Learned, from the 2013 Embedded Linux Conference:
Automation is important, and continuous integration is "very valuable", Rose said. He suggested building for every platform all of the time, even for "things you don't use any more". SpaceX does that and has found interesting problems when building unused code. Unit tests are run from the continuous integration system any time the code changes. "Everyone here has 100% unit test coverage", he joked, but running whatever tests are available, and creating new ones is useful. When he worked on video games, they had a test to just "warp" the character to random locations in a level and had it look in the four directions, which regularly found problems.
That whole article is very eye-opening in how different the testing culture is at SpaceX versus Boeing.
<snip>
I'd be very interested to know how many of these processes Boeing has in place.
SpaceX benefits immensely from running a modern software stack, allowing the use of modern tools, enabling the use of modern methodologies.
I suspect Boeing is hamstrung by their preference for legacy systems. How are these systems programmed? A 20-year-old subsystem might be perfectly fine. But since its creation the software world has evolved dramatically.
-
#1354
by
gaballard
on 06 Mar, 2020 20:32
-
SpaceX: Lessons Learned, from the 2013 Embedded Linux Conference:
Automation is important, and continuous integration is "very valuable", Rose said. He suggested building for every platform all of the time, even for "things you don't use any more". SpaceX does that and has found interesting problems when building unused code. Unit tests are run from the continuous integration system any time the code changes. "Everyone here has 100% unit test coverage", he joked, but running whatever tests are available, and creating new ones is useful. When he worked on video games, they had a test to just "warp" the character to random locations in a level and had it look in the four directions, which regularly found problems.
That whole article is very eye-opening in how different the testing culture is at SpaceX versus Boeing.
<snip>
I'd be very interested to know how many of these processes Boeing has in place.
SpaceX benefits immensely from running a modern software stack, allowing the use of modern tools, enabling the use of modern methodologies.
I suspect Boeing is hamstrung by their preference for legacy systems. How are these systems programmed? A 20-year-old subsystem might be perfectly fine. But since its creation the software world has evolved dramatically.
Totally agree. And it's fair to point out Musk was primarily a software guy when he started SpaceX, so that definitely helped them get on the right path early on. I think another quote from that link I posted sums it all up pretty well:
The SpaceX team has had to face the common problem that "no one outside of software understands software".
-
#1355
by
SoftwareDude
on 06 Mar, 2020 21:23
-
What I would have like to know:
1. Why did the emulators being wrong not break Boeing's software? The emulators were the spec? I am confused about this. Was both the spec wrong and the emulators wrong or there was no spec? They just hacked to the emulator? Or what?
2. What are the Black-box testing requirements? How many paths, happy and not happy, does NASA require [Black-box testing[/i] of. It didn't sound like there are any not happy paths tested, and no specific requirements, which I find astounding. They seem happy to talk about unit tests which are a kind of white-box test and are meaningless regarding testing functionality.
3. Is it possible, that the Flight computers don't have good software development ecologies like code coverage tools, libraries, and IDEs? Do they use an Eclipse plug-in? I was shocked when NASA thought that incomplete code coverage for unit-tests/white-box tests is common but they are going to fix that.
4. Were the emulators emulating an older version of Atlas/Centaur? I was also shocked that Boeing's Starliner development group didn't have anyone on staff who knew enough to know that the emulators weren't... uh emulating. There must be people who know about Atlas V payload interfaces; can they hire them?
5. Which brings up another question, does Boeing have on staff test engineers who only write test software?
6. Regarding the contract for Starliner. Why is NASA so hesitant to call for re-flight when Boeing says they are happy to do it? This speaks volumes about who has to pay for it.
7. What was Boeing's criteria for being ready for OFT? They thought they had the "happy path" working? Good enough?
-
#1356
by
JonathanD
on 06 Mar, 2020 22:05
-
I think it's safe to say we are lucky that they had the MET problem, otherwise who knows when we would have found out about all the rest of this stuff. Yikes.
-
#1357
by
dglow
on 06 Mar, 2020 22:57
-
<...> who knows when we would have found out about all the rest of this stuff.
Moments after the service module separated from the capsule, presumably
-
#1358
by
SoftwareDude
on 06 Mar, 2020 23:23
-
<...> who knows when we would have found out about all the rest of this stuff.
Moments after the service module separated from the capsule, presumably
No one knows what could have happened and when. We only know that the first thing that was supposed to happen after separation didn't.
-
#1359
by
dglow
on 06 Mar, 2020 23:39
-
<...> who knows when we would have found out about all the rest of this stuff.
Moments after the service module separated from the capsule, presumably
No one knows what could have happened and when. We only know that the first thing that was supposed to happen after separation didn't.
I think they're pretty sure what
could have happened had they not resolved the jet mapping issue.