-
#720
by
erioladastra
on 24 Jan, 2020 00:45
-
Many of the issues being discussed here are inter-related:
Should the code check for inconsistencies even if can't determine which value is correct?
Would the astronauts, had they been onboard, have noticed in time?
Would they know what to do in such an event, in a timely manner?
Is it safer to go sub-orbital, and enter in the Indian ocean, or proceed to orbit?
Un-crewed software has, over many years, converged on a solution that adddresses all these concerns:
1) The code is full of self-checks. But if one fails, it does not try to correct the problem, it drops into "safe mode". Coders can add as many assertions as they wish without having to figure out what the spacecraft should do in each case.
2) Safe mode is as simple as possible and is super-intensively tested. Its only goals are stable attitude, power positive, minimal use of consumables, and enable communication.
3) It's the job of humans to figure out how to get the spacecraft out of safe mode.
Such an architecture would have solved all the problems on the mission, plus many more conjectured here.
There were lots of ways safe mode could have been triggered. Timer's disagree, excessive fuel consumption, radio link not as expected, etc.
Two of the main features of safe mode would have helped save the mission - don't use excessive fuel, and get in a good attitude for comms.
If astronauts had been on board, there is no question they would notice. The screen would say "Safe mode entered, manual input required".
There is also no question that the astronauts would know what to do. Since safe mode is the result of almost all problems, it's one they would surely train for - safe mode during orbital insertion, safe mode during ISS approach, etc.
This also helps with the question of whether it's better to re-enter in the middle of nowhere, or press to orbit. If the craft goes into safe mode while still sub-orbital, the astronauts could assess the situation and decide what is safer. If all thrusters are working fine under manual control, press on. But if they are not certain they can re-enter later, maybe better to passively re-enter in the Indian ocean.
Robotic missions have spent decades making all these tradeoffs work, and as far as I know almost every mission has gone into safe mode at least once (often soon after launch, when tasks are being performed for real for the first time). But I have no idea how similar the software is for crewed and un-crewed crafts. Maybe it should be more similar than it appears to be.
Would Safe Mode cause Starliner to burn up in the atmosphere because the insertion orbit brings it back if it cannot be reset without it happening?
Safe mode, which generally doesn't make sense for a crewed vehicle (that is what the humans do, adding more software complexity increases risk of something going wrong - my opinion) would have resulted in loss of vehicle and mission in this case.
-
#721
by
freddo411
on 24 Jan, 2020 02:44
-
Many of the issues being discussed here are inter-related:
Should the code check for inconsistencies even if can't determine which value is correct?
Would the astronauts, had they been onboard, have noticed in time?
Would they know what to do in such an event, in a timely manner?
Is it safer to go sub-orbital, and enter in the Indian ocean, or proceed to orbit?
Un-crewed software has, over many years, converged on a solution that adddresses all these concerns:
1) The code is full of self-checks. But if one fails, it does not try to correct the problem, it drops into "safe mode". Coders can add as many assertions as they wish without having to figure out what the spacecraft should do in each case.
2) Safe mode is as simple as possible and is super-intensively tested. Its only goals are stable attitude, power positive, minimal use of consumables, and enable communication.
3) It's the job of humans to figure out how to get the spacecraft out of safe mode.
Such an architecture would have solved all the problems on the mission, plus many more conjectured here.
There were lots of ways safe mode could have been triggered. Timer's disagree, excessive fuel consumption, radio link not as expected, etc.
Two of the main features of safe mode would have helped save the mission - don't use excessive fuel, and get in a good attitude for comms.
If astronauts had been on board, there is no question they would notice. The screen would say "Safe mode entered, manual input required".
There is also no question that the astronauts would know what to do. Since safe mode is the result of almost all problems, it's one they would surely train for - safe mode during orbital insertion, safe mode during ISS approach, etc.
This also helps with the question of whether it's better to re-enter in the middle of nowhere, or press to orbit. If the craft goes into safe mode while still sub-orbital, the astronauts could assess the situation and decide what is safer. If all thrusters are working fine under manual control, press on. But if they are not certain they can re-enter later, maybe better to passively re-enter in the Indian ocean.
Robotic missions have spent decades making all these tradeoffs work, and as far as I know almost every mission has gone into safe mode at least once (often soon after launch, when tasks are being performed for real for the first time). But I have no idea how similar the software is for crewed and un-crewed crafts. Maybe it should be more similar than it appears to be.
Would Safe Mode cause Starliner to burn up in the atmosphere because the insertion orbit brings it back if it cannot be reset without it happening?
Safe mode, which generally doesn't make sense for a crewed vehicle (that is what the humans do, adding more software complexity increases risk of something going wrong - my opinion) would have resulted in loss of vehicle and mission in this case.
That's only true due to the odd decision to fly into a sub orbital trajectory. That choice was sub optimal, as demonstrated by the failures.
Safe mode makes a lot of sense for vehicles in orbit.
-
#722
by
Semmel
on 24 Jan, 2020 08:54
-
*snip*
Look, I'm just going to say it. The only reason Boeing didn't do an in-flight abort was to save money by not having to purchase another Atlas V and all associated costs with an IFA mission.
*snip*
You do not have to use the launch vehicle to do an in-flight abort test. All other in-flight abort tests prior to SpaceX's IFA test have used a much cheaper stand-in rocket booster for the in-flight abort test.
Correct. Apollo and, more recently, Orion (2019-07-02), used a stand-in LV for their test. Dragon is the only spacecraft to ever use it's designated standard LV to perform an IFA test. Boeing could easily have done the same as Apollo and Orion, but chose not to.
F9 was probably cheaper than any alternative. No development of new interfaces. No development of new hardware, everything is just an other copy of existing stuff. One off developments are expensive. Not to mention, this was the 4th flight of that booster. It was practically free. The only cost was the second stage (which lacked its most expensive part, the engine) and the lack of an other use in the future of S1. But thats not real cost, thats just accounting trickery if they have more boosters than they need anyway. So using F9 for the abort test could have been a straight up economical decision, not a "test as you fly" decision.
-
#723
by
clongton
on 24 Jan, 2020 13:03
-
Boeing could have used the same LV as Orion. Just too cheap to do it.
-
#724
by
Cherokee43v6
on 24 Jan, 2020 13:26
-
*snip*
Look, I'm just going to say it. The only reason Boeing didn't do an in-flight abort was to save money by not having to purchase another Atlas V and all associated costs with an IFA mission.
*snip*
You do not have to use the launch vehicle to do an in-flight abort test. All other in-flight abort tests prior to SpaceX's IFA test have used a much cheaper stand-in rocket booster for the in-flight abort test.
Correct. Apollo and, more recently, Orion (2019-07-02), used a stand-in LV for their test. Dragon is the only spacecraft to ever use it's designated standard LV to perform an IFA test. Boeing could easily have done the same as Apollo and Orion, but chose not to.
F9 was probably cheaper than any alternative. No development of new interfaces. No development of new hardware, everything is just an other copy of existing stuff. One off developments are expensive. Not to mention, this was the 4th flight of that booster. It was practically free. The only cost was the second stage (which lacked its most expensive part, the engine) and the lack of an other use in the future of S1. But thats not real cost, thats just accounting trickery if they have more boosters than they need anyway. So using F9 for the abort test could have been a straight up economical decision, not a "test as you fly" decision.
Add to that SpaceX's vertical integration. They're not going to go outside for something they can do themselves.
On a related note, SpaceX's original plan was to use the second grasshopper follow-on vehicle with three engines to perform the IFA back in 2015. Once the decision was made to do it with a 'flight vehicle' instead of a boilerplate like was used in the static abort test the amount of time meant that they would have reflown boosters available to provide a higher fidelity test.
-
#725
by
LouScheffer
on 24 Jan, 2020 13:28
-
Safe mode, which generally doesn't make sense for a crewed vehicle (that is what the humans do, adding more software complexity increases risk of something going wrong - my opinion) would have resulted in loss of vehicle and mission in this case.
On the contrary, safe mode, had it been implemented, would have saved this mission entirely (IMO, of course). What were the two main problems? Excessive fuel use, and difficulty communicating to fix the problem. Safe mode solves both these issues - it goes for a stable attitude, minimizing fuel consumption, and establishes communication. The the ground would have had a half hour, or more, so send commands. Once they did that, there would have been the normal amount of fuel, and the mission could have continued to docking.
Even a sub-orbital trajectory (at least in LEO) gives plenty of time for ground and/or astronaut intervention, provided the spacecraft does nothing stupid in the meantime. And that's what safe mode is, don't do something stupid....
-
#726
by
laszlo
on 24 Jan, 2020 13:40
-
Boeing could have used the same LV as Orion. Just too cheap to do it.
Boeing is a publicly-traded company responsible to its shareholders. As long as the customer agrees with no IFA demo, spending extra money on an unrequested exercise is financially irresponsible, as well as a waste of time.
You may not agree with the technical decision jointly made by NASA and Boeing, but financially it was the correct thing for Boeing, as guardian of their stockholders' money, to do. It wasn't just being cheap.
-
#727
by
ShaunML09
on 24 Jan, 2020 14:06
-
Boeing appears to have over-complicated the Starliner due the need for orbital insertion by Starliner (versus second stage) - just looking at the thruster comparison for the exact same mission objective, its crazy how complicated the Starliner is compared to Crew Dragon. Elon's "no part is the best part" theory appeared to have played out here.
Also, their software design is still objectively bad given the MET timer issue setting off a cascade of failures including failed orbit insertion burn, overuse of thrusters, and TDRS communications.
I'm also baffled by the fact that for Starliner to connect to the TDRS satellites it had to be where it was supposed to be, and when it wasn't where it was supposed to be, it couldn't connect to TDRS to fix its critical failure (MET Timer). And the only way at that point in the mission to connect to TDRS was to know where it located, which it couldn't because it couldn't connect to TDRS due its critical failure (MET timer). Talk about a circular logic of failure.
-
#728
by
mn
on 24 Jan, 2020 14:12
-
Safe mode, which generally doesn't make sense for a crewed vehicle (that is what the humans do, adding more software complexity increases risk of something going wrong - my opinion) would have resulted in loss of vehicle and mission in this case.
On the contrary, safe mode, had it been implemented, would have saved this mission entirely (IMO, of course). What were the two main problems? Excessive fuel use, and difficulty communicating to fix the problem. Safe mode solves both these issues - it goes for a stable attitude, minimizing fuel consumption, and establishes communication. The the ground would have had a half hour, or more, so send commands. Once they did that, there would have been the normal amount of fuel, and the mission could have continued to docking.
Even a sub-orbital trajectory (at least in LEO) gives plenty of time for ground and/or astronaut intervention, provided the spacecraft does nothing stupid in the meantime. And that's what safe mode is, don't do something stupid....
If it doesn't know where or when it is, there is little guarantee that it would be able to establish communications, as we see that they in fact had difficulty establishing communications initially, and perhaps only got lucky that they were able to establish communications in time to save part of the mission.
-
#729
by
LouScheffer
on 24 Jan, 2020 18:53
-
If it doesn't know where or when it is, there is little guarantee that it would be able to establish communications, as we see that they in fact had difficulty establishing communications initially, and perhaps only got lucky that they were able to establish communications in time to save part of the mission.
This "didn't know where it was" seems COMPLETELY implausible. That's the ENTIRE POINT of an inertial navigation system. It knew where it was on the pad, and it could correctly detect the Earth rotation, or it would fail pre-launch tests. Even if it gets no updates whatsoever, it knows its location within a few km and its orientation within a few degrees. That's plenty good enough for aiming antennas. And even if has some idea of where is should be based on time from launch, this likely can't include orientation, plus it should wildly disagree with the INS. And assuming it has multiple INS systems, they should agree with each other and not the time based estimate.
Overall, I cannot see how it can possible not know its approximate location and orientation. Otherwise, spacecraft, crewed and un-crewed, have been incorporating useless INS systems for decades....
-
#730
by
ShaunML09
on 24 Jan, 2020 19:50
-
If it doesn't know where or when it is, there is little guarantee that it would be able to establish communications, as we see that they in fact had difficulty establishing communications initially, and perhaps only got lucky that they were able to establish communications in time to save part of the mission.
This "didn't know where it was" seems COMPLETELY implausible. That's the ENTIRE POINT of an inertial navigation system. It knew where it was on the pad, and it could correctly detect the Earth rotation, or it would fail pre-launch tests. Even if it gets no updates whatsoever, it knows its location within a few km and its orientation within a few degrees. That's plenty good enough for aiming antennas. And even if has some idea of where is should be based on time from launch, this likely can't include orientation, plus it should wildly disagree with the INS. And assuming it has multiple INS systems, they should agree with each other and not the time based estimate.
Overall, I cannot see how it can possible not know its approximate location and orientation. Otherwise, spacecraft, crewed and un-crewed, have been incorporating useless INS systems for decades....
Boeing specifically said in the press conference that Starliner's "eyes were not open at that time" - which is why its TDRS antennas were not aimed in the correct location, causing a loss of communication with the vehicle for 8 minutes while it overtaxed its thrusters. Once they regained communication, they reset the MET and took control of the vehicle.
-
#731
by
LouScheffer
on 24 Jan, 2020 20:08
-
This "didn't know where it was" seems COMPLETELY implausible. That's the ENTIRE POINT of an inertial navigation system. It knew where it was on the pad, and it could correctly detect the Earth rotation, or it would fail pre-launch tests. Even if it gets no updates whatsoever, it knows its location within a few km and its orientation within a few degrees. That's plenty good enough for aiming antennas. And even if has some idea of where is should be based on time from launch, this likely can't include orientation, plus it should wildly disagree with the INS. And assuming it has multiple INS systems, they should agree with each other and not the time based estimate.
Overall, I cannot see how it can possible not know its approximate location and orientation. Otherwise, spacecraft, crewed and un-crewed, have been incorporating useless INS systems for decades....
Boeing specifically said in the press conference that Starliner's "eyes were not open at that time" - which is why its TDRS antennas were not aimed in the correct location, causing a loss of communication with the vehicle for 8 minutes while it overtaxed its thrusters. Once they regained communication, they reset the MET and took control of the vehicle.
But the entire point of having an INS is that it does not need "eyes open". That's why it's called "Inertial". Eyes would help, but we are talking about reducing the error from km to meters with GPS, and degrees to milli-degrees with star trackers, completely unnecessary for pointing antennas. And again, the whole point of an INS is to know your position and orientation, at least approximately, SPECIFICALLY WHEN outside input is unavailable or unreliable.
-
#732
by
HVM
on 24 Jan, 2020 20:14
-
Whole time they said in the stream that it had "bad attitude" and thrusters firings that didn't make any sense, demonstrated that. StarLiner de facto didn't know its orientation, heck it didn't know the time... You think that Boeing avionic team must have some sophisticated back ups, when actually just their data retrieval seems to lack basic error and sanity checks. But I agree it should know just using IMUs, but clearly it did not.
-
#733
by
saliva_sweet
on 24 Jan, 2020 20:24
-
On the contrary, safe mode, had it been implemented, would have saved this mission entirely (IMO, of course). What were the two main problems? Excessive fuel use, and difficulty communicating to fix the problem. Safe mode solves both these issues - it goes for a stable attitude, minimizing fuel consumption, and establishes communication. The the ground would have had a half hour, or more, so send commands. Once they did that, there would have been the normal amount of fuel, and the mission could have continued to docking.
Again, I doubt this. ISS missions have instantaneous (i.e. very brief) launch windows. Even if the Starliner tanks had been full after recovery to orbit it would have been a LOM as the vehicle would have been in the wrong orbit to go to ISS.
-
#734
by
ShaunML09
on 24 Jan, 2020 21:07
-
This "didn't know where it was" seems COMPLETELY implausible. That's the ENTIRE POINT of an inertial navigation system. It knew where it was on the pad, and it could correctly detect the Earth rotation, or it would fail pre-launch tests. Even if it gets no updates whatsoever, it knows its location within a few km and its orientation within a few degrees. That's plenty good enough for aiming antennas. And even if has some idea of where is should be based on time from launch, this likely can't include orientation, plus it should wildly disagree with the INS. And assuming it has multiple INS systems, they should agree with each other and not the time based estimate.
Overall, I cannot see how it can possible not know its approximate location and orientation. Otherwise, spacecraft, crewed and un-crewed, have been incorporating useless INS systems for decades....
Boeing specifically said in the press conference that Starliner's "eyes were not open at that time" - which is why its TDRS antennas were not aimed in the correct location, causing a loss of communication with the vehicle for 8 minutes while it overtaxed its thrusters. Once they regained communication, they reset the MET and took control of the vehicle.
But the entire point of having an INS is that it does not need "eyes open". That's why it's called "Inertial". Eyes would help, but we are talking about reducing the error from km to meters with GPS, and degrees to milli-degrees with star trackers, completely unnecessary for pointing antennas. And again, the whole point of an INS is to know your position and orientation, at least approximately, SPECIFICALLY WHEN outside input is unavailable or unreliable.
I agree they "should" know but Boeing specifically stated Starliner didn't know where it was at that time so I'm really unclear what we are discussing here. As outlined above, I think its a fundamental and dangerous software issue.
Starliner clearly knew later on where it was, but not until after the significant issues had already happened.
See Minute 25-27 min - Chilton very clearly states Starliner did not know where it was when it separated from Atlas second stage due to MET timer and as a result, could not communicate with TDRS.
"we got off that Atlas V not where we expected to be"
"This is a point in the mission where we tell the spacecraft where it is - not where it opens its eyes and looks, which is most of the rest of the mission" "it was further from TDRS than it thought"
"from an attitude perspective...because the vehicle wasn't where we expected it to be and it was not where it thought was, it wasn't quite pointing its antennas at TDRS quite right"
"you add those factors together and it took a little more time to connect than we expected"
-
#735
by
abaddon
on 24 Jan, 2020 21:10
-
Boeing could have used the same LV as Orion. Just too cheap to do it.
Boeing is a publicly-traded company responsible to its shareholders. As long as the customer agrees with no IFA demo, spending extra money on an unrequested exercise is financially irresponsible, as well as a waste of time.
You may not agree with the technical decision jointly made by NASA and Boeing, but financially it was the correct thing for Boeing, as guardian of their stockholders' money, to do. It wasn't just being cheap.
Pad abort was also an optional milestone, and yet Boring chose to do that one, apparently squandering shareholder funds. Or maybe it’s a bad argument.
-
#736
by
HVM
on 24 Jan, 2020 21:40
-
"from an attitude perspective...because the vehicle wasn't where we expected it to be and it was not where it thought was, it wasn't quite pointing its antennas at TDRS quite right"
Is it possible that whole FTINU data retrieval was scrambled and Starliner's IMUs were initialized/adjusted erroneously?
At least this make now sense:
-
#737
by
freddo411
on 24 Jan, 2020 22:19
-
"from an attitude perspective...because the vehicle wasn't where we expected it to be and it was not where it thought was, it wasn't quite pointing its antennas at TDRS quite right"
Is it possible that whole FTINU data retrieval was scrambled and Starliner's IMUs were initialized/adjusted erroneously?
At least this make now sense:
I can't help thinking that Boeing's efforts have descended into Monty Python Cheese shop territory
-
#738
by
erioladastra
on 25 Jan, 2020 00:36
-
Whole time they said in the stream that it had "bad attitude" and thrusters firings that didn't make any sense, demonstrated that. StarLiner de facto didn't know its orientation, heck it didn't know the time... You think that Boeing avionic team must have some sophisticated back ups, when actually just their data retrieval seems to lack basic error and sanity checks. But I agree it should know just using IMUs, but clearly it did not.
No that is not correct but I can't tell if the wrong term was used in the press conference or what. But there is a big difference between knowing where you are (which Starliner did know) and it not being the right or expected (which I guess some people have called bad).
-
#739
by
erioladastra
on 25 Jan, 2020 00:37
-
Boeing appears to have over-complicated the Starliner due the need for orbital insertion by Starliner (versus second stage) - just looking at the thruster comparison for the exact same mission objective, its crazy how complicated the Starliner is compared to Crew Dragon. Elon's "no part is the best part" theory appeared to have played out here.
Also, their software design is still objectively bad given the MET timer issue setting off a cascade of failures including failed orbit insertion burn, overuse of thrusters, and TDRS communications.
I'm also baffled by the fact that for Starliner to connect to the TDRS satellites it had to be where it was supposed to be, and when it wasn't where it was supposed to be, it couldn't connect to TDRS to fix its critical failure (MET Timer). And the only way at that point in the mission to connect to TDRS was to know where it located, which it couldn't because it couldn't connect to TDRS due its critical failure (MET timer). Talk about a circular logic of failure.
That is not how it works - it didn't have to be where it was planned to be - it can find TDRS but there are other issues (still being analyzed).