-
#160 by Lars-J on 21 Dec, 2019 19:42
-
I guess Soyuz has had success with an abort system that I think has been used 3 times (?) but I don't think we know how many injuries and deaths have occurred due to the abort system propellants.
Soyuz SAS has killed 3 ground crew (1966) and saved 6 flight crew (1975, 1983, 2018).
The 1975 abort happened after the launch abort system was jettisoned, so it does not apply here. It used the spacecrafts own thrusters to separate.
https://en.wikipedia.org/wiki/Soyuz_7K-T_No.39 -
#161 by clongton on 21 Dec, 2019 20:00
-
Yep, the other thing bugging me. Why is there a need to communicate with Earth at all? ... so why can't they have fully autonomous flight control software that would monitor and manage the flight?
There is no "need" to communicate with earth (for flight control). Starliner does have fully autonomous flight control software, but it failed. The communication that was attempted was to override the autonomous flight control software that had failed and take manual control of the spacecraft. But the communications dropout prevented that until too much fuel had been expended to make reaching ISS tenable. Had crew been aboard they would have done exactly what ground control attempted, but without the comm dropout, they would have succeeded where ground control did not. There would not have been the time lapse where excessive propellant got burn off. Crew would have taken control, oriented the spacecraft and executed the insertion burn, long before any comm gap had finished. By the time the spacecraft had come out of the comm drop the spacecraft would have been well on its way to the ISS, -
#162 by Jcc on 21 Dec, 2019 20:28
-
We simply don't have enough information to make that kind of judgment call right now. Let's not get ahead of ourselvesCould this anomaly be the result of human error based on the recent discussions on this thread?
I don't see how it could be anything but human error.
I think the subtlety of the "I don't see how it could be anything but human error" comment was missed. Its what HAL told Dave Bowman when Mission Control told Dave there was a discrepancy between HAL and the ground-based 9000 unit....
Oops. Meant to just like the post, but ended up responding. Human error! -
#163 by marsbase on 21 Dec, 2019 20:31
-
NASA Administrator Bridenstine has said that an uncrewed docking mission to the ISS is not required for a crewed mission to be launched. But will that crewed mission then have to dock manually? I don't think you would want an automated docking to take place for the first time with crew on board. And since all subsequent missions of Starliner to the ISS would be crewed, that would mean Starliner could never use it's automated docking system.
-
#164 by ShaunML09 on 21 Dec, 2019 20:33
-
Yep, the other thing bugging me. Why is there a need to communicate with Earth at all? ... so why can't they have fully autonomous flight control software that would monitor and manage the flight?
There is no "need" to communicate with earth (for flight control). Starliner does have fully autonomous flight control software, but it failed. The communication that was attempted was to override the autonomous flight control software that had failed and take manual control of the spacecraft. But the communications dropout prevented that until too much fuel had been expended to make reaching ISS tenable. Had crew been aboard they would have done exactly what ground control attempted, but without the comm dropout, they would have succeeded where ground control did not. There would not have been the time lapse where excessive propellant got burn off. Crew would have taken control, oriented the spacecraft and executed the insertion burn, long before any comm gap had finished. By the time the spacecraft had come out of the comm drop the spacecraft would have been well on its way to the ISS,
I really don't see it this way -
Starliner has an autonomous system that relies upon NASA/Boeing to tell it where it is - they specifically referenced the fact that at the time the Starliner did not "have its eyes open" and it relies upon the ground to provide altitude. Of course because it wasn't where it was expected, the ground couldn't provide that data due to misalignment of TDRSS. -
#165 by AJW on 21 Dec, 2019 21:07
-
Crew would have taken control, oriented the spacecraft and executed the insertion burn, long before any comm gap had finished. By the time the spacecraft had come out of the comm drop the spacecraft would have been well on its way to the ISS,
I thought that voice comm also went through TDRS, so crew would have been onboard a ship with malfunctioning thrusters and no communications with the ground. How are you so confident that a crew would have had time to diagnose the problem, take control of the ship, and with no ground comms make a decision to boost to a higher, more dangerous orbit, rather than taking the safer de-orbit path that they were already on and wait until comms returned? I would have to believe that crew's first reaction would have been to treat this as an emergency. The would have declared it, and if they didn't already know comms were out, they soon would. Next step is to evaluate the emergency and take first steps to mitigate. Key to most emergencies is to not act rashly, and once in a stable state, begin a second evaluation of the situation. Pressing on to the ISS after an emergency that even a day later may not be fully understood and no ground comms makes no rational sense to me. -
#166 by Jcc on 21 Dec, 2019 21:10
-
NASA Administrator Bridenstine has said that an uncrewed docking mission to the ISS is not required for a crewed mission to be launched. But will that crewed mission then have to dock manually? I don't think you would want an automated docking to take place for the first time with crew on board. And since all subsequent missions of Starliner to the ISS would be crewed, that would mean Starliner could never use it's automated docking system.
To my logic, they crew would have to dock manually only if the automated docking doesn’t work. If that doesn’t work, they go home, loss of mission.
If the Russian partners want to be cautious, they might have the ISS crew go into their Soyuz, just in case Starliner rams the ISS. -
#167 by JEF_300 on 21 Dec, 2019 21:23
-
Perhaps it's time to tone back the speculation (and discussion of speculation, and discussion of discussion of speculation) about the origin of the problem, and start the long wait for analysis.
-
#168 by LouScheffer on 21 Dec, 2019 21:25
-
I also write software for a living, and strongly disagree with this. I do the best I can, but I make mistakes. Fortunately, the software goes to others to be tested before it is released. The quality of these tests, and the feedback from them, has a huge function on the quality of the final results.Prove that any of this has stemmed from "poor corporate/managerial culture".
I've had good and bad management. One thing I do is write control software. The quality of my control software has nothing to do with the quality of my management. In fact, management has exactly nothing to do with any of it. I don't know if that's the case inside Boeing, and neither do you.
Exactly. You and ONLY you are responsible for the quality of the product you provide. Blaming lousy product quality on bad management says less about management than it does about your own character.
Here is where management comes in. Management can assign really good folks to QA, who creatively think of ways to stress your software, or they can assign folks who will just test precisely what you fixed. They can think of QA as an asset that helps them maintain quality (and pay them well), or treat them as an overhead that should be minimized. If there is a workforce reduction, they can decide whether to fire a QA person or a coder. If a QA person finds a problem, they can reward them for preventing a customer failure, or castigate them as nit-pickers who are impacting the schedule. They can consider a move from (say) QA to coding as a promotion, or as a lateral move. There are any number of ways management can affect QA, and hence quality.
Writing software might seem like an activity where you and only you determine quality. But it's not - every programmer has blind spots and makes mistakes. It's a team effort to create quality software, and management has a big impact on the effectiveness of the team. -
#169 by LouScheffer on 21 Dec, 2019 21:48
-
I agree the astronauts would likely land and not press on. Put yourself in their place. You are in a brand new ship. The first thing you try results in unexpected thruster firings. You shut them off - now what? You'd have to think, what happens if I press on to ISS? Can I be *absolutely* sure the problem will not recur? If the same thing happens again after you raise orbit, it could be disastrous - not enough fuel for re-entry, and you die. If it re-occurs when right next to ISS, it could be worse - you could kill yourself AND the station crew. You would need to be absolutely sure the problem would not re-occur, and I don't see how you could be so certain in just a few minutes, even with the help of the ground. So the only sensible thing would be to re-enter as soon as possible, and live to fight another day.Crew would have taken control, oriented the spacecraft and executed the insertion burn, long before any comm gap had finished. By the time the spacecraft had come out of the comm drop the spacecraft would have been well on its way to the ISS,
I thought that voice comm also went through TDRS, so crew would have been onboard a ship with malfunctioning thrusters and no communications with the ground. How are you so confident that a crew would have had time to diagnose the problem, take control of the ship, and with no ground comms make a decision to boost to a higher, more dangerous orbit, rather than taking the safer de-orbit path that they were already on and wait until comms returned? I would have to believe that crew's first reaction would have been to treat this as an emergency. The would have declared it, and if they didn't already know comms were out, they soon would. Next step is to evaluate the emergency and take first steps to mitigate. Key to most emergencies is to not act rashly, and once in a stable state, begin a second evaluation of the situation. Pressing on to the ISS after an emergency that even a day later may not be fully understood and no ground comms makes no rational sense to me.
A very similar event happened on Gemini 8. They got unexpected thruster firings, swapped to a backup system, and got it under control. They even diagnosed it down to a single stuck thruster. They then re-entered at first opportunity, since they could no longer trust the primary attitude control and were down to a single backup system. The pilot on that mission was Neil Armstrong, and he was praised for making the right decision, even though some of the other astronauts grumbled that if had had more of the right stuff, the remaining portion of the mission could have possibly been saved. -
#170 by TheRadicalModerate on 21 Dec, 2019 21:53
-
I also write software for a living, and strongly disagree with this. I do the best I can, but I make mistakes. Fortunately, the software goes to others to be tested before it is released. The quality of these tests, and the feedback from them, has a huge function on the quality of the final results.Prove that any of this has stemmed from "poor corporate/managerial culture".
I've had good and bad management. One thing I do is write control software. The quality of my control software has nothing to do with the quality of my management. In fact, management has exactly nothing to do with any of it. I don't know if that's the case inside Boeing, and neither do you.
Exactly. You and ONLY you are responsible for the quality of the product you provide. Blaming lousy product quality on bad management says less about management than it does about your own character.
Here is where management comes in. Management can assign really good folks to QA, who creatively think of ways to stress your software, or they can assign folks who will just test precisely what you fixed. They can think of QA as an asset that helps them maintain quality (and pay them well), or treat them as an overhead that should be minimized. If there is a workforce reduction, they can decide whether to fire a QA person or a coder. If a QA person finds a problem, they can reward them for preventing a customer failure, or castigate them as nit-pickers who are impacting the schedule. They can consider a move from (say) QA to coding as a promotion, or as a lateral move. There are any number of ways management can affect QA, and hence quality.
Writing software might seem like an activity where you and only you determine quality. But it's not - every programmer has blind spots and makes mistakes. It's a team effort to create quality software, and management has a big impact on the effectiveness of the team.
I assume that Boeing Space, being mostly a defense contractor, still uses a One Spec to Rule Them All kind of software methodology:
1) Gather requirements.
2) Write the spec.
3) Design and code the software to the spec.
4) Test to the spec.
5) For problems discovered, modify the spec through some soul-destroying process.
6) Goto 3
Notice that this is an infinite loop, and therefore a reasonable approximation to the actual Boeing process.
In contrast, companies that were founded some time in the last 15 years use something like an Agile process:
a) Gather key requirements and throw them on a big list.
b) Determine a minimum viable product from the contents of the big list.
c) Code to that MVP, likely using test-driven software development.
d) QA is mostly playing with the MVP and not worrying too much about the final product yet.
e) Decide if this is something that you could actually ship. If so, goto h
f) Take output from QA and the big list to build the next iteration.
g) goto c
h) If the customer is NASA or DoD, now you write the spec and clear it with the customer.
i) Do formal QA against the spec.
This is obviously an oversimplification, because you have to conform to NASA or the DoD's commitment milestones, but it's really different from the One Spec to Rule Them All methodology that was cutting-edge in 1975.
Assuming that the MET problem is really just a configuration error, I'm not sure which methodology would have been more likely to catch it; maybe they're both equally bad. But the agile methodology puts a very high premium on usability, and a usable system is one where it's hard to make a configuration error.
I believe that the massive, massive cost and schedule overruns that the incumbents are incurring are likely largely driven by their discomfort with the fact that software now dictates their mechanical engineering practices and not the other way around. In contrast, the next-generation companies take this as an article of faith, and it guides all of their methodological practices.
That said, adapting Agile to mission-critical embedded systems is no picnic. However, there now appear to be existence proofs that it's possible. -
#171 by ZachF on 21 Dec, 2019 22:04
-
I also write software for a living, and strongly disagree with this. I do the best I can, but I make mistakes. Fortunately, the software goes to others to be tested before it is released. The quality of these tests, and the feedback from them, has a huge function on the quality of the final results.Prove that any of this has stemmed from "poor corporate/managerial culture".
I've had good and bad management. One thing I do is write control software. The quality of my control software has nothing to do with the quality of my management. In fact, management has exactly nothing to do with any of it. I don't know if that's the case inside Boeing, and neither do you.
Exactly. You and ONLY you are responsible for the quality of the product you provide. Blaming lousy product quality on bad management says less about management than it does about your own character.
Here is where management comes in. Management can assign really good folks to QA, who creatively think of ways to stress your software, or they can assign folks who will just test precisely what you fixed. They can think of QA as an asset that helps them maintain quality (and pay them well), or treat them as an overhead that should be minimized. If there is a workforce reduction, they can decide whether to fire a QA person or a coder. If a QA person finds a problem, they can reward them for preventing a customer failure, or castigate them as nit-pickers who are impacting the schedule. They can consider a move from (say) QA to coding as a promotion, or as a lateral move. There are any number of ways management can affect QA, and hence quality.
Writing software might seem like an activity where you and only you determine quality. But it's not - every programmer has blind spots and makes mistakes. It's a team effort to create quality software, and management has a big impact on the effectiveness of the team.
I assume that Boeing Space, being mostly a defense contractor, still uses a One Spec to Rule Them All kind of software methodology:
1) Gather requirements.
2) Write the spec.
3) Design and code the software to the spec.
4) Test to the spec.
5) For problems discovered, modify the spec through some soul-destroying process.
6) Goto 3
Notice that this is an infinite loop, and therefore a reasonable approximation to the actual Boeing process.
In contrast, companies that were founded some time in the last 15 years use something like an Agile process:
a) Gather key requirements and throw them on a big list.
b) Determine a minimum viable product from the contents of the big list.
c) Code to that MVP, likely using test-driven software development.
d) QA is mostly playing with the MVP and not worrying too much about the final product yet.
e) Decide if this is something that you could actually ship. If so, goto h
f) Take output from QA and the big list to build the next iteration.
g) goto c
h) If the customer is NASA or DoD, now you write the spec and clear it with the customer.
i) Do formal QA against the spec.
This is obviously an oversimplification, because you have to conform to NASA or the DoD's commitment milestones, but it's really different from the One Spec to Rule Them All methodology that was cutting-edge in 1975.
Assuming that the MET problem is really just a configuration error, I'm not sure which methodology would have been more likely to catch it; maybe they're both equally bad. But the agile methodology puts a very high premium on usability, and a usable system is one where it's hard to make a configuration error.
I believe that the massive, massive cost and schedule overruns that the incumbents are incurring are likely largely driven by their discomfort with the fact that software now dictates their mechanical engineering practices and not the other way around. In contrast, the next-generation companies take this as an article of faith, and it guides all of their methodological practices.
That said, adapting Agile to mission-critical embedded systems is no picnic. However, there now appear to be existence proofs that it's possible.
Having software under the same roof as engineering, production, and management also massively helps... The cost savings end up exceeding the sum of their parts. At modern bloated MIC companies you're looking at multiple roofs for each one of those categories. -
#172 by ncb1397 on 21 Dec, 2019 22:14
-
A very similar event happened on Gemini 8. They got unexpected thruster firings, swapped to a backup system, and got it under control. They even diagnosed it down to a single stuck thruster. They then re-entered at first opportunity, since they could no longer trust the primary attitude control and were down to a single backup system.
That backup system was the re-entry control system. Mission rules stated that they had to re-enter. None of that would necessarily apply to this situation. If the astronauts command the system to stop and it stops and they then manually test the RCS and everything is working normally and very little fuel as used, it is a totally different ball game than Gemini 8. It wasn't that they made a choice to end the mission, fuel was depleted because of the stuck thruster using it and the thrusting required to counteract it for a long period of time. There was total loss of attitude control while this was actually controlling attitude too precisely. -
#173 by Lee Jay on 21 Dec, 2019 22:27
-
Exactly. You and ONLY you are responsible for the quality of the product you provide. Blaming lousy product quality on bad management says less about management than it does about your own character.
This is absolutely, categorically false.
And yet, it's absolutely true, for me.QuoteIt has been proven mathematically that poor process (which is set by management)
At least for me, process is set by workers, and only signed-off on by management. Management doesn't understand the details or the work well enough to set the process. We explain the process we choose to them, and they comment. Then we agree, or adjust until we do. -
#174 by ChrisWilson68 on 21 Dec, 2019 22:28
-
We simply don't have enough information to make that kind of judgment call right now. Let's not get ahead of ourselvesCould this anomaly be the result of human error based on the recent discussions on this thread?
I don't see how it could be anything but human error.
Much like anybody stating the issue is company wide.
Any comparison to the Max problem is completely clueless
Yes, there is absolutely no way you could say that multiple major problems stemming from poor corporate/managerial culture across several divisions with increasing frequency can be the result of poor overall corporate/managerial culture.
Prove that any of this has stemmed from "poor corporate/managerial culture".
I've had good and bad management. One thing I do is write control software. The quality of my control software has nothing to do with the quality of my management. In fact, management has exactly nothing to do with any of it. I don't know if that's the case inside Boeing, and neither do you.
I've spend my whole career in software, and I completely disagree. If management has nothing to do with the quality of the software, management isn't doing management right.
-
#175 by Lee Jay on 21 Dec, 2019 22:29
-
I also write software for a living, and strongly disagree with this. I do the best I can, but I make mistakes. Fortunately, the software goes to others to be tested before it is released. The quality of these tests, and the feedback from them, has a huge function on the quality of the final results.Prove that any of this has stemmed from "poor corporate/managerial culture".
I've had good and bad management. One thing I do is write control software. The quality of my control software has nothing to do with the quality of my management. In fact, management has exactly nothing to do with any of it. I don't know if that's the case inside Boeing, and neither do you.
Exactly. You and ONLY you are responsible for the quality of the product you provide. Blaming lousy product quality on bad management says less about management than it does about your own character.
For me as well.QuoteHere is where management comes in. Management can assign really good folks to QA, who creatively think of ways to stress your software, or they can assign folks who will just test precisely what you fixed.
Fortunately, I don't have that limitation. The people who I check and who check me, are assigned by us. In other words, I seek out who I want to do the checking, and that's always the person who knows the most about the topic. -
#176 by arachnitect on 21 Dec, 2019 22:38
-
I agree the astronauts would likely land and not press on. Put yourself in their place. You are in a brand new ship. The first thing you try results in unexpected thruster firings. You shut them off - now what? You'd have to think, what happens if I press on to ISS? Can I be *absolutely* sure the problem will not recur? If the same thing happens again after you raise orbit, it could be disastrous - not enough fuel for re-entry, and you die. If it re-occurs when right next to ISS, it could be worse - you could kill yourself AND the station crew. You would need to be absolutely sure the problem would not re-occur, and I don't see how you could be so certain in just a few minutes, even with the help of the ground. So the only sensible thing would be to re-enter as soon as possible, and live to fight another day.Crew would have taken control, oriented the spacecraft and executed the insertion burn, long before any comm gap had finished. By the time the spacecraft had come out of the comm drop the spacecraft would have been well on its way to the ISS,
I thought that voice comm also went through TDRS, so crew would have been onboard a ship with malfunctioning thrusters and no communications with the ground. How are you so confident that a crew would have had time to diagnose the problem, take control of the ship, and with no ground comms make a decision to boost to a higher, more dangerous orbit, rather than taking the safer de-orbit path that they were already on and wait until comms returned? I would have to believe that crew's first reaction would have been to treat this as an emergency. The would have declared it, and if they didn't already know comms were out, they soon would. Next step is to evaluate the emergency and take first steps to mitigate. Key to most emergencies is to not act rashly, and once in a stable state, begin a second evaluation of the situation. Pressing on to the ISS after an emergency that even a day later may not be fully understood and no ground comms makes no rational sense to me.
A very similar event happened on Gemini 8. They got unexpected thruster firings, swapped to a backup system, and got it under control. They even diagnosed it down to a single stuck thruster. They then re-entered at first opportunity, since they could no longer trust the primary attitude control and were down to a single backup system. The pilot on that mission was Neil Armstrong, and he was praised for making the right decision, even though some of the other astronauts grumbled that if had had more of the right stuff, the remaining portion of the mission could have possibly been saved.
I'm pretty confident they would have at least put it in orbit and flown it to one of the selected landing sites. Even in an abort, the first step would be to take manual control and get good attitude. That would have gotten comm reestablished quicker. Problem was understood almost immediately. Fly the spacecraft and buy time.
Whether or not they would then press on to ISS? Who knows? -
#177 by Lars-J on 21 Dec, 2019 22:41
-
I also write software for a living, and strongly disagree with this. I do the best I can, but I make mistakes. Fortunately, the software goes to others to be tested before it is released. The quality of these tests, and the feedback from them, has a huge function on the quality of the final results.Prove that any of this has stemmed from "poor corporate/managerial culture".
I've had good and bad management. One thing I do is write control software. The quality of my control software has nothing to do with the quality of my management. In fact, management has exactly nothing to do with any of it. I don't know if that's the case inside Boeing, and neither do you.
Exactly. You and ONLY you are responsible for the quality of the product you provide. Blaming lousy product quality on bad management says less about management than it does about your own character.
Here is where management comes in. Management can assign really good folks to QA, who creatively think of ways to stress your software, or they can assign folks who will just test precisely what you fixed. They can think of QA as an asset that helps them maintain quality (and pay them well), or treat them as an overhead that should be minimized. If there is a workforce reduction, they can decide whether to fire a QA person or a coder. If a QA person finds a problem, they can reward them for preventing a customer failure, or castigate them as nit-pickers who are impacting the schedule. They can consider a move from (say) QA to coding as a promotion, or as a lateral move. There are any number of ways management can affect QA, and hence quality.
Writing software might seem like an activity where you and only you determine quality. But it's not - every programmer has blind spots and makes mistakes. It's a team effort to create quality software, and management has a big impact on the effectiveness of the team.
I’m also a software developer in a large organization, and I had to quote all of this because it is 100% true. Management priorities make a huge difference. My opinion about the quality of my work is just that - my opinion - until it has been thoroughly tested by QA. -
#178 by LouScheffer on 21 Dec, 2019 22:47
-
The job of management is to support the workers, if they need it. If they are doing more than that, then either they are bad or the workers are. If management is involved directly in the actual work of the company then something has gone horribly wrong.
In my experience, that defines mediocre management. In good management, the manager knows what is going on technically and helps the people who do the details. Managers who could do this on large projects are legendary (Werner von Braun, Robert Oppenheimer) and get results they would never have gotten with a hand-off management style. -
#179 by Jeff Lerner on 21 Dec, 2019 23:05
-
I’m a retired IT Project Manager...managed software development and maintenance projects for over 25 years...was deemed to be successful at it by my management ....my projects came in on time, on budget and with quality...finding “bugs” didn’t start and end with our QA teams...finding bugs started at the Requirements stage, Business analysis, technical analysis, coding, testing, etc...the further along the development cycle, the costlier it was to fix a problem..
I also know that the project development process is a 3 legged stool...in my experience, you can have a project that comes in on time, on budget or is of high quality...pick any two of those three, because those priorities from your clients will drive your development process..
IMHO, NASA (the client) is under tremendous pressure to get manned flight from U.S. soil flying again ASAP..my concern is that NASA is prepared to let quality slide from both Boeing (Developer) and SpaceX (Developer) to get Commercial Manned space flying ASAP...
