-
#1060 by vt_hokie on 10 Feb, 2020 01:39
-
Boeing might be having a hard time maintaining a top programming team because so many younger people probably think it's cooler to work for SpaceX or Tesla or Google, or to be somewhere else in Silicon Valley, instead of working for their grandparents' or great-grandparents' kind of company. I don't have the stats in front of me, but the average age of SpaceX and Google employees is unusually low, somewhere around 30, and an old established company like Boeing is probably more accommodating and welcoming of older employees (AARP does research and can validate such distinctions). ...
I thought that was supposed to be Boeing's strength, but I guess there are two ways to look at it. While SpaceX follows the startup model of hiring cheap young fresh-outs, chewing 'em up, and spitting 'em out before being saddled with a lot of legacy costs, more experienced engineers and developers could ostensibly give a more established company a leg up on quality. But then you read about the MCAS fiasco where software was outsourced to Indian programmers making $9/hr, and that assumption doesn't hold up. So, I don't know. I do know that while younger people do look at companies like SpaceX as cool, it doesn't take so long for things like work/life balance, salary, benefits, and retirement savings to start to look kind of cool too! -
#1061 by groknull on 10 Feb, 2020 03:17
-
Boeing might be having a hard time maintaining a top programming team because so many younger people probably think it's cooler to work for SpaceX or Tesla or Google, or to be somewhere else in Silicon Valley, instead of working for their grandparents' or great-grandparents' kind of company. I don't have the stats in front of me, but the average age of SpaceX and Google employees is unusually low, somewhere around 30, and an old established company like Boeing is probably more accommodating and welcoming of older employees (AARP does research and can validate such distinctions). ...
I thought that was supposed to be Boeing's strength, but I guess there are two ways to look at it. While SpaceX follows the startup model of hiring cheap young fresh-outs, chewing 'em up, and spitting 'em out before being saddled with a lot of legacy costs, more experienced engineers and developers could ostensibly give a more established company a leg up on quality. But then you read about the MCAS fiasco where software was outsourced to Indian programmers making $9/hr, and that assumption doesn't hold up. So, I don't know. I do know that while younger people do look at companies like SpaceX as cool, it doesn't take so long for things like work/life balance, salary, benefits, and retirement savings to start to look kind of cool too!
Somewhat off topic, but needs addressing... (my bold above)
Not all startups "hire cheap young fresh-outs, chew 'em up, and spit them out".
Not all companies that do "hire, chew, spit" are startups. The former Big-8 accounting firms were known for that, especially in the consulting groups that dealt with software. (My teams often fixed the crap code those groups produced.)
Part of the startup culture, especially in Silicon Valley, is working in a fast paced environment with other bright people, doing something interesting, and having your product live or die by market acceptance (in contrast to being axed by a company executive). The intrinsic rewards (fast, interesting, worthwhile, fun) are part of the work/life balance in a startup.
Some young people are not cut out for life in a startup. Some are. Still others, who are not so physically young, thrive on it, and in it, too.
Extrinsic rewards (salary, big office, car, pension) are top level goals for some people, but not all. For others, like those who thrive in startups, those are not top goals, but are often results of actual personal goals (intrinsic reward stuff).
Where this applies to Boeing, (and some other companies), is that negative intrinsic rewards (frustration, haphazard/schizophrenic leadership, arbitrary schedule pressure, understaffing, underfunding, lack of empowerment, etc.) can drive down productivity far more, and faster, than extrinsic rewards can improve it. Recent articles in the media have mentioned "corrosive culture" in reference to Boeing.
The SpaceX approach is probably less of "bring 'em in, chew 'em up, spit 'em out", but rather "bring 'em in, give them something really interesting to do, watch them thrive". Some will stay, some won't. That is very different from "spit 'em out".
Quality is not a direct result of experience or the average age of the workforce. It is a direct result of leadership. An excellent culture and great leaders can produce a team that does consistently excellent work. A bad (e.g. "corrosive") culture and bad leadership basically guarantees low quality.
The insidious thing is that the process has hysteresis. Evidence of bad leadership and decaying company culture is not immediately apparent. Former high quality work persists until the declining culture eventually beats the optimism and energy out of the employees. The executives look good because morale is high and quarterly profits are good. Until it isn't. The bad news for Boeing is the recovery has hysteresis too. Quality does not magically rise. Trust is not restored overnight. Employees don't come to work the next day energetic and optimistic.
So the question is what is Boeing going to do? Improve the company image (whitewashing), or really do a major purge and rebuild the company culture in a positive way? -
#1062 by Comga on 10 Feb, 2020 03:35
-
I don't want to get into an argument over my example, but you're making my point. For some reason they assumed the two test instruments were wrong. In reality the problem was with reflective null corrector used during manufacturing. A simple knife-edge test would have confirmed the test instruments were correct and the mirror had a spherical aberration.
The point is there was a lack of proper quality control. Probably because of overconfidence. I've seen this a lot when I worked in IT.
Those statements are technically incorrect but the point is spot on. Sweeping problems under the rug and hoping they and their ilk don’t come back to bite you is a recipe for disaster. -
#1063 by king1999 on 10 Feb, 2020 05:01
-
Many were assumptions IMO, not "publicly available". The CEO was shown the door not because of these problems, many of which were under the previous CEO's watch. He was let go because of his pushing FAA to speed up the MAX return to flight process [edit zubenelgenubi: Potty-mouth Language!].FF, those are pretty significant claims against Boeing management and it's software development process, are there sources that you can point to which give us something other than your informed assertion that it's this bad? I'd ask the same question if this were a statement about the practices at SpaceX, ULA or any anywhere else.
There has been extensive reporting and disclosure by the FAA, former employees, whistle-blowers, and congress regarding internal processes on software development within Boeing all of which is widely and publicly availble. Up to and including emails being leaked from within the company describing a culture where major programs are "worked on by monkeys and managed by clowns" direct quote of a Boeing employee.
This is OT and outside the scope of this site with the exception that it was worth pointing out its a specific corporate culture mindset and management mindset regarding this issue. And that it's systemic.
Case and point Boeing's investors have spoken and the previous executive management team and the CEO have been shown the door as a result of choosing to foster this type of culture and choosing a cheapskate crackpot way of developing critical software for new products.
Also edit zubenelgenubi: fixed quote attribution -
#1064 by SoftwareDude on 10 Feb, 2020 05:21
-
Boeing might be having a hard time maintaining a top programming team because so many younger people probably think it's cooler to work for SpaceX or Tesla or Google, or to be somewhere else in Silicon Valley, instead of working for their grandparents' or great-grandparents' kind of company. I don't have the stats in front of me, but the average age of SpaceX and Google employees is unusually low, somewhere around 30, and an old established company like Boeing is probably more accommodating and welcoming of older employees (AARP does research and can validate such distinctions). If the latter is the case, then a communication gap among the differing generations could be heavily at play in more ways than one.
This has been going on long before anyone ever heard of SpaceX or Blue Origin. It's 30 years of hiring entry-level programmers. and giving them too much responsibility. Then when things go South, which they invariably do, hire expensive consultants to do 85% of the work. Then the consultants leave, and there is no way to upgrade what they have. This is not just Boeing by the way.
That's mostly speculation, but no one has brought up the possibility of a need for intercession from further afield, by industrial psychologists. Edit to add or professional consultants whose sole purpose is to help companies grow and thrive. -
#1065 by RonM on 10 Feb, 2020 06:04
-
I don't want to get into an argument over my example, but you're making my point. For some reason they assumed the two test instruments were wrong. In reality the problem was with reflective null corrector used during manufacturing. A simple knife-edge test would have confirmed the test instruments were correct and the mirror had a spherical aberration.
The point is there was a lack of proper quality control. Probably because of overconfidence. I've seen this a lot when I worked in IT.
Those statements are technically incorrect but the point is spot on. Sweeping problems under the rug and hoping they and their ilk don’t come back to bite you is a recipe for disaster.
It's in the NASA report.
-
#1066 by llanitedave on 10 Feb, 2020 14:22
-
What you have is nothing right now but a chance to fix things and have some kind of program in the future.
Without being argumentative, what else do we have at this point?QuoteThe administrator wants dissimilar redundancy, well he has it with Dragon and Soyuz...
Wrong. Without going too OT here no, the situation is much worse.
Soyuz is arguably getting more dangerous with every flight due to deteriorating quality control due in turn to corruption within Russia. Holes in the spacecraft. Exploding boosters.
Dragon exploded. Yes I know they claim its fixed. Dragon still has super high pressure COPVs embedded in the body of the spacecraft for its LAS. I don't trust this system and I don't trust that there aren't more bugs hiding.
The truth is what we have is dissimilar hazards not dissimilar reliability and broken risk assessment and quality control across all three systems. And I have yet to see anybody really stepping up here and actually fixing this nonsense and that includes admin Jim.
The alternative is to have less than nothing when starliner slams into the space station or kills its crew and a future earth centric anti science congress responds by canceling all HSF programs and funding. Software and communications issues are what lead to the circumstances that caused the MIR progress collision. Quality control and risk assessment failure are what led to 51L and 107.
Looks to me like canceling STS and not building the J130 in favor of the stick and a bunch of paper vehicles was a really bad idea. Gee who could have guessed it except the entire Augustine commission all of the industry engineering rank and file and this entire websites' user base.
For reference I'm not mad at you or anyone else on here I'm mad at the people who allow this nonsense to happen and I'm sick of my taxes being frakked away on infinite nothings that never fly or become death traps worse than the device they were supposed to replace. It's even worse because Congress was told about this multiple times after Columbia and after CXP and here we are again, so was NASA and yet the lack of safety culture that brought down two vehicles is right back again. Twenty year cycle as usual.
Do you seriously think the STS was safer? -
#1067 by SWGlassPit on 10 Feb, 2020 14:48
-
I've seen a number of folks asking questions on how this compares to STS flight software.
While I don't have any info on CST software, here's a good article discussing some of the development and sustaining work on STS flight software: https://www.fastcompany.com/28121/they-write-right-stuff
Of particular note, the roughly 420,000 lines of HAL/S on shuttle were maintained by a staff of 260 people at a cost of $35M/year. That's the sustaining cost, not the initial development cost.
Shuttle software was profoundly expensive, and in particular, the processes used with it (and the speed thereof) would drive probably 95% of the modern software development community completely insane. The specs alone were around 40,000 pages of written material. -
#1068 by dj_fan on 10 Feb, 2020 17:26
-
I've seen a number of folks asking questions on how this compares to STS flight software.
While I don't have any info on CST software, here's a good article discussing some of the development and sustaining work on STS flight software: https://www.fastcompany.com/28121/they-write-right-stuff
Of particular note, the roughly 420,000 lines of HAL/S on shuttle were maintained by a staff of 260 people at a cost of $35M/year. That's the sustaining cost, not the initial development cost.
Shuttle software was profoundly expensive, and in particular, the processes used with it (and the speed thereof) would drive probably 95% of the modern software development community completely insane. The specs alone were around 40,000 pages of written material.
And even then, an IBM programmer on the Shuttle program told me in the early 1990s that there were two different occasions where he thought that *software problems* during STS missions were so unexpected that they were worried they could lose the Orbiter. I don't recall the specifics, but one related to state vectors and the other telemetry/displays. -
#1069 by FinalFrontier on 10 Feb, 2020 17:38
-
What you have is nothing right now but a chance to fix things and have some kind of program in the future.
Without being argumentative, what else do we have at this point?QuoteThe administrator wants dissimilar redundancy, well he has it with Dragon and Soyuz...
Wrong. Without going too OT here no, the situation is much worse.
Soyuz is arguably getting more dangerous with every flight due to deteriorating quality control due in turn to corruption within Russia. Holes in the spacecraft. Exploding boosters.
Dragon exploded. Yes I know they claim its fixed. Dragon still has super high pressure COPVs embedded in the body of the spacecraft for its LAS. I don't trust this system and I don't trust that there aren't more bugs hiding.
The truth is what we have is dissimilar hazards not dissimilar reliability and broken risk assessment and quality control across all three systems. And I have yet to see anybody really stepping up here and actually fixing this nonsense and that includes admin Jim.
The alternative is to have less than nothing when starliner slams into the space station or kills its crew and a future earth centric anti science congress responds by canceling all HSF programs and funding. Software and communications issues are what lead to the circumstances that caused the MIR progress collision. Quality control and risk assessment failure are what led to 51L and 107.
Looks to me like canceling STS and not building the J130 in favor of the stick and a bunch of paper vehicles was a really bad idea. Gee who could have guessed it except the entire Augustine commission all of the industry engineering rank and file and this entire websites' user base.
For reference I'm not mad at you or anyone else on here I'm mad at the people who allow this nonsense to happen and I'm sick of my taxes being frakked away on infinite nothings that never fly or become death traps worse than the device they were supposed to replace. It's even worse because Congress was told about this multiple times after Columbia and after CXP and here we are again, so was NASA and yet the lack of safety culture that brought down two vehicles is right back again. Twenty year cycle as usual.
Do you seriously think the STS was safer?
STS was a working well understood system. It was inherently dangerous by design but given how well understood it was by the time it retired its very likely it could have flown for another five years without an anomaly. However, as ASAP had pointed out it would have been getting gradually more dangerous as the vehicles aged.
With that said the point is:
1. STS was well understood.
2. STS actually worked.
Myself and many other have said for years that any HSF gap was totally unacceptable. Ideally STS should have been extended and NASA should have sought to build a J130 type of vehicle to provide a backstop incase of commercial crew delays. Some people scoffed at this and yet here we are. The commercial crew delay materialized and were many times worse than expected, the shuttle was stood down too soon making the gap even longer and more crippling. And there is no CALV at all to ISS or anywhere else at least until SpaceX makes a successful operational flight to ISS with crew since starliner is probably out of the running for now.
Assuming that happens the gap will close at 9 years and some change, which is outrageous considering the amount of money wasted on all the delays on CXP and on buying soyuz seats from an enemy government. Just to illustrate how ridiculous this situation is we have Russia under heavy sanctions as a result of their actions in Ukraine and in 2016 and on top of that they are building a new generation of nuclear weapons designed to defeat our defense systems.
And yet we are buying rides to a station we paid for from them. Really stupid. And we are doing it because of these poor decisions decisions which were totally unnecessary.
That was the point I'm making. We had a great number of options Aug Com and others put on the table to avoid the current situation and literally every single one of these was ignored by congress and nasa and we ended up where we are now as a result.
It did not have to be this way. It is this way because stupidity won the day. -
#1070 by TorenAltair on 10 Feb, 2020 23:26
-
I again read today GAO's ruling after the complaints from SNC that Dreamchaser wasn't chosen for the crew program. We call this Realsatire in Germany (not sure if that's a loanword in English as well, something like real-life comedy)
[NASA] "recognized Boeing's higher price but also considered Boeing's proposal to be the strongest of all three proposals in terms of technical approach, management approach and past performance, and to offer the crew transportation system with most utility and highest value to the government." -
#1071 by Lemurion on 11 Feb, 2020 00:13
-
I again read today GAO's ruling after the complaints from SNC that Dreamchaser wasn't chosen for the crew program. We call this Realsatire in Germany (not sure if that's a loanword in English as well, something like real-life comedy)
[NASA] "recognized Boeing's higher price but also considered Boeing's proposal to be the strongest of all three proposals in terms of technical approach, management approach and past performance, and to offer the crew transportation system with most utility and highest value to the government."
I really wonder if anyone would be able to say that now? -
#1072 by ulm_atms on 11 Feb, 2020 00:23
-
I again read today GAO's ruling after the complaints from SNC that Dreamchaser wasn't chosen for the crew program. We call this Realsatire in Germany (not sure if that's a loanword in English as well, something like real-life comedy)
[NASA] "recognized Boeing's higher price but also considered Boeing's proposal to be the strongest of all three proposals in terms of technical approach, management approach and past performance, and to offer the crew transportation system with most utility and highest value to the government."
I really wonder if anyone would be able to say that now?
With or without a straight face?
But I guarantee that Boeing would tell you that it's 100% still true. I have no doubt after listening to the Friday news conference recording. -
#1073 by SoftwareDude on 11 Feb, 2020 00:24
-
I've seen a number of folks asking questions on how this compares to STS flight software.
I wrote some utilities used to upload software into the Shuttle computers during countdown from Wright Patterson Air Force Base. The problem was the Shuttle Computers, AP-101B was the late 1960s avionics hardware made by IBM with the initial version having 128 KW (24-bit words) of "core" memory. There was an upgrade to the AP-101S version with 500 KW of semiconductor memory.
While I don't have any info on CST software, here's a good article discussing some of the development and sustaining work on STS flight software: https://www.fastcompany.com/28121/they-write-right-stuff
Of particular note, the roughly 420,000 lines of HAL/S on shuttle were maintained by a staff of 260 people at a cost of $35M/year. That's the sustaining cost, not the initial development cost.
Shuttle software was profoundly expensive, and in particular, the processes used with it (and the speed thereof) would drive probably 95% of the modern software development community completely insane. The specs alone were around 40,000 pages of written material.
Modern programmers are several orders of magnitude more productive than the best programmers maintaining shuttle software. Programming in HAL/S is done on an IBM 370 computer; unlike anything, most programmers can imagine. The Utilities I wrote were in assembler.
In 2005, it was known that United Launch Alliance supported the HAL/S language. I hope there is not an AP-101 in Starliner but wouldn't be surprised because Boeing still uses them in the F-15E; F-15 programming is done in Jovial.
-
#1074 by zubenelgenubi on 11 Feb, 2020 01:03
-
Repeating our fearless leader's words from the thread OP:
Civil and respectful at all times and remain on topic please.
-
#1075 by ShaunML09 on 11 Feb, 2020 01:08
-
Can anybody reconcile the fact that these egregious software errors were discovered during flight, but not during testing prior to launch?
That is the most concerning and frightening aspect of this whole debacle. Only when things went wrong did they thoroughly check the thruster software? Nobody thought to do a thorough review prior to launch??? That NASA or Boeing wouldn’t have reviewed the critical event software is unfathomable. -
#1076 by ChrisWilson68 on 11 Feb, 2020 01:38
-
I don’t suppose anyone over at SNC has knocked on Bridenstine’s door yet and asked “how dissimilar would you like your redundancy, sir? Ours comes with stubby wings...”
While only one party seems to exhibit the behavior at this time, I could imagine both Boeing and SpaceX becoming complacent without a third party nipping at their heels.
Dream Chaser would have a ton of time/work to get back in the crew game, but it could be a reasonable question at this stage.
As much as I am alarmed at the poor performance of Boeing, SNC doesn't have any kind of a track record doing this kind of thing and a crew Dream Chaser would cost billions of dollars and many years more.
If you really wanted to have two entirely different vehicles and you really wanted to drop Boeing, no matter the cost or delay, I think you're better off going with Blue Origin over SNC. Or even with Google over SNC. At least Google understands large, complex engineering projects.
And about the comment about two competitors being complacent if they don't have a third nipping at their heels -- that makes no sense to me. Why not three being complacent without a fourth? Or 100 being complacent without a 101st? I don't think there's some magic about 3 versus 2 that keeps complacency from happening.
-
#1077 by TrevorMonty on 11 Feb, 2020 02:06
-
SNC have their work cutout getting DC cargo version ready. Once that has some flight history they would be in better position to do crew version.
-
#1078 by JAFO on 11 Feb, 2020 04:49
-
The same thing was said about Space X when they were building Falcon 1.
As much as I am alarmed at the poor performance of Boeing, SNC doesn't have any kind of a track record doing this kind of thing and a crew Dream Chaser would cost billions of dollars and many years more.
-
#1079 by octavo on 11 Feb, 2020 07:05
-
We are about due for another crew accident..
Apollo 1 to Challenger - 6941 days
Challenger to Columbia - 6213 days
Columbia to today - 6217 days
Especially with the myriad of different new crew systems that are planned for use in the next ~36 months. It really is a matter of when(and what, who, how many, etc.), not if, and with BLEO flights the risk just goes up. Sure, Boeing should do another OFT, but NASA should pay them fair value for cargo. Let SpaceX take the risk with only 1 uncrewed orbital test flight.
edit: Maybe Doug Loverro should change his pin from counting down to December 31st 2024 to counting up from February 1st, 2003.
I suppose you were only counting NASA crews, but there was another American crew fatality on 31 October 2014 when Mike Alsbury was killed while test piloting the SS2.
