-
STS-122: Post ECO Issue Troubleshooting/Pre Launch Flow Latest
by
Chris Bergin
on 10 Dec, 2007 15:43
-
New thread to lead up to the NET Jan 2 launch date.
http://www.nasaspaceflight.com/content/?cid=5305 - based on L2 information, with some of what is a wonderful e-mail/memo written by Wayne Hale. We have a ton of info flowing, so there will be more articles to follow.
All STS-122 latest into this thread.
-
#1
by
brihath
on 10 Dec, 2007 16:05
-
Every time I see Wayne Hale at a Press Conference or read one of his memos/emails, I am impressed by the level of calm professionalism he represents. He is an excellent model for all the professionals who work in the space business, whether NASA or contractors. I don't work in the industry, but to have a mentor or leader of his caliber would be, I think, very rewarding.
-
#2
by
Joffan
on 10 Dec, 2007 16:10
-
Echoing brihath; Wayne Hale's memo is a model of thoughtful and professional response to a very difficult problem. His generalization out to other unused safety systems is an example of refusing to accept illusory comforts that is 100% the right attitude. NASA is very fortunate to have him, and very wise (or lucky) in bringing him into his current role.
-
#3
by
Rob in KC
on 10 Dec, 2007 16:11
-
brihath - 10/12/2007 11:05 AM
Every time I see Wayne Hale at a Press Conference or read one of his memos/emails, I am impressed by the level of calm professionalism he represents. He is an excellent model for all the professionals who work in the space business, whether NASA or contractors. I don't work in the industry, but to have a mentor or leader of his caliber would be, I think, very rewarding.
Couldn't agree more. The guy is really valuable to the program and having read the full orginal memo on L2, I gained even more respect for him and the entire team, both for and against launching with working ECO sensors or not.
Another great article.
-
#4
by
Pete at Edwards
on 10 Dec, 2007 17:29
-
Really great article.
Was interested to also read about how they are taking this forward. Any idea how long the Time-Domain Reflectometer troubleshooting will take?
-
#5
by
JMS
on 10 Dec, 2007 17:51
-
Excellent article Chris. Thank you.
And I'll add, I'm another unabashed Wayne Hale admirer.
The definition of calm reasoned leadership.
-
#6
by
vt_hokie
on 10 Dec, 2007 18:02
-
JMS - 10/12/2007 1:51 PM
Excellent article Chris. Thank you.
Indeed, thanks for the updates! Man, you spend a couple of days away from a computer, and you miss a lot! So much to catch up on...
-
#7
by
uko
on 10 Dec, 2007 18:41
-
Great article Chris! Great words from Wayne hale.. I also admire him!
-
#8
by
Chris Bergin
on 10 Dec, 2007 19:03
-
Pete at Edwards - 10/12/2007 6:29 PM
Really great article.
Was interested to also read about how they are taking this forward. Any idea how long the Time-Domain Reflectometer troubleshooting will take?
Oh, thanks everyone
We don't know the timescale yet. They had a meeting this morning and they have more later. Big, big effort.
-
#9
by
kimmern123
on 10 Dec, 2007 19:13
-
Terrific article, Chris! Hale is definitely one of the best men NASA could've placed in this job in this phase.
-
#10
by
JayP
on 10 Dec, 2007 20:18
-
Pete at Edwards - 10/12/2007 12:29 PM
Really great article.
Was interested to also read about how they are taking this forward. Any idea how long the Time-Domain Reflectometer troubleshooting will take?
TDR testing shouldn't take long once they can get in there. Telephone line-men use it all the time to find breaks.
-
#11
by
kimmern123
on 10 Dec, 2007 20:34
-
Has the TDR testing been done durin any of the previous ECO-sensor events? If it's not such a difficult test then why hasn't it been done before? My guess it's because the team have considered other things as the cause and only now have started to look at the circuitry
-
#12
by
joncz
on 10 Dec, 2007 21:02
-
Two reasons:
First, up until now, the ECO failures didn't repeat on the retanking. This failure on ECO #3 has repeated itself once and NASA hopes it will occur when they do their tanking test.
Second, TDR is an intrusive test. It requires the sender/receiver unit be introduced into the system to send the echo pulse; it can't be done on a circuit that has other traffic live on it. In fact, I expect NASA will need multiple TDR units so they can characterize good circuits from the failed one. This means NASA is likely to disconnect the Point Sensor Box (eliminate echoes from that side of the circuit) and run the tanking test with TDRs in its place.
-
#13
by
avollhar
on 10 Dec, 2007 21:33
-
I can only add as many others my deepest respect for the way Wayne Hale is dealing with this issue! A brave man standing up and saying: to hell with schedule, we have to understand this and fix it rather than waiving it.
I just hope he gets the support from higher level personnel, he certainly deserves it.
-
#14
by
on 10 Dec, 2007 22:55
-
Is there going to be a press conference tomorrow after the team meets?
-
#15
by
hyper_snyper
on 10 Dec, 2007 23:31
-
Question about the article.
The catastrophic failure that would happen if the tank ran dry....what would it look like? From Hale's memo it sounds like it would be a danger to the vehicle. I always thought it would be engine out and the motor would be permanently damaged but the vehicle would be okay. Are we looking at something like a violent explosion or rupture of the SSME were this to happen?
-
#16
by
chksix
on 10 Dec, 2007 23:40
-
The engine will tear itself apart and send shrapnel through the aft compartment. Very bad day indeed!
-
#17
by
Chris Bergin
on 10 Dec, 2007 23:46
-
hyper_snyper - 11/12/2007 12:31 AM
Question about the article.
The catastrophic failure that would happen if the tank ran dry....what would it look like? From Hale's memo it sounds like it would be a danger to the vehicle. I always thought it would be engine out and the motor would be permanently damaged but the vehicle would be okay. Are we looking at something like a violent explosion or rupture of the SSME were this to happen?
I quoted some L2 content on that into the previous article:
http://www.nasaspaceflight.com/content/?cid=5304About a third way through.
Basically, it's LOV/C via explosion/fire etc. Destroying the vehicle from the aft. Crew wouldn't stand a chance.
-
#18
by
joncz
on 10 Dec, 2007 23:55
-
-
#19
by
hey you
on 11 Dec, 2007 00:51
-
In the preceding (now frozen) thread, in the discussions about the STS 114 Engine Cut-off Sensor Anomaly
Documentation, the one line that stood out slightly more than many of the others to me is:
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
O-7. Specific issues with two of the PSB’s tested during the STS-114 troubleshooting have yet to be resolved. The PSB S/N 112 power converter has been noisy. This may be related to power converter instability or limit cycle regulation. Age may affect capacitor properties that can also possibly affect stability. This should be investigated further to ensure high reliability. In addition, the potential for copper path damage between the signal conditioner card circuitry and electrical connector in PSB S/N 111 should be assessed.
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
The comments about ***stability*** and noise are issues that sometimes can be associated with intermittent
behavior of electronic circuits. If a circuit is right on the edge of breaking into oscillation (to throw buzzwords
around: very small phase margin and/or very small gain margin) small pertubations can cause drastically
different circuit responses. At this very top level, the same verbage could be used to describe the overall
operation the ECO sensor *system* (as someone pointed out in an earlier post). Likewise, the noise that is mentioned may or may not be anything else than a parasitic oscillation. Some experience (lets not bother to
qualify what and/or how much) leads to a willingness by at least one person to describe such circuit operation
as "squirrelly". In other words, chaotic, unpredictable. Hmmm, that sure sounds familar although at best that is
only a correlation and is not anything on which to claim cause and effect. However, it seems sufficient to include
it in a fault tree when an effort is undertaken to isolate the cause of the problem.
In the mode of "modified brainstorming" [unlike normal brainstorming in which negative or critical remarks
are not allowed, such remarks are allowed if used to be building blocks], it seems to me that NASA should
put together about 4 teams (remarks about the economics farther below). One of the teams, if at all possible,
would be those that put together the STS 114 anomaly document. Two of the teams should be from
universities with individuals specializing in control systems and circuit design. One of these would be
from a more theoretically oriented institution while the second would be from a place taking a more practical
pragmatic approach. The fourth team would be an outside consulting firm with established expertise in
control systems and circuit design.
The product would include a design review presentation, computer simulations (including worst case for
both the circuit and control system requirements), parasitic effects due to how the units and system are
physically constructed. Each team would first independently present to NASA (projecting a half day -- if
it is more than there is more going on inside the circuitry than I am guessing.) After this (following NASA
feedback) the 4 teams would meet jointly with NASA sharing, exchanging viewpoints, and critiquing the
other teams approach to the problem. At this point, it is hoped, but not guaranteed, that some clarity will
have developed for a constructive path to take to resolve this entire situation in a timely, safe, and
effective manner.
BTW, in addition, I would also expect a very large number of Monte Carlo simulations to be run. The computer
models for devices would have to be extensive enough to reflect the full operating range (including nonlinearities
such as transistors turning on and off). Some challenges might be encountered for things like supply
line variations, for instance, but lets take one step at a time.
As for finances, we could do a simplified analysis based on lost opportunity costs due to postponements.
An effort such as putting four teams together for such a short intense effort would not be large compared to the
hidden costs lost in the delays that are happening.
Finally, it is good to see the discussion from multiple postings have delved down to a deeper level
of detail. That is the good news. The possible bad news is that there might be several more levels. I would
expect at least one more deeper and more detailed level of logic diagrams plus another level to get to
circuit diagrams.
The good news is that things appear to being looked at more closely. The bad news is that it might take
a much deeper level of looking. There are a lot of details that could be brought out for review when one
gets to this level. Sometimes, they have been taken into account. Other times,...........
So, a one line summary: parasitic oscillatiions inside a unit could (no guarantee) give unpredictable "chaotic
appearing" operation similar to what is being seen with the ECO sensor system.
