SpaceX COTS Demo 1 Updates

[Swatch]

I was referring to people on this forum dancing around the point.    There processes (more than just ground ops) is my area of concern

A person with expertise got a lot more out of that news than I did.  I couldn't figure out how they were seeing a cracked nozzle inside an assembled rocket. 

So this is my guess: you think they have old pictures of the broken nozzle from days or weeks ago, and only now is somebody noticing.  Something went wrong to let them assemble it with the broken nozzle in the first place.  Thus the bad process, and a drop in confidence of competence, meaning they might make another error.  Is that right?

Yes,  many process errors

1.  bad hardware was created and passed inspection

2.  Bad hardware was put into a flight vehicle.

3.  The review of closeout photos was not completed before the move to the next major phase in the ground flow

It sounds like you're working under the assumption that the damage was a preexisting condition that simply wasn't caught.   What would your opinion on the matter be if the damage had developed after integration with the vehicle?  In that situation, it seems their process caught it at exactly the right time.  I don't believe they've released information that implies one situation over the other, so I'm just playing devil's advocate.

[pippin]

In that case something's probably seriously wrong with the integration process...

[cuddihy]

"judgment" by the way, not "judgement ".

Touche.

I just can't help it on the "autocorrect, but wrong word," but it's no different than a typo, you're right.

[Namechange User]

Fascinating thread from a social and psychological perspective. 

From a technical perspective, Jim is quite correct in his comments.  Process is vital in this business and it appears anyway that something escaped somewhere along the line.  While likely not the end of the world and they are investigating and seemingly doing due diligence, some of the comments thus far are intriguing by some posters. 

[Kabloona]

I was referring to people on this forum dancing around the point.    There processes (more than just ground ops) is my area of concern

A person with expertise got a lot more out of that news than I did.  I couldn't figure out how they were seeing a cracked nozzle inside an assembled rocket. 

So this is my guess: you think they have old pictures of the broken nozzle from days or weeks ago, and only now is somebody noticing.  Something went wrong to let them assemble it with the broken nozzle in the first place.  Thus the bad process, and a drop in confidence of competence, meaning they might make another error.  Is that right?

If you go back and read some of the previous posts and the "press" releases, what happened was that SpaceX took "closeout" photos inside the F9 interstage as a final inspection. A closeout inspection is routine and is the last thing you do before bolting the door closed, because it's the last chance you have to catch any hardware problems in the interstage before launch. So if I read the statement correctly, these are not "old pictures from weeks ago."

The closeout inspection did what it was supposed to do: it found a problem. And about this "process" thing. Hey, people, anyone who's been in this business knows that there's no such thing as perfection. The process of design, manufacturing, inspection, operation, and improvement is fundamentally a human process, of imperfect people using imperfect materials to try to do make a super-complex machine that's incredibly intolerant of imperfection, and thus incredibly difficult to do succesfully. That's true of Shuttle, F9, and every other large rocket that ever flew. The SpaceX closeout inspection process did what it was supposed to do: it found an imperfection. Now they'll fix it. How is this different from any other program? A potentially bad weld? QA maybe missed something? Wow, that's never happened on any other program...

[FinalFrontier]

Not a peep from Spacex on today's supposed launch attempt. No idea if there will be an attempt today. 3 inch crack may be a rumor.

[ugordan]

Not a peep from Spacex on today's supposed launch attempt. No idea if there will be an attempt today.

There won't be any. Try to keep up instead of overlooking information available and repeated over the last few pages of the thread.

[HIPAR]

A revelation of the 'facts' should help us understand the 'width' of that line separating human imperfection from incompetence. 

---  CHAS

[butters]

The nature of any "process problem" depends on the root cause of the cracks. Without knowing how the cracks happened, there's no way of knowing what, if anything, was wrong with the process. All we know right now is that there is an anomaly with the vehicle which was identified before launch and is being addressed.

In this context, as I see it, the "process" is a learning process, about assimilating new information and responding appropriately. SpaceX has an experience problem. They haven't built up a large knowledge base about their operations that helps them foresee any potential issues before they occur. That can only come with flight history.

[Jim]

It sounds like you're working under the assumption that the damage was a preexisting condition that simply wasn't caught.   What would your opinion on the matter be if the damage had developed after integration with the vehicle?  In that situation, it seems their process caught it at exactly the right time.  I don't believe they've released information that implies one situation over the other, so I'm just playing devil's advocate.

Still a process failure. 
1.  The damage occurred
2.   the incident that caused the damage wasn't caught right away or still is unknown.

[robertross]

The nature of any "process problem" depends on the root cause of the cracks. Without knowing how the cracks happened, there's no way of knowing what, if anything, was wrong with the process. All we know right now is that there is an anomaly with the vehicle which was identified before launch and is being addressed.

In this context, as I see it, the "process" is a learning process, about assimilating new information and responding appropriately. SpaceX has an experience problem. They haven't built up a large knowledge base about their operations that helps them foresee any potential issues before they occur. That can only come with flight history.

huh??

I guess some people don't understand what porosity in a weld is.

If you have porosity, you have a weak spot. That weak spot can initiate or propogate a crack.

[Kabloona]

The nature of any "process problem" depends on the root cause of the cracks. Without knowing how the cracks happened, there's no way of knowing what, if anything, was wrong with the process. All we know right now is that there is an anomaly with the vehicle which was identified before launch and is being addressed.

In this context, as I see it, the "process" is a learning process, about assimilating new information and responding appropriately. SpaceX has an experience problem. They haven't built up a large knowledge base about their operations that helps them foresee any potential issues before they occur. That can only come with flight history.

Agreed, and every vehicle program has had to climb the same learning curve. We all know that no orbital vehicle/team ever had all its processes ironed out after one launch.

[Namechange User]

The nature of any "process problem" depends on the root cause of the cracks. Without knowing how the cracks happened, there's no way of knowing what, if anything, was wrong with the process. All we know right now is that there is an anomaly with the vehicle which was identified before launch and is being addressed.

In this context, as I see it, the "process" is a learning process, about assimilating new information and responding appropriately. SpaceX has an experience problem. They haven't built up a large knowledge base about their operations that helps them foresee any potential issues before they occur. That can only come with flight history.

Yes and no.  While I admit I have not scoured Facebook, Twitter and the rest of the internet to find out what SpaceX is saying, these cracks appear to be on a weld AFAIK.  There are really three possibilities to cause these cracks as I see it at this point:

1.  Impact damage from "something".  I consider this unlikely because if whatever hit it (a foot, tool, etc) would leave other witness marks.  To the best of my knowledge that has not been disclosed and it would have to be a hard hit to crack a weld, increasing the likelihood of additional witness marks.

2.  Low cycle fatigue.  Given this nozzle has been shipped to various facilities and rolled to the pad and back some number of times, I suppose it is possible.  This would be a design issue that should have likely been discovered during qualification and certification.  I also see this as a low likely hood since the area I believe this is in should be a low stress area, etc.

3.  A process escape.  When the weld is complete it should be reviewed by quality and/or materials and process (M&P or whatever they may call it) personnel.  This inspection generallly involves the use of X-rays or other NDE techniques to look for weld penetraion and no porosity, etc.  Perhaps upon further review of the data, this is what they discovered and are now investigating.  Based on what I have seen published here, this seems quite possible to me. 

So, in summary, it happens.  This business is hard and unforgiving.  Regardless of the reason for this, which I'm sure we will ultimately hear about one way or the other, that is why process is vital and important.  They found it and they are investigating.   

Like I said earlier, I have found some the comments fascinating on this thread for a variety of reasons beyond technical discussion. 

[moose103]

I read the statement correctly, these are not "old pictures from weeks ago."

So this is the other school of thought.  That they can and did just peer inside the assembled rocket on the pad.  So rather than old closeout pictures and a problem they missed earlier, you believe it might be new pictures and a new problem.  Meaning they didn't MISS a cracked nozzle.

In this case we can imagine: maybe it banged against something during the static fire, or a manufacturing issue caused it to be brittle, or...

Wow this really is speculating with ALMOST no information isn't it.  Even Shotwell didn't really know what was going on, so how do I!!!!

[Kabloona]

I read the statement correctly, these are not "old pictures from weeks ago."

So this is the other school of thought.  That they can and did just peer inside the assembled rocket on the pad.  So rather than old closeout pictures and a problem they missed earlier, you believe it might be new pictures and a new problem.  Meaning they didn't MISS a cracked nozzle.

In this case we can imagine: maybe it banged against something during the static fire, or a manufacturing issue caused it to be brittle, or...

Wow this really is speculating with ALMOST no information isn't it.  Even Shotwell didn't really know what was going on, so how do I!!!!

Look, this isn't that complicated:

1. You inspect and take closeout photos just before you "close" whatever compartment it is that you're "closing," whenever that happens in the process flow. So, by design, this is the last time you ever look inside the area before launch, and then you bolt the cover on, or whatever. Someone was looking at the closeout photos, maybe someone back in Hawthorne, and saw something they didn't like. They opened up the compartment on the pad and did another visual and confirmed that, yes, there were in fact cracks.

2. Obviously, the cracks occurred before the closeout photos were taken. Exactly when they occurred and how is under investigation.

[KSC Engineer]

Jay Barbree is usually pretty good with his research.

That may have been the case back in the 80's, but if you listen to his questions at any shuttle pressers you'll quickly realize he isn't as up to par on what's really happening in space anymore.

Interesting....good to know.   I have not followed things for a while so that makes sense.  Thanks

[iamlucky13]

PS. You guys with the dremel tool, don't forget about the stiffening ring, you can use it as a straight edge, and you'll need a hot glue gun to put it back on.

I forgot about the stiffening ring. Isn't there actually two of them?

Robertross - I'll have to read the paper you posted later today when I have time.

Small update from the Orlando Sentinel this morning. Looks like Wednesday is out:
http://www.orlandosentinel.com/news/space/os-spacex-launch-advancer-20101206,0,1049791.story

SpaceX President Gwynne Shotwell said that If the nozzle had to be replaced, the launch could slip to Friday or Saturday.

[Robotbeat]

Just received by email what Chris Bergin posted here:
http://forum.nasaspaceflight.com/index.php?topic=22041.msg667156#msg667156

(EDIT: Yes, Chris Bergin posted this previously.)

[Robotbeat]

Just received by email:

UPDATE:  COTS Demo 1 Launch Activities
 
SpaceX engineers are analyzing two small cracks in the aft end of the 2nd stage engine nozzle extension.  These cracks are in a region near the end of the nozzle extension where there is very little stress and so they would not cause a flight failure by themselves.  However, further investigation is warranted to ensure that these cracks are not symptomatic of a more serious problem.
A decision on whether or not to attempt launch on Wednesday will be provided this evening [Tuesday].

The bell shaped Merlin Vacuum nozzle extension is made of niobium sheet alloy, measures 9 feet tall and 8 feet at the base diameter, and thins out to about twice the thickness of a soda can at the end.  Although made of an exotic refractory alloy metal with a melting temperature high enough to boil steel, this component is geometrically the simplest part of the engine.

It is important to note that the niobium nozzle extension increases the efficiency of the Merlin engine in vacuum and is installed by default on all upper stage Merlin engines, but that efficiency increase is not required for this mission.  The nozzle extension is most helpful when launching very heavy satellites or to maximize throw mass to distant destinations like Mars.  The most likely path forward is that we will trim off the thinnest portion of the nozzle extension, which is where the cracks are located, perform a thorough systems check and resume launch preparation.

Chris posted this yesterday.

You are correct. I thought it was a little different, but it isn't.

[KSC Engineer]

I guess he missed this one or knows something we don't.

Given what I've gathered so far, let's just say I'd trust nblackwell over Jay on this. You might consider there may be are other posters here involved with SpaceX one way or another, i.e. it's not Jim vs. everyone else.

If there's overly tolerant behavior for SpaceX' processes here, that doesn't mean "conspiracy" theories need to be made on the other side on how they get it easier at every step, compared to ULA or whatnot.

I watched the presser again and based on the comments here - what she might have said was "we are not flying over Europe" versus a short "The FAA has approved our flight."   This would have been a more clear answer but it does sound like Jay was off the mark on this one.  As to conspiracy theories - anyone who doesn't believe political favorites do not receive easier processes than others is either new at this or very naive.  It is clear they are the political favorite at this time due to the desire of the WH and NASA to promote "commercialization of space".   Don't kid yourself.