Author Topic: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread  (Read 958771 times)

Offline wronkiew

  • Full Member
  • *
  • Posts: 186
  • 34.502327, -116.971697
  • Liked: 105
  • Likes Given: 125
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1500 on: 06/09/2014 07:25 PM »
YouTube hasn't been updated in a while. IainCole, can you turn auto-generation back on?

I switched it back to 3hr and it's still saying it's duplicate. I'm going to move it back to 1hr again

Ok, never mind. The last MMB change was 33 hours ago. I've made clock fixes since then, but those don't currently trigger the uploader.

Offline mhenderson

  • Member
  • Posts: 69
  • USA
  • Liked: 101
  • Likes Given: 18
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1501 on: 06/09/2014 08:06 PM »
The last MMB change was 33 hours ago. /snip/

Hmmm. Does that mean we are
a) Done?
b) Losing interest?
c) Too busy with our lives?
d) Running out of easy fixes and needing new tools (bitflip automation)?
e) In a summer slump?

Offline princess

  • Member
  • Posts: 65
  • Liked: 106
  • Likes Given: 25
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1502 on: 06/09/2014 08:18 PM »
Hmmm. Does that mean we are

f. Busy on the error detection files!

Over the weekend I've been changing the raw.ts file so that all the fixes we found in raw_final_fixed.ts are incorporated. It turned out to be a LOT harder than I expected, but I believe I have it now. The raw_final_fixed.ts actually had a run of 30456 bytes which was a duplicate of an earlier section within the file, so it didn't line up with raw.ts. Finding that one and fixing it was a great amount of fun...

I'm going to post two files, and because of the 5Mb limit I'll have to use two posts to do it. The first file is raw_aligned.ts. This is basically SpaceX's raw.ts with the following changes:

1. At offset 0x382a8, insert 56 bytes and save the file.
2. At offset 0xd7250, insert 56 bytes and save the file.
3. At offset 0x215d4c, insert 56 bytes and save the file.
4. At offset 0x3571ec, insert 56 bytes and save the file.
5. At offset 0x3dc0ac, insert 56 bytes and save the file.

To do the insertion I simply copied the range after the insertion point 56 bytes forward, thus leaving a 56-byte duplicate at each insertion point. The result is a 4481732 byte long file with MD5 checksum 0821a52d195f9566ef71921fd777bc14. The insertions were done to re-align the TS stream with the 188-byte packet offsets.

(continued in second post)

Offline princess

  • Member
  • Posts: 65
  • Liked: 106
  • Likes Given: 25
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1503 on: 06/09/2014 08:26 PM »
Here is my second file, rocket.ts. As the name suggests it's my best guess at what the rocket actually transmitted. It was made from 3 main parts:

1. 1355 TS packets from raw_aligned.ts at the start. This is the 16 P-frames that were transmitted before the first I-frame.
2. The raw_final_fixed.ts file was then appended to these packets.
3. From offset 0x3e03b8 onwards, the contents of raw_aligned.ts were substituted instead so that the duplicate data is removed.

The resulting file is 4481732 bytes long and aligns byte-for-byte with raw_aligned.ts. The file was then fixed up further (as far as I could) so that it's as good as I can reasonably get it without making too many blind guesses. The file is good enough to play in recent versions of VLC.

The aim is that people can now compare raw_aligned.ts with rocket.ts, and any differences are bitflip errors. Please take into account the insertions done in raw_aligned.ts if this affects the error pattern, but as the error block size appears to be 56 bytes anyway then it should hopefully not matter.

Offline cscott

  • Senior Member
  • *****
  • Posts: 2949
  • Liked: 2054
  • Likes Given: 664
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1504 on: 06/09/2014 08:32 PM »
The aim is that people can now compare raw_aligned.ts with rocket.ts, and any differences are bitflip errors. Please take into account the insertions done in raw_aligned.ts if this affects the error pattern, but as the error block size appears to be 56 bytes anyway then it should hopefully not matter.

Fantastic!

Just checking my understanding -- the MMB fixes are *not* included in rocket.ts, right?  So the bitflip errors found when comparing rocket.ts with raw_aligned.ts are just the transport stream errors.

What I'd like to do is to take our "best attempt at a fix" for a specific iframe, convert it back into the .ts format, and then start looking at thost bitflips, to see if we've inadvertently "rediscovered" some triple flip patterns.  If we have, this would make me more confident in the "most errors are triple flips" hypothesis.

(I was hoping to start doing some ts-file-parser hacking this weekend, but family stuff took precedence...)
« Last Edit: 06/09/2014 08:33 PM by cscott »

Offline princess

  • Member
  • Posts: 65
  • Liked: 106
  • Likes Given: 25
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1505 on: 06/09/2014 08:40 PM »
Just checking my understanding -- the MMB fixes are *not* included in rocket.ts, right?  So the bitflip errors found when comparing rocket.ts with raw_aligned.ts are just the transport stream errors.

That's right, I've not included any MMB fixes in rocket.ts. The aim is that the bitflips present are ones that we're fairly confident of, and also that enough are present that we can discern any patterns. We definitely don't have the entire list - otherwise the resultant video would be perfect! :)

What I'd like to do is to take our "best attempt at a fix" for a specific iframe, convert it back into the .ts format, and then start looking at thost bitflips, to see if we've inadvertently "rediscovered" some triple flip patterns.  If we have, this would make me more confident in the "most errors are triple flips" hypothesis.

As far as I know there are a lot of MMB fixes that don't map directly back to the TS file, they're playing tricks in ffmpeg - for example, I think that redirecting a block to inherit from a different one isn't possible in MPEG4.

Also, the MMB fixes are done visually and not at the bitstream level. While the resulting frames look very good to the human eye, they're not what the rocket transmitted and can't be used for error detection purposes.

Offline moralec

Amazing work princess!!

Ok so how do you think we should proceed?
Should we  go ahead and subsitute raw_final_fixed.ts with one of your newer versions? If thatīs the case, do you think the existing MMB codes will still work?

Offline cscott

  • Senior Member
  • *****
  • Posts: 2949
  • Liked: 2054
  • Likes Given: 664
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1507 on: 06/09/2014 08:48 PM »
As far as I know there are a lot of MMB fixes that don't map directly back to the TS file, they're playing tricks in ffmpeg - for example, I think that redirecting a block to inherit from a different one isn't possible in MPEG4.

Also, the MMB fixes are done visually and not at the bitstream level. While the resulting frames look very good to the human eye, they're not what the rocket transmitted and can't be used for error detection purposes.

Yup.  So, concretely what I'd like to begin by doing is taking a "lightly damaged" iframe (one without a lot of MMB fixes) and trying to manually figure out a sequence of triple flips which would recreate the effect of the MMB fixes.

In theory this could be done semi-automatically -- taking the manual .png output of our work and using it as the "target" of a brute force search that only does triple flips.  Basically looking for the "pure triple flip" sequence which best matches our "by eye" fixes.  Dunno if that search space is reasonable yet.  Like I said, the first step is to try to do a small sample manually to get a feel for the issues.

Offline princess

  • Member
  • Posts: 65
  • Liked: 106
  • Likes Given: 25
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1508 on: 06/09/2014 08:58 PM »
Amazing work princess!!

Ok so how do you think we should proceed?
Should we  go ahead and subsitute raw_final_fixed.ts with one of your newer versions? If thatīs the case, do you think the existing MMB codes will still work?

Why thank you moralec! No, I don't think we need to replace the TS file that people are using for the video recovery. It's good enough and the new file only makes a few data-related changes - mainly the extra P-frames at the start (which have no I-frame to reference to), and the duplicate run in P-frames 270/271.

It looks like this duplicate area was put in because the start of frame 270 is just atrociously corrupted, so in the rocket.ts file I actually don't have the start of P-frame 270 as I honestly can't decide where to put it. I'd advise the MMB people that 270 might not start in the right place, and 269 might have junk at the end. It seems as though the duplicate bytes were inserted in order to use some of 271's data to shore up 270 in raw_final_fixed.ts, so this area of the video might pause slightly.

In summary, the rocket.ts file is mainly for error detection but also might be of academic interest, or might be useful for people to attempt a more thorough recovery of difficult frames by working from the raw MPEG4 stream using ffmpeg.

Offline cscott

  • Senior Member
  • *****
  • Posts: 2949
  • Liked: 2054
  • Likes Given: 664
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1509 on: 06/09/2014 09:01 PM »
Ok so how do you think we should proceed?
Should we  go ahead and subsitute raw_final_fixed.ts with one of your newer versions? If thatīs the case, do you think the existing MMB codes will still work?
Why thank you moralec! No, I don't think we need to replace the TS file that people are using for the video recovery. [...] In summary, the rocket.ts file is mainly for error detection but also might be of academic interest, or might be useful for people to attempt a more thorough recovery of difficult frames by working from the raw MPEG4 stream using ffmpeg.

I will presumably have to figure out how to translate MMBs from raw_final_fixed.ts to rocket.ts in order to pursue the triple-flip research I want to do, so if it comes to it I might be able to help translate stuff.  But that's a ways in the future yet.

Offline princess

  • Member
  • Posts: 65
  • Liked: 106
  • Likes Given: 25
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1510 on: 06/09/2014 09:11 PM »
I will presumably have to figure out how to translate MMBs from raw_final_fixed.ts to rocket.ts in order to pursue the triple-flip research I want to do, so if it comes to it I might be able to help translate stuff.  But that's a ways in the future yet.

Well the "X" parts of the MMB command will map directly back, as that's just an XOR command on a direct offset. There may be other ones that can map back - maybe if you change the luma or chroma enough to not need a new luma_size or chroma_size field? Michaelni would be able to help here.

As for mappings - I can provide mappings between the MPEG4 bit number and the TS file offset. You would apply the MMBs to rocket.ts, and leave raw_aligned.ts alone. The idea is this:

 - raw_aligned.ts represents what was received in Elon's plane, properly aligned to MPEG TS packet boundaries. It's the equivalent of forensic evidence in the investigation, and should remain as untouched as we possibly can.

 - rocket.ts represents the group's best idea of what was actually transmitted from the rocket's antenna. As we're initially using it for error pattern analysis I've tried to be conservative in the bitflips - if I wasn't sure of one, I've left it as it is in raw_aligned.ts. As we become more confident of other bitflips we would accumulate them in this file.

I hope that clears things up!


Offline Quialiss

  • Member
  • Posts: 75
  • Liked: 82
  • Likes Given: 29
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1511 on: 06/09/2014 09:55 PM »
The last MMB change was 33 hours ago. /snip/

Hmmm. Does that mean we are
c) Too busy with our lives?

I've been working on part 10 fixing up some of the odd errors in the p frames, good work to do in ten minute stints.  They're done-ish now and the sequence looks a bit cleaner, so up on the wiki they go. 

Curious notes:  The issues where many blocks would be off in luma are frequently fixed by making sure to -3 data that LOOKS empty but isn't (sizes of <10).  They frequently don't show up as errors, but they can confuse things a lot.  An error at the top of the frame can start screwing up blocks... at the bottom, leaving the majority of the blocks in the middle looking perfect.  But sometimes there doesn't seem to be an obvious error and I end up fixing all the sections individually. 

Offline moralec

Amazing work princess!!

Ok so how do you think we should proceed?
Should we  go ahead and subsitute raw_final_fixed.ts with one of your newer versions? If thatīs the case, do you think the existing MMB codes will still work?

Why thank you moralec! No, I don't think we need to replace the TS file that people are using for the video recovery. It's good enough and the new file only makes a few data-related changes - mainly the extra P-frames at the start (which have no I-frame to reference to), and the duplicate run in P-frames 270/271.

It looks like this duplicate area was put in because the start of frame 270 is just atrociously corrupted, so in the rocket.ts file I actually don't have the start of P-frame 270 as I honestly can't decide where to put it. I'd advise the MMB people that 270 might not start in the right place, and 269 might have junk at the end. It seems as though the duplicate bytes were inserted in order to use some of 271's data to shore up 270 in raw_final_fixed.ts, so this area of the video might pause slightly.

In summary, the rocket.ts file is mainly for error detection but also might be of academic interest, or might be useful for people to attempt a more thorough recovery of difficult frames by working from the raw MPEG4 stream using ffmpeg.

Speaking of academic interest, I think we may want to start posting all these files in the wiki, and work on a detailed description of what we did. We don't need to do it now,  be it may be convenient to do it sooner than later.... what do you think about this?  Should I open a page just for this purpose? Or do you prefer to leave this for the end of the process?


Offline princess

  • Member
  • Posts: 65
  • Liked: 106
  • Likes Given: 25
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1513 on: 06/09/2014 10:32 PM »
Speaking of academic interest, I think we may want to start posting all these files in the wiki, and work on a detailed description of what we did. We don't need to do it now,  be it may be convenient to do it sooner than later.... what do you think about this?  Should I open a page just for this purpose? Or do you prefer to leave this for the end of the process?

A wiki page is always helpful for people to get a summary of an issue, and also to find the files without having to wade through the thread here. I've often put links to files in the wiki just so I can find them again!

So yes, if in doubt wiki it. We can always edit and tidy things up in the future, and it helps to write things down while they're fresh in people's minds.

As for the files - raw_aligned.ts should remain pristine and untouched unless someone can prove one of the insertions needs moving, whereas rocket.ts will almost certainly have many revisions as we find bitflips. The thing is that rocket.ts will always give a worse video than using MMB commands, so working on it won't give impressive visual results. However, improving it will give us more confidence in our error pattern analysis.

Lastly, maybe I'm strange but I like to think that recovering rocket.ts is getting us closer to knowing the "last words" of a dying rocket stage. It did its job perfectly, finally achieving what Elon and his team had hoped for it, and was desperate to tell the world, frantically transmitting the news of its success before bravely succumbing to the stormy seas. I may be anthropomorphising things too much, but I think we owe it to that tenacious machine to reconstruct its last historic message to the world.

Offline Quialiss

  • Member
  • Posts: 75
  • Liked: 82
  • Likes Given: 29
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1514 on: 06/09/2014 10:41 PM »
It looks like this duplicate area was put in because the start of frame 270 is just atrociously corrupted, so in the rocket.ts file I actually don't have the start of P-frame 270 as I honestly can't decide where to put it. I'd advise the MMB people that 270 might not start in the right place, and 269 might have junk at the end. It seems as though the duplicate bytes were inserted in order to use some of 271's data to shore up 270 in raw_final_fixed.ts, so this area of the video might pause slightly.

IIRC, SwissCheese and I worked on those frames.. from our notes on the wiki, 269 and 270 were completely unrecoverable in fix8 version, and some data became recoverable in the raw_final_fixed.ts, though they're still missing large chunks of data.  But they don't appear to pause(outside of what's caused by missing a third of a frame) and the timestamps all look correctly recovered.

Offline princess

  • Member
  • Posts: 65
  • Liked: 106
  • Likes Given: 25
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1515 on: 06/09/2014 10:52 PM »
IIRC, SwissCheese and I worked on those frames.. from our notes on the wiki, 269 and 270 were completely unrecoverable in fix8 version, and some data became recoverable in the raw_final_fixed.ts, though they're still missing large chunks of data.  But they don't appear to pause(outside of what's caused by missing a third of a frame) and the timestamps all look correctly recovered.

The TS timestamps for 270 were altered to make the frame look correct, but I don't know what happened with the MPEG4 stream.

Basically at that point in the data the stream was all fine with no errors, and then suddenly from around offset 0x003dbb04 to offset 0x003dd284 there was a huge burst of corruption that knocked out 32 TS packets. If you look at it with a TS analyser it's very bad, just totally mangled.

Anyway, the boundary between 269 and 270 occurred in the middle of this. Usually the corruption is light enough that you can kind of eyeball the closest packet, or break out the Hamming-distance analysis, but in this case it's useless. Whoever did the byte duplicate in raw_final_fixed.ts was basically just pasting the start of 271 into this and resetting the timestamp to be correct. It's the TS equivalent of saying "this macroblock is shot to pieces, use one that looks similar".

So yes, this will allow extra data to be recovered - the good macroblocks in 270 after the corruption burst finished. But be aware that the end of 269 and the start of 270 are mangled beyond recovery.

Offline rocketguy101

  • Member
  • Full Member
  • ****
  • Posts: 837
    • Strib's Rocket Page
  • Liked: 207
  • Likes Given: 681
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1516 on: 06/09/2014 11:28 PM »
Lastly, maybe I'm strange but I like to think that recovering rocket.ts is getting us closer to knowing the "last words" of a dying rocket stage. It did its job perfectly, finally achieving what Elon and his team had hoped for it, and was desperate to tell the world, frantically transmitting the news of its success before bravely succumbing to the stormy seas. I may be anthropomorphising things too much, but I think we owe it to that tenacious machine to reconstruct its last historic message to the world.
I love this description!!!  Around here we refer to Space Shuttles as "she" so I think you hit it perfectly!  BTW I stand in awe of your work here, I don't understand 1/10th of what you guys do, but I appreciate your passion and enjoy seeing the fruits of your labors...
David

Offline moralec

IIRC, SwissCheese and I worked on those frames.. from our notes on the wiki, 269 and 270 were completely unrecoverable in fix8 version, and some data became recoverable in the raw_final_fixed.ts, though they're still missing large chunks of data.  But they don't appear to pause(outside of what's caused by missing a third of a frame) and the timestamps all look correctly recovered.

The TS timestamps for 270 were altered to make the frame look correct, but I don't know what happened with the MPEG4 stream.

Basically at that point in the data the stream was all fine with no errors, and then suddenly from around offset 0x003dbb04 to offset 0x003dd284 there was a huge burst of corruption that knocked out 32 TS packets. If you look at it with a TS analyser it's very bad, just totally mangled.

Anyway, the boundary between 269 and 270 occurred in the middle of this. Usually the corruption is light enough that you can kind of eyeball the closest packet, or break out the Hamming-distance analysis, but in this case it's useless. Whoever did the byte duplicate in raw_final_fixed.ts was basically just pasting the start of 271 into this and resetting the timestamp to be correct. It's the TS equivalent of saying "this macroblock is shot to pieces, use one that looks similar".

So yes, this will allow extra data to be recovered - the good macroblocks in 270 after the corruption burst finished. But be aware that the end of 269 and the start of 270 are mangled beyond recovery.

Lighting?

Offline aero

  • Senior Member
  • *****
  • Posts: 2988
  • 92129
  • Liked: 796
  • Likes Given: 282
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1518 on: 06/10/2014 03:54 AM »
Lightning.
Retired, working interesting problems

Offline mhenderson

  • Member
  • Posts: 69
  • USA
  • Liked: 101
  • Likes Given: 18
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1519 on: 06/10/2014 07:00 AM »
Relevant chatter?

Lightning or otherwise, I imagine there's a significant buildup of electric potential on / around a falling rocket that is re-igniting engines. Rockets falling from space are enveloped in a ball of plasma. The atmosphere separates ionic charges into layers. Are the igniters piezo-electric like my grill?

** Even in good weather ** discharges are possible and perhaps likely to occur again. The techniques developed here may become standard operating procedure.

Is it valid to cut / paste data from adjacent frames to allow the codec to continue?

IMHO, absolutely yes. That is a necessary part of the restoration. Let the brave little booster sing. 

Tags: