Author Topic: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread  (Read 400252 times)

Offline Lars_J

  • Senior Member
  • *****
  • Posts: 5914
  • Liked: 498
  • California
Yes, the video interpretation should be in it own thread. I'm interested in reading about the data recovery efforts here - and not having it mixed in with interpretations (mostly wrong) based on what people think they are seeing.

Online mvpel

  • Full Member
  • **
  • Posts: 270
  • Liked: 320
  • New Hampshire
Its journey started with a huge plume of water, and ended exactly the same way.  ;D

Offline ajmartin

  • Member
  • Posts: 4
  • Liked: 25
I'm trying to find flipped bits.

The problem I'm having is, it's almost almost impossible to tell if I've found a flipped bit. Just because the next few blocks or rows improve that doesn't mean you found a flipped one. It's very well possible you just found by luck a way to get the alignment of the next macroblocks right, but there's still a subtle propagating error.

The advantage of finding a flipped bit is, you preserve the most information possible in the frame. But it's only possible in regions where the error rate is less then perhaps 1 flipped bit per 1000.

Looking at the null packets, I noticed some patterns in the bit errors that might be useful for this kind of effort.

Specifically, many (but not all) of the bit-flips come in groups of 3, with one bit flipped, then 13 unaffected, then the next two also flipped.  This is most obvious in a hex-dump of the null packets, which are supposed to be filled with 0xff bytes: patterns such as "7f fc" or "fb ff e7" are common.  But the same bit-flip patterns apply where errors overlay the TS headers, and presumably also when they occur in image data.

Combinations (with XOR) of these 3-bit errors are also seen; mathematically, these error patterns are multiples of the polynomial x^15 + x + 1 over the binary field.  This could be the result of FEC coding applied to a signal with too many errors for successful correction.  I haven't figured it out completely but there are indications that the data may have been coded in 28-byte or 56-byte blocks (not respecting the TS packet boundaries); in particular, errors that don't fit this pattern seem to be found near offsets that are multiples of 28 bytes.

There is still the problem of figuring out if the change helped or not, but this should help to reduce the search space.  In low-error regions, there's a good chance that some of the flaws can be fixed with a single instance of this pattern.

Online moralec

  • Full Member
  • *
  • Posts: 116
  • Liked: 136
  • Washington, DC, United States
EDIT: I concur that this discussion should go in a parallel thread.

Let start the discussion! http://forum.nasaspaceflight.com/index.php?topic=34794.0

Online mvpel

  • Full Member
  • **
  • Posts: 270
  • Liked: 320
  • New Hampshire
I found the missing p-frame between i-frames 150 and 169, it is just following the i-frame. Here the debug log at the end of the i-frame, the next frame is a p-frame but keeps the number 000 when it should have 001. Should we try to find the position in the ts file and hand-edit the "ghost" p-frame header?

I: 47 43 E8 3F 07 10 00 32 86 C7 7E 00 00 00 01 E0 00 00 81 80 07 21 01 93 CC FD FF FF 00 00 01 B0 03 00 00 01 B5 0D 0F 00 00 01 00 00 00 01 20

This is the first bit of the transport stream for the iframe out of rawsplit_part9.ts - the 000001B0 is a visual sequence start marker.

The next start sequence we see is this,. at 0x115:

0x0115 : 00 00 01 B6 14

The 000001B6 code means that it's a video object plane start marker, and the next two bits tell us whether it's a "VOP-I" or a "VOP-P" - the four bits after the B6 are 0001, and since the first two are 00, we know it's an iframe that follows.

This page has a good explanation of these pieces: http://stackoverflow.com/questions/4082080/parsing-mpeg4-frames-from-rtp-packets

At the 0x5184 offset in rawsplit_part9.ts:

P: 47 43 E8 3C 07 10 00 32 92 82 7E 00 00 00 01 E0 00 00 81 80 07 21 01 93 FB E9 FF FF 00 00 01 B6 55 38 0C 06 42 9A 7F 7E 2A 00 93 E3 C5 78 29

This is a video object plane indicated by 000001B6, and the first two bits of the highlighted "5"  (0101) are "01", indicating that it's a pframe. You can see this continue on down the line for subsequent pframes:

P: 47 43 E8 37 07 10 00 32 9E 3D 7E 00 00 00 01 E0 00 00 81 80 07 21 01 95 2A D5 FF FF 00 00 01 B6 55 F3 BE 04 42 99 62 9D A3 70 61 5F B7 94 1F

P: 47 43 E8 3B 07 10 00 32 A9 F8 7E 00 00 00 01 E0 00 00 81 80 07 21 01 95 59 C1 FF FF 00 00 01 B6 56 AF 6C 06 53 57 38 76 4A E1 99 55 02 D2 39

P: 47 43 E8 31 07 10 00 32 B5 B3 7E 00 00 00 01 E0 00 00 81 80 07 21 01 95 88 AD FF FF 00 00 01 B6 57 6B 1E 06 53 02 00 82 7D B0 60 48 08 0A 03


Now, I see that the bits after the VOP type indicator, is a counter incrementing on each pframe, but I don't seem to have the right documentation on hand to tell me how many bits it's supposed to be. It looks like it skipped an increment, but I'm not sure if the counter is limited to that one nybble or not. I seem to recall Shanuson figured that out in a post a while back, I'll check more and see what I can find, or if we're lucky Michael Niedermyer will stop in. :)

(Edit: checked rawsplit-6, and it is a much longer than four bit counter. )

AJMartin - that's one heck of a first post!! Nice find! There were some other areas which had 56-byte multiples, which didn't make any sense in the context of a 188-byte transport stream packet. It looks like you may have solved a mystery here. We haven't gotten further information from SpaceX about the characteristics of the radio transmission from the booster, but it sounds like that would fit together with what you've observed.
« Last Edit: 05/21/2014 04:18 AM by mvpel »

Online meekGee

  • Global Moderator
  • Senior Member
  • *****
  • Posts: 3217
  • Liked: 740

Looking at the null packets, I noticed some patterns in the bit errors that might be useful for this kind of effort.

Specifically, many (but not all) of the bit-flips come in groups of 3, with one bit flipped, then 13 unaffected, then the next two also flipped.  This is most obvious in a hex-dump of the null packets, which are supposed to be filled with 0xff bytes: patterns such as "7f fc" or "fb ff e7" are common.  But the same bit-flip patterns apply where errors overlay the TS headers, and presumably also when they occur in image data.

Combinations (with XOR) of these 3-bit errors are also seen; mathematically, these error patterns are multiples of the polynomial x^15 + x + 1 over the binary field.  This could be the result of FEC coding applied to a signal with too many errors for successful correction.  I haven't figured it out completely but there are indications that the data may have been coded in 28-byte or 56-byte blocks (not respecting the TS packet boundaries); in particular, errors that don't fit this pattern seem to be found near offsets that are multiples of 28 bytes.

There is still the problem of figuring out if the change helped or not, but this should help to reduce the search space.  In low-error regions, there's a good chance that some of the flaws can be fixed with a single instance of this pattern.

If there was a bigger "Like" button, I'd use it.  Consider this a super-like.

Online mvpel

  • Full Member
  • **
  • Posts: 270
  • Liked: 320
  • New Hampshire
A 300% like-to-post ratio - is that record? I wonder how high it'll get when the Europeans sign on? :)

(Edit: now it's 1,200% like-to-post.  :D )

I went through rawsplit-9 with Wireshark and bvi, and found a few dozen instances where Shanuson's fixing tools were too conservative to pound a particularly mangled transport stream header into conformance, as he had mentioned. I did this by hand in the attached file, and it now shows no protocol warnings in Wireshark.

If someone more conversant with ffmpeg could take a look at it and compare it to rawsplit-9 to see what comes out the other end of it, I'd appreciate it. I'm curious if that might help the ghost pframe reported earlier, or if it will make any noticeable difference in the frames that come out or change any offsets. In any case, a clean transport stream is better than dirty, one would assume.

If it's worth the effort, we can come up with a more aggressive transport stream header solver.
« Last Edit: 05/21/2014 12:56 PM by mvpel »

Offline meadows.st

  • Full Member
  • *
  • Posts: 111
  • Liked: 62
  • Vancouver BC, Canada
Coming up on the 150,000 view mark for this thread/effort. Noting it as there's a small active team of people really working hard on this, but also a HUGE amount of people watching that work. Always nice to have a large appreciative audience, both inside and outside the space industry.

You'd be astonished how many views are from IP addresses in a place called Hawthorne, California! ;D

I was curious how this thread ranked overall in the history of SpaceX threads (currently as of this writing, reads are >180,000) so I sorted the other SpaceX dedicated threads and only the 100,000+ and Missions sections have threads that exceed 180,000. Currently this thread is the 27th most read SpaceX thread and I think the NSF server (sorry Chris  ;D) will explode when you all get the video restored "even" to the level of SwissCheese's latest gif (which is awesome BTW). see:
I updated http://spacexlanding.wikispaces.com/Frames+169-188 :)

Graphic representation of the historical significance of the work that the video restoration group is doing (see pic).

Added: Updated context with consolidated "conversations" (multiple threads on same topic now rolled-up into a single "conversation").
« Last Edit: 05/24/2014 11:33 PM by meadows.st »
The difference between stupidity and genius is that genius has its limits. - Albert Einstein? (first ref (1961) attributed to "unknown")

Offline arnezami

  • Full Member
  • **
  • Posts: 275
  • Liked: 258
I'm trying to find flipped bits.

The problem I'm having is, it's almost almost impossible to tell if I've found a flipped bit. Just because the next few blocks or rows improve that doesn't mean you found a flipped one. It's very well possible you just found by luck a way to get the alignment of the next macroblocks right, but there's still a subtle propagating error.

The advantage of finding a flipped bit is, you preserve the most information possible in the frame. But it's only possible in regions where the error rate is less then perhaps 1 flipped bit per 1000.

Looking at the null packets, I noticed some patterns in the bit errors that might be useful for this kind of effort.

Specifically, many (but not all) of the bit-flips come in groups of 3, with one bit flipped, then 13 unaffected, then the next two also flipped.  This is most obvious in a hex-dump of the null packets, which are supposed to be filled with 0xff bytes: patterns such as "7f fc" or "fb ff e7" are common.  But the same bit-flip patterns apply where errors overlay the TS headers, and presumably also when they occur in image data.

Combinations (with XOR) of these 3-bit errors are also seen; mathematically, these error patterns are multiples of the polynomial x^15 + x + 1 over the binary field.  This could be the result of FEC coding applied to a signal with too many errors for successful correction.  I haven't figured it out completely but there are indications that the data may have been coded in 28-byte or 56-byte blocks (not respecting the TS packet boundaries); in particular, errors that don't fit this pattern seem to be found near offsets that are multiples of 28 bytes.

There is still the problem of figuring out if the change helped or not, but this should help to reduce the search space.  In low-error regions, there's a good chance that some of the flaws can be fixed with a single instance of this pattern.
Incredibly great find!!!

I hope we can use it somehow. But finding patterns in the errors is potentially HUGE!

Offline arnezami

  • Full Member
  • **
  • Posts: 275
  • Liked: 258
There's a new Online editor up for doing p-frame stuff here: http://spacex2.slapbet.org/

It should be fairly self explanatory for those doing p-frame work, it's also very new and is missing some stuff (like log / error info) which I will get to when I can.

Have a play.

Sir. You are a magician! I love the slider :)

Offline saliva_sweet

  • Member
  • Posts: 97
  • Liked: 98
Looking at the null packets, I noticed some patterns in the bit errors that might be useful for this kind of effort.

Specifically, many (but not all) of the bit-flips come in groups of 3, with one bit flipped, then 13 unaffected, then the next two also flipped.  This is most obvious in a hex-dump of the null packets, which are supposed to be filled with 0xff bytes: patterns such as "7f fc" or "fb ff e7" are common.  But the same bit-flip patterns apply where errors overlay the TS headers, and presumably also when they occur in image data.

Combinations (with XOR) of these 3-bit errors are also seen; mathematically, these error patterns are multiples of the polynomial x^15 + x + 1 over the binary field.  This could be the result of FEC coding applied to a signal with too many errors for successful correction.  I haven't figured it out completely but there are indications that the data may have been coded in 28-byte or 56-byte blocks (not respecting the TS packet boundaries); in particular, errors that don't fit this pattern seem to be found near offsets that are multiples of 28 bytes.

There is still the problem of figuring out if the change helped or not, but this should help to reduce the search space.  In low-error regions, there's a good chance that some of the flaws can be fixed with a single instance of this pattern.

That is brilliant. And something that never occurred to me. It could be incredibly important. Just yesterday (posted here: http://forum.nasaspaceflight.com/index.php?topic=34597.msg1201317#msg1201317) through a lot of trial and error I ended up doing this to frame72: X:11807:80,X:11822:80. We could really use more statistics though. Need to know what is the probability of single flips vs patterned flips. Perhaps a graph showing probability of flipped bit in positions following another flipped bit going up to 28 bytes.

Offline wronkiew

  • Full Member
  • *
  • Posts: 173
  • Liked: 102
  • 34.502327, -116.971697
Just in case someone else runs into this, I misplaced the -threads option and it was causing ffmpeg to ignore it. This is on OS X, so apparently the threads thing is not just Windows. Here is the command line I used successfully:

./ffmpeg -threads 1 -r 44999/3003 -mmb `cat mmb_starting_from_209.txt` -debug mb_pos_size -err_detect ignore_err  -i fixed_edit8_part_209.ts -f image2 fixed_edit8_part_209_%03d.png

Offline Shanuson

  • Full Member
  • *
  • Posts: 110
  • Liked: 75
A 300% like-to-post ratio - is that record? I wonder how high it'll get when the Europeans sign on? :)

I went through rawsplit-9 with Wireshark and bvi, and found a few dozen instances where Shanuson's fixing tools were too conservative to pound a particularly mangled transport stream header into conformance, as he had mentioned. I did this by hand in the attached file, and it now shows no protocol warnings in Wireshark.

If someone more conversant with ffmpeg could take a look at it and compare it to rawsplit-9 to see what comes out the other end of it, I'd appreciate it. I'm curious if that might help the ghost pframe reported earlier, or if it will make any noticeable difference in the frames that come out or change any offsets. In any case, a clean transport stream is better than dirty, one would assume.

If it's worth the effort, we can come up with a more aggressive transport stream header solver.


My tools are more or less my hands and a few python scripts. :D and yes, i was conservative or not concerned about that problem yet. Did you only fix some 4byte TS-headers? or more? And only for some part or the whole stream? If only for one part. could you apply those changes to the Part_X.ts file so we can more easily merge your and my changes together?

@ajmartin
nice find. That could explain why there were 56 byte missing in raw.ts at 5 points. Never 28 though.
« Last Edit: 05/21/2014 07:21 AM by Shanuson »

Online mvpel

  • Full Member
  • **
  • Posts: 270
  • Liked: 320
  • New Hampshire
My tools are more or less my hands and a few python scripts. :D and yes, i was conservative or not concerned about that problem yet. Did you only fix some 4byte TS-headers? or more? And only for some part or the whole stream? If only for one part. could you apply those changes to the Part_X.ts file so we can more easily merge your and my changes together?

In the part9 I uploaded, it was strictly the transport stream headers, since that's what was easily discernable with Wireshark, and I only dealt with that one segment of the stream. Working through the procedure by hand, though, gave me a sense of what the algorithm would need to be to implement it, and extend it to the PES and MPEG headers. I'm currently thinking that breaking out the entire stream into an array of nested C structs will make it easier to manipulate and parse, and apply the known anchor points to the bits.

I recall you mentioning a section where the 188-byte alignment goes out the window at one point - something about AJMartin's 28- or 56-byte frames, perhaps? Do you have the offset where that occurs?

Online SwissCheese

  • Full Member
  • *
  • Posts: 125
  • Liked: 210
Hello everyone, latest update of the gif  :) (frames 169-173,175-177,180-181,183-199)

It looks really cool!

Note that many p-frames only need very little correction, some even don't need any.

One issue seems to be corrupt motion vectors (see the legs), would it be possible to implement something to change the motion vectors values? or at least set them to 0?

(Click the image to make the gif animate).
« Last Edit: 05/21/2014 06:08 PM by Chris Bergin »

Tags: