Author Topic: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread  (Read 1201522 times)

There's a new Online editor up for doing p-frame stuff here: http://spacex2.slapbet.org/

It should be fairly self explanatory for those doing p-frame work, it's also very new and is missing some stuff (like log / error info) which I will get to when I can.

Have a play.

Online Asmegin

  • Member
  • Posts: 37
  • Canada
  • Liked: 26
  • Likes Given: 3
New editor looks nice - love the slider feature!

Doesn't have any bit info when you click on a block yet, so can't really use it right now.

Offline Maciej Olesinski

  • Full Member
  • *
  • Posts: 136
  • Liked: 34
  • Likes Given: 15
You guyz are doing amazing job!

From above gif I can tell that stage was landing with wild angle (tipped to right from viewer perspective) and doing very nice hoover slam.

Offline meadows.st

  • Full Member
  • *
  • Posts: 158
  • Toronto ON, Canada
  • Liked: 90
  • Likes Given: 5422
You guyz are doing amazing job!

From above gif I can tell that stage was landing with wild angle (tipped to right from viewer perspective) and doing very nice hoover slam.

I think we need the "video interpretation/analysis" thread sooner rather than later given the stellar work that the BFE "bit flippers extraordinaire" group is doing.


So, if the mods want to move this response, please do.

re: "wild angle" hypothesis.  I do not agree since the frame 188 image I posted above appears to have a virtually round plume (the yellow arcs were drawn as a circle and positioned to bound the outer limits of the visible plume). 
I would expect this plume to be significantly elliptical (I understand it is already distorted by the fish-eye camera) if the stage were descending at a "wild angle" (unless our use of the term is very different).

EDIT: After re-reading your comment it occurred to me that you might have meant to write "mild angle"? I can accept the hypothesis of a mild angle especially given the reported weather conditions.
« Last Edit: 05/21/2014 12:00 am by meadows.st »
“A little rudder far from the rocks is a lot better than a lot of rudder close to the rocks.” L. David Marquet

Offline Lars_J

  • Senior Member
  • *****
  • Posts: 6160
  • California
  • Liked: 677
  • Likes Given: 195
Yes, the video interpretation should be in it own thread. I'm interested in reading about the data recovery efforts here - and not having it mixed in with interpretations (mostly wrong) based on what people think they are seeing.

Offline mvpel

  • Full Member
  • ****
  • Posts: 1125
  • New Hampshire
  • Liked: 1303
  • Likes Given: 1685
Its journey started with a huge plume of water, and ended exactly the same way.  ;D
"Ugly programs are like ugly suspension bridges: they're much more liable to collapse than pretty ones, because the way humans (especially engineer-humans) perceive beauty is intimately related to our ability to process and understand complexity. A language that makes it hard to write elegant code makes it hard to write good code." - Eric S. Raymond

Offline ajmartin

  • Member
  • Posts: 4
  • Liked: 25
  • Likes Given: 179
I'm trying to find flipped bits.

The problem I'm having is, it's almost almost impossible to tell if I've found a flipped bit. Just because the next few blocks or rows improve that doesn't mean you found a flipped one. It's very well possible you just found by luck a way to get the alignment of the next macroblocks right, but there's still a subtle propagating error.

The advantage of finding a flipped bit is, you preserve the most information possible in the frame. But it's only possible in regions where the error rate is less then perhaps 1 flipped bit per 1000.

Looking at the null packets, I noticed some patterns in the bit errors that might be useful for this kind of effort.

Specifically, many (but not all) of the bit-flips come in groups of 3, with one bit flipped, then 13 unaffected, then the next two also flipped.  This is most obvious in a hex-dump of the null packets, which are supposed to be filled with 0xff bytes: patterns such as "7f fc" or "fb ff e7" are common.  But the same bit-flip patterns apply where errors overlay the TS headers, and presumably also when they occur in image data.

Combinations (with XOR) of these 3-bit errors are also seen; mathematically, these error patterns are multiples of the polynomial x^15 + x + 1 over the binary field.  This could be the result of FEC coding applied to a signal with too many errors for successful correction.  I haven't figured it out completely but there are indications that the data may have been coded in 28-byte or 56-byte blocks (not respecting the TS packet boundaries); in particular, errors that don't fit this pattern seem to be found near offsets that are multiples of 28 bytes.

There is still the problem of figuring out if the change helped or not, but this should help to reduce the search space.  In low-error regions, there's a good chance that some of the flaws can be fixed with a single instance of this pattern.

Offline moralec

EDIT: I concur that this discussion should go in a parallel thread.

Let start the discussion! http://forum.nasaspaceflight.com/index.php?topic=34794.0

Offline mvpel

  • Full Member
  • ****
  • Posts: 1125
  • New Hampshire
  • Liked: 1303
  • Likes Given: 1685
I found the missing p-frame between i-frames 150 and 169, it is just following the i-frame. Here the debug log at the end of the i-frame, the next frame is a p-frame but keeps the number 000 when it should have 001. Should we try to find the position in the ts file and hand-edit the "ghost" p-frame header?

I: 47 43 E8 3F 07 10 00 32 86 C7 7E 00 00 00 01 E0 00 00 81 80 07 21 01 93 CC FD FF FF 00 00 01 B0 03 00 00 01 B5 0D 0F 00 00 01 00 00 00 01 20

This is the first bit of the transport stream for the iframe out of rawsplit_part9.ts - the 000001B0 is a visual sequence start marker.

The next start sequence we see is this,. at 0x115:

0x0115 : 00 00 01 B6 14

The 000001B6 code means that it's a video object plane start marker, and the next two bits tell us whether it's a "VOP-I" or a "VOP-P" - the four bits after the B6 are 0001, and since the first two are 00, we know it's an iframe that follows.

This page has a good explanation of these pieces: http://stackoverflow.com/questions/4082080/parsing-mpeg4-frames-from-rtp-packets

At the 0x5184 offset in rawsplit_part9.ts:

P: 47 43 E8 3C 07 10 00 32 92 82 7E 00 00 00 01 E0 00 00 81 80 07 21 01 93 FB E9 FF FF 00 00 01 B6 55 38 0C 06 42 9A 7F 7E 2A 00 93 E3 C5 78 29

This is a video object plane indicated by 000001B6, and the first two bits of the highlighted "5"  (0101) are "01", indicating that it's a pframe. You can see this continue on down the line for subsequent pframes:

P: 47 43 E8 37 07 10 00 32 9E 3D 7E 00 00 00 01 E0 00 00 81 80 07 21 01 95 2A D5 FF FF 00 00 01 B6 55 F3 BE 04 42 99 62 9D A3 70 61 5F B7 94 1F

P: 47 43 E8 3B 07 10 00 32 A9 F8 7E 00 00 00 01 E0 00 00 81 80 07 21 01 95 59 C1 FF FF 00 00 01 B6 56 AF 6C 06 53 57 38 76 4A E1 99 55 02 D2 39

P: 47 43 E8 31 07 10 00 32 B5 B3 7E 00 00 00 01 E0 00 00 81 80 07 21 01 95 88 AD FF FF 00 00 01 B6 57 6B 1E 06 53 02 00 82 7D B0 60 48 08 0A 03


Now, I see that the bits after the VOP type indicator, is a counter incrementing on each pframe, but I don't seem to have the right documentation on hand to tell me how many bits it's supposed to be. It looks like it skipped an increment, but I'm not sure if the counter is limited to that one nybble or not. I seem to recall Shanuson figured that out in a post a while back, I'll check more and see what I can find, or if we're lucky Michael Niedermyer will stop in. :)

(Edit: checked rawsplit-6, and it is a much longer than four bit counter. )

AJMartin - that's one heck of a first post!! Nice find! There were some other areas which had 56-byte multiples, which didn't make any sense in the context of a 188-byte transport stream packet. It looks like you may have solved a mystery here. We haven't gotten further information from SpaceX about the characteristics of the radio transmission from the booster, but it sounds like that would fit together with what you've observed.
« Last Edit: 05/21/2014 04:18 am by mvpel »
"Ugly programs are like ugly suspension bridges: they're much more liable to collapse than pretty ones, because the way humans (especially engineer-humans) perceive beauty is intimately related to our ability to process and understand complexity. A language that makes it hard to write elegant code makes it hard to write good code." - Eric S. Raymond

Offline meekGee

  • Senior Member
  • *****
  • Posts: 14158
  • N. California
  • Liked: 14046
  • Likes Given: 1392

Looking at the null packets, I noticed some patterns in the bit errors that might be useful for this kind of effort.

Specifically, many (but not all) of the bit-flips come in groups of 3, with one bit flipped, then 13 unaffected, then the next two also flipped.  This is most obvious in a hex-dump of the null packets, which are supposed to be filled with 0xff bytes: patterns such as "7f fc" or "fb ff e7" are common.  But the same bit-flip patterns apply where errors overlay the TS headers, and presumably also when they occur in image data.

Combinations (with XOR) of these 3-bit errors are also seen; mathematically, these error patterns are multiples of the polynomial x^15 + x + 1 over the binary field.  This could be the result of FEC coding applied to a signal with too many errors for successful correction.  I haven't figured it out completely but there are indications that the data may have been coded in 28-byte or 56-byte blocks (not respecting the TS packet boundaries); in particular, errors that don't fit this pattern seem to be found near offsets that are multiples of 28 bytes.

There is still the problem of figuring out if the change helped or not, but this should help to reduce the search space.  In low-error regions, there's a good chance that some of the flaws can be fixed with a single instance of this pattern.

If there was a bigger "Like" button, I'd use it.  Consider this a super-like.
ABCD - Always Be Counting Down

Offline mvpel

  • Full Member
  • ****
  • Posts: 1125
  • New Hampshire
  • Liked: 1303
  • Likes Given: 1685
A 300% like-to-post ratio - is that record? I wonder how high it'll get when the Europeans sign on? :)

(Edit: now it's 1,200% like-to-post.  :D )

I went through rawsplit-9 with Wireshark and bvi, and found a few dozen instances where Shanuson's fixing tools were too conservative to pound a particularly mangled transport stream header into conformance, as he had mentioned. I did this by hand in the attached file, and it now shows no protocol warnings in Wireshark.

If someone more conversant with ffmpeg could take a look at it and compare it to rawsplit-9 to see what comes out the other end of it, I'd appreciate it. I'm curious if that might help the ghost pframe reported earlier, or if it will make any noticeable difference in the frames that come out or change any offsets. In any case, a clean transport stream is better than dirty, one would assume.

If it's worth the effort, we can come up with a more aggressive transport stream header solver.
« Last Edit: 05/21/2014 12:56 pm by mvpel »
"Ugly programs are like ugly suspension bridges: they're much more liable to collapse than pretty ones, because the way humans (especially engineer-humans) perceive beauty is intimately related to our ability to process and understand complexity. A language that makes it hard to write elegant code makes it hard to write good code." - Eric S. Raymond

Offline meadows.st

  • Full Member
  • *
  • Posts: 158
  • Toronto ON, Canada
  • Liked: 90
  • Likes Given: 5422
Coming up on the 150,000 view mark for this thread/effort. Noting it as there's a small active team of people really working hard on this, but also a HUGE amount of people watching that work. Always nice to have a large appreciative audience, both inside and outside the space industry.

You'd be astonished how many views are from IP addresses in a place called Hawthorne, California! ;D

I was curious how this thread ranked overall in the history of SpaceX threads (currently as of this writing, reads are >180,000) so I sorted the other SpaceX dedicated threads and only the 100,000+ and Missions sections have threads that exceed 180,000. Currently this thread is the 27th most read SpaceX thread and I think the NSF server (sorry Chris  ;D) will explode when you all get the video restored "even" to the level of SwissCheese's latest gif (which is awesome BTW). see:
I updated http://spacexlanding.wikispaces.com/Frames+169-188 :)

Graphic representation of the historical significance of the work that the video restoration group is doing (see pic).

Added: Updated context with consolidated "conversations" (multiple threads on same topic now rolled-up into a single "conversation").
« Last Edit: 05/24/2014 11:33 pm by meadows.st »
“A little rudder far from the rocks is a lot better than a lot of rudder close to the rocks.” L. David Marquet

Offline arnezami

  • Full Member
  • **
  • Posts: 285
  • Liked: 267
  • Likes Given: 378
I'm trying to find flipped bits.

The problem I'm having is, it's almost almost impossible to tell if I've found a flipped bit. Just because the next few blocks or rows improve that doesn't mean you found a flipped one. It's very well possible you just found by luck a way to get the alignment of the next macroblocks right, but there's still a subtle propagating error.

The advantage of finding a flipped bit is, you preserve the most information possible in the frame. But it's only possible in regions where the error rate is less then perhaps 1 flipped bit per 1000.

Looking at the null packets, I noticed some patterns in the bit errors that might be useful for this kind of effort.

Specifically, many (but not all) of the bit-flips come in groups of 3, with one bit flipped, then 13 unaffected, then the next two also flipped.  This is most obvious in a hex-dump of the null packets, which are supposed to be filled with 0xff bytes: patterns such as "7f fc" or "fb ff e7" are common.  But the same bit-flip patterns apply where errors overlay the TS headers, and presumably also when they occur in image data.

Combinations (with XOR) of these 3-bit errors are also seen; mathematically, these error patterns are multiples of the polynomial x^15 + x + 1 over the binary field.  This could be the result of FEC coding applied to a signal with too many errors for successful correction.  I haven't figured it out completely but there are indications that the data may have been coded in 28-byte or 56-byte blocks (not respecting the TS packet boundaries); in particular, errors that don't fit this pattern seem to be found near offsets that are multiples of 28 bytes.

There is still the problem of figuring out if the change helped or not, but this should help to reduce the search space.  In low-error regions, there's a good chance that some of the flaws can be fixed with a single instance of this pattern.
Incredibly great find!!!

I hope we can use it somehow. But finding patterns in the errors is potentially HUGE!

Offline arnezami

  • Full Member
  • **
  • Posts: 285
  • Liked: 267
  • Likes Given: 378
There's a new Online editor up for doing p-frame stuff here: http://spacex2.slapbet.org/

It should be fairly self explanatory for those doing p-frame work, it's also very new and is missing some stuff (like log / error info) which I will get to when I can.

Have a play.

Sir. You are a magician! I love the slider :)

Offline saliva_sweet

  • Full Member
  • ****
  • Posts: 614
  • Liked: 476
  • Likes Given: 1826
Looking at the null packets, I noticed some patterns in the bit errors that might be useful for this kind of effort.

Specifically, many (but not all) of the bit-flips come in groups of 3, with one bit flipped, then 13 unaffected, then the next two also flipped.  This is most obvious in a hex-dump of the null packets, which are supposed to be filled with 0xff bytes: patterns such as "7f fc" or "fb ff e7" are common.  But the same bit-flip patterns apply where errors overlay the TS headers, and presumably also when they occur in image data.

Combinations (with XOR) of these 3-bit errors are also seen; mathematically, these error patterns are multiples of the polynomial x^15 + x + 1 over the binary field.  This could be the result of FEC coding applied to a signal with too many errors for successful correction.  I haven't figured it out completely but there are indications that the data may have been coded in 28-byte or 56-byte blocks (not respecting the TS packet boundaries); in particular, errors that don't fit this pattern seem to be found near offsets that are multiples of 28 bytes.

There is still the problem of figuring out if the change helped or not, but this should help to reduce the search space.  In low-error regions, there's a good chance that some of the flaws can be fixed with a single instance of this pattern.

That is brilliant. And something that never occurred to me. It could be incredibly important. Just yesterday (posted here: http://forum.nasaspaceflight.com/index.php?topic=34597.msg1201317#msg1201317) through a lot of trial and error I ended up doing this to frame72: X:11807:80,X:11822:80. We could really use more statistics though. Need to know what is the probability of single flips vs patterned flips. Perhaps a graph showing probability of flipped bit in positions following another flipped bit going up to 28 bytes.

Offline wronkiew

  • Full Member
  • *
  • Posts: 186
  • 34.502327, -116.971697
  • Liked: 105
  • Likes Given: 125
Just in case someone else runs into this, I misplaced the -threads option and it was causing ffmpeg to ignore it. This is on OS X, so apparently the threads thing is not just Windows. Here is the command line I used successfully:

./ffmpeg -threads 1 -r 44999/3003 -mmb `cat mmb_starting_from_209.txt` -debug mb_pos_size -err_detect ignore_err  -i fixed_edit8_part_209.ts -f image2 fixed_edit8_part_209_%03d.png

Online Shanuson

  • Full Member
  • ***
  • Posts: 395
  • Liked: 327
  • Likes Given: 2542
A 300% like-to-post ratio - is that record? I wonder how high it'll get when the Europeans sign on? :)

I went through rawsplit-9 with Wireshark and bvi, and found a few dozen instances where Shanuson's fixing tools were too conservative to pound a particularly mangled transport stream header into conformance, as he had mentioned. I did this by hand in the attached file, and it now shows no protocol warnings in Wireshark.

If someone more conversant with ffmpeg could take a look at it and compare it to rawsplit-9 to see what comes out the other end of it, I'd appreciate it. I'm curious if that might help the ghost pframe reported earlier, or if it will make any noticeable difference in the frames that come out or change any offsets. In any case, a clean transport stream is better than dirty, one would assume.

If it's worth the effort, we can come up with a more aggressive transport stream header solver.


My tools are more or less my hands and a few python scripts. :D and yes, i was conservative or not concerned about that problem yet. Did you only fix some 4byte TS-headers? or more? And only for some part or the whole stream? If only for one part. could you apply those changes to the Part_X.ts file so we can more easily merge your and my changes together?

@ajmartin
nice find. That could explain why there were 56 byte missing in raw.ts at 5 points. Never 28 though.
« Last Edit: 05/21/2014 07:21 am by Shanuson »

Offline mvpel

  • Full Member
  • ****
  • Posts: 1125
  • New Hampshire
  • Liked: 1303
  • Likes Given: 1685
My tools are more or less my hands and a few python scripts. :D and yes, i was conservative or not concerned about that problem yet. Did you only fix some 4byte TS-headers? or more? And only for some part or the whole stream? If only for one part. could you apply those changes to the Part_X.ts file so we can more easily merge your and my changes together?

In the part9 I uploaded, it was strictly the transport stream headers, since that's what was easily discernable with Wireshark, and I only dealt with that one segment of the stream. Working through the procedure by hand, though, gave me a sense of what the algorithm would need to be to implement it, and extend it to the PES and MPEG headers. I'm currently thinking that breaking out the entire stream into an array of nested C structs will make it easier to manipulate and parse, and apply the known anchor points to the bits.

I recall you mentioning a section where the 188-byte alignment goes out the window at one point - something about AJMartin's 28- or 56-byte frames, perhaps? Do you have the offset where that occurs?
"Ugly programs are like ugly suspension bridges: they're much more liable to collapse than pretty ones, because the way humans (especially engineer-humans) perceive beauty is intimately related to our ability to process and understand complexity. A language that makes it hard to write elegant code makes it hard to write good code." - Eric S. Raymond

Offline SwissCheese

  • Full Member
  • *
  • Posts: 166
  • Liked: 256
  • Likes Given: 107
Hello everyone, latest update of the gif  :) (frames 169-173,175-177,180-181,183-199)

It looks really cool!

Note that many p-frames only need very little correction, some even don't need any.

One issue seems to be corrupt motion vectors (see the legs), would it be possible to implement something to change the motion vectors values? or at least set them to 0?

(Click the image to make the gif animate).
« Last Edit: 05/21/2014 06:08 pm by Chris Bergin »

Offline CuddlyRocket

^ Amazing, awesome, astonishing - yes, I'm impressed with the achievement here (and more to come!).

Hopefully, there'll be some appropriate thank you for all this work coming the workers' way. A guided tour or an invite to a launch, perhaps?

Tags:
 

Advertisement NovaTech
Advertisement Northrop Grumman
Advertisement
Advertisement Margaritaville Beach Resort South Padre Island
Advertisement Brady Kenniston
Advertisement NextSpaceflight
Advertisement Nathan Barker Photography
0