Author Topic: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread  (Read 1201693 times)

Offline mlindner

  • Software Engineer
  • Senior Member
  • *****
  • Posts: 2908
  • Space Capitalist
  • Silicon Valley, CA
  • Liked: 2204
  • Likes Given: 818
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1120 on: 05/29/2014 10:29 am »
Just want to say this is an amazing job you guys have done! I had to drop out from doing it because real life intervened in my ability to have time to assist. Hopefully I'll have time to catch up on the some 30 pages that have been posted since I last helped.

Just as a quick question, is there still need to repair I-frames? I notice that many of the I-frames have not advanced much in repair state from when I was last involved.
LEO is the ocean, not an island (let alone a continent). We create cruise liners to ride the oceans, not artificial islands in the middle of them. We need a physical place, which has physical resources, to make our future out there.

Offline arnezami

  • Full Member
  • **
  • Posts: 285
  • Liked: 267
  • Likes Given: 378
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1121 on: 05/29/2014 10:43 am »
Hi guys,

Little update.

I'm experimenting with logging detailed info about all the decoded bits in the macroblocks (and their interpretation).

Attached is a preliminary image showing the first few marcroblocks of frame 169.

Red is macroblock-header. The cbpy represents which lum-blocks will contain DCTs, cbpc does for chrom-blocks.
Yellow is DC-values (4 lum, 2 chrom). So this is either 8x8px info or 16x16px info. Bascicaly ranges from 0 to 255 each.
Green is DCT-values inside each of these 6 blocks. So this is sub 8x8px or 16x16px info.

You can see that not every block has transformation data (the green sub-8/16-pixel data).

The bits themselves are not logged yet, but that should be relatively easy to do.

Just wanted to share. :)

Regards,

arnezami

PS. I am considering a macroblock editor/analyser of sorts. Where you can see each bit and where it belongs to given the current starting bitposition of a (macro)block. Then you can see what a flipped bit will actually do to the interpretation by the decoder. Not sure yet how best to do that.
« Last Edit: 05/29/2014 03:59 pm by arnezami »

Offline arnezami

  • Full Member
  • **
  • Posts: 285
  • Liked: 267
  • Likes Given: 378
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1122 on: 05/29/2014 10:56 am »
Just want to say this is an amazing job you guys have done! I had to drop out from doing it because real life intervened in my ability to have time to assist. Hopefully I'll have time to catch up on the some 30 pages that have been posted since I last helped.

Just as a quick question, is there still need to repair I-frames? I notice that many of the I-frames have not advanced much in repair state from when I was last involved.
Hi mlindner! Glad you're back :)

There is definitely still a need for work on the I-frames. I think many moved their focus to the P-frames because making video is so tempting and rewarding ;)

But improving the I-frames clearly makes the 19 P-frames that follow much better!

Your excellent work on frame 169 shows that!

Regards,

arnezami
« Last Edit: 05/29/2014 10:59 am by arnezami »

Offline Shanuson

  • Full Member
  • ***
  • Posts: 395
  • Liked: 327
  • Likes Given: 2542
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1123 on: 05/29/2014 11:21 am »
@arnezami
Would it be possible to add a option to manipulate the VOP-quant and vop_fcode_forward part of a p-frame header?
Some seem to be messed up, and at least vop-quant seems to have a large effect on the whole frame.

Cheers
Shanuson

Online FutureSpaceTourist

  • Global Moderator
  • Senior Member
  • *****
  • Posts: 48178
  • UK
    • Plan 28
  • Liked: 81683
  • Likes Given: 36941
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1124 on: 05/29/2014 11:42 am »
Thread has just passed 250,000 views!

Keep up the amazing work - seeing the legs deploy on the latest video is a wonderful result.

Offline arnezami

  • Full Member
  • **
  • Posts: 285
  • Liked: 267
  • Likes Given: 378
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1125 on: 05/29/2014 12:18 pm »
@arnezami
Would it be possible to add a option to manipulate the VOP-quant and vop_fcode_forward part of a p-frame header?
Some seem to be messed up, and at least vop-quant seems to have a large effect on the whole frame.

Cheers
Shanuson
Hi Shanuson,

Maybe. Currently the mmb-command work in the "scope" after the VOP-quant etc have been decoded. Maybe/probably we are too late there. So I'm not sure if it's possible using the mmb-commands. But maybe using a different command. @michaelni: do you know how to do this?

Regards,

arnezami

Offline Lourens

  • Full Member
  • *
  • Posts: 156
  • The Netherlands
  • Liked: 206
  • Likes Given: 304
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1126 on: 05/29/2014 12:39 pm »
arnezami, could you post a patch? I was about to start doing the exact same thing :). Hopefully the additional output will allow a kind of guided bit flipping, where we can quickly discard flips that yield obvious nonsense.

Offline SwissCheese

  • Full Member
  • *
  • Posts: 166
  • Liked: 256
  • Likes Given: 107
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1127 on: 05/29/2014 01:08 pm »
I adapted part 10 to the new ts, wasn't too difficult. I made a few correction on the way :)

I think we should keep all the work put in the wiki with the previous ts files as an archive and create new pages with the new ts. Meanwhile I'll put the modification here.

edit: forum does not like big gif it seems...
« Last Edit: 05/29/2014 03:34 pm by SwissCheese »

Offline SwissCheese

  • Full Member
  • *
  • Posts: 166
  • Liked: 256
  • Likes Given: 107
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1128 on: 05/29/2014 03:27 pm »
And the same for part 9 (13/20 frames corrected) :)

Offline moralec

arnezami, could you post a patch? I was about to start doing the exact same thing :). Hopefully the additional output will allow a kind of guided bit flipping, where we can quickly discard flips that yield obvious nonsense.

Yes we can do that. I'll put the current pages on archive and create new empty ones. :)


Online Comga

  • Senior Member
  • *****
  • Posts: 6466
  • Liked: 4572
  • Likes Given: 5136
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1130 on: 05/29/2014 04:07 pm »
Here's an example of frame 94 with redrawn timestamps. The top part is the isolated pframe, the middle is the output from ffmpeg, and the bottom is repainted.

Only the digits which are known with certainty are modified, so this should be a faithful representation of those pixels. The 1/10 second place is interlaced between 6 and 7. The last digit is not known, so it is left scrambled.

If the first decimal place is interlaced between 6 and 7 one would expect that the second decimal place is interlaced between 9 and 0.  It does look like that.  With the preceding or following frames is that not sufficiently certain?

edit: Forgot to say that I am amazed at the terrific work people are doing here.
« Last Edit: 05/29/2014 04:12 pm by Comga »
What kind of wastrels would dump a perfectly good booster in the ocean after just one use?

Offline arnezami

  • Full Member
  • **
  • Posts: 285
  • Liked: 267
  • Likes Given: 378
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1131 on: 05/29/2014 04:12 pm »
arnezami, could you post a patch? I was about to start doing the exact same thing :). Hopefully the additional output will allow a kind of guided bit flipping, where we can quickly discard flips that yield obvious nonsense.

Yes we can do that. I'll put the current pages on archive and create new empty ones. :)

@moralec: I think you meant to reply on this post by SwissCheese (quoted below) right?  (instead of Laurens post)
@moralec/SwissCheese: would it be a good idea to number the I-frames to 1,21,41,61 etc? And the P-frames in between? Since we expect each I-frame to contain exactly 19 P-frames right?

I adapted part 10 to the new ts, wasn't too difficult. I made a few correction on the way :)

I think we should keep all the work put in the wiki with the previous ts files as an archive and create new pages with the new ts. Meanwhile I'll put the modification here.

edit: forum does not like big gif it seems...

@Lourens: I will try but there are still quite a few problems with this version I noticed. For example you can see that bits are missing. Thats not good ;). It's also very hacky atm. Will keep you updated.
« Last Edit: 05/29/2014 04:16 pm by arnezami »

Offline mvpel

  • Full Member
  • ****
  • Posts: 1125
  • New Hampshire
  • Liked: 1303
  • Likes Given: 1685
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1132 on: 05/29/2014 04:19 pm »
I adapted part 10 to the new ts, wasn't too difficult. I made a few correction on the way :)

It looks great! Does the stability of non-moving parts of the image mean that you guys were able to fix up bad motion vectors, or is this just an unusually good sequence?
"Ugly programs are like ugly suspension bridges: they're much more liable to collapse than pretty ones, because the way humans (especially engineer-humans) perceive beauty is intimately related to our ability to process and understand complexity. A language that makes it hard to write elegant code makes it hard to write good code." - Eric S. Raymond

Offline mhenderson

  • Member
  • Posts: 71
  • USA
  • Liked: 101
  • Likes Given: 18
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1133 on: 05/29/2014 04:21 pm »
@arnesami - should we adopt a numbering convention of PART.FRAME like 10.01 - 10.20? This allows for new pFrames to be found without changing all frames after.

Offline wronkiew

  • Full Member
  • *
  • Posts: 186
  • 34.502327, -116.971697
  • Liked: 105
  • Likes Given: 125
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1134 on: 05/29/2014 04:21 pm »
All the digits as drawn by the camera.

Offline wronkiew

  • Full Member
  • *
  • Posts: 186
  • 34.502327, -116.971697
  • Liked: 105
  • Likes Given: 125
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1135 on: 05/29/2014 04:30 pm »
Here's an example of frame 94 with redrawn timestamps. The top part is the isolated pframe, the middle is the output from ffmpeg, and the bottom is repainted.

Only the digits which are known with certainty are modified, so this should be a faithful representation of those pixels. The 1/10 second place is interlaced between 6 and 7. The last digit is not known, so it is left scrambled.

If the first decimal place is interlaced between 6 and 7 one would expect that the second decimal place is interlaced between 9 and 0.  It does look like that.  With the preceding or following frames is that not sufficiently certain?

edit: Forgot to say that I am amazed at the terrific work people are doing here.

The video only runs at ~15 frames per second, so the hundredths digit would be incremented by ~0.067 per (non-interlaced) frame. That makes its value in any particular frame unpredictable. I'm going to put together a table of digit-to-digit transitions that should make it easier to figure out what the p-frame is trying to draw without knowing what exactly was in the previous frame.

Offline Lourens

  • Full Member
  • *
  • Posts: 156
  • The Netherlands
  • Liked: 206
  • Likes Given: 304
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1136 on: 05/29/2014 04:50 pm »
arnezami, could you post a patch? I was about to start doing the exact same thing :). Hopefully the additional output will allow a kind of guided bit flipping, where we can quickly discard flips that yield obvious nonsense.

@Lourens: I will try but there are still quite a few problems with this version I noticed. For example you can see that bits are missing. Thats not good ;). It's also very hacky atm. Will keep you updated.
Okay, I'll go and have a look at the FFmpeg source myself as well, will post when I have something.

Offline meadows.st

  • Full Member
  • *
  • Posts: 158
  • Toronto ON, Canada
  • Liked: 90
  • Likes Given: 5422
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1137 on: 05/29/2014 05:00 pm »
Coming up on the 150,000 view mark for this thread/effort. Noting it as there's a small active team of people really working hard on this, but also a HUGE amount of people watching that work. Always nice to have a large appreciative audience, both inside and outside the space industry.

You'd be astonished how many views are from IP addresses in a place called Hawthorne, California! ;D

I was curious how this thread ranked overall in the history of SpaceX threads (currently as of this writing, reads are (1) >180,000) so I sorted the other SpaceX dedicated threads and only the 100,000+ and Missions sections have threads that exceed 180,000. Currently this thread is the (2) 27th most read SpaceX thread and I think the NSF server (sorry Chris  ;D) will explode when you all get the video restored "even" to the level of SwissCheese's latest gif (which is awesome BTW). see:


Graphic representation of the historical significance of the work that the video restoration group is doing (see pic).

Added: Updated context with consolidated "conversations" (multiple threads on same topic now rolled-up into a single "conversation").

1) Thread currently at >252,000 reads which puts it at 15th most read SpaceX thread of all time and 13th most read SpaceX conversation[1]. See original link for images of the overall ranking.

2) 27th to 15th in the space of <1 week. Y'all's latest video (oh yeah, and EM's tweet ;) ) really ramped up the hit rate.  I think this is further evidence that the "complete" video is going to rival a live launch hit rate. :D

Phenomenal work everyone!

[1] "conversations" = multiple threads on same topic rolled-up into a single line (e.g. General SpaceX and Falcon Discussion consists of 9 threads for a total of >2.2M views.
“A little rudder far from the rocks is a lot better than a lot of rudder close to the rocks.” L. David Marquet

Offline princess

  • Member
  • Posts: 65
  • Liked: 106
  • Likes Given: 25
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1138 on: 05/29/2014 05:46 pm »
I've spent today going over Shanuson's complete TS file, cross-referencing it with the original raw TS file and fixing up the non-data packet types. All packets of type 0x0000 and 0x0020 are now present with the correct contents and no continuity errors. All padding packets have been identified and set as valid. All packets in the file have a PID of either 0x0000, 0x0020, 0x03e8 or 0x1fff.

I'm using a deliberately old version of VLC (from Ubuntu 12.10) to test with as it was far less error-tolerant back then - although the old VLC can't get a picture from the TS file yet, the file now reports no TS-level errors, only MPEG4 errors. It plays in newer VLC (Ubuntu 14.04) as well as it plays in mplayer or ffmpeg.

Please note that this fix won't affect the data recovery effort (as no data packets have been changed), but it does help if you're trying to get the TS file to play in a video player program.

Offline princess

  • Member
  • Posts: 65
  • Liked: 106
  • Likes Given: 25
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1139 on: 05/29/2014 06:30 pm »
On the subject of recovering the images of the clock in the bottom left corner, I think the TS recovery people might be able to help with that!

To understand why, I need to explain a little about how TS data packets work. Each packet is 188 bytes long, and they have a short header at the start that can define a variable-length adaptation field - basically, it says "reserve space at the start of this packet for extra stuff". Then the actual MPEG4 data comes after the adaptation field.

In the last data packet of a video frame. the encoder will use this feature to ensure that the packet only contains the right amount of data. For example, if there was only 88 bytes of data left to encode in the last packet, the encoder would specify that the last data packet has an adaptation field of 100 bytes long. It then pads the fields with 0xff bytes, and puts the MPEG4 data at the END of the packet. So the packet looks like this:

Header length | ffffffffffff.....ffffffff | MPEG 4 data

Why am I telling you all of this? Well, you can see that if there are bitflips in the "Header length" part, we might not be able to sync up the last piece of MPEG4 data. If the header is too long the MPEG 4 data gets lost, and if it's too short there is a huge run of 0xff bytes which can confuse the decoder mightily.

As the clock is on the last line of macroblocks, it's very susceptible to this kind of corruption. Also, these kind of problems don't register in ffmpeg or VLC as TS errors, so we don't know they're happening.

So what can we do? Well, if we're having problems decoding the last line of macroblocks in an I-frame, it might be possible that it's actually a TS-level problem. We should examine the I-frames where they seem fine but the clock is being stubbornly corrupt, and look to see what the correct TS header length should be. As the packets are only 188 bytes long it should be fairly trivial to brute-force the search and see which makes the best MPEG4 data for the clock.

Tags:
 

Advertisement NovaTech
Advertisement Northrop Grumman
Advertisement
Advertisement Margaritaville Beach Resort South Padre Island
Advertisement Brady Kenniston
Advertisement NextSpaceflight
Advertisement Nathan Barker Photography
1