Author Topic: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread  (Read 964322 times)

Offline arnezami

  • Full Member
  • **
  • Posts: 284
  • Liked: 262
  • Likes Given: 346
Over at the reddit thread somebody posted his very useful analysis. I will quote it here:

Quote
Argh. I have some MPEG-2/MPEG-4 experience, and had a go at fixing the file too. Sadly, it's no better than the version SpaceX posted, in terms of useable frames and recognizable objects in the frame. I think there's just too much lost data (and the I-frames made up too much of the missing data) to get much back. But I'd love to be wrong!

The order and detail of my efforts was:

 * Identify discontinuities in the file -- where 0x47 sync bytes jumped offset. Unlike the SpaceX file (which apparently removed data), I added 0xFFs to pad the data to align the next identifiable packet to a 188 byte multiple.

 * Identify the different PIDs in the file. There were the usual suspects: 0x0000 (PAT: Program Allocation Table), 0x0020 (PMT: Program Map Table referenced by the PAT), 0x03e8 (PES video stream, referenced by the PMT), and 0x1fff (padding packets to align the transport stream bit clock with the Program Clock Reference).

 * Identify video PES streams by start code and ID (0x000001e0). I created a template packet that looked like this (hex): 47 43 e8 3_ 07 10 00 __ __ __ __ __ 00 00 01 E0 00 00 81 80 07 I tested each packet against this template, computing the Hamming distance and printing out packets below 23 bits difference. This showed me all the likely PES headers, which I could examine and repair. The five "__" bytes contain the Program Clock Reference value for that point in the transport stream. The PCR advances at 90kHz, with a 300-count fraction/extension that advances at 27MHz. In this file, the PCR advances at multiples of 6006 (decimal) or 66.733 milliseconds. I managed to recover 276 PCR values, and the time stamps span 318 frames (at 6006 PCR counts per frame) or 21.221 seconds. For PCR values that were not multiples of 6006, but were clearly close or off by one or two flipped bits, I flipped them back (hex file editor) to get the interval right.

 * I also normalized the Presentation Time Stamp (PTS) values, which appear to lag the PCR of the packet by exactly 10000 counts (111.111 milliseconds). I used the PCR and PTS intervals to cross-check values (looking for multiples of 6006 in the PCR and a difference of +10000 between the PCR and PTS) and fixed anything that was out of sorts.

 * I identified all transport stream packets as being either PAT, PMT, video, or padding, and coerced bit errors in the header to have the correct sync byte (0x47) and flags (turning off the transport stream error indicator, transport priority, and scrambling bits. I used the continuity counter value to tell whether successive packets might actually be the same PID (usually 0x3e8, video). Based on all this information, I coerced PIDs to the most likely value, and turned any obviously junk packets into padding (null) packets. I fixed up the continuity counter on the packets of each PID so they were incrementing without gaps.

 * I then dug into the MPEG-4 visual object sequence and visual object structures. I saw damage on some of the bits that described the dimensions of the image frame, which were causing VLC and ffmpeg some consternation. So I normalized those.

At this point, I ran out of time. I have a lot of other projects I should be working on. So I share all this in hopes others can build on this, or point out a fatal flaw in my approach and improve on it.

Here's the file I've hand-edited to correct all the PCR and PTS timing issues I could find:

edited.ts

And here's that file run through a script to coerce the transport stream headers into more sensible values (which might or might not actually be an improvement):

coerced.ts

Good luck! And I hope SpaceX gets better weather and position for their RF link on the next launch. Being a software radio nerd, I can appreciate the challenges they face catching several megabits of data from an object they don't want to be too close to when it comes down.

    - Jared

Offline michaelni

  • Member
  • Posts: 28
  • Liked: 23
  • Likes Given: 0
here are two videos which where transcoded with a modified FFmpeg.
The modifications force the decoder to continue decoding frames even once errors where detected, this should show more parts of the frames at the expense of also significnatly more random damaged and ugly looking blocks.
Also motion compensation and error concealment where disabled, the way neither correct nor incorrect parts of previous frames would leak in the current frame.
Overall the video certainly is less pleasing to watch that way but it should contain more information for analysis and be easier to make sense of if someone wants to analize it frame by frame

The first is based on raw.ts, the second is based on the coerced.ts file from Jarred which was posted here
https://ffmpeg.org/~michael/space-x-video/raw.ts-nomc-ignoreerr.mkv
and
https://ffmpeg.org/~michael/space-x-video/Jared-coerced.ts-nomc-ignoreerr.mkv

Offline michaelni

  • Member
  • Posts: 28
  • Liked: 23
  • Likes Given: 0
and here are teh 3 videos, raw.ts and the 2 from jared decoded with ffmpeg with error concealment tuned a bit (that is favoring predicting from the previous frame with MV 0,0), which might look slightly more pleasing than the original, but shouldnt contain more information

https://ffmpeg.org/~michael/space-x-video/raw.ts-tunedec.mkv
https://ffmpeg.org/~michael/space-x-video/Jared-edited.ts-tunedec.mkv
https://ffmpeg.org/~michael/space-x-video/Jared-coerced.ts-tunedec.mkv

Offline arnezami

  • Full Member
  • **
  • Posts: 284
  • Liked: 262
  • Likes Given: 346
here are two videos which where transcoded with a modified FFmpeg.
The modifications force the decoder to continue decoding frames even once errors where detected, this should show more parts of the frames at the expense of also significnatly more random damaged and ugly looking blocks.
Also motion compensation and error concealment where disabled, the way neither correct nor incorrect parts of previous frames would leak in the current frame.
Overall the video certainly is less pleasing to watch that way but it should contain more information for analysis and be easier to make sense of if someone wants to analize it frame by frame

The first is based on raw.ts, the second is based on the coerced.ts file from Jarred which was posted here
https://ffmpeg.org/~michael/space-x-video/raw.ts-nomc-ignoreerr.mkv
and
https://ffmpeg.org/~michael/space-x-video/Jared-coerced.ts-nomc-ignoreerr.mkv

Wooow! I didn't realize this at first. But we have Michael Niedermayer in the house!  8)

One of the creators of ffmpeg itself. Gulp.

Hope we can work together to figure this out. I like your approach though. Currently I'm focusing on the I-frames and altering the decoder to give me more information about what is wrong with certain (macro)blocks. But I've seen the decoder source code for the first time today, while you may know it by heart.

Ok guys. I think we're in good hands now ;)

Regards,

arnezami

Offline Chris Bergin

Interesting work guys! Keep up the good work!

Offline Helodriver

  • Full Member
  • ****
  • Posts: 1000
  • Liked: 5609
  • Likes Given: 585
From that cleanup it appears the video definitely ends before the engine quits and definitely before the vehicle begins to topple. Still no definitive answer as to what the "8 seconds" entailed. Nice work on the video.

Offline Marams

  • Member
  • Posts: 4
  • Liked: 2
  • Likes Given: 0
here are two videos which where transcoded with a modified FFmpeg.
The modifications force the decoder to continue decoding frames even once errors where detected, this should show more parts of the frames at the expense of also significnatly more random damaged and ugly looking blocks.
Also motion compensation and error concealment where disabled, the way neither correct nor incorrect parts of previous frames would leak in the current frame.
Overall the video certainly is less pleasing to watch that way but it should contain more information for analysis and be easier to make sense of if someone wants to analize it frame by frame

The first is based on raw.ts, the second is based on the coerced.ts file from Jarred which was posted here
https://ffmpeg.org/~michael/space-x-video/raw.ts-nomc-ignoreerr.mkv
and
https://ffmpeg.org/~michael/space-x-video/Jared-coerced.ts-nomc-ignoreerr.mkv

Nice work. The following frames from the raw.ts where quite interesting because they clearly show the time stamp in the lower left corner. There are a lot frames with clearly visible last numbers of time stamps.

MECO was around 19:28:19
landing video started at 19:35:03 - 1s or so
landing video ended at 19:35:19

In every frame there is 19:35 in the lower left corner with the exact same pixels. For crypto-analysis a known plain text would help for sure :)

Offline michaelni

  • Member
  • Posts: 28
  • Liked: 23
  • Likes Given: 0
Hope we can work together to figure this out. I like your approach though. Currently I'm focusing on the I-frames and altering the decoder to give me more information about what is wrong with certain (macro)blocks. But I've seen the decoder source code for the first time today, while you may know it by heart.

If we assume that a frame has been damaged by a flipped bit, then in general the decoder will not detect it when reading that bit but rather will either show artifacts and continue without a detected error or more likely get desynced and detect some kind or inconsistency like a out of range value or incorrect marker bit at a later point. For example if theres an error detected in MB at location XxY the actual bit error that caused it could be a few macroblocks before it,
MBs are 16x16 pixel in mpeg4 and are stored in raster scan order left to right and top to bottom.
mpeg4 (A)SP uses variable length coding (huffman coding with a few unused codes), so bit errors can throw things out of sync (vlc not matching up between encoder and decoder or symbols not matching up, that is for example a MB type could end up being decoded as a motion vector after a bit error). If detected inconsistencies are ignored then the decoder can end up falling into sync again by chance and continue to decode the bitstream somewhat reasonably. After such resync various values like motion vectors, quantization parameters or values used for AC/DC prediction could/would be wrong though causing some artifacts but could/should still contain some human recognizeable features of the original.

I suspect if the number of flipped bits per frame is small enough and nothing is missing of a frame, someone with alot of time could find the bits by trial and error and by starting from the location in raster scan order where artifacts are first vissible. error concealment would have to be disabled in the used decoder though so that one can see where the artifacts start.
The error concealment code in ffmpeg by default conceals several MBs prior to a detected error.
Error concealment can be disabled with -ec 0


Offline arnezami

  • Full Member
  • **
  • Posts: 284
  • Liked: 262
  • Likes Given: 346
Nice work. The following frames from the raw.ts where quite interesting because they clearly show the time stamp in the lower left corner. There are a lot frames with clearly visible last numbers of time stamps.

MECO was around 19:28:19
landing video started at 19:35:03 - 1s or so
landing video ended at 19:35:19

In every frame there is 19:35 in the lower left corner with the exact same pixels. For crypto-analysis a known plain text would help for sure :)

Great find!!

Offline Michael Bloxham

  • Regular
  • Full Member
  • ****
  • Posts: 639
  • Auckland, New Zealand
  • Liked: 6
  • Likes Given: 6
Looks like the stage is still going reasonably fast when it hits the water, and plunges deep into the water?

Offline mmeijeri

  • Senior Member
  • *****
  • Posts: 7457
  • Martijn Meijering
  • NL
  • Liked: 76
  • Likes Given: 185
We will be vic-toooooo-ri-ous!!!

Offline YetAnotherLurker

  • Member
  • Posts: 11
  • Liked: 1
  • Likes Given: 7
Won't play for me with either Windows Media Player or DivX. What software can I use to play it?

VLC Media Player

Offline mlindner

  • Software Engineer
  • Senior Member
  • *****
  • Posts: 2090
  • Space Capitalist
  • Silicon Valley, CA -- previously in Ann Arbor, MI
  • Liked: 748
  • Likes Given: 290
Internal combustion engine in space. It's just a Bad Idea.TM - Robotbeat

Offline mheney

  • The Next Man on the Moon
  • Global Moderator
  • Full Member
  • *****
  • Posts: 779
  • Silver Spring, MD
  • Liked: 378
  • Likes Given: 200
Xine on a linux box works as well.  I believe you can download a version of xine for Windows ...

Offline MTom

  • Full Member
  • ***
  • Posts: 378
  • EU / Hungary
  • Liked: 107
  • Likes Given: 435
Nice work. The following frames from the raw.ts where quite interesting because they clearly show the time stamp in the lower left corner. There are a lot frames with clearly visible last numbers of time stamps.

MECO was around 19:28:19
landing video started at 19:35:03 - 1s or so
landing video ended at 19:35:19

In every frame there is 19:35 in the lower left corner with the exact same pixels. For crypto-analysis a known plain text would help for sure :)

Great find!!

And this is placed on the bottom of the frames.
This sounds me (without any knowledge of video) as a good point: could it be a good reference point for the correction? (The top of the frames have a good quality on the raw video, after identifiing the bottom lines could it be like a "picture frame" for the work?)
« Last Edit: 05/02/2014 10:48 PM by MTom »

Offline Jcc

  • Full Member
  • ****
  • Posts: 895
  • Liked: 210
  • Likes Given: 108
Xine on a linux box works as well.  I believe you can download a version of xine for Windows ...

I just downloaded the latest RealPlayer update "RealPlayer Cloud" on win7, and it plays mkv files.

Offline AS-503

  • Full Member
  • ****
  • Posts: 428
  • Orion Fab Team
  • Colorado USA
  • Liked: 160
  • Likes Given: 114
All Windows based OS can benefit greatly from installing a complete CODEC package like K-Lite (mega). This will not only allow playback of virtually everything, but also enable thumbnails and super-charge Windows Media Player to playback almost all video and audio file extensions.

Offline MarsInMyLifetime

  • Full Member
  • **
  • Posts: 278
  • Austin, TX
    • Light of Day
  • Liked: 132
  • Likes Given: 137
FWIW, I was able to watch these mkv files using J River Media Center which seems to have (or get) the right codec.
Don

Offline IslandPlaya

  • Full Member
  • ****
  • Posts: 582
  • Outer Hebrides
  • Liked: 163
  • Likes Given: 166
Just download VLC player. It saves downloading all them dodgy codec packs.
If VLC can't play it then forget it...

Offline MP99

Just download VLC player. It saves downloading all them dodgy codec packs.
If VLC can't play it then forget it...

Concur.

Cheers, Martin

Tags: