Author Topic: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread  (Read 916411 times)

Offline wronkiew

  • Full Member
  • *
  • Posts: 186
  • 34.502327, -116.971697
  • Liked: 105
  • Likes Given: 121
I am not a video specialist but looking at these timestamps there is ~1.3 seconds difference between each I frame so most likely there should be no I frame between frame 2 and 4.

Gah! You're right. It's too early in the morning for such advanced math.


 2   53 19:35:04.9
-3   does not exist
 4   73 19:35:06.3
 5   93 19:35:07.6
 6  113 19:35:08.9
-7a 13? 19:35:10.2
-7b 150 19:35:11.5
 8  169 19:35:12.9

Offline AncientU

  • Senior Member
  • *****
  • Posts: 5844
  • Liked: 3666
  • Likes Given: 5107
I am not a video specialist but looking at these timestamps there is ~1.3 seconds difference between each I frame so most likely there should be no I frame between frame 2 and 4.

Gah! You're right. It's too early in the morning for such advanced math.


 2   53 19:35:04.9
-3   does not exist
 4   73 19:35:06.3
 5   93 19:35:07.6
 6  113 19:35:08.9
-7a 13? 19:35:10.2
-7b 150 19:35:11.5
 8  169 19:35:12.9

Would be consistent number-sequence wise if the first column was incremented by 20 (33, 53, 73, 93, 113, 133-7a, 153-7b, etc.) and the seconds incremented by 1.3.  This drops i frame 3 as suggested by many and also fixes7a/b.  There are also 20-ish gaps in the higher frames as well as 1.3-1.4s increments.  Somewhere in the 150-170 range, an extra DC offset  of about 6 was added.
« Last Edit: 05/09/2014 06:09 PM by AncientU »
"If we shared everything [we are working on] people would think we are insane!"
-- SpaceX friend of mlindner

Offline theshadow27

  • Member
  • Posts: 28
  • Liked: 27
  • Likes Given: 7
Ok, first try at auto-marking bad MBs based on ffmpeg log output and primitive analysis.

Blocks which were reported as errors by ffmpeg (modified, with debug on) are marked black. Internally, this tracks:
  --  the -1 and -2 override blocks from MBM,
  -- dc clipped at
  -- ac-tex at
  -- dquant at
  -- stuffing at
  -- Error at MB
although these are all painted black as the colors were confusing.

Blocks that were NOT reported as errors but probably are bad are translucent red.
   -- currently tests for the block having less than 5 individual colors.



Next step is to begin (hopefully) intelligent mutation of the MBM string to achieve a better overall score. Sample image is of frame 4 with saliva_sweet MBMs (from wiki).


Edit: Here's the Java source if anyone else wants to play. http://pastebin.com/bKgzCRqD The mammoth inner class at the bottom is for writing the animated gif :)
« Last Edit: 05/09/2014 06:20 PM by theshadow27 »

Offline Swatch

  • Full Member
  • **
  • Posts: 273
  • Official Aerospace Engineer as of June 13th, 2009
  • Cincinnati
    • ProjectApollo/NASSP: Virtual Systems and Flight Simulation of the Apollo Program
  • Liked: 28
  • Likes Given: 16
There also appears to be, on average, 20 frames between each I-frame.  Like xcg001, I'm no video specialist, so I'm not sure if that should stay relatively consistant or not.

Edit:  What AncientU said...
« Last Edit: 05/09/2014 06:15 PM by Swatch »
Ex-Rocket Scientist in Training, now Rocket Scientist!
M-F trying to make the world of the future a smaller place through expanding horizons...

Offline wronkiew

  • Full Member
  • *
  • Posts: 186
  • 34.502327, -116.971697
  • Liked: 105
  • Likes Given: 121
Here's what I have so far. It's about 2-3 seconds of video almost up to splashdown:

[youtube]<object width="425" height="350"><param name="movie" value="http://www.youtube.com/v/CxheyOUAlG8"></param><param name="wmode" value="transparent"></param><embed src="http://www.youtube.com/v/CxheyOUAlG8" type="application/x-shockwave-flash" wmode="transparent" width="425" height="350"></embed></object>[/youtube]

This looks good. Was this only adding the improved iframes to the video as described earlier?

I took raw.ts-nomc-ignoreerr.mkv from way back at the beginning of this thread and broke it up in to individual frames. With motion turned off, what is left is block updates and something approximating a delta between the current frame and nearby i- and p-frames. A fraction of the intermediate frames are useable with a bit of shifting around. I added in the fixed up i-frames and cut out the bad blocks. I then went sequentially through the frames adding deltas and alpha-blending the block updates.

I think the video could be vastly improved with some more tweaking, more intermediate frames, and a longer run time. However, editing the frames is obviously rather time consuming. If anyone is interested in helping out, I can post my project files to the wiki.
« Last Edit: 05/09/2014 06:23 PM by wronkiew »

Offline moralec

There also appears to be, on average, 20 frames between each I-frame.  Like xcg001, I'm no video specialist, so I'm not sure if that should stay relatively consistant or not.

Edit:  What AncientU said...

A bit off topic, but I updated the Iframe sequence that was on the wiki, to match this speed.
« Last Edit: 05/09/2014 07:09 PM by moralec »

Offline Y Combinator

  • Member
  • Posts: 5
  • Liked: 1
  • Likes Given: 1
There also appears to be, on average, 20 frames between each I-frame.  Like xcg001, I'm no video specialist, so I'm not sure if that should stay relatively consistant or not.

Edit:  What AncientU said...

A bit off topic, but I updated the Iframe sequence that was on the wiki, to match this speed.

I believe this is 15 FPS video; it looks like your animated GIF is assuming 30 FPS and as such is too fast.  Check the timestamps against a stopwatch.

Offline moralec

There also appears to be, on average, 20 frames between each I-frame.  Like xcg001, I'm no video specialist, so I'm not sure if that should stay relatively consistant or not.

Edit:  What AncientU said...

A bit off topic, but I updated the Iframe sequence that was on the wiki, to match this speed.

I believe this is 15 FPS video; it looks like your animated GIF is assuming 30 FPS and as such is too fast.  Check the timestamps against a stopwatch.

Yep, you are right. I updated it to a new version.

Offline Swatch

  • Full Member
  • **
  • Posts: 273
  • Official Aerospace Engineer as of June 13th, 2009
  • Cincinnati
    • ProjectApollo/NASSP: Virtual Systems and Flight Simulation of the Apollo Program
  • Liked: 28
  • Likes Given: 16
Dang... Frame 4 is tantalizing because of the leg deploy.

Also, that pace feels more accurate moralec.
Ex-Rocket Scientist in Training, now Rocket Scientist!
M-F trying to make the world of the future a smaller place through expanding horizons...

Offline AncientU

  • Senior Member
  • *****
  • Posts: 5844
  • Liked: 3666
  • Likes Given: 5107
There also appears to be, on average, 20 frames between each I-frame.  Like xcg001, I'm no video specialist, so I'm not sure if that should stay relatively consistant or not.

Edit:  What AncientU said...

A bit off topic, but I updated the Iframe sequence that was on the wiki, to match this speed.

I believe this is 15 FPS video; it looks like your animated GIF is assuming 30 FPS and as such is too fast.  Check the timestamps against a stopwatch.

Yep, you are right. I updated it to a new version.
Might also want to drop i frame 3.  Leg deployment starts just after 2 and is about a quarter of the way done by frame 4, 1.3s later. Engine exhaust hitting sea surface happens between 8 and 9.  Splashdown at around 10. "Settled" vertically in sea at 13.  Not sure when engine start happens/happened.  Leg deployment was supposed to be half way through a 20ish second burn IIRC -- leg deployment to splash (about 2 -10) is 7 or 8 x 1.3s = 9.1s-10.4s which checks.

Ed: Note -- 3 could be already dropped
« Last Edit: 05/09/2014 07:45 PM by AncientU »
"If we shared everything [we are working on] people would think we are insane!"
-- SpaceX friend of mlindner

Offline mlindner

  • Software Engineer
  • Full Member
  • ****
  • Posts: 1929
  • Space Capitalist
  • Silicon Valley, CA -- previously in Ann Arbor, MI
  • Liked: 526
  • Likes Given: 210
My 2 cents on frame 3. Because of how garbled it is, its possible that the corruption is not in the data itself but in the frame header. The iframe header has lots of specifications that define how even numbers are defined (whether to use reversible or non-reversible codings). That would cause the entire frame to be decoded wrong.

Edit: Just checked, yeah the frame header is corrupted. If we fix it, large parts of the image may pop out. Other frames may also have corrupted headers but not bad enough to block header decoding. This would result in odd settings for the image still though.

I've been doing some inspection of the transport stream and iframe3 in a hex editor. iframe3 bears no resemblance to any other iframe in its data content. I don't think its actually an iframe or even a fame. I think the decoder got confused and tried to extract the data as an iframe. There are indeed 13 iframes though, we're just not extracting the first one for whatever reason.

Also the Postion in the Video suggest that interpretation:
iFrame2 at 1146612
iFrame3 (which probably is no iFrame) at 1207712
iFrame 4 at 1446284

all other iFrames are also separeted by around 300000 bits.

Exceptions: between IF 6 and 7 it is twice as much, Are we sure there is no Iframe between these 2?
Also IF1 is at 847880. Could this mean there are 3 more iframes before that? 85/30 is ~ 3.

Cheers,
Shanuson

I do not know for the beginning, but for iframe 6 the blocks are "heavier"  (more bits)  than in the other iframes. The file is roughly twice larger than the other ones.

I have the feeling that there is at least two additional iframes before IF1. In the original video there is a section of the rocket surrounded by clouds, with no sea visible.

Further investigation.

I think there is only a single iframe not being decoded. I found all the relevant frame headers and everything at 0x03E3EC position in the file.

0x000001B0 = visual_object_sequence_start_code
0x03       = profile and level indication
0x000001B5 = visual_object_start_code

There are exactly 13 of these patterns in this sequence in the transport stream. I did some copy and paste searching of the starts of the various iframe file dumps and each one lines up to one of these code sets. Importantly, we entirely skip the first set of these codes. The first iframe is the second set of these. I'm trying to figure out why its not decoding, but not there yet.

We also need to somehow disable iframe3 from being used as that will screw up any video that could possibly be correct that would otherwise reference iframe2.

I think the first iframe is a lost cause. Where the user data section should start there is instead a "video_session_error_code" which according to the spec: "The video_session_error_code has been allocated for use by a media interface to indicate where uncorrectable errors have been detected."

Even replacing the code with the user data start code and the data itself with another frame's user data, the output is garbage. The decoder barfs everywhere and can't even get to decoding macroblocks.
Internal combustion engine in space. It's just a Bad Idea.TM - Robotbeat

Offline Digitalchromakey

  • Member
  • Posts: 23
  • Phuket
  • Liked: 5
  • Likes Given: 2
I-Frame 6

Looks like the RHS Leg is being deployed with the LHS leg already locked in position.

Needs a lot more work to be more certain.

0:0:583,43:0:5530,
00:01:6129,02:01:6129,21:1:8584,22:1:8796,
4:3:12577,14:3:19199,15:03:-1,18:03:19433,20:3:14083,23:3:14220,42:3:21474,
0:04:-1,31:04:24629,32:4:15280,
6:5:17702,7:5:-1,13:05:27662,14:05:-1,42:0:5530,17:05:28599,18:5:13888,24:05:-1,25:05:29375,
28:7:40520,29:7:40520,32:07:-1,37:07:41235,
1:8:37105,
1:9:37105,
26:10:61670,27:10:40399,
17:11:64694,
0:13:56967'
0:16:71551,1:10:11678,
1:11:11678,
5:12:12185,18:12:58656,
07:13:-1,13:13:60221,14:13:13373,15:13:60221,
19:14:59308,
12:15:72126,
0:17:71551,2:17:71610,
0:18:71551,7:18:72226,
0:19:121762,
1:28:202509




« Last Edit: 05/09/2014 08:05 PM by Digitalchromakey »

Offline SwissCheese

  • Full Member
  • *
  • Posts: 164
  • Liked: 249
  • Likes Given: 90
So I tried to get something with iframe 4, but still a lot to do. I have to stop now for a few days so good luck to those who wish to improve it ;)

From try1.ts

5:1:4200,00:03:-1,
02:03:10968,10:03:-1,
17:06:22427,14:11:-1,
38:16:65876,19:17:-1,
00:19:78514,28:20:-1,
0:28:129509

At 17911 and 55810 there are also valid sequences, which I do not know exactly where to put, and I did not really try to improve the bottom of the frame.

Also some sequences from http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195103#msg1195103 could help if put at the accurate place.
« Last Edit: 05/09/2014 08:27 PM by SwissCheese »

Offline cscott

  • Senior Member
  • *****
  • Posts: 2848
  • Liked: 1971
  • Likes Given: 664
I think the first iframe is a lost cause.

Perhaps you could turn your attention to the missing iframe "7a"?  Maybe there's a bit flipped somewhere which is causing your search for 0x000001B0 / 0x03 / 0x000001B5 not to turn it up?  (I'm assuming the existing iframe 7 is "7b" based on the timestamp image, someone correct me if i'm wrong.)

Offline theshadow27

  • Member
  • Posts: 28
  • Liked: 27
  • Likes Given: 7
Have the brute-force code running on frame 4 checking 1-bit modifications, with a threshold of eliminating 4 or more erroneous blocks (starting with 00:10:-1,00:14:-1,00:17:71044,00:24:-1,00:27:114198,08:15:61126,19:05
:20944,21:13:50333,25:14:57756,35:05:-1,36:15:-1,43:06:27582). It is running pretty slowly now, about 77ms per bit (and there are a lot of bits!) but it has produced some interesting results (31 solutions and counting for a 1-bit flip).

I'm not very good at playing with the numbers manually, perhaps someone else wants to see if any of these can be improved? These are the best so far:

00:10:-1,00:14:-1,00:17:71044,00:24:-1,00:27:114198,08:
15:61126,19:05:20944,21:13:50333,25:14:57756,35:05:-1,36:15:-1,43:06:27582,X:10838:32

00:10:-1,00:14:-1,00:17:71044,00:24:-1,00:27:114198,08:
15:61126,19:05:20944,21:13:50333,25:14:57756,35:05:-1,36:15:-1,43:06:27582,X:9308:16

00:10:-1,00:14:-1,00:17:71044,00:24:-1,00:27:114198,08:
15:61126,19:05:20944,21:13:50333,25:14:57756,35:05:-1,36:15:-1,43:06:27582,X:9278:128

00:10:-1,00:14:-1,00:17:71044,00:24:-1,00:27:114198,08:
15:61126,19:05:20944,21:13:50333,25:14:57756,35:05:-1,36:15:-1,43:06:27582,X:4548:32

00:10:-1,00:14:-1,00:17:71044,00:24:-1,00:27:114198,08:
15:61126,19:05:20944,21:13:50333,25:14:57756,35:05:-1,36:15:-1,43:06:27582,X:4416:32
« Last Edit: 05/09/2014 08:33 PM by Chris Bergin »

Offline Swatch

  • Full Member
  • **
  • Posts: 273
  • Official Aerospace Engineer as of June 13th, 2009
  • Cincinnati
    • ProjectApollo/NASSP: Virtual Systems and Flight Simulation of the Apollo Program
  • Liked: 28
  • Likes Given: 16
I think the first iframe is a lost cause.

Perhaps you could turn your attention to the missing iframe "7a"?  Maybe there's a bit flipped somewhere which is causing your search for 0x000001B0 / 0x03 / 0x000001B5 not to turn it up?  (I'm assuming the existing iframe 7 is "7b" based on the timestamp image, someone correct me if i'm wrong.)

Not sure how long it would take to brute-force through the file in the area of concern (from pos 2054276 to 2940884), performing a bit-flip at each bit along the way and performing a search for 0x000001B0 / 0x03 / 0x000001B5 within the bits +/- the total message length.  If you don't see anything, flip the bit back and move onto the next bit.  When you see a new hit for 0x000001B0 / 0x03 / 0x000001B5, record the file location and then go back later to see if maybe a new iframe materializes.

Frustrating because I know I used to be able to do all I said above, but I've been away from coding for too long to do it in any expedient manner.
« Last Edit: 05/09/2014 08:19 PM by Swatch »
Ex-Rocket Scientist in Training, now Rocket Scientist!
M-F trying to make the world of the future a smaller place through expanding horizons...

Offline theshadow27

  • Member
  • Posts: 28
  • Liked: 27
  • Likes Given: 7
Not sure how long it would take to brute-force through the file in the area of concern, performing a bit-flip at each bit along the way and performing a search for 0x000001B0 / 0x03 / 0x000001B5 within the bits +/- the total message length.  If you don't see anything, flip the bit back and move onto the next bit.  When you see a new hit for 0x000001B0 / 0x03 / 0x000001B5, record the file location and then go back later to see if maybe a new iframe materializes.

Frustrating because I know I used to be able to do all I said above, but I've been away from coding for too long to do it in any expedient manner.

The brute-force I just posted is for frame 4, not the mysterious missing frame. I'm just flipping bits in the MBs that ffmpeg identified.

Offline Oersted

  • Member
  • Full Member
  • ****
  • Posts: 847
  • Liked: 447
  • Likes Given: 264
How much intimate knowledge of video compression and codecs and whatnot would have never been discovered if it wasn't for this garbled video! :-)

Online Chris Bergin

Careful with the long number strings folks. So long they break the site width. A split point half way works.

Offline michaelni

  • Member
  • Posts: 28
  • Liked: 23
  • Likes Given: 0
Third, there when a block from a static area (faring, clock, smudge) is out of place but parsed correctly, it should have a high correlation with the same block in a "known good" frame using an appropriate fourier transform. Since there are relatively few blocks (600) a complete distance matrix could be constructed with 380k convolutions (or half that with a good mask and bad-block detection). Main problem here is determining the fastest/most accurate transform for block difference. Maybe the ffmpeg guys have some advice here?

Comparing blocks this way can be a bit tricky, as the DC values could be wrong as they depend on previous blocks and one row or column of AC values could be wrong when ac prediction is used. Also if the QP is wrong the block could be dequantized with a wrong scale factor. Also the blocks could be subjected to cliping when they are too bright or dark. The cliping issue could be avoided by comparing the coefficients before the IDCT.
If one did some block brute force comparisson, one could for each pair of blocks to compare rescale one to compensate for a possible QP error, adjust its DC value and if it uses ac prediction maybe use the block above or to he left if its static too to remove its influence, whats left could be compared by some means like sum of squared differences or sum of absolute differences of the coefficients.

Tags: