SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread

[princess]

I would like to wish you a good sleep. I liked your marathon.

Thank you (and deruch!) for your kind words! My plan now is to go through all the fixed TS files we have and see if we're missing any more P-frames. The two timestamps are very regular: the PTS always lags the PCR by exactly 10,000 ticks, and each frame (I-frame or P-frame) is always 6006 ticks after the last one. We can use this to see if there are any "holes" in the timestamp sequencing, which would imply a missing P-frame.

Once we've got every P-frame and set their timestamps correctly, Shanuson should then be able to give the whole lot a final check and hopefully declare the TS files stable. Then we will need the ffmpeg wizards to port their -mmb changes to the new files.

The -mmb lines will need changing because part of the TS recovery work allows us to pull out more of the MPEG4 stream. In some places, TS-level corruption causes part of the valid data stream to be skipped as padding. Once we have the fixed TS files, you'll all have more macroblocks (and more corruption!) to fix.

[Shanuson]

Two new P-frames in part 11!

After tonight's fixing marathon, I decided to go back over my part 11 attempt and I found the two extra P-frames that I was missing before. I've attached the fixed part 11 below. If the mmb / ffmpeg experts could give it a look, I'd be most grateful.

I hope everyone didn't mind me posting lots of results - I've just written some custom tools in C++ that help me find the problems, calculate timestamps and so on. It's made it a lot quicker to fix up the TS problems and find P-frames hiding in the corruption. I'm going to turn in for the night now, so it'll give you all a chance to have a look at all the fixed up things and see what you can make from them!

Hi Princess, thank you for all that work.

i looked into part_11:
The following P-Frame headers are not 100% correct:

Pos - bit distance from the correct header - how the header looks lile - PTC distance in dec
bit distance and PTC distance should be 0.

You identified all correct positions, but correct the complite header it seems. We now what it should look like since all those bits in the header have to be the same for all headers. (except the counter and PTS part - but you got that one correct).

trying to find header: 2
          -> ['47 43 E8 3x 07 10 00 34 73 75 7E 00 00 00 01 E0 00 00 81 80 07 21 01 A3 7F B5 FF FF 00 00 01 B6 5x ']
0x7870 6 -> ['47 43 e8 37 07 10 00 34 73 75 00 00 00 00 01 e0 00 00 81 80 07 21 01 a3 7f b5 ff ff 00 00 01 b6 52 '] 0
0xb330 11 -> ['47 43 e8 3f 07 10 00 34 7f 30 7e 00 00 00 01 e0 00 00 81 80 07 21 01 a3 ae a1 ff ff 00 00 01 b6 53 '] 6006

trying to find header: 13
          -> ['47 43 E8 3x 07 10 00 34 f4 7e 7E 00 00 00 01 E0 00 00 81 80 07 21 01 A7 83 D9 FF FF 00 00 01 B6 5x ']
0x2f98c 11 -> ['47 43 e8 3c 07 1b 00 34 f4 7e 00 00 00 00 01 e0 00 00 81 80 07 20 01 a6 83 d9 ff ff 00 00 01 b6 5a '] 0
0x415c 14 -> ['47 43 e8 34 07 10 00 34 67 ba 7e 00 00 00 01 e0 00 00 81 80 07 21 01 a3 50 c9 ff ff 00 00 01 b6 51 '] -72072

trying to find header: 14
          -> ['47 43 E8 3x 07 10 00 35 00 39 7E 00 00 00 01 E0 00 00 81 80 07 21 01 A7 B2 C5 FF FF 00 00 01 B6 5x ']
0x162b4 13 -> ['47 43 e8 34 07 10 00 34 a2 61 7e 00 00 00 01 e0 00 00 81 80 07 21 01 a5 3b 65 ff ff 00 00 01 b6 55 '] -48048
0x3344c 13 -> ['47 43 e8 34 07 d0 00 35 00 39 00 00 00 00 01 e0 00 00 81 80 0a 21 01 a7 b2 c5 ff ff 00 00 01 b6 68 '] 0

also my CC continuity check failed at pos: 0x2F98C (CC is = C in your file but should be a 4)

and finally there are 56 times where a TS header that starts with 4703e8 continues not with 1X but something else. This is ok for the last TS-packet of a frame when the frame data does not fill the 184 byte completely.
I attached a file where all those 56 parts are listed and marked the correct tails. all the other packets should be 4703e81x.

I hope this helps to improve your tools further.
Btw. I put a part_11.ts file online yesterday or the day before that was fixed. You can use that to compare your version to it.

Thanks again for all the hard work. Even the final edit on your versions will be much faster than starting from raw... So your work is much appreciated anyways.

I will check the rest of your upload tonight. Have to go to work now.

Cheers
Shanuson

[arnezami]

Best to make it explicit with the license Lar suggested earlier:  "can we have it licensed under CC-BY-SA version 4, please" and "WITHOUT the NC or ND clauses attached, please" — and for the reasons he articulated so well.

I think I would like to see the original video licensed CC-BY-SA. The reason is that this would allow Wikipedia to use the final video for educational purposes. Wikimedia has sample letters to use for requesting a free license for content. I have used this to successfully request CC licenses from SpaceX in the past, though they were a smaller company then.

See http://commons.wikimedia.org/wiki/Commons:Email_templates

That's a good idea! Because all this is really educational.

[CuddlyRocket]

Focus should continue to be on the fine work in fixing the video. I'll ensure the rest, but the non official response I got first of all today was "LOL! You're doing us a favor!" <---seriously!

I wonder if Elon will mention this at the new Dragon reveal Thursday night? Or perhaps they'll show the latest version to the crowd whilst they're waiting for the main event of the evening to start?

[Oersted]

Thanks again for all the hard work. Even the final edit on your versions will be much faster than starting from raw... So your work is much appreciated anyways.

It is a normal part of any work process that error checkers comb through the large job of any participant. It is good to see that you guys (and princesses) have informally set up a good-natured, friendly system for improving on each others' work. A new set of eyes always see a few surprises... 

[Chris Bergin]

Focus should continue to be on the fine work in fixing the video. I'll ensure the rest, but the non official response I got first of all today was "LOL! You're doing us a favor!" <---seriously!

I wonder if Elon will mention this at the new Dragon reveal Thursday night? Or perhaps they'll show the latest version to the crowd whilst they're waiting for the main event of the evening to start?

Thursday is going to be more of a Hollywood style event. Some of the rumors I'm hearing are absolutely crazy for this reveal. I don't think a technical effort on an ocean landing video would fit with Robert Downey Jr using robots and lasers to pull a sheet off a new spaceship.

[mhenderson]

Proof of concept, please?

I applaud the "header work" by mvpel, Shanuson, princess, et al. Bravo / Brava, my hat is off to you!

Progress is not always linear and forward. Sometimes it is necessary to step backward to find the best solution.

Will the updated .ts now cause rework of the mmp effort? We have seen remarkable progress and the cumulative result (per the mp4 that SwissCheese posted yesterday) is damn fine. Watching those legs bounce teaches a lot about the turbulent environment at the base of a falling rocket! And yet the vide leaves me begging for more / better. Color consistency, filling in the gaps, wronkiew's clock overlay...

Am I correct by inferring that header fixes - finding new macroblocks, removing errant "ignore this part" flags, and other logical corrections - may change POS values and render manual X: bitflips redundant?

If so, let's get a decision on the direction forward and switch horses. Meanwhile, stop any work on the legacy raw_edit8.ts and point the resources at the new, mobettah .ts.

A proof of concept comparing a few frames from each .ts would go along way to re-energizing an effort on the new stuff.

[SwissCheese]

Hi everyone,

One p-frame looks very strange, with "too intense" colors, could anyone come with an explanation and maybe a solution to this?

It's frame 227 / part 12-19

mmb: 0:0:-1,30:1:10681,10:3:-1,31:3:19630

[saliva_sweet]

Hi everyone,

One p-frame looks very strange, with "too intense" colors, could anyone come with an explanation and maybe a solution to this?

It's frame 227 / part 12-19

mmb: 0:0:-1,30:1:10681,10:3:-1,31:3:19630

I saw that too and wondered. Could it be a corrupted VOP quant in the frame header?

[mvpel]

One p-frame looks very strange, with "too intense" colors, could anyone come with an explanation and maybe a solution to this?

To me, looks too smooth and consistent to me to be a MPEG artifact. It seems that if it were an error in the encoding that it would be more blocky and vary more wildly, but it's a fairly steady sprinkle of red, green, and blue across the frame.

I'm wondering if it might be a burst of noise that affected the electronics of the NTSC camera itself and the RGB sensor, prior to the video signal reaching the digitizer and MPEG encoder.

If that's the case, and someone with more familiarity with video capture and encoding will want to comment on this, then it would seem that the only feasible workaround would be to ignore that pframe.

[princess]

my CC continuity check failed at pos: 0x2F98C (CC is = C in your file but should be a 4)
and finally there are 56 times where a TS header that starts with 4703e8 continues not with 1X but something else. This is ok for the last TS-packet of a frame when the frame data does not fill the 184 byte completely.
I attached a file where all those 56 parts are listed and marked the correct tails. all the other packets should be 4703e81x.
I hope this helps to improve your tools further.

Shanuson, thank you very much for that feedback, I have updated my tool to fix the issues you found, and I now feel it's ready for release.

I've decided to post my "tsrepair" program here for a few reasons - firstly, I feel bad about the amount of data we're uploading to the server here! Also it is hard to see the difference between edits, and it's hard to merge them too. It also makes it hard for us to crowd-source the work.

The tsrepair program accepts a "-fix" command line argument in a similar way as the "-mmb" option in ffmpeg. It allows people to take one of the TS files and apply some changes to it using a simple text string that can be posted here or on the wiki. Example usage is like this:

tsrepair rawsplit_part_9.ts -fix:203,noaf/205,pay,8 rawsplit_part_9_fixed.ts > fix.txt

The first filename is the name of the input file, the second filename is the name of the output TS file. The program produces one line of output per packet, which in the example above is redirected to the file "fix.txt".

The program will take the input file, apply any manual fixes that you specify, and then run some automatic fix passes to do the bulk of the heavy lifting. This can be suppressed with the "-nofix" option. In fact, the way I work is to use the -nofix option to analyse the file first:

tsrepair rawsplit_part_9.ts -nofix > nofix.txt

Then run a repair pass (as detailed above), and show the differences between the two log files using a visual diff program like meld (on Linux) or WinDiff or something:

meld nofix.txt fix.txt

The program is written in C++ and I've developed it in Linux, but it should port easily to Windows for those people that need it.

The fix options that I currently have are:

Part 3:
-fix:114,pid,3e8/114,pay,5/114,noaf/133,noaf/160,pay,2/162,
pcr,5913898/162,ptsauto/162,pes/162,pframe/178,noaf/199,pid,3e8/199,pay,7/199,af,66/
199,nopcr/200,pid,1fff/200,pay,4/243,pid,1fff/243,pay,15/
255,noaf/257,noaf/321,pcr,5925910/321,ptsauto/323,pay,12/349,
noaf/364,pay,11/405,pes/405,ptsauto/405,pframe/
720,pes/720,pframe/815,noaf/945,noaf/947,pay,2/1124,ptsauto/1245,noaf[

Part 4:
-fix:195,noaf/326,pes/326,pcr,6046030/326,pframe/361,noaf/364,noaf/
402,pes/402,pframe/818,noaf/863,noaf/907,noaf/955,pid,1fff/956,pcr,6094078/1275,pes/1275,pcr,6118102/
1316,noaf/1387,noaf/1415,pid,1fff/1415,pay,7/1511,pay,15/1529,noaf/1530,noaf/1535,noaf

Part 5:
-fix:175,pid,3e8/175,pay,11/176,pay,12/311,pay,5/366,pid,3e8/419,noaf/
451,noaf/506,pid,3e8/506,pay,0/506,noaf/508,pay,1/535,af,137/654,noaf/715,ptsauto/
715,pframe/767,af,43/801,noaf/828,noaf/829,noaf/959,noaf/1047
,noaf/1057,noaf/1247,pid,1fff/1247,pay,8/1371,noaf/1434,noaf/1539,noaf

Part 6:
-fix:4,noaf/39,noaf/500,noaf/1042,pay,7/1042,pid,3e8/1043,pay,8/1043,pid,1fff/
1044,pid,3e8/1044,noaf/1044,pay,8/1046,noaf/1272,pay,15/1274
,pay,1/1462,noaf/1505,pid,1fff/1507,pid,3e8/1507,pay,12/1604,noaf/1622,noaf/1644,noaf

Part 7:
-fix:44,noaf/131,pay,7/131,pid,3e8/132,pay,8/351,noaf/419,pay,12/451,pid,1fff/
547,noaf/683,pay,9/741,noaf/759,noaf/789,noaf/826,noaf/955,pid,3e8/955,pay,5/956,pid,1fff/
956,pay,13/978,pid,1fff/978,pay,3/979,pid,3e8/
979,pay,6/979,pes/979,pcr,6460444/979,ptsauto/
979,pframe/1056,pid,1fff/1056,pay,11/1173,noaf/1328,pid,3e8/1328,pay,3/1329,pid,1fff/
1329,pay,1/1403,noaf/1466,noaf/1474,noaf/1493,noaf/1534,pid,1fff/1534,pay,9

Part 8:
-fix:0,pid,1fff/105,pid,03e8/106,pid,1fff/163,pay,10/163,pes/163,
pcr,6514498/163,ptsauto/163,pframe/187,noaf/412,pid,03e8/412,pay,4/412,noaf/601,noaf/816,pid,03e8/
816,pay,8/817,pid,03e8/818,pid,03e8/898,noaf/907,noaf/977,noaf/1158,pay,11

Part 9:
-fix:203,noaf/205,pay,8/227,noaf/371,noaf/400,pes/400,pcr,6652636/400,
ptsauto/400,pframe/478,pid,1fff/576,pay,2/576,noaf/691,pid,1fff/691,noaf/702,pid,1fff/
1115,pes/1115,ptsauto/1115,pframe/1119,noaf/1238,noaf/1285,noaf/1418,noaf

Part 10:
-fix:75,noaf/474,noaf/710,noaf/726,af,7/726,pcr,6796780/726,ptsauto/726,pes/
726,pframe/765,noaf/755,noaf/1037,pframe/1065,pay,6/1077,noaf

Part 11:
-fix:103,noaf/164,pes/164,pcr,6874858/164,pframe/887,noaf/920,noaf/962,noaf
/966,noaf/980,noaf/1037,pay,4/1037,pes/1037,pcr,6940924/1037,ptsauto/1037,pframe/
1065,noaf/1113,pid,1fff/1114,pay,7/1116,pay,9/1117,pes/1117,
pcr,6946930/1117,ptsauto/1117,pframe/1141,noaf/1322,pid,3e8/1324,pay,2/1324,noaf/1515,ptsauto

Part 14:
-fix:718,pes/718,ptsauto/719,pay,11/912,pid,3e8/912,pay,9/913,pid
,3e8/915,pay,12/1214,noaf/1240,pid,3e8/1241,pay,15/1274,pay,0/1322,pay,15/1505,pay,8/1506,pay,9

Part 15:
-fix:22,pid,3e8/27,pay,15/108,pid,3e8/108,pay,14/108,pes/
108,pcr,7349332/108,ptsauto/108,pframe/124,pid,3e8/127,pid,3e8

I'll update the Wiki and if anyone needs me to describe the -fix options I can do that too. Please take a look and let me know what you think. The code is in the public domain, no copyright is asserted and you're free to do what you like with the code.

[Chris Bergin]

We are slowly adding frames to the movie, so I put an updated version. Big thanks in particular to the anonymous people who cleaned part 248!

97 frames (some of them only very partially recovered) out of ~270 were used.

So, based on where we are right this moment, is this still a very good representation of where the progress currently is?

There's a very good reason for asking

[wronkiew]

We are slowly adding frames to the movie, so I put an updated version. Big thanks in particular to the anonymous people who cleaned part 248!

97 frames (some of them only very partially recovered) out of ~270 were used.

So, based on where we are right this moment, is this still a very good representation of where the progress currently is?

There's a very good reason for asking

Yes, I think that is still a good representation of our progress to date. There have been some minor improvements since then, but SwissCheese's video has more polish. I posted my most recent auto-build for comparison.

[Chris Bergin]

PS Sorry Princess. The forum simply can't handle those mega long strings, so I've had to cut them....but I'm sure people can just merge them when using (they are otherwise intact).

[Chris Bergin]

We are slowly adding frames to the movie, so I put an updated version. Big thanks in particular to the anonymous people who cleaned part 248!

97 frames (some of them only very partially recovered) out of ~270 were used.

So, based on where we are right this moment, is this still a very good representation of where the progress currently is?

There's a very good reason for asking

Yes, I think that is still a good representation of our progress to date. There have been some minor improvements since then, but SwissCheese's video has more polish. I posted my most recent auto-build for comparison.

Very good, thanks!

[princess]

PS Sorry Princess. The forum simply can't handle those mega long strings, so I've had to cut them....but I'm sure people can just merge them when using (they are otherwise intact).

No problem Chris, thanks for fixing it up. It looked OK in preview!

[Chris Bergin]

And here's the youtube version of SwissCheese's video, uploaded to a clean account with nothing else in it, etc. This is a requested version (as opposed to the version that is attached as a file on the thread - which is over 4,000 downloads already )

This is easier to share than the link.

[Joffan]

Tweeted this  video out
https://twitter.com/Joffan7/status/471710560006848512

[SwissCheese]

We are slowly adding frames to the movie, so I put an updated version. Big thanks in particular to the anonymous people who cleaned part 248!

97 frames (some of them only very partially recovered) out of ~270 were used.

So, based on where we are right this moment, is this still a very good representation of where the progress currently is?

There's a very good reason for asking

Yes, I think that is still a good representation of our progress to date. There have been some minor improvements since then, but SwissCheese's video has more polish. I posted my most recent auto-build for comparison.

You should set the frame rate manually (14.98 fps) when reconstructing the movie with option -framerate 44999/3003, it seems to be too quick now.

(my command line: ffmpeg -framerate 44999/3003 -i img_spx_%03d.png -c:v mpeg4 -q:v 3 spacex_landing.mp4)

[saliva_sweet]

One p-frame looks very strange, with "too intense" colors, could anyone come with an explanation and maybe a solution to this?

It's frame 227 / part 12-19

mmb: 0:0:-1,30:1:10681,10:3:-1,31:3:19630

I checked the VOP quants in this section and that frame does look like a bit of an outlier. Here are the VOP quants in fixed_edit8_part_209.ts:

1  00100
2  00010
3  00001
4  00001
5  00010
6  00010
7  00010
8  00001
9  00001
10 00000
11 00001
12 00001
13 00001
14 00001
15 00001
16 00001
17 00001
18 00001
19 00111
20 00001

My first guess is that the correct VOP quant is 00001, and there's a double flip, but other values are possible. I attach a ts file where that VOP quant is set to 00001. It's at offset x43A9B so other values can be tried. I can't check it myself as I'm yet to establish a proper setup for multiframe work. Let me know how it looks.