Author Topic: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread  (Read 954613 times)

Offline meadows.st

  • Full Member
  • *
  • Posts: 152
  • Vancouver BC, Canada
  • Liked: 88
  • Likes Given: 3810
Hi guys,

Since this is the subject I've been spending a lot of time on thinking I will react too. But I havent got a lot of time right now. So I will probably be more elaborate later.

What this whole problem boils down to is three phases (per frame):

1) Extracting: Correcting the predictable streamheaders and I/P-frame headers. This way we (or actually the decoder) can extract all the video data without missing packets.

2) Positioning: Finding all the macroblock-data (that is: finding all bitpositions where each mb-data starts) and then assigning the right macroblocks to the these starting bitpositions.

3) Repairing: Correcting left-over DC-values errors either by bit flipping or by changing the DC-values after decoding. This should implcitly also fix "bad inheritance": where a macro inherents its DC base value from the wrong macroblock (that is: left or top).

I believe that for 2 there is an algorithm that will find all likely correct starting positions of macroblocks, but it will not be able correctly assign macroblocks to these bitpositions. I believe it would be helpful if we auto-generate all the likely starting positions for each frame and make these available as a static file to the online editor. This would allow the user to choose from good starting positions.

As for 3, I think there may be a way to correct DC-values somewhat automatically. However bitflipping is a better way when possible of course. Massively correcting DC-values should only be done if 1 and 2 has been done for a frame. If this works as I hope this will really give it a "finished look".

There is also the issue with the MV's for P-frames. I haven't looked into this too much.

I will go into more detail when I have more time ;).

Regards,

arnezami

You said it much better than I did. I concur, fix the data 'mechanically' first for the best starting state, then devise algorithms to narrow down the probable solution space.  I am eager to read your more detailed thoughts.
A little rudder far from the rocks is a lot better than a lot of rudder close to the rocks. L. David Marquet

Offline mvpel

  • Full Member
  • ****
  • Posts: 1124
  • New Hampshire
  • Liked: 1295
  • Likes Given: 1686
I was thinking about the movie, could someone with some artistic talent (not my case :P) make 1-2 title images, and maybe also some credits images at the end (for example a few before/after images, printscreens of the wiki, web application, ...)? It should have the same format as the images, i.e. 704x480.

I nominate for the soundtrack. And definitely . Start with a few flashed images of the Wright Brothers and Mercury-Redstone, Gemini, Saturn I... some sexy Falcon 9 rollout and countdown shots, the launch, a loving slo-mo of the huge splash of water that gifted us the immensely useful spots on the camera lens and body... then some retro burn video or animations to set the context, and then the landing... ;D It'll be sweet.
"Ugly programs are like ugly suspension bridges: they're much more liable to collapse than pretty ones, because the way humans (especially engineer-humans) perceive beauty is intimately related to our ability to process and understand complexity. A language that makes it hard to write elegant code makes it hard to write good code." - Eric S. Raymond

Offline inverted_strawberry

  • Member
  • Posts: 1
  • Liked: 2
  • Likes Given: 0
I posted this on reddit (http://www.reddit.com/r/spacex/comments/24bsn2/first_stage_landing_video/) and Destructor1701 suggested to re-post this here. I don't think I will be checking back here a lot so if you have any questions then please get in touch over reddit.

So I may be late to the party but I gave it a serious try. I managed to recover a good number of p-frames, and a cleaned up a few i-frames better than in the original repair1.ts file. Here we go: extracted i-frames are here: https://imgur.com/a/Fu5by
As you see I was unable to recover most of the earlier frames, so I tried grafting i-frame 6 on top of 3,4, and 5. This means the content of this video may not be what the camera actually recorded but could help in making sense of the motion (p-frames).
Edit: Here's the .ts in case anyone is interested: https://www.dropbox.com/s/37h85hxxqlgv79z/map1.ts

Offline mhenderson

  • Member
  • Posts: 69
  • USA
  • Liked: 101
  • Likes Given: 18
scriptmonkey420 on Reddit has posted SwissCheese's latest gif dissected to individual frames here:
http://imgur.com/a/Xv8ln#0

Really cool.

Images 40-43 appear to be out of sequence. I think the order should be 41, 40, 43, 42 (These correspond to frames 206 - 211 with gap). Were they in this order in the GIF?

Offline saliva_sweet

  • Full Member
  • ****
  • Posts: 555
  • Liked: 424
  • Likes Given: 1345
Hi all
ive added a new option to the custom ffmpeg on github,
-debug mb_pos_bruteforce

Could you make a new /github.com/mlindner/ffmpeg-spacex/archive/spacexdebug1.tar.gz with these changes. I don't have much hope for this, but frame 72 is causing desperation to set in. I'm grabbing at straws here. Thinking of going down to sub-macroblock level.

Offline michaelni

  • Member
  • Posts: 28
  • Liked: 23
  • Likes Given: 0
I think what would also be very useful is to able to tell ffmpeg only to decode either luminosity, chrom1 or chrom2. So either only the first 4 dc's are decoded or only the fifth or only the sixth.

Then we can add 3 checkboxes in the online version. That way you can much more clearly see where things go wrong colorwise or intensity wise. Without our eyes being districated by either the intensity or (one of) the two colors.

you can achive that by removing chroma or luma after decoding using for example:
-vf mp=eq2=1:0:0:1
-vf mp=eq2=1:1:0:0
or
-vf lutyuv=y=128:u=128
-vf lutyuv=y=128:v=128
-vf lutyuv=v=128:u=128

Offline morningdew76

  • Member
  • Posts: 11
  • McLean, VA
  • Liked: 12
  • Likes Given: 7
I've been attempting to tune this bash script, with an eye towards performance.

This script is searching for two bits between 8800 and 9065 on frame 52.  It's a lot of combinations, and I am getting roughly 600 frames per minute.

Quote
for i in {8800..9064} ; do for j in {8801..9065} ; do ffmpeg.exe -debug mb_pos_size -s:0 704:480 -mmb X:76768:80,X:22120:80,X:45038:80,X:42196:80,X:66234:80,X:50298:80,X:$i:80,X:$j:80, 0:7:-2:-10:-10:-10:-10:8:-5,2:7:15506,9:7:-1,15:7:16391,17:7:-1,28:7:16704, 35:7:-2:-10:-10:-10:-10:8:-5,29:9:21074,34:9:-1,39:9:21626,5:28:-1::63,10:28:80196 -i iframe52.mpg4-img 2> output-$i-$j.txt -f image2 /dev/null -y ; done ; done

The output images have been entirely disabled, and the directive to ignore errors has been removed.  I am not 100% sure that the latter is "safe" but it does significantly increase the speed of the process.

Note that there are spaces in the mmb string that would need to be removed to run this.

After the run is done, I first do a grep for 42:03:9061 , then a grep for 00:29:81738 .  The second one is just in case the first "known good" position ends up changing due to correctly flipped bits.  Edit: This second test is probably worthless.

If anybody has suggestions for how to speed this up, I am all ears!  The search space can likely be reduced but I'm really unsure of what would be advisable.

Just sticking to two bits right now... going for three in a search space this size would take something like 21 days at a rate of 600 tries per minute.
« Last Edit: 05/23/2014 07:30 PM by morningdew76 »

Offline maximlevitsky

  • Member
  • Posts: 85
  • Liked: 86
  • Likes Given: 103
I've been attempting to tune this bash script, with an eye towards performance.

This script is searching for two bits between 8800 and 9065 on frame 52.  It's a lot of combinations, and I am getting roughly 600 frames per minute.

Quote
for i in {8800..9064} ; do for j in {8801..9065} ; do ffmpeg.exe -debug mb_pos_size -s:0 704:480 -mmb X:76768:80,X:22120:80,X:45038:80,X:42196:80,X:66234:80,X:50298:80,X:$i:80,X:$j:80, 0:7:-2:-10:-10:-10:-10:8:-5,2:7:15506,9:7:-1,15:7:16391,17:7:-1,28:7:16704, 35:7:-2:-10:-10:-10:-10:8:-5,29:9:21074,34:9:-1,39:9:21626,5:28:-1::63,10:28:80196 -i iframe52.mpg4-img 2> output-$i-$j.txt -f image2 /dev/null -y ; done ; done

The output images have been entirely disabled, and the directive to ignore errors has been removed.  I am not 100% sure that the latter is "safe" but it does significantly increase the speed of the process.

Note that there are spaces in the mmb string that would need to be removed to run this.

After the run is done, I first do a grep for 42:03:9061 , then a grep for 00:29:81738 .  The second one is just in case the first "known good" position ends up changing due to correctly flipped bits.  Edit: This second test is probably worthless.

If anybody has suggestions for how to speed this up, I am all ears!  The search space can likely be reduced but I'm really unsure of what would be advisable.

Just sticking to two bits right now... going for three in a search space this size would take something like 21 days at a rate of 600 tries per minute.


you could pipe output of the for loop to grep so you don't create temp files. dunno how much will this save in terms of performance.

Offline Lourens

  • Full Member
  • *
  • Posts: 156
  • The Netherlands
  • Liked: 206
  • Likes Given: 304
For what it's worth, I'm running at about 750 calls per minute, with all output to a RAM disk and subsequent call to a separate program to analyse the resulting image. This is on GNU/Linux on a three year old laptop, with ignore errors enabled. Putting intermediate output on a RAM disk may help a bit, since disk writes are slow. Beyond that I think it's probably mostly the overhead of starting up FFmpeg all the time.

Offline morningdew76

  • Member
  • Posts: 11
  • McLean, VA
  • Liked: 12
  • Likes Given: 7
you could pipe output of the for loop to grep so you don't create temp files. dunno how much will this save in terms of performance.

That's a great idea, I'll try to figure that out (my Linux skills.. not so great).

I also realized that the nested loops weren't running efficiently-

Quote
for i in {8800..9009} ; do for j in $(eval echo "{$(($i+1))..9010}") ; do ffmpeg.exe -debug mb_pos_size -s:0 704:480 -mmb X:76768:80,X:22120:80,X:45038:80,X:42196:80,X:66234:80,X:50298:80,X:$i:80,X:$j:80, 0:7:-2:-10:-10:-10:-10:8:-5,2:7:15506,9:7:-1,15:7:16391,17:7:-1,28:7:16704, 35:7:-2:-10:-10:-10:-10:8:-5,29:9:21074,34:9:-1,39:9:21626,5:28:-1::63,10:28:80196 -i iframe52.mpg4-img 2> output-$i-$j.txt -f image2 /dev/null -y ; done ; done

This change halves the number of combinations to search.

For what it's worth, I'm running at about 750 calls per minute, with all output to a RAM disk and subsequent call to a separate program to analyse the resulting image. This is on GNU/Linux on a three year old laptop, with ignore errors enabled. Putting intermediate output on a RAM disk may help a bit, since disk writes are slow. Beyond that I think it's probably mostly the overhead of starting up FFmpeg all the time.

I had thought about using a RAM disk, and I'm probably losing a lot to Windows/MinGW overhead.  Too bad I can't run Linux on this machine.
« Last Edit: 05/23/2014 08:43 PM by morningdew76 »

Offline Jamsta

  • Member
  • Posts: 73
  • Liked: 21
  • Likes Given: 1115
From L2:

<snip>

You've also got a fan in Elon. He's been full of praise for the video work.
« Last Edit: 05/23/2014 08:33 PM by Chris Bergin »

Offline Chris Bergin

I was thinking about the movie, could someone with some artistic talent (not my case :P) make 1-2 title images, and maybe also some credits images at the end (for example a few before/after images, printscreens of the wiki, web application, ...)? It should have the same format as the images, i.e. 704x480.

Edit: a first version of a movie with all recovered frames (the other are not used)


When you compare that to the initial video provided, it's absolutely stunning. Amazing work guys.

Offline moralec

« Last Edit: 05/23/2014 08:43 PM by moralec »

Offline aero

  • Senior Member
  • *****
  • Posts: 2974
  • 92129
  • Liked: 793
  • Likes Given: 280
Wonder if the individual movie or gif frames should be trademarked? "SpaceX?"
Retired, working interesting problems

Offline AncientU

  • Senior Member
  • *****
  • Posts: 6217
  • Liked: 4008
  • Likes Given: 5546
Legs for days....

http://spacexlanding.wikispaces.com/Frames+72-91
Can you animate these frames?  Legs are starting to open frames 85-90...
"If we shared everything [we are working on] people would think we are insane!"
-- SpaceX friend of mlindner

Offline Asmegin

  • Member
  • Posts: 35
  • Canada
  • Liked: 21
  • Likes Given: 3
Is there a way to add a section for each Frameset page on the wiki that automatically creates a 'Entire Segment' input from the data entered?

Offline princess

  • Member
  • Posts: 65
  • Liked: 106
  • Likes Given: 25
Newbie here - I have experience working with MPEG-TS streams and I've written a few custom tools that can fix corruption at the transport stream layer. Is it worth me working on the split TS files that shanuson created from the raw edit8.ts?

I've run his files through the tools, and they fix a lot of stuff like bad PIDs, bad program continuity counters, wrong adaptation field lengths, things like that. The start positions of various badly-corrupted P-frames become visible, but they're still quite damaged.

I guess I'm asking, would the recovery effort benefit from this?

Offline dorkmo

  • Full Member
  • ****
  • Posts: 690
  • Liked: 318
  • Likes Given: 819
heyo,
seems like theres some sort of small flame event on frames 74 and 75. is from that the engines or perhaps do the legs have some sort of pyrotechnic release system? thought it was odd since it was so brief.

Offline AncientU

  • Senior Member
  • *****
  • Posts: 6217
  • Liked: 4008
  • Likes Given: 5546
heyo,
seems like theres some sort of small flame event on frames 74 and 75. is from that the engines or perhaps do the legs have some sort of pyrotechnic release system? thought it was odd since it was so brief.
Post link or images?
Thanks.
"If we shared everything [we are working on] people would think we are insane!"
-- SpaceX friend of mlindner

Offline deruch

  • Senior Member
  • *****
  • Posts: 2286
  • California
  • Liked: 1810
  • Likes Given: 3911
Newbie here - I have experience working with MPEG-TS streams and I've written a few custom tools that can fix corruption at the transport stream layer. Is it worth me working on the split TS files that shanuson created from the raw edit8.ts?

I've run his files through the tools, and they fix a lot of stuff like bad PIDs, bad program continuity counters, wrong adaptation field lengths, things like that. The start positions of various badly-corrupted P-frames become visible, but they're still quite damaged.

I guess I'm asking, would the recovery effort benefit from this?

Excellent, I've been trying to clean them up some.  But I'm totally in the dark.  I've done what I can for parts _11 and _12.  But I'm only really playing with a hex editor.  So I don't even know if what I'm doing is helpful, beyond the fact that it's cleaned up a bunch of the continuity errors.  It did return a few additional frames, but I can't see them (no computer Fu, i'm capable of following exact directions but that's it).  I've attached my attempts.  Someone let me know if they're any improvement or if I'm just wasting my time.  Not that that will necessarily keep me from goofing around with them.
Shouldn't reality posts be in "Advanced concepts"?  --Nomadd

Tags: