Author Topic: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread  (Read 948124 times)

Offline wronkiew

  • Full Member
  • *
  • Posts: 186
  • 34.502327, -116.971697
  • Liked: 105
  • Likes Given: 121
Interpolated descent between frames 150 and 168.

Many thanks to SwissCheese for demonstrating how to apply p-frames.

Offline arnezami

  • Full Member
  • **
  • Posts: 282
  • Liked: 262
  • Likes Given: 345
Video time  8)

I've used the new multiframe command on fixed_edit8.ts using all known mmb's for the I-frames that work on edit8 (from the wiki). I then reencoded it back into a video.

I used this command:

Quote
./ffmpeg -r 44999/3003 -mmb `cat mmb.txt` -debug mb_pos_size -err_detect ignore_err  -i fixed_edit8.ts -c:v mpeg4 -q:v 3 -tag:v xvid spacex_langing_nsf_20140517.avi

The mmb.txt file is attached. As you can see it's simply all the mmb's concatenated with '=' between them and 'FRAMExxx:' in front of each.

[edit] Almost forgot: for some reason the framenumbers are offset by 8. I think there are some frames before the first I-frame that cause this.

If you just want all the pictures you can do this:

Quote
./ffmpeg -r 44999/3003 -mmb `cat mmb.txt` -debug mb_pos_size -err_detect ignore_err  -i fixed_edit8.ts -f image2 img-%03d.png

The video (also attached) already looks very cool :) At some point you can actually see it flying!



Of course the P-frames disturb the video heavily. So there is still a lot of work to do.

Enjoy!

arnezami

@SwissCheese: I've tried to add the P-frame mmb's you provided. But somehow this doesn't work: the changed P-frames and the P-frames after that stopped working. We'll have to figure out why. I thought that should have worked.

[edit] Added youtube video.
« Last Edit: 05/17/2014 04:57 PM by arnezami »

Offline moralec

Video time  8)

I've used the new multiframe command on fixed_edit8.ts using all known mmb's for the I-frames that work on edit8 (from the wiki). I then reencoded it back into a video.

I used this command:

Quote
./spx/ffmpeg-spacex-spacexdebug1/ffmpeg -r 44999/3003 -mmb `cat mmb.txt` -debug mb_pos_size -err_detect ignore_err  -i fixed_edit8.ts -c:v mpeg4 -q:v 3 -tag:v xvid spacex_langing_nsf_20140517.avi

The mmb.txt file is attached. As you can see it's simply all the mmb's concatenated with =FRAMExxx: between them and just FRAMExxx: at the beginning.

[edit] Almost forgot: for some reason the framenumbers are offset by 8. I think there are some frames before the first I-frame that cause this.

If you just want all the pictures you can do this:

Quote
./spx/ffmpeg-spacex-spacexdebug1/ffmpeg -r 44999/3003 -mmb `cat mmb.txt` -debug mb_pos_size -err_detect ignore_err  -i fixed_edit8.ts -f image2 img-%03d.png

The video (also attached) already looks very cool :) At some point you can actually see it flying!

Of course the P-frames disturb the video heavily. So there is still a lot of work to do.

Enjoy!

arnezami

@SwissCheese: I've tried to add the P-frame mmb's you provided. But somehow this doesn't work: the changed P-frames and the P-frames after that stopped working. We'll have to figure out why. I thought that should have worked.

That things looks amazing!!!!!!!!!

Offline mvpel

  • Full Member
  • ****
  • Posts: 1124
  • New Hampshire
  • Liked: 1295
  • Likes Given: 1686
I don't know if it's a good idea to go this way, but I was stuck trying to repair P-frames: as soon as I touched them (tried to correct them with matlab), they would not be used during the video encoding (even when simply reading and rewriting a frame, without changing anything).

Are there some headers and metadata around the pframes that might be able to be repaired in a similar way as the transport stream and iframe headers were? I can't seem to find a good explanation so far as to how pframes are encoded, though. If there's some reference points like timestamp values and what have you, that might help resuscitate some of them.

Edit: NICE jobs on that video and animation, you two! I think that explains why the orange color in iframe 209 wants to get pulled down farther than you'd expect - it's reflecting off the rocket body.

Edit: By the way, at the very beginning of the video are we perhaps seeing some evidence that the first couple of iframes were almost completely fogged in? Until we get up to iframe 32 which SwissCheese combed out, there's a huge amount of grey MBs in there, and then just a hint of orange as the engine glow passes through the fog. What if we hardcoded the first iframe or two to just a solid grey foggy expanse and see what happens as the pframes come through?

Would the triangles at the lower corners be visible in dense fog, also? Seems like they would be, but that would depend on how the housing is designed.
« Last Edit: 05/17/2014 08:53 PM by mvpel »
"Ugly programs are like ugly suspension bridges: they're much more liable to collapse than pretty ones, because the way humans (especially engineer-humans) perceive beauty is intimately related to our ability to process and understand complexity. A language that makes it hard to write elegant code makes it hard to write good code." - Eric S. Raymond

Offline arnezami

  • Full Member
  • **
  • Posts: 282
  • Liked: 262
  • Likes Given: 345
Hi guys,

I've added a youtube video link to my previous post.

Regards,

arnezami

Offline Sohl

  • Full Member
  • **
  • Posts: 241
  • Liked: 77
  • Likes Given: 214
Video time  8)

That is fantastic!

But I was wondering, is there a relatively simple way to coerce the hue/color values in the p-frames to something in the blue-to-gray/white range before re-encoding the video?  Well, and maybe some yellow/orange near the frame center when the plume is visible.

Offline Asmegin

  • Member
  • Posts: 35
  • Canada
  • Liked: 21
  • Likes Given: 3
Amazing video! Nice to see all the hard work starting to come together.

Will the work on the P-Frames be more automatic or manual similar to the I-Frames? I'm sure we have enough people here with a bit of knowledge to help work on them if lots of manual labour is required. Otherwise it's all up to the few smart guys here :D

The next daytime landing won't be for many months, so there is lots of time to produce a good product. The upcoming Orbcomm launch will likely be in the early morning darkness, and the following two Asiasat launches won't have legs. Which means we have to wait until CRS-4, which I believe is scheduled for early August.

Offline adrianwyard

  • Full Member
  • ****
  • Posts: 962
  • Liked: 177
  • Likes Given: 228
Apologies if that has come up before (this is a big thread!) but has any consideration been given to post-processing the final 'best' rendering of the actual data?

It's a good bet that others have tried to recover highly corrupted MPEG streams before so there may be filters and techniques already developed out there beyond NSF. But some ideas:

+ Restrict the color palette to those we know are real. For those crazy colored larger macroblocks, substitute a color by averaging that of the neighbors. A really simple version of this would be to just forgo color and go black-and-white.

+ Detect hard 90 degree edges (of macroblocks) and blur those away; we know those edges didn't exist in the camera source image.

+ At some point you need to determine if there's any useful engineering data in this project. If so, then you'd be cautious about adding blur and generating colors, if if not, then post-processing choices would be geared to producing the most pleasing perceived video. You may find that adding generated 'snow' produces subjectively 'better' video because we're more used to seeing signals degraded this way than bright-green macroblocks.

Just my $0.02. Thanks everyone for your hard work; it may be just a footnote, but this landing is going in the history books.

Offline meadows.st

  • Full Member
  • *
  • Posts: 152
  • Vancouver BC, Canada
  • Liked: 88
  • Likes Given: 3768
Amazing video! Nice to see all the hard work starting to come together.

Will the work on the P-Frames be more automatic or manual similar to the I-Frames? I'm sure we have enough people here with a bit of knowledge to help work on them if lots of manual labour is required. Otherwise it's all up to the few smart guys here :D

The next daytime landing won't be for many months, so there is lots of time to produce a good product. The upcoming Orbcomm launch will likely be in the early morning darkness, and the following two Asiasat launches won't have legs. Which means we have to wait until CRS-4, which I believe is scheduled for early August.

First, let me say that I echo the comment on the Amazing video and simulation and offer any brute-force labor that may be required for P-Frames.

re: "The next daytime landing", I first started to do a calculation for what the light might be like at between ~1000m and 2000m ASL where the landing burn should start on 27 May, 2014 and then I found SunCalc.net that seems to indicate that the landing area should be between "civil twilight and sunrise" between 03:00 and 03:30 EDT on 27 May, 2014.  If the launch actually happens, this light should make for some reasonable (but perhaps not great) footage.  I have attached a screenshot of the resulting map presented by the SunCalc simulation.

EDIT: Actually, I suspect the launch images will be spectacular in the early light if the weather is clear and the further East the landing is located, the more light there will be for the landing (so no pressure) ;).

@Comga gives more detailed (and contradictory) info here:
http://forum.nasaspaceflight.com/index.php?topic=33089.msg1199888#msg1199888

EDIT2: Sadly I think that SunCalc is giving the time in my local timezone (PDT) so unfortunately I (now) agree with @Comga that the flight will still be in darkness. :(  In my own defence, I am used to higher latitudes (49N) and it starts to get light at 4AM around here as we approach the summer solstice. (added updated image with time in EDT)
« Last Edit: 05/17/2014 06:06 PM by meadows.st »
A little rudder far from the rocks is a lot better than a lot of rudder close to the rocks. L. David Marquet

Offline Llian Rhydderch

  • Full Member
  • ****
  • Posts: 901
  • Terran Anglosphere
  • Liked: 612
  • Likes Given: 5976
With the "NSF community-edited video" getting further along, it is probably not too early to ask this question:

What will be the license attached to any video that comes of this community editing effort by NSFers?

It would be great if the finished product ends up with one of the acceptable Creative Commons licenses on it that would make the video okay for Wikimedia and all the Wikipedia Foundation projects.  Two of the acceptable licenses for use there include the Creative Commons Attribution-Sharealike 3.0 Unported License (CC-BY-SA) and the GNU Free Documentation License (GFDL). 

Given that this started out as a SpaceX raw file, with what license I don't know, what will the edited/cleaned final version license be?  ... or should it be?


Edited:  fixed a single-letter typo
« Last Edit: 05/17/2014 08:22 PM by Llian Rhydderch »
Re arguments from authority on NSF:  "no one is exempt from error, and errors of authority are usually the worst kind.  Taking your word for things without question is no different than a bracket design not being tested because the designer was an old hand."
"You would actually save yourself time and effort if you were to use evidence and logic to make your points instead of wrapping yourself in the royal mantle of authority.  The approach only works on sheep, not inquisitive, intelligent people."

Offline arnezami

  • Full Member
  • **
  • Posts: 282
  • Liked: 262
  • Likes Given: 345
Edit: On other news, I tried to apply your very long mmb code for frame 92 (the frame formally known as iframe # 5) to the newest version (the one obtained from edit5.ts) and unfortunately is not working anymore. AS you where the main author, maybe you will be able to figure out what has changed. I'm guessing a small adjustment should be sufficient to get that frame visible again! It could also be a problem on how the code was pasted in the forum: I believe this happened to Saliva_sweet
This is my best attempt so far:

0:0:550,14:0:-1,
25:10:17410,
17:11:33650,
17:12:37648,
25:12:39009,29:12:-1,32:12:-1,10:14:-1,
17:14:52765,18:14:-1,24:14:-1,21:15:-1,22:15:-1,
34:15:56061,24:16:-1,
27:16:61466,42:19:-1,
43:19:83153,43:20:-1,
1:21:90585,11:22:-1,
12:22:99201,26:22:-1,
17:23:105394,22:24:-1,
23:24:111581,0:27:-1,
16:27:124815,24:27:-1,
25:27:125621,28:28:-1,
3:29:140154

Hope that helps others to make it better again. I've tried to use as much as possible from the last (much better) attempt. This though is using edit8.

But somehow I feel (while more data is certainly added to this frame) something got lost too. Not sure. Particularly around the legs. But I'm not so good at this. Hope I'm wrong.

Please help.

Regards,

arnezami
« Last Edit: 05/17/2014 10:00 PM by arnezami »

Offline Asmegin

  • Member
  • Posts: 35
  • Canada
  • Liked: 21
  • Likes Given: 3

This is my best attempt so far:

0:0:550,14:0:-1,
25:10:17410,
17:11:33650,
17:12:37648,
25:12:39009,29:12:-1,32:12:-1,10:14:-1,
17:14:52765,18:14:-1,24:14:-1,21:15:-1,22:15:-1,
34:15:56061,24:16:-1,
27:16:61466,42:19:-1,
43:19:83153,43:20:-1,
1:21:90585,11:22:-1,
12:22:99201,26:22:-1,
17:23:105394,22:24:-1,
23:24:111581,0:27:-1,
16:27:124815,24:27:-1,
25:27:125621,28:28:-1,
3:29:140154


Bit more of legs: 19:13:42998, 18:14:47626
« Last Edit: 05/17/2014 10:57 PM by Asmegin »

Offline SwissCheese

  • Full Member
  • *
  • Posts: 164
  • Liked: 249
  • Likes Given: 90
Interpolated descent between frames 150 and 168.

Many thanks to SwissCheese for demonstrating how to apply p-frames.

It looks good! How did you do the interpolation? Did you use information from the P-frames? I was thinking about interpolating the moving vector fields for the non-usable P-frames, or somehow better using them, but I'm stuck...
« Last Edit: 05/17/2014 11:04 PM by SwissCheese »

Offline mvpel

  • Full Member
  • ****
  • Posts: 1124
  • New Hampshire
  • Liked: 1295
  • Likes Given: 1686

This is my best attempt so far:
Bit more of legs: 19:13:42998, 18:14:47626

This looks very close to the first attempt! If someone can tease out more of the ocean, and perhaps find dirt spot #3 again, we'd be in good shape.
"Ugly programs are like ugly suspension bridges: they're much more liable to collapse than pretty ones, because the way humans (especially engineer-humans) perceive beauty is intimately related to our ability to process and understand complexity. A language that makes it hard to write elegant code makes it hard to write good code." - Eric S. Raymond

Offline Asmegin

  • Member
  • Posts: 35
  • Canada
  • Liked: 21
  • Likes Given: 3

This is my best attempt so far:
Bit more of legs: 19:13:42998, 18:14:47626

This looks very close to the first attempt! If someone can tease out more of the ocean, and perhaps find dirt spot #3 again, we'd be in good shape.

Dirt spot: 26:10:31150. Kinda messes up the rest though.
« Last Edit: 05/17/2014 11:19 PM by Asmegin »

Offline wronkiew

  • Full Member
  • *
  • Posts: 186
  • 34.502327, -116.971697
  • Liked: 105
  • Likes Given: 121

This is my best attempt so far:
Bit more of legs: 19:13:42998, 18:14:47626

This looks very close to the first attempt! If someone can tease out more of the ocean, and perhaps find dirt spot #3 again, we'd be in good shape.

Dirt spot: 26:10:31150

Love that dirt spot! They should put them in the camera for the next flight too.

Offline mvpel

  • Full Member
  • ****
  • Posts: 1124
  • New Hampshire
  • Liked: 1295
  • Likes Given: 1686
Better if they turn off interlacing and enable CRC instead. :D
"Ugly programs are like ugly suspension bridges: they're much more liable to collapse than pretty ones, because the way humans (especially engineer-humans) perceive beauty is intimately related to our ability to process and understand complexity. A language that makes it hard to write elegant code makes it hard to write good code." - Eric S. Raymond

Offline somepitch

  • Full Member
  • *
  • Posts: 172
  • Vancouver
  • Liked: 179
  • Likes Given: 407
Coming up on the 150,000 view mark for this thread/effort. Noting it as there's a small active team of people really working hard on this, but also a HUGE amount of people watching that work. Always nice to have a large appreciative audience, both inside and outside the space industry.

You'd be astonished how many views are from IP addresses in a place called Hawthorne, California! ;D

How many from Centennial, CO or Evry-Courcouronnes, France...?  ;D

Offline Asmegin

  • Member
  • Posts: 35
  • Canada
  • Liked: 21
  • Likes Given: 3

This is my best attempt so far:

0:0:550,14:0:-1,
25:10:17410,
17:11:33650,
17:12:37648,
25:12:39009,29:12:-1,32:12:-1,10:14:-1,
17:14:52765,18:14:-1,24:14:-1,21:15:-1,22:15:-1,
34:15:56061,24:16:-1,
27:16:61466,42:19:-1,
43:19:83153,43:20:-1,
1:21:90585,11:22:-1,
12:22:99201,26:22:-1,
17:23:105394,22:24:-1,
23:24:111581,0:27:-1,
16:27:124815,24:27:-1,
25:27:125621,28:28:-1,
3:29:140154


Bit more of legs: 19:13:42998, 18:14:47626
Dirt: 26:10:31150

I found what appears to be more of the left leg @ 17:13:42121 and 19:13:42516, but they only kindaaa fit and really mess up everything else. Maybe someone else can find the correct ones??

Offline Jakusb

  • Full Member
  • ****
  • Posts: 785
  • NL
  • Liked: 573
  • Likes Given: 270
A repost for convenience. :)
For those new:
The image has annotated most of the common dirt groups that reoccur in all frames. It helps align the macroblocks. :)
« Last Edit: 05/18/2014 12:32 AM by Jakusb »

Tags: