Author Topic: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread  (Read 917031 times)

Offline SwissCheese

  • Full Member
  • *
  • Posts: 164
  • Liked: 249
  • Likes Given: 90
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1180 on: 05/30/2014 07:09 PM »

Ok, the first part of this is done. Unfortunately in PNG format the zip file is 85 MB, so trying to auto-upload it to the wiki is probably pushing it. No pun intended. Instead, I put it on Google Drive at this url:

https://drive.google.com/file/d/0B_wTAwfYg03FQ3p2UVlkZE9MckU/edit?usp=sharing

Also on BTSync at B76OWSNRL5HWE47N7SHT4ZIX2B3B63IXT

This is ~250 frames in PNG format generated by scraping the wiki and then running it through ffmpeg. Why is this useful? Well, you can turn these into a movie file with this command:

ffmpeg -y -start_number 40 -i landing_base_set/frame-%03d.png -c:v libx264 -r 15 -pix_fmt yuv420p out.mp4

Also, you can run a batch process on these images to do post-processing or add annotations.

The zip archive will be regenerated every hour if the wiki has seen any updates.

I updated the auto-build script to renumber the frames to follow the wiki. Frames 181-200 are now taken from rawsplit_final.ts.

Also the BTSync share includes a video version with the frame-rate fixed as SwissCheese suggested.

SwissCheese, can you share a PNG of the title card?

Here you have! In the movie, I replaced frames 1 to 40 by the title image.


My current thinking on this is that rather than putting a duplicate timestamp in a matte, I'll redraw the known timestamps using the bitmap numerals already displayed. No interpolated timestamps added.

When I get a chance I also want to put together a system that automatically generates png sequences from the mmbs, zips them up, and uploads them to the wiki. That way anyone can download the frames and do their own batch processing on them.

Ok, the first part of this is done. Unfortunately in PNG format the zip file is 85 MB, so trying to auto-upload it to the wiki is probably pushing it. No pun intended. Instead, I put it on Google Drive at this url:

https://drive.google.com/file/d/0B_wTAwfYg03FQ3p2UVlkZE9MckU/edit?usp=sharing

Also on BTSync at B76OWSNRL5HWE47N7SHT4ZIX2B3B63IXT

This is ~250 frames in PNG format generated by scraping the wiki and then running it through ffmpeg. Why is this useful? Well, you can turn these into a movie file with this command:

ffmpeg -y -start_number 40 -i landing_base_set/frame-%03d.png -c:v libx264 -r 15 -pix_fmt yuv420p out.mp4

Also, you can run a batch process on these images to do post-processing or add annotations.

The zip archive will be regenerated every hour if the wiki has seen any updates.

Don't suppose you can also put a video somewhere on google drive too? If so I can probably automate pushing it to youtube or something.
« Last Edit: 05/30/2014 07:52 PM by IainCole »

Offline wronkiew

  • Full Member
  • *
  • Posts: 186
  • 34.502327, -116.971697
  • Liked: 105
  • Likes Given: 121
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1182 on: 05/30/2014 08:00 PM »
Here you have! In the movie, I replaced frames 1 to 40 by the title image.

Thank you!

Offline wronkiew

  • Full Member
  • *
  • Posts: 186
  • 34.502327, -116.971697
  • Liked: 105
  • Likes Given: 121
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1183 on: 05/30/2014 08:01 PM »
Don't suppose you can also put a video somewhere on google drive too? If so I can probably automate pushing it to youtube or something.

I can take care of this tonight.

Offline Chris Bergin

Sorry if someone already posted this, but your efforts are now linked on the SpaceX website :)

http://www.spacex.com/news/2014/04/29/first-stage-landing-video

And we hear Elon specifically mentioned you guys at the post Dragon V2 reveal Q&A!
« Last Edit: 05/30/2014 08:08 PM by Chris Bergin »

Offline Cinder

  • Full Member
  • ****
  • Posts: 603
  • Liked: 62
  • Likes Given: 284
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1185 on: 05/30/2014 09:20 PM »
I'd put the SpaceX logo above.

Ok, the first part of this is done. Unfortunately in PNG format the zip file is 85 MB, so trying to auto-upload it to the wiki is probably pushing it. No pun intended. Instead, I put it on Google Drive at this url:

https://drive.google.com/file/d/0B_wTAwfYg03FQ3p2UVlkZE9MckU/edit?usp=sharing

Also on BTSync at B76OWSNRL5HWE47N7SHT4ZIX2B3B63IXT

This is ~250 frames in PNG format generated by scraping the wiki and then running it through ffmpeg. Why is this useful? Well, you can turn these into a movie file with this command:

ffmpeg -y -start_number 40 -i landing_base_set/frame-%03d.png -c:v libx264 -r 15 -pix_fmt yuv420p out.mp4

Also, you can run a batch process on these images to do post-processing or add annotations.

The zip archive will be regenerated every hour if the wiki has seen any updates.

I updated the auto-build script to renumber the frames to follow the wiki. Frames 181-200 are now taken from rawsplit_final.ts.

Also the BTSync share includes a video version with the frame-rate fixed as SwissCheese suggested.

SwissCheese, can you share a PNG of the title card?

Here you have! In the movie, I replaced frames 1 to 40 by the title image.
The pork must flow.

Don't suppose you can also put a video somewhere on google drive too? If so I can probably automate pushing it to youtube or something.

I can take care of this tonight.

Cool, I think I pretty much have everything set up. All I need is a public accessible URL to hit that has the latest video. Also you need to send a POST to a url when it's uploaded so I know when it's changed, that do-able?

Offline catdlr

  • Member
  • Senior Member
  • *****
  • Posts: 5117
  • Viewed launches since the Redstones
  • Marina del Rey, California, USA
  • Liked: 1803
  • Likes Given: 1250
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1187 on: 05/30/2014 09:32 PM »
Sorry if someone already posted this, but your efforts are now linked on the SpaceX website :)

http://www.spacex.com/news/2014/04/29/first-stage-landing-video

And we hear Elon specifically mentioned you guys at the post Dragon V2 reveal Q&A!

YES!!  Congratulations to you Chris and the great efforts of the NSF team.
Tony De La Rosa

Offline SwissCheese

  • Full Member
  • *
  • Posts: 164
  • Liked: 249
  • Likes Given: 90
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1188 on: 05/30/2014 09:36 PM »
Hi guys,

Important note to everyone using the wiki/online editor: the online editor has been updated lately. This means you have to use -3s instead of -1s for P-frames. Just keep that in mind.

I am currently replacing all -1s with -3s for all P-frames on the wiki and re-uploading all those images. Again. :o

Regards,

arnezami

PS. The -1s are actually causing the gray "bands" in the picture...

The online editor gives a correct image with the -3 but does not seem to update the macroblock information and errors. Is it on purpose?

Hi guys,

Important note to everyone using the wiki/online editor: the online editor has been updated lately. This means you have to use -3s instead of -1s for P-frames. Just keep that in mind.

I am currently replacing all -1s with -3s for all P-frames on the wiki and re-uploading all those images. Again. :o

Regards,

arnezami

PS. The -1s are actually causing the gray "bands" in the picture...

The online editor gives a correct image with the -3 but does not seem to update the macroblock information and errors. Is it on purpose?

I think I know what this might be, I'll take a look in a bit.

Offline wronkiew

  • Full Member
  • *
  • Posts: 186
  • 34.502327, -116.971697
  • Liked: 105
  • Likes Given: 121
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1190 on: 05/30/2014 09:43 PM »
Don't suppose you can also put a video somewhere on google drive too? If so I can probably automate pushing it to youtube or something.

I can take care of this tonight.

Cool, I think I pretty much have everything set up. All I need is a public accessible URL to hit that has the latest video. Also you need to send a POST to a url when it's uploaded so I know when it's changed, that do-able?

Yep. URL?

Offline SwissCheese

  • Full Member
  • *
  • Posts: 164
  • Liked: 249
  • Likes Given: 90
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1191 on: 05/30/2014 09:44 PM »
Hi guys,

Important note to everyone using the wiki/online editor: the online editor has been updated lately. This means you have to use -3s instead of -1s for P-frames. Just keep that in mind.

I am currently replacing all -1s with -3s for all P-frames on the wiki and re-uploading all those images. Again. :o

Regards,

arnezami

PS. The -1s are actually causing the gray "bands" in the picture...

The online editor gives a correct image with the -3 but does not seem to update the macroblock information and errors. Is it on purpose?

I think I know what this might be, I'll take a look in a bit.

Oh and while you are here, would it be possible to add a slider to enhance the image contrast? It helps a lot for repairing the p-frames but I have to use an image viewer to do this currently.

Thank you again for your great work!

Don't suppose you can also put a video somewhere on google drive too? If so I can probably automate pushing it to youtube or something.

I can take care of this tonight.

Cool, I think I pretty much have everything set up. All I need is a public accessible URL to hit that has the latest video. Also you need to send a POST to a url when it's uploaded so I know when it's changed, that do-able?

Yep. URL?

Sent in PM as it contains an API key =)

Hi guys,

Important note to everyone using the wiki/online editor: the online editor has been updated lately. This means you have to use -3s instead of -1s for P-frames. Just keep that in mind.

I am currently replacing all -1s with -3s for all P-frames on the wiki and re-uploading all those images. Again. :o

Regards,

arnezami

PS. The -1s are actually causing the gray "bands" in the picture...

The online editor gives a correct image with the -3 but does not seem to update the macroblock information and errors. Is it on purpose?

I think I know what this might be, I'll take a look in a bit.

Oh and while you are here, would it be possible to add a slider to enhance the image contrast? It helps a lot for repairing the p-frames but I have to use an image viewer to do this currently.

Thank you again for your great work!

If I even had the slightest idea how to do that I'd be all over it =)

Hi guys,

Important note to everyone using the wiki/online editor: the online editor has been updated lately. This means you have to use -3s instead of -1s for P-frames. Just keep that in mind.

I am currently replacing all -1s with -3s for all P-frames on the wiki and re-uploading all those images. Again. :o

Regards,

arnezami

PS. The -1s are actually causing the gray "bands" in the picture...

The online editor gives a correct image with the -3 but does not seem to update the macroblock information and errors. Is it on purpose?

I think I know what this might be, I'll take a look in a bit.

Oh and while you are here, would it be possible to add a slider to enhance the image contrast? It helps a lot for repairing the p-frames but I have to use an image viewer to do this currently.

Thank you again for your great work!

If I even had the slightest idea how to do that I'd be all over it =)

hmmmm this might do it http://camanjs.com/

Hi guys,

Important note to everyone using the wiki/online editor: the online editor has been updated lately. This means you have to use -3s instead of -1s for P-frames. Just keep that in mind.

I am currently replacing all -1s with -3s for all P-frames on the wiki and re-uploading all those images. Again. :o

Regards,

arnezami

PS. The -1s are actually causing the gray "bands" in the picture...

The online editor gives a correct image with the -3 but does not seem to update the macroblock information and errors. Is it on purpose?

Can you give me a specific example of what isn't working and what you expect to see?

Offline SwissCheese

  • Full Member
  • *
  • Posts: 164
  • Liked: 249
  • Likes Given: 90
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1196 on: 05/30/2014 10:13 PM »
Hi guys,

Important note to everyone using the wiki/online editor: the online editor has been updated lately. This means you have to use -3s instead of -1s for P-frames. Just keep that in mind.

I am currently replacing all -1s with -3s for all P-frames on the wiki and re-uploading all those images. Again. :o

Regards,

arnezami

PS. The -1s are actually causing the gray "bands" in the picture...

The online editor gives a correct image with the -3 but does not seem to update the macroblock information and errors. Is it on purpose?

Can you give me a specific example of what isn't working and what you expect to see?

Just take any p-frame with errors, and write 0:0:-3 (should discard the whole frame) the yellow squares stay and the information on top of the frame still shows the discarded information.

(difficult to write from a mobile phone)

Hi guys,

Important note to everyone using the wiki/online editor: the online editor has been updated lately. This means you have to use -3s instead of -1s for P-frames. Just keep that in mind.

I am currently replacing all -1s with -3s for all P-frames on the wiki and re-uploading all those images. Again. :o

Regards,

arnezami

PS. The -1s are actually causing the gray "bands" in the picture...

The online editor gives a correct image with the -3 but does not seem to update the macroblock information and errors. Is it on purpose?

Can you give me a specific example of what isn't working and what you expect to see?

Just take any p-frame with errors, and write 0:0:-3 (should discard the whole frame) the yellow squares stay and the information on top of the frame still shows the discarded information.

(difficult to write from a mobile phone)

That seems to be what's coming back from the log :/ Dunno if it's specific to the online editor?

Offline mlindner

  • Software Engineer
  • Full Member
  • ****
  • Posts: 1929
  • Space Capitalist
  • Silicon Valley, CA -- previously in Ann Arbor, MI
  • Liked: 526
  • Likes Given: 210
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1198 on: 05/31/2014 01:26 AM »
Just want to say this is an amazing job you guys have done! I had to drop out from doing it because real life intervened in my ability to have time to assist. Hopefully I'll have time to catch up on the some 30 pages that have been posted since I last helped.

Just as a quick question, is there still need to repair I-frames? I notice that many of the I-frames have not advanced much in repair state from when I was last involved.
Hi mlindner! Glad you're back :)

There is definitely still a need for work on the I-frames. I think many moved their focus to the P-frames because making video is so tempting and rewarding ;)

But improving the I-frames clearly makes the 19 P-frames that follow much better!

Your excellent work on frame 169 shows that!

Regards,

arnezami

Where's the most recent version of the source code people have been using? Presumably people are no longer using my out-of-date repo anymore?
Internal combustion engine in space. It's just a Bad Idea.TM - Robotbeat

Offline moralec

Just want to say this is an amazing job you guys have done! I had to drop out from doing it because real life intervened in my ability to have time to assist. Hopefully I'll have time to catch up on the some 30 pages that have been posted since I last helped.

Just as a quick question, is there still need to repair I-frames? I notice that many of the I-frames have not advanced much in repair state from when I was last involved.
Hi mlindner! Glad you're back :)

There is definitely still a need for work on the I-frames. I think many moved their focus to the P-frames because making video is so tempting and rewarding ;)

But improving the I-frames clearly makes the 19 P-frames that follow much better!

Your excellent work on frame 169 shows that!

Regards,

arnezami

Where's the most recent version of the source code people have been using? Presumably people are no longer using my out-of-date repo anymore?

Is good to see you back mlindner! Let me try yo give you a fast update.
- we have are still using MMB codes to recover I and (most recently) p frames. We now have many partially fixed, but none as good as your original iframe.
- There has been several rounds of improvements on the .ts files since you left. After coerced it came try1, edit5 and edit8 (all with more frames recovered).  The latest one is called raw_final_fixed.ts. From that one we were finally able to extract all I and P frames from the video. The zip file was posted here some pages back.
- There have been also improvements on the FFmpeg utility. The most important is that it now supports multi-frame MMB codes to fix both frames and Iframes. It's still hosted in GitHub. (http://spacexlanding.wikispaces.com/Custom+FFmpeg)
- We also have a terrific online tool to fix the frames online, that is capable of identifying errors, showing previews and logs -among other things. You can find it here: http://spacex2.slapbet.org/

You should be able to update yourself pretty fast by reading the wiki home page.
http://spacexlanding.wikispaces.com

Tags: