Author Topic: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread  (Read 954668 times)

Offline Lourens

  • Full Member
  • *
  • Posts: 156
  • The Netherlands
  • Liked: 206
  • Likes Given: 304
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1060 on: 05/27/2014 07:11 PM »
Swiss and/or Shanuson, can you tell us which of the .ts files are currently "stable" and ready for mb fixing? I took a look at frame 131. The fixed_edit8_part_131.ts presumably gives < 19 frames. I downloaded rawsplit_part_8_fixed.ts and found that the mb fixes on http://spacexlanding.wikispaces.com/Frames+131-149 resulted in a scrambled image. I can go ahead and port the fixes to rawsplit_part_8_fixed.ts, but I want to make sure that is our more-or-less final version before I put more time into it.

rawsplit_part_8_fixed.ts will be used to create a final raw file. at least that is my idea.
Also all files currently active could still/will be modified before put back into a final raw file.
Working on the 6 Ts files I assigned _fixed to should be save.

Would it be possible to have a fixed .ts that is directly comparable to the original raw.ts? Or very specifically, on the one hand our best guess at the correct stream so far, and on the other hand the original raw.ts with apparent holes (the 56-byte gaps that were found here and there) filled with zeros?

I've been thinking about any error correction that may have been applied to the stream, inspired by ajmartin's post. There's a Reed-Solomon code that packs 28-byte chunks of data into 32-byte code words and can repair up to two broken bytes, and detect more than that. That might explain the 28-byte or 56-byte pattern and the gaps (error detected, packet dropped), but not the bit-pattern ajmartin mentioned. It seems to be pretty common to have two nested codes however, so perhaps there's a bit-level code within that. Anyway, detecting these kinds of patterns would be easier to attempt if we had a raw and a fixed stream side-by-side to compare. I've tried with rawsplit_part_2.ts and rawsplit_part_2_fixed.ts, but as I understand it the rawsplit_part_2.ts is already an extract from a patched-up file.

Alternatively Chris, if SpaceX can answer the question "Which (forward) error correction mechanisms were used in getting the TS stream from the camera to the web?" that might be helpful too.

The reason that this is important is that we'll get to a point where we have all the undamaged macroblocks located and put into the right place, and we want to try to improve the broken ones. That amounts to flipping bits and seeing what the result looks like, but there are problems with that:

1) There are a lot of possible combination to try. If an error correction code was used then that information could possibly allow us to discard some of them (e.g. if a single-error correcting code was used at the bit level, then any single-bit flips will have already been fixed and don't need to be tried, and there may be a more limited number of combinations of flipped bits to check).

2) It's difficult to figure out whether the result is any good. Of course it should decode without errors, but there are lots of combinations of bits that give a correctly decodable block, and only one correct one. We can weed out some bad ones by seeing if the block matches up with neighbouring blocks (I call it the jigsaw-check), and even better is just looking at it and seeing if it makes sense. Having a lot of people willing to judge potential solutions will be very helpful in that stage.

3) If the block is damaged badly enough, you end up flipping so many bits that you're effectively just painting in the gaps. If it gets to that, then there are much better ways of doing so than flipping bits and digging through the results. Knowing the FEC mechanism may make it easier to limit the search to small errors only, so that if something usable pops out there's still a reasonable chance that it's really the original block.

Finally, maybe there should be a separate thread for how to release the results, licensing, presentation, etc.? It's becoming a fairly big topic, and I'm sure it's only going to grow. Then we can keep this one for technical discussion on the repair process.

Offline SwissCheese

  • Full Member
  • *
  • Posts: 164
  • Liked: 249
  • Likes Given: 92
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1061 on: 05/27/2014 07:22 PM »
Swiss and/or Shanuson, can you tell us which of the .ts files are currently "stable" and ready for mb fixing? I took a look at frame 131. The fixed_edit8_part_131.ts presumably gives < 19 frames. I downloaded rawsplit_part_8_fixed.ts and found that the mb fixes on http://spacexlanding.wikispaces.com/Frames+131-149 resulted in a scrambled image. I can go ahead and port the fixes to rawsplit_part_8_fixed.ts, but I want to make sure that is our more-or-less final version before I put more time into it.

I hope it's the final version, but you can never be sure it seems :D

I also started earlier fixing the i-frame (there is more data, so the bit positions have mostly just to be shifted), but without the data available on the online editor it takes much more time (counting the squares to find the good positions instead of just clicking on the image...), so I stopped and went back to the other parts :P

I attach the mmb sequence where I stopped, if you want to continue fixing it.


Offline princess

  • Member
  • Posts: 65
  • Liked: 106
  • Likes Given: 25
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1062 on: 05/27/2014 08:03 PM »
OK, here is my second attempt at the part 10 TS file after feedback from mvpel. I've managed to recover an extra P-frame (starting at offset 0x00021528 into the file) and I've set the PCR, PTS and MPEG4 headers correctly as far as I can see. Feedback would be much appreciated!

Offline princess

  • Member
  • Posts: 65
  • Liked: 106
  • Likes Given: 25
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1063 on: 05/27/2014 08:12 PM »
And as an extra bonus for all you lovely people, here is my fixed up Part 9 TS file. I've recovered two P-frames, corrected their PCR and PTS timestamps, and set the MPEG4 headers so that ffmpeg can find them. As usual, let me know your feedback!

It would be great if one of the -mmb wizards (SwissCheese?) could take a look and see if it makes a difference to their efforts, as that's a bit out of my league...

Offline princess

  • Member
  • Posts: 65
  • Liked: 106
  • Likes Given: 25
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1064 on: 05/27/2014 08:38 PM »
Here's my fixed Part 6 TS file. This one was fairly straightforward with no new P-frames uncovered, so this is less of a priority.

Offline mhenderson

  • Member
  • Posts: 69
  • USA
  • Liked: 101
  • Likes Given: 18
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1065 on: 05/27/2014 08:49 PM »
My $.02. Perhaps I am being naive, but here's the case for NOT requesting a license:

The original post at spacex.com reads as follows:
"We welcome anyone to download the raw file to make improvements directly—if you are successful, please feel free to post your video at: http://www.reddit.com/r/spacex/comments/24bsn2/first_stage_landing_video/. "

The source files have been downloaded with no license restriction thousands of times.

Reddit.com is a commercial website. Republishing and some commercial use of derivative work is allowed.

Folks from Hawthorne have actively participated in this forum. NSF is a commercial website. (See those Site Booster ads >>>)

Work in progress is openly posted to the wiki. No cease and desist letter so far.

Requesting a license == inviting lawyers to the party. Not a good idea without deep pockets.

This ship set sail five weeks ago and has met / exceeded the highest hopes and expectations. Nobody's gonna risk a Streisand Effect by squelching this little exercise in fan fixin'. 

Edit:  Oh yeah, I neglected to mention the CEO of SpaceX tweeted a link to a copy of our derivative work and publicly said "Great progress by @NASASpaceflight members repairing the Falcon 9 ocean landing video". I think we got a green light there.
« Last Edit: 05/27/2014 09:05 PM by mhenderson »

Offline princess

  • Member
  • Posts: 65
  • Liked: 106
  • Likes Given: 25
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1066 on: 05/27/2014 09:05 PM »
I'm on a roll tonight! Here is my fixed Part 5 TS file. One of the P-frame headers was slightly corrupt and the PTS timestamp was wrong, so I've corrected that too. It probably won't affect ffmpeg that much but it's nice to have the timestamps correct anyway.

Offline princess

  • Member
  • Posts: 65
  • Liked: 106
  • Likes Given: 25
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1067 on: 05/27/2014 10:14 PM »
Another TS file! Here's part 4 fixed up. A few P-frame fixes, mainly timestamps and the odd bit of header corruption.

Offline princess

  • Member
  • Posts: 65
  • Liked: 106
  • Likes Given: 25
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1068 on: 05/27/2014 10:59 PM »
Finishing up the unclaimed TS parts from the wiki, here is part 3 fixed up. It had a lot of corruption and a few P-frames have had timestamps and headers corrected.

Offline princess

  • Member
  • Posts: 65
  • Liked: 106
  • Likes Given: 25
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1069 on: 05/27/2014 11:29 PM »
Two new P-frames in part 11!

After tonight's fixing marathon, I decided to go back over my part 11 attempt and I found the two extra P-frames that I was missing before. I've attached the fixed part 11 below. If the mmb / ffmpeg experts could give it a look, I'd be most grateful.

I hope everyone didn't mind me posting lots of results - I've just written some custom tools in C++ that help me find the problems, calculate timestamps and so on. It's made it a lot quicker to fix up the TS problems and find P-frames hiding in the corruption. I'm going to turn in for the night now, so it'll give you all a chance to have a look at all the fixed up things and see what you can make from them!

Online Chris Bergin


Folks from Hawthorne have actively participated in this forum. NSF is a commercial website. (See those Site Booster ads >>>)


Kinda. No one is earning any money off this thing (as much as we should be given the size of the place now). I think I'm personally $30K down since starting it....don't worry folks, we're self sustaining now, just about.

Oh and I'd do it all again if I was given a time machine. Though probably not by starting it by maxing out a card to get it built! ;D

Quote

Edit:  Oh yeah, I neglected to mention the CEO of SpaceX tweeted a link to a copy of our derivative work and publicly said "Great progress by @NASASpaceflight members repairing the Falcon 9 ocean landing video". I think we got a green light there.

Yep, that's a good call. Focus should continue to be on the fine work in fixing the video. I'll ensure the rest, but the non official response I got first of all today was "LOL! You're doing us a favor!" <---seriously! :)

Offline Lee Jay

  • Elite Veteran
  • Global Moderator
  • Senior Member
  • *****
  • Posts: 6673
  • Liked: 980
  • Likes Given: 139
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1071 on: 05/28/2014 01:26 AM »
Exactly.  When I thought of SpaceX exerting some sort of rights over this effort, the expression that came to my mind was "biting the hand that feeds you".  So glad to hear they are being entirely rational about the whole thing!

The effort is amazing, by the way.

Offline moralec

Wow, many updates!
Guys: if you need to add new files to the online editor, just let me know :) I can gladly take care of all the arrangements :)

Offline deruch

  • Senior Member
  • *****
  • Posts: 2286
  • California
  • Liked: 1813
  • Likes Given: 3911
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1073 on: 05/28/2014 03:25 AM »
Wow Princess!  Charger day, huh?

Shouldn't reality posts be in "Advanced concepts"?  --Nomadd

Offline arnezami

  • Full Member
  • **
  • Posts: 284
  • Liked: 262
  • Likes Given: 346
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1074 on: 05/28/2014 03:35 AM »
Two new P-frames in part 11!

After tonight's fixing marathon, I decided to go back over my part 11 attempt and I found the two extra P-frames that I was missing before. I've attached the fixed part 11 below. If the mmb / ffmpeg experts could give it a look, I'd be most grateful.

I hope everyone didn't mind me posting lots of results - I've just written some custom tools in C++ that help me find the problems, calculate timestamps and so on. It's made it a lot quicker to fix up the TS problems and find P-frames hiding in the corruption. I'm going to turn in for the night now, so it'll give you all a chance to have a look at all the fixed up things and see what you can make from them!
Hi princess,

I would like to wish you a good sleep. I liked your marathon. :)

Regards,

arnezami

Offline arnezami

  • Full Member
  • **
  • Posts: 284
  • Liked: 262
  • Likes Given: 346
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1075 on: 05/28/2014 03:48 AM »
Hi guys,

My "macroblock data finder" kinda works, but it doesn't seem to be much more effective than what we have already done.

You can try it out (its attached to the post). You will have to edit  the variables $filename_ts and $ffmpeg_binary in order for it to do what you want. It outputs an mmb and puts -2s between the "winner" mb-sequences it found. Notice this x,y positioning is just so the sequences dont overlap and many times the y goes over 29. This is only useful for finding new mb-data, not for positioning them or for auto generating mmbs. Its fixed to frame 0 (I-frame) but you can change that in the code.

So back to the drawing board...

I'm currently working on ways to recover bad macroblocks. Or at least finding their starting positions. I've still got some ideas. ;)

Regards,

arnezami
« Last Edit: 05/28/2014 03:53 AM by arnezami »

Offline Llian Rhydderch

  • Full Member
  • ****
  • Posts: 917
  • Terran Anglosphere
  • Liked: 644
  • Likes Given: 6131
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1076 on: 05/28/2014 03:54 AM »

Folks from Hawthorne have actively participated in this forum. NSF is a commercial website. (See those Site Booster ads >>>)


Kinda. No one is earning any money off this thing (as much as we should be given the size of the place now). I think I'm personally $30K down since starting it....don't worry folks, we're self sustaining now, just about.

Oh and I'd do it all again if I was given a time machine. Though probably not by starting it by maxing out a card to get it built! ;D

Quote

Edit:  Oh yeah, I neglected to mention the CEO of SpaceX tweeted a link to a copy of our derivative work and publicly said "Great progress by @NASASpaceflight members repairing the Falcon 9 ocean landing video". I think we got a green light there.

Yep, that's a good call. Focus should continue to be on the fine work in fixing the video. I'll ensure the rest, but the non official response I got first of all today was "LOL! You're doing us a favor!" <---seriously! :)

I absolutely agree, Chris, that the NSF team here are doing SpaceX a very large favor.  And wonderfully, and somewhat counter-intuitively, the way this works is that those people volunteering so much good time here are also receiving benefits of various sorts from doing the work.*

But on the license question, I do thinkit is a very good idea to "make the implicit, explicit".  They know, and we know, (and channeling Dick Cheney) they know that we know and we know that they know.  But all of that is in our heads; it's implicit.

Best to make it explicit with the license Lar suggested earlier:  "can we have it licensed under CC-BY-SA version 4, please" and "WITHOUT the NC or ND clauses attached, please" — and for the reasons he articulated so well.

Best,
 Llian

* fun, bragging rights, the opportunity to contribute to something that is meaningful, etc.—heck, I see some of the video guys able to write a book about the topic based on all that was learned here in this project.  All of this is common in the open source world, and literally millions of person hours are donated to such efforts each month.
Re arguments from authority on NSF:  "no one is exempt from error, and errors of authority are usually the worst kind.  Taking your word for things without question is no different than a bracket design not being tested because the designer was an old hand."
"You would actually save yourself time and effort if you were to use evidence and logic to make your points instead of wrapping yourself in the royal mantle of authority.  The approach only works on sheep, not inquisitive, intelligent people."

Online wronkiew

  • Full Member
  • *
  • Posts: 186
  • 34.502327, -116.971697
  • Liked: 105
  • Likes Given: 125
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1077 on: 05/28/2014 05:52 AM »

My current thinking on this is that rather than putting a duplicate timestamp in a matte, I'll redraw the known timestamps using the bitmap numerals already displayed. No interpolated timestamps added.

When I get a chance I also want to put together a system that automatically generates png sequences from the mmbs, zips them up, and uploads them to the wiki. That way anyone can download the frames and do their own batch processing on them.

Ok, the first part of this is done. Unfortunately in PNG format the zip file is 85 MB, so trying to auto-upload it to the wiki is probably pushing it. No pun intended. Instead, I put it on Google Drive at this url:

https://drive.google.com/file/d/0B_wTAwfYg03FQ3p2UVlkZE9MckU/edit?usp=sharing

Also on BTSync at B76OWSNRL5HWE47N7SHT4ZIX2B3B63IXT

This is ~250 frames in PNG format generated by scraping the wiki and then running it through ffmpeg. Why is this useful? Well, you can turn these into a movie file with this command:

ffmpeg -y -start_number 40 -i landing_base_set/frame-%03d.png -c:v libx264 -r 15 -pix_fmt yuv420p out.mp4

Also, you can run a batch process on these images to do post-processing or add annotations.

The zip archive will be regenerated every hour if the wiki has seen any updates.

Online wronkiew

  • Full Member
  • *
  • Posts: 186
  • 34.502327, -116.971697
  • Liked: 105
  • Likes Given: 125
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1078 on: 05/28/2014 06:00 AM »

Ok, the first part of this is done. Unfortunately in PNG format the zip file is 85 MB, so trying to auto-upload it to the wiki is probably pushing it. No pun intended. Instead, I put it on Google Drive at this url:

I should mention that I padded the frame sequence to reserve space for extra p-frames. Here is the mapping:

32 -> 40
52 -> 60
72 -> 80
92 -> 100
112 -> 120
131 -> 140
150 -> 160
169 -> 180
189 -> 200
209 -> 220
229 -> 240
248 -> 260
268 -> 280

Online wronkiew

  • Full Member
  • *
  • Posts: 186
  • 34.502327, -116.971697
  • Liked: 105
  • Likes Given: 125
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1079 on: 05/28/2014 06:32 AM »

Best to make it explicit with the license Lar suggested earlier:  "can we have it licensed under CC-BY-SA version 4, please" and "WITHOUT the NC or ND clauses attached, please" — and for the reasons he articulated so well.

I think I would like to see the original video licensed CC-BY-SA. The reason is that this would allow Wikipedia to use the final video for educational purposes. Wikimedia has sample letters to use for requesting a free license for content. I have used this to successfully request CC licenses from SpaceX in the past, though they were a smaller company then.

See http://commons.wikimedia.org/wiki/Commons:Email_templates

Tags: