Author Topic: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread  (Read 1201459 times)

Offline mvpel

  • Full Member
  • ****
  • Posts: 1125
  • New Hampshire
  • Liked: 1303
  • Likes Given: 1685
You did really great! Those headers were totally mangled, but after your fixes your result file is pretty clean from a TS point of view.

Just that one duplicate continuity counter - it was odd, the lead-in and tail-out of that section were quite clean but the numbers just didn't line up. Maybe that's a side effect of earlier alignment efforts.

Quote
I hope you don't mind but I've processed it a little more and fixed a couple of PCC discontinuities that were remaining. I've also removed the AF from any packets where either the AF is longer than the packet, or when the AF shows a wildly invalid PTS time. This is hopefully the correct thing to do, but please take a look and let me know what you think.

Why would I mind? :D

I've been putting some thought into parsing code to deal with the PES headers and the frame headers, but haven't had time to deal with it. Had a big deadline yesterday for a symposium presentation I'll be doing the second week of June and have a pile of essays to write for a course I'm taking - the work I did yesterday was to wind down after a 60-hour workweek.

I agree clearing the AF bit in packets where it's clear that its definitely not an adaptation field that follows is the right thing to do. As Shanuson points out there's a clear pattern to AFs which we can use to figure them out. There were sections where there were a really wild number of flipped bits, and figuring out whether or not an AF is valid wasn't going to be a "wind down" sort of thing for me to think about last night. There were one or two that showed up clearly in Wireshark that I fixed, but I'm glad you were able to take care of the rest of it.

I got the info I needed to extract the frames with ffmpeg, but I wound up with a completely blank iframe for some reason that I didn't have time to investigate, and I haven't downloaded MichaelNi's patched ffmpeg yet. It did deliver a full 20 frames, though.
"Ugly programs are like ugly suspension bridges: they're much more liable to collapse than pretty ones, because the way humans (especially engineer-humans) perceive beauty is intimately related to our ability to process and understand complexity. A language that makes it hard to write elegant code makes it hard to write good code." - Eric S. Raymond

Offline arnezami

  • Full Member
  • **
  • Posts: 285
  • Liked: 267
  • Likes Given: 378
Question: when we use -mmb X:offset:pattern, are we flipping bits before or after the entropy decoding stage? Is this simply equivalent to flipping bits in the input file? Or is it essentially just a low-level way of editing the MB directly?
From what I understand it is equivalent to flipping the bits in the input file. It happens well before the decoding of any blocks start.

Offline mvpel

  • Full Member
  • ****
  • Posts: 1125
  • New Hampshire
  • Liked: 1303
  • Likes Given: 1685
Legs for days....

http://spacexlanding.wikispaces.com/Frames+72-91
Can you animate these frames?  Legs are starting to open frames 85-90...

If you paste the attached MMB into the http://spacex2.slapbet.com/ frame 72 entire-segment box, and then hit the play button at the right end of the slider bar below the image, you'll see the beginning of the legs opening - this should not be used for ongoing work, as I simply nulled out some of the intervening pframes which were making it difficult to see the leg movement and combined all the work up to this point.

It's VERY cool to see!!
« Last Edit: 05/24/2014 04:51 pm by mvpel »
"Ugly programs are like ugly suspension bridges: they're much more liable to collapse than pretty ones, because the way humans (especially engineer-humans) perceive beauty is intimately related to our ability to process and understand complexity. A language that makes it hard to write elegant code makes it hard to write good code." - Eric S. Raymond

Offline morningdew76

  • Member
  • Posts: 20
  • McLean, VA
  • Liked: 48
  • Likes Given: 219
Brute force bit flipper inside the decoder:

Another idea is to do the bruteforce bitflipping inside the decoder. The main reason to do the brute forcing inside the code of the decoder is that it is likely to be much faster. Lets say you have four blocks of -1 (for example 15:12, 16:12, 17:12 and 18:12 in I-frame 150). To the left and right of it is are good macroblocks (14:12 at 45301 and 19:12 at 46628). You remove the four -1s so it will give an error again. Now you say to the decoder (in broteforce mode) try bitflipping 1-3 bits between the starting position of 15:12 (45472) and the starting position of 19:12 (46628) and each time you do this try to decode from 15:12 and see if you find 4 macroblocks and end at exactly at 19:12 (46628).

The bash script that I have been using now spits out roughly 900 combinations per minute in, using one thread.  This produces a rate sufficient for a 4-bit search over a 50-bit area in a matter four hours, and works pretty much exactly how you described.. The difference being that I am grepping the output of FFmpeg.  A few hundred candidates were produced, of which 40 or so look good.  My next step is to add generation of the following P-frame to cross-reference these 40 candidates.

The size of the search space follows "bit-space choose bit-size", which worked out to "50 choose 4" for the 4-bit run.

Moving this searching into FFmpeg is an obvious next step, I sincerely hope it's possible!

Offline Untribium

  • Member
  • Posts: 32
  • Switzerland
  • Liked: 32
  • Likes Given: 82
Actually, it occurs to me that the straight up "ps" approach to tracking workers may have problems with the above script, because the workers are constantly starting and stopping ffmpeg, which ps is looking for.

Another easy trick to track spawned workers is just to use PHP.

To check workers you can use:
while [ `ps aux | grep process_one\.php | grep -v grep | wc -l` -ge $threads ]

And to spawn workers you can use:
# spawn a worker(via php passthrough for easy thread tracking) on the current offset
php /root/process_one.php $i &

process_one.php looks like this:

-snip-

Again, the overhead php adds to spawning a process in this case is negligible because the workers will be running for a good while each and you're spawning thousands, not millions or billions.

Looks like this: http://coldattic.info/shvedsky/pro/blogs/a-foo-walks-into-a-bar/posts/7 might be an option as well. Generate the addresses using seq and then pipe it to xargs which then runs a fixed number of ffmpeg processes at a time. I'll check it out tomorrow, should get some sleep first :)

I am interested to see the results.  I'm just so busy that my "recreation" basically involves this thread and a few other news sites at the moment.  Maybe you have found something that will be very useful to my endeavors in the future, at least on one level of scale!  Although I must admit, I do enjoy coding this type of thing.

Okay, I looked up some xargs stuff...
The single flip should be pretty simple, something like this should do it:

Quote
seq 8800 9100 | xargs -n 1 -I BITPOS --max-procs=4 ffmpeg -threads 1 -debug mb_pos_size -err_detect ignore_err -s:0 704:480 -mmb ...

In the mmb you can now use BITPOS as the parameter for example X:BITPOS:08. Unfortunately, I can't check whether it works and actually runs faster right now...
Also, I'm having some trouble when it comes to multiple flips. The problem is, that you can only -I one parameter in xargs, so some bash trickery (like creating an array with the params in) will be necessary. Now if we get the bitflipping built into the decoder as arnezami suggested (which would be awesome!), that will be kinda pointless, but I least I now know how to move and rename a bunch of numbered files at once (using multiple processes even!) :)
« Last Edit: 05/24/2014 05:38 pm by Untribium »

Offline The Roadie

  • Full Member
  • ****
  • Posts: 426
  • Portland, Oregon
  • Liked: 2327
  • Likes Given: 98
Why are we succeeding?

I think there are two main reasons why we are succeeding in what was deemed to be impossible. The first is that we really went outside-the-box with our approach. The second is that our techniques are such that we can collaborate in our efforts.
I may be a noob here, but as an old-timer on IRC, Usenet, and forums, I've been over-the-top impressed with this project. Every forum with a critical mass of talent and altruistic members seems to adopt a project sooner or later that defines the spirit of the group. And it's often called the Epic Thread. And mention of it can go viral, and it's talked about at trade meetings, cocktail parties, and campfires for a generation.

I'm certain I missed a few here in the past, but this one seems to be the Thread of 2014 in my opinion. Carry on, and thanks!
"A human being should be able to...plan an invasion..conn a ship..solve equations, analyze a new problem..program a computer, cook a tasty meal.."-RAH

Offline princess

  • Member
  • Posts: 65
  • Liked: 106
  • Likes Given: 25
OK, here's my fixed version of the part 15 TS file. As far as I can see there's a damaged P-frame starting in packet 108 (offset 0x4f50 bytes into the file), but it's damaged at the MPEG4 level and I'm only fixing TS-level things at the moment.

I've set its AF to 7 bytes so that the MPEG4 data will be aligned properly, but I've not put a PCR in it because it's very damaged (and as far as I can see, ffmpeg doesn't seem to mind the lack of a PCR).

It should be ready for MPEG4 experts to start playing with it - if anyone wants to correct the header of this packet, a new P-frame should hopefully emerge. Let us know how you go!

Online Shanuson

  • Full Member
  • ***
  • Posts: 395
  • Liked: 327
  • Likes Given: 2542
OK, here's my fixed version of the part 15 TS file. As far as I can see there's a damaged P-frame starting in packet 108 (offset 0x4f50 bytes into the file), but it's damaged at the MPEG4 level and I'm only fixing TS-level things at the moment.

I've set its AF to 7 bytes so that the MPEG4 data will be aligned properly, but I've not put a PCR in it because it's very damaged (and as far as I can see, ffmpeg doesn't seem to mind the lack of a PCR).

It should be ready for MPEG4 experts to start playing with it - if anyone wants to correct the header of this packet, a new P-frame should hopefully emerge. Let us know how you go!

Here how the header looks like in rawsplit_part_15.ts:
47 65 28 20 87 80 00 38 12 2A 53 00 9E 00 5E E7 02 00 81 80 07 21 09 BF FA B9 FF 47 00 50 05 96 4D

Here is the P-frame header how it should look like:
47 43 E8 3E 07 10 00 38 12 2a 7E 00 00 00 01 E0 00 00 81 80 07 21 01 BF FA 89 FF FF 00 00 01 B6 5D

34 bits flipped between them (not counting the CC and the last half byte D)
Actually, i never checked if there is more header after the  00 00 01 b6 5 which we should repair.
I will do that when the rest is done.

Cheers Shanuson

Edit: Ok I looked at your fixed part15 file. There were still many unfixed Ts-packets, most with the correct PID and CC but wrong flags set. The TS-Header of a data packet should really only look like 47 03 e8 1X, with 2 exceptions, the first and last of a frame which should be easily identified.

« Last Edit: 05/24/2014 10:51 pm by Shanuson »

Offline princess

  • Member
  • Posts: 65
  • Liked: 106
  • Likes Given: 25

Edit: Ok I looked at your fixed part15 file. There were still many unfixed Ts-packets, most with the correct PID and CC but wrong flags set. The TS-Header of a data packet should really only look like 47 03 e8 1X, with 2 exceptions, the first and last of a frame which should be easily identified.

That's great, thank you for taking a look. Your changes all look good! I'd missed that packet with the huge AF at offset 0x52fc, good catch.

For the other "unfixed" packets I'd just left the erroneous Payload Unit Start Indicator bits in there. I agree with you that they should be removed - I had left them in there because (as far as I can see) they don't trip ffmpeg up, and if I'm in doubt I like to change as little as possible. I'm always worried about changing too much, it always feels nicer to make as small a change as I can. However, in future I can remove these too if you think it makes it better.

Thank you for your feedback, much appreciated!

Offline mvpel

  • Full Member
  • ****
  • Posts: 1125
  • New Hampshire
  • Liked: 1303
  • Likes Given: 1685
For the other "unfixed" packets I'd just left the erroneous Payload Unit Start Indicator bits in there. I agree with you that they should be removed - I had left them in there because (as far as I can see) they don't trip ffmpeg up, and if I'm in doubt I like to change as little as possible. I'm always worried about changing too much, it always feels nicer to make as small a change as I can.

It's not so much the size of the change, but the quality of the change that's important. There's a true and original state underneath all that corruption, and that's where we should aim as long as we can aim with confidence and precision.

"Ugly programs are like ugly suspension bridges: they're much more liable to collapse than pretty ones, because the way humans (especially engineer-humans) perceive beauty is intimately related to our ability to process and understand complexity. A language that makes it hard to write elegant code makes it hard to write good code." - Eric S. Raymond

Offline princess

  • Member
  • Posts: 65
  • Liked: 106
  • Likes Given: 25
There's a true and original state underneath all that corruption, and that's where we should aim as long as we can aim with confidence and precision.

That's a good way to look at it! OK, I will make sure to fix the Payload Unit Start Indicators in future ;)

Offline wronkiew

  • Full Member
  • *
  • Posts: 186
  • 34.502327, -116.971697
  • Liked: 105
  • Likes Given: 125
This is just a proof of concept. I deciphered a few of the timestamps on the images and interpolated the rest. Due to interlacing and propagating errors, even if we get good mbs in the pframes the timestamps are still going to be gibberish most of the time. So, I propose we just redraw them. Any concerns about this?

Offline Jakusb

  • Full Member
  • ****
  • Posts: 1207
  • NL
  • Liked: 1215
  • Likes Given: 637

This is just a proof of concept. I deciphered a few of the timestamps on the images and interpolated the rest. Due to interlacing and propagating errors, even if we get good mbs in the pframes the timestamps are still going to be gibberish most of the time. So, I propose we just redraw them. Any concerns about this?

Seems like a very good idea, with a few constraints:
- they need to be correct
- they are only applied at time of generating the movie file, so not to mess up de frame reconstruction process. In the end it is much better to have them fixed instead of reapplied.

But if you are at it, is it easy for you to also add a side view impression of how the stage would look at a every frame?

I think it would really help the viewers better understand what they are looking at. Including frames that are too corrupted. Also it would trigger a discussion on what happens in every frame.

Offline deruch

  • Senior Member
  • *****
  • Posts: 2422
  • California
  • Liked: 2006
  • Likes Given: 5634
This is just a proof of concept. I deciphered a few of the timestamps on the images and interpolated the rest. Due to interlacing and propagating errors, even if we get good mbs in the pframes the timestamps are still going to be gibberish most of the time. So, I propose we just redraw them. Any concerns about this?

Another route to go would be to put the frame# in the upper corner (I remember one of the previous gifs posted did this).
Shouldn't reality posts be in "Advanced concepts"?  --Nomadd

Offline Lourens

  • Full Member
  • *
  • Posts: 156
  • The Netherlands
  • Liked: 206
  • Likes Given: 304
Just a quick note before I have to go do other stuff. I'm working on brute-force bit flipping a bit. As I expected, making a good objective function is not so easy, looking at the sum of squared errors across block edges is a good first stab, but it needs more work for sure. The combinatorics is the second problem, there are just way too many combinations of bits to flip, and some sort of clever search algorithm needs to be devised. Also it would really help to know a bit more about the error correction method used by the telemetry stream.

I have one idea for speeding things up though, which I haven't tried yet but will. What if you concatenate n copies of the I-frame together into a single file, each with a different bit flipped, and then put that into FFmpeg. Would that yield n separate frames in a single call? If so, that would amortise the overhead I think.

Excellent work on the TS repair by the way, that will really help because it will allow us to take things one frame at a time. That has to be easier than solving the whole problem in one go.

Offline CuddlyRocket

This is just a proof of concept. I deciphered a few of the timestamps on the images and interpolated the rest. Due to interlacing and propagating errors, even if we get good mbs in the pframes the timestamps are still going to be gibberish most of the time. So, I propose we just redraw them. Any concerns about this?

Reminds me of the Doctor Who Restoration Team who fix old (1960s onwards) episodes. With the credits, they know what they're supposed to say; they know what font etc they were in, so they simply make crisp new sets and slot them in. They use another trick for the titles, most of which would have been identical for each episode. They make a compilation of the best bits from individual episodes and use that for all the episodes.

However, ...

Seems like a very good idea, with a few constraints:
- they need to be correct
- they are only applied at time of generating the movie file, so not to mess up de frame reconstruction process. In the end it is much better to have them fixed instead of reapplied.

IMO this is an historical document. First, you should obtain the best version of the data that is present. Ideally, someone should detail all the individual steps to get from the original data to the present so that future historians can verify its correctness (hope you're all keeping notes! :) ). Once that has been done and archived, then you can start on these and similar techniques to improve the appearance of the video.

Offline mhenderson

  • Member
  • Posts: 71
  • USA
  • Liked: 101
  • Likes Given: 18
For the final work product (if there is such a thing) I suggest a video that repeats the sequence several times interleaved with some text screens that explain who / what / why at a high level.

1) Begin with a brief paragraph about the circumstances and the challenge.
2) Full video first pass, show the original video before restoration.
3) Another paragraph, introduce the NSF forum and crowdsourced effort.
4) A series of before / after shots of individual frames to indicate progress with captions highlighting tools used, round-the-globe contributions, people contributing from their own skillsets....
5) Full video, showing a "true" restoration (no enhancements)
6) Full video, show a side-by-side presentation. in the top left corner of screen (20%), show a graphic animation of the landing process in the main frame bottom right (80%) show the restored video again. For the graphic animation, perhaps  the YouTube video by Oberg N titled "F9R in KSP" is pretty close to what we would need.   Perhaps this can be captioned with the interpretation.
7) A final pass, full screen of the team's video including enhancements such as a frame counter at the bottom right and wronkiew's nice clock in the bottom left.
8) End with credits thanking core people - slow scroll - and every username on the forum - fast scroll.
« Last Edit: 05/25/2014 03:17 pm by mhenderson »

Offline MP99

This is just a proof of concept. I deciphered a few of the timestamps on the images and interpolated the rest. Due to interlacing and propagating errors, even if we get good mbs in the pframes the timestamps are still going to be gibberish most of the time. So, I propose we just redraw them. Any concerns about this?

As just an interested onlooker, my opinion is just to correct as much as possible, but not to overwrite anything like timestamps.

Otherwise, someone somewhere is gonna do a "moon landings were fake" meme on this, highlighting the bits that don't match the rest of the stream. (Yes, yes - but you know that they will.)

Cheers, Martin

Offline meadows.st

  • Full Member
  • *
  • Posts: 158
  • Toronto ON, Canada
  • Liked: 90
  • Likes Given: 5422
This is just a proof of concept. I deciphered a few of the timestamps on the images and interpolated the rest. Due to interlacing and propagating errors, even if we get good mbs in the pframes the timestamps are still going to be gibberish most of the time. So, I propose we just redraw them. Any concerns about this?

I agree with a number of other posters (Jakusb, CR) that this is an historic document and should this be restored as carefully as possible, however, I also agree that this timestamp information is important. I would like to see any intensional additions (other than bitflips) to be included not as overlays but rather as additions in a larger screen such that the restored video is the "picture in picture" or as an overlay in an extended sequence such as mhenderson suggests here.

For the final work product (if there is such a thing) I suggest a video that repeats the sequence several times interleaved with some text screens that explain who / what / why at a high level.

1) Begin with a brief paragraph about the circumstances and the challenge.
2) Full video first pass, show the original video before restoration.
3) Another paragraph, introduce the NSF forum and crowdsourced effort.
4) A series of before / after shots of individual frames to indicate progress with captions highlighting tools used, round-the-globe contributions, people contributing from their own skillsets....
5) Full video, showing a "true" restoration (no enhancements)
6) Full video, show a side-by-side presentation. in the top left corner of screen (20%), show a graphic animation of the landing process in the main frame bottom right (80%) show the restored video again. For the graphic animation, perhaps  the YouTube video by Oberg N titled "F9R in KSP" is pretty close to what we would need.   Perhaps this can be captioned with the interpretation.
7) A final pass, full screen of the team's video including enhancements such as a frame counter at the bottom right and wronkiew's nice clock in the bottom left.
8) End with credits thanking core people - slow scroll - and every username on the forum - fast scroll.

“A little rudder far from the rocks is a lot better than a lot of rudder close to the rocks.” L. David Marquet

Offline mvpel

  • Full Member
  • ****
  • Posts: 1125
  • New Hampshire
  • Liked: 1303
  • Likes Given: 1685
This is just a proof of concept. I deciphered a few of the timestamps on the images and interpolated the rest. Due to interlacing and propagating errors, even if we get good mbs in the pframes the timestamps are still going to be gibberish most of the time. So, I propose we just redraw them. Any concerns about this?

I also agree that putting a clear timestamp on a ribbon outside of the original frame, rather than overlapping it, would be the way to go.

I agree with CuddlyRocket that this is a historical document, because I believe that the ability to reuse at least 70% of the cost of the launch vehicle - which is what this video represents - will lead to a revolutionary reduction in the cost of access to space, leading to a new era of human spaceflight and a permanent human presence not only in LEO, but on the Moon, at various Lagrange points, and ultimately Mars and beyond. In that future, these few frames of video will be treasured as the first steps down that path, and and even the gibberish will have historic significance in the decades and centuries to come. The Star Spangled banner is in tatters, but even the bits of shredded string were meticulously preserved and carefully displayed.

"Ugly programs are like ugly suspension bridges: they're much more liable to collapse than pretty ones, because the way humans (especially engineer-humans) perceive beauty is intimately related to our ability to process and understand complexity. A language that makes it hard to write elegant code makes it hard to write good code." - Eric S. Raymond

Tags:
 

Advertisement NovaTech
Advertisement Northrop Grumman
Advertisement
Advertisement Margaritaville Beach Resort South Padre Island
Advertisement Brady Kenniston
Advertisement NextSpaceflight
Advertisement Nathan Barker Photography
1