Author Topic: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread  (Read 850472 times)

Offline lgjy98d

  • Member
  • Posts: 10
  • Liked: 6
  • Likes Given: 0
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1360 on: 06/04/2014 04:11 PM »
You could add a hash of the mmb to the URL.

The problem is that the MMBs used to create the hash in the URL may not match the MMBs the proxy script (spxi) sees when it pulls json for the sheet.  If a user changes the MMB, the sheet will request a new image immediately, but the sheet doesn't immediately save the changed MMB to Google's servers, so spxi will return images for the wrong MMB.

I suppose spxi could calculate a hash the same way as the spreadsheet (no simple native functions in google sheets for hashing, so we'd have to get creative), and if they don't match, loop for a bit, pulling new json for the sheet, trying its best to get updated data in the allotted amount of time;  But there's no guarantee it would ever return the correct image;  It could be some time before spxi actually sees the changed MMB.

That said, it's probably better than nothing.
I'd say that Google Spreadsheet is no way replacement for online editor, but is good enough representation of state of the repair efforts that reduces amount of manual wiki page editing.

I.e. with Spreadsheets the workflow is:
* use data from spreadsheet cell (either manually copying data out and use in standalone ffmpeg, or automatically with online editor "Load Frame MMBs From Sheet" function)
* improve it until satisfied with the result
* paste updated MMBs back into spreadsheet

Frames images in the Spreadsheet should update automatically after 5-10 minutes, freeing one time to share accomplishments with this forum.

BTW, I've inserted widget embedding "latest" video of spacexlandingrestore channel in http://spacexlanding.wikispaces.com/raw_final_fixedMMB page. Should it be placed on frontpage?

Offline mhenderson

  • Member
  • Posts: 69
  • USA
  • Liked: 101
  • Likes Given: 18
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1361 on: 06/04/2014 04:12 PM »
The autobuilt video is updated with fixes.

Oh that is looking sooooooooo sweet. Congrats and thanks.

What are your thoughts on adding a second pass at a slower frame rate (3 fps perhaps?).

I played with the last iFrame -- 15 (281) -- last night and the contrast is easily improved from the grey cast it has now. That would enhance the video by not ending on a washed out image. Even shifting the first pos to 0:0:559 looks better. There may be a more elegant way to handle it. Add a couple chroma / luma touchups in rows 11 and 14 and you have a nice image.

I did not post an update on the wiki because I was trying to get more macroblocks out of the voids. Hopefully tonight unless someone else gets to it first.

Offline Helodriver

  • Full Member
  • ****
  • Posts: 948
  • Liked: 5085
  • Likes Given: 530
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1362 on: 06/04/2014 04:17 PM »
Gwynne Shotwell just used a clip of the repaired video in a presentation she gave today.
« Last Edit: 06/04/2014 04:19 PM by Helodriver »

Offline Quialiss

  • Member
  • Posts: 75
  • Liked: 82
  • Likes Given: 29
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1363 on: 06/04/2014 04:56 PM »
I've touched up frames 185-190 (part 10) to remove the large gray areas.  186 still looks rather sketchy but it's an improvement. 

The gray areas seem to be screwed up inheritance due to missing blocks, which either require manual matching to the DC values of a good block, or using -1 instead of -3 in select places. 

The darker areas (lots more of these in frames 191+) are corrected with +32 luma, which screams bitflip.

Offline JohnKiel

  • Member
  • Posts: 15
  • United States
  • Liked: 10
  • Likes Given: 0
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1364 on: 06/04/2014 05:03 PM »
Gwynne Shotwell just used a clip of the repaired video in a presentation she gave today.

I believe you're referring to this?
https://www.youtube.com/watch?feature=player_detailpage&v=sYocHwhfFDc

Around 9:40
« Last Edit: 06/05/2014 04:12 AM by JohnKiel »

Offline moralec

Isntī just amazing to see our video there? congratulations everybody!!!!


Offline Quialiss

  • Member
  • Posts: 75
  • Liked: 82
  • Likes Given: 29
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1366 on: 06/04/2014 06:30 PM »
The few seconds before 'our' video are from earlier in the descent, I'd recognize that dust spot anywhere.  No timestamps to tell when that was in relation to the start of the recovered video, though...   

Offline Alpha Control

  • Veteran
  • Full Member
  • ****
  • Posts: 1172
  • Washington, DC
  • Liked: 106
  • Likes Given: 69
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1367 on: 06/04/2014 06:37 PM »
Hi guys,

I think it would be really cool to include a repair-video of at least one iframe (but probably several) in the final video compilation. This is similar to the one from Quialiss.



But instead of just showing the luma/chroma fixes, start with the original (corrupted) iframe, then add one corrected position at a time and then show the luma/chroma fixes one at-a-time. Then I think any John Doe will understand it. (i can probably make a script that creates such a repair-video for any iframe). I think this would also work very well for an article (edited by Chris) explaining the way we repaired it and how things went chronologically.

Since the repair of the iframe 14 was so well received I will try frame 15. Its the last part of the video people will see, so I will try to make it look good. Don't get your hopes up though, this one is a toughy. ;)

Btw the reason why the repair mmb's for frame 14 (and probably also 15) are quite large is they have one or several complete "horizontal bars" of -1s" in them. This effectively removes all inherited data from the top, which you have to restore for (almost) every macroblock just below it. It's very tedious work, but in the end it's rewarding :).

Regards,

arnezami

arnezami,

I think that's a great idea. The video-editing John Does among us (myself included) would find that very illuminating. One suggestion I have would be to label the frames in the upper right corner, to provide sequencing. For example, "Orig" on the raw original frame, then "Repair 1", "Repair 2", etc. Something like that. What do you think?
Space launches attended:
Antares/Cygnus ORB-D1 Wallops Island, VA Sept 2013 | STS-123 KSC, FL March 2008 | SpaceShipOne Mojave, CA June 2004

Oh I do like the idea behind this:

https://www.youtube.com/channel/UCyZDgyJBYz3OXD3JbDJzNww

That is a great one stop page to see the progress over time.

One thing I've noticed is that the channel is putting out 'updated' videos at a really high rate... it tends to over-saturate the 'cool' effect of watching it develop.

So while a few builds a day is cool up front, perhaps it is worthwhile to delete or archive all but one video per day every couple days.  That way someone who goes there can truly appreciate the evolution of the video from initial to final form without being inundated by small deltas.

I'd be happy to add an alternate build, perhaps daily? IainCole, are you able to handle a second video update post?

I could, but having it as an alternative rather than a replacement kinda defeats the object of what Swatch was saying. Unless I upload it to a different channel (which I don't think is a good idea). Unfortunately the tool I'm using to do it doesn't allow you to put it into a specific playlist.

Offline JohnKiel

  • Member
  • Posts: 15
  • United States
  • Liked: 10
  • Likes Given: 0
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1369 on: 06/04/2014 07:17 PM »
Partially because I could, and partially because I thought it would be interesting, I created a read only view of all the frame sheets.  (Warning: it can take some time to load.)

https://docs.google.com/spreadsheets/d/1xEtLQy3tvjv3cyz_wMQssuSFwHYVbMQ5-FdSz-uCsKA/edit?pli=1#gid=0

It's kinda neat to look at the differences between the last p-frame of a part and the following i-frame.

Offline Req

  • Full Member
  • ***
  • Posts: 379
  • Liked: 338
  • Likes Given: 2581
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1370 on: 06/04/2014 07:27 PM »
Partially because I could, and partially because I thought it would be interesting, I created a read only view of all the frame sheets.  (Warning: it can take some time to load.)

https://docs.google.com/spreadsheets/d/1xEtLQy3tvjv3cyz_wMQssuSFwHYVbMQ5-FdSz-uCsKA/edit?pli=1#gid=0

It's kinda neat to look at the differences between the last p-frame of a part and the following i-frame.

This view is really cool.

Offline Quialiss

  • Member
  • Posts: 75
  • Liked: 82
  • Likes Given: 29
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1371 on: 06/04/2014 07:35 PM »
Partially because I could, and partially because I thought it would be interesting, I created a read only view of all the frame sheets.  (Warning: it can take some time to load.)

https://docs.google.com/spreadsheets/d/1xEtLQy3tvjv3cyz_wMQssuSFwHYVbMQ5-FdSz-uCsKA/edit?pli=1#gid=0

It's kinda neat to look at the differences between the last p-frame of a part and the following i-frame.

This is awesome!  It should also be helpful in doing a final tweaking of some of the i frames to better match up.  To pick a minor issue, 121 looks just a hair too dark.. not surprising considering the data gaps at the top of the frame. 

I also note that there's a surprising amount of valid p frame data in part 2 considering the i frame is complete garbage.  Engine glow in the clouds looks lovely even with only a couple rows of data.  :)
« Last Edit: 06/04/2014 07:43 PM by Quialiss »

Offline mhenderson

  • Member
  • Posts: 69
  • USA
  • Liked: 101
  • Likes Given: 18
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1372 on: 06/04/2014 08:59 PM »
I also note that there's a surprising amount of valid p frame data in part 2 considering the i frame is complete garbage.  Engine glow in the clouds looks lovely even with only a couple rows of data.  :)

Which begs the question: Is it possible to use the complete and final pFrame from the prior set as a replacement for the iFrame in the next?  Just throwin' it out there.

Online saliva_sweet

  • Full Member
  • ****
  • Posts: 508
  • Liked: 380
  • Likes Given: 1248
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1373 on: 06/04/2014 09:01 PM »
It should also be helpful in doing a final tweaking of some of the i frames to better match up.  To pick a minor issue, 121 looks just a hair too dark.. not surprising considering the data gaps at the top of the frame. 

I post an mmb here that, I think, fixes the top row (including the crucial 0:0 MB). But alas, as usual, it throws off existing dc corrections.

Offline JohnKiel

  • Member
  • Posts: 15
  • United States
  • Liked: 10
  • Likes Given: 0
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1374 on: 06/04/2014 10:03 PM »
Which begs the question: Is it possible to use the complete and final pFrame from the prior set as a replacement for the iFrame in the next?  Just throwin' it out there.

Once every bit of original data is teased out, I think the next logical step is using pixels from previous (and future) frames as an approximation for missing blocks in the current frame.

Offline princess

  • Member
  • Posts: 65
  • Liked: 106
  • Likes Given: 25
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1375 on: 06/04/2014 10:56 PM »
Is it possible to use the complete and final pFrame from the prior set as a replacement for the iFrame in the next?

I think a cooler thing to do would be to use both the blocks and the motion vectors from the previous P-frame. Just repeat the motion vector again so that you get the same approximate motion, otherwise the duplicated P-frame image would look like an obvious pause or stutter in the stream.

Offline ajmartin

  • Member
  • Posts: 4
  • Liked: 25
  • Likes Given: 3
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1376 on: 06/04/2014 11:43 PM »
I've done some more work on error modeling, and I thought I should post an update.  I determined the original values for as much of the "boilerplate" data as I could (null packets, TS headers, PAT and PMT packets, VOL headers, etc.) and located all the bit-flips there in raw.ts, then measured statistics and tested hypotheses.

The postulated 56-byte FEC coding blocks turn out to be composed of alternate 25-byte and 31-byte "sub-blocks", starting at (zero-based) offsets of the form 56n + 29 and 56n + 54.  Single bit-flips are found only in the first and last 15 bits of these blocks, with possible rare exceptions in high-error regions.  All other bit-flip errors are combinations of "triple flips" (i.e., bit shifts of 0x8003) that do not cross the block boundaries.

(That does not include frame-shift errors such as the one originally found by Shanuson, which are surprisingly common - I found at least 20 of them just in the "boilerplate" data, by amounts ranging from 1 bit to 12 bits and in both directions.  But these seem to occur only after high-error regions, which may have damaged the framing information that was used in decoding the signal.  They affect entire "sub-blocks" and garbage bits are shifted in at one end.)

I suspect that many of the single-bit flips (X mmb's) that have been found to improve the images are "workarounds" that affect the lengths of VLCs in such a way that the rest of the bit-stream is interpreted closer to its original meaning, but not true "corrections" that restore the original data.

Finding the latter will be more difficult, but may produce even better results.  (It won't be possible everywhere, of course, due to the high error rates in some parts of the stream.)  If anyone would like to make the attempt, then I would recommend a few things to improve the chance of success:
  • Use the error model: Look for "triple flips" and "single flips" according to the sub-block structure described above.  Look for adjacent pairs of "triple flips" (i.e., bit shifts of 0x18005) before more complicated combinations.
  • Work from raw.ts: This is perhaps the hardest change to make from the current approach.  But the original 56-byte alignment and the original values of corrupted TS-headers are needed to properly identify "triple flips".  (Of course, the TS-headers still need to be "virtually" fixed, or the TS-layer decoding bypassed entirely.)
  • Try to identify errors "early": Knowing the first point in the stream where the decoding is clearly erroneous will help to narrow the search.  Perhaps it would help to produce a "coefficient-by-coefficient" visualization of the decoding process for damaged macroblocks, rendering the image after each DCT coefficient is decoded (with undecoded coefficients set to zero)?
  • If it's working and you're feeling ambitious, try to automate this coefficient-by-coefficient evaluation, and use a Viterbi-like algorithm to correct more complicated error patterns.

  • Good luck to all with your continued efforts at recovering the video... the progress so far is impressive!

    Oh, and I noticed something else that seems interesting: the values of two of the header fields, PCR and vop_time_increment, consistently indicate that the video recording started 61 seconds before the start of the file.  (Presumably the rocket was too far away for a useful signal to be received during that time.)

    (Edit: corrected reference to first finding of a frame-shift error.)
    « Last Edit: 06/04/2014 11:54 PM by ajmartin »

    Offline deruch

    • Full Member
    • ****
    • Posts: 1704
    • California
    • Liked: 1284
    • Likes Given: 1879
    Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    « Reply #1377 on: 06/05/2014 12:24 AM »
    Gwynne Shotwell just used a clip of the repaired video in a presentation she gave today.

    Around 17:27

    I think it was at 9:40.
    Shouldn't reality posts be in "Advanced concepts"?  --Nomadd

    Online cscott

    • Senior Member
    • *****
    • Posts: 2636
    • Liked: 1839
    • Likes Given: 658
    Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    « Reply #1378 on: 06/05/2014 02:41 AM »
    Work from raw.ts: This is perhaps the hardest change to make from the current approach.  But the original 56-byte alignment and the original values of corrupted TS-headers are needed to properly identify "triple flips".  (Of course, the TS-headers still need to be "virtually" fixed, or the TS-layer decoding bypassed entirely.)

    I suspect the ship has already sailed and no-ones eager to go back to raw.ts.  But I bet it *would* be helpful if someone would compile a list of the adjusted start/end positions of the FEC coding blocks referenced to the current .ts and including the discovered bit shifts, etc, like princess did:
    For Quialiss and the other people tweaking MMB values, here are the changes needed to map the old bit positions to new ones:

    Offline Quialiss

    • Member
    • Posts: 75
    • Liked: 82
    • Likes Given: 29
    Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    « Reply #1379 on: 06/05/2014 11:22 AM »
    TS fixers:  How horrified would you be to know that macroblock 0:0 starts at 67 on some of the p frames instead of the normal 66?  I've noticed nothing visually unusual about the frames that start at 67, but...

    Tags: