Author Topic: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread  (Read 948249 times)

Offline mvpel

  • Full Member
  • ****
  • Posts: 1124
  • New Hampshire
  • Liked: 1295
  • Likes Given: 1686
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1000 on: 05/25/2014 04:53 PM »
This is just a proof of concept. I deciphered a few of the timestamps on the images and interpolated the rest. Due to interlacing and propagating errors, even if we get good mbs in the pframes the timestamps are still going to be gibberish most of the time. So, I propose we just redraw them. Any concerns about this?

I also agree that putting a clear timestamp on a ribbon outside of the original frame, rather than overlapping it, would be the way to go.

I agree with CuddlyRocket that this is a historical document, because I believe that the ability to reuse at least 70% of the cost of the launch vehicle - which is what this video represents - will lead to a revolutionary reduction in the cost of access to space, leading to a new era of human spaceflight and a permanent human presence not only in LEO, but on the Moon, at various Lagrange points, and ultimately Mars and beyond. In that future, these few frames of video will be treasured as the first steps down that path, and and even the gibberish will have historic significance in the decades and centuries to come. The Star Spangled banner is in tatters, but even the bits of shredded string were meticulously preserved and carefully displayed.

"Ugly programs are like ugly suspension bridges: they're much more liable to collapse than pretty ones, because the way humans (especially engineer-humans) perceive beauty is intimately related to our ability to process and understand complexity. A language that makes it hard to write elegant code makes it hard to write good code." - Eric S. Raymond

Offline wronkiew

  • Full Member
  • *
  • Posts: 186
  • 34.502327, -116.971697
  • Liked: 105
  • Likes Given: 121
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1001 on: 05/25/2014 08:00 PM »
Thanks for the feedback on the timestamp reconstruction. It sounds like adding mattes to the video and drawing timestamps and/or simulation graphics there would be preferable.

Online AncientU

  • Senior Member
  • *****
  • Posts: 6178
  • Liked: 3970
  • Likes Given: 5461
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1002 on: 05/25/2014 08:19 PM »
For the final work product (if there is such a thing) I suggest a video that repeats the sequence several times interleaved with some text screens that explain who / what / why at a high level.

1) Begin with a brief paragraph about the circumstances and the challenge.
2) Full video first pass, show the original video before restoration.
3) Another paragraph, introduce the NSF forum and crowdsourced effort.
4) A series of before / after shots of individual frames to indicate progress with captions highlighting tools used, round-the-globe contributions, people contributing from their own skillsets....
5) Full video, showing a "true" restoration (no enhancements)
6) Full video, show a side-by-side presentation. in the top left corner of screen (20%), show a graphic animation of the landing process in the main frame bottom right (80%) show the restored video again. For the graphic animation, perhaps  the YouTube video by Oberg N titled "F9R in KSP" is pretty close to what we would need.   Perhaps this can be captioned with the interpretation.
7) A final pass, full screen of the team's video including enhancements such as a frame counter at the bottom right and wronkiew's nice clock in the bottom left.
8) End with credits thanking core people - slow scroll - and every username on the forum - fast scroll.
The full restoration should be edited in with launch video, separation of first stage, supersonic retro burn, and then this landing. SpaceX has much unreleased footage and might be willing to collaborate on the effort.  The film of Wright brother's first flight in 1903 (link below) has spooky parallels with the SpaceX effort.  Even some of the phraseology is directly applicable.  Maybe this should be a separate product, but one or both will be seen by millions (billions?).
"If we shared everything [we are working on] people would think we are insane!"
-- SpaceX friend of mlindner

Online Shanuson

  • Full Member
  • **
  • Posts: 282
  • Liked: 197
  • Likes Given: 498
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1003 on: 05/25/2014 09:07 PM »
Decoding parts of the I-Frame and P-Frame Header to find out how the part after 00 00 01 b6 should look like for P-Frames.
Summary at the end!!!!

First row is  hex, second row is binary value. Then a bunch of explanation.

Video Object Layer Part: This sets certain flags used later in decoding the  video object plane header:
   00       00       01       20       00       c4       8d   
00000000 00000000 00000001 00100000 00000000 11000100 10001101

video object layer start code
random_accessible_vol
video object type indication -> simple object type
is object layer identifier
object layer verid -> 0001 -> newpred_enable is 0, also reduced_resolution_vop_enable is 0
object layer priority
aspect ratio info -> 1:1
vol control parameter
chroma format -> 4:2:0

   c0       00       46       56       c0       1e       c0       01       xx       xx       
11000000 00000000 01000110 01010110 11000000 00011110 11000000 00000001 xxxxxxxx xxxxxxxx

low delay
vdv_parameters -> yes
first half bit rate
marker bit
latter half bit rate
marker bit
first half vbv buffer size
marker bit
latter half vbv buffer size
first half vbv occupancy
marker bit
latter half vbv occupancy

   9a       fc       7c       2e       ee       2c       08       78       28       c7
10011010 11111100 01111100 00101110 11101110 00101100 00001000 01111000 00101000 11000111

marker bit
video_object_layer_shape -> rectangular
marker bit
vop_time_increment_resolution = 44999
marker_bit
fixed_vop_rate
fixed_vop_time_increment = 3003
marker_bit
video_object_layer_width = 704
marker_bit
video_object_layer_height = 480
marker_bit
interlaced -> no (it is true that this bit is 0 for all 15 I-Frames)
obmc_disable  -> yes
sprite_enable  -> no
not_8_bit -> no ( is 8 bit?) -> quant_precision = 5
quant_type -> no
complexity_estimation_disable -> yes
resync_marker_disable -> yes
data_partitioned -> no
scalability -> no
stuffing
Next comes userdata until we hit the vop_start_code 0x000001b6.

P-Frame VOP header part:
(1/5)x       xx       x(e/c)    0(even x) (4/5)x
 0x01xxxx xxxxxxxx xxxx11x0  0000xxx0      010xxxxx

vop_coding type - 00 = I-Frame ; 01 = P-frame
modulo time base = 0
marker bit
vop_time_increment <- this is increasing from frame to frame by 3003 (fixed_vop_time_increment) modulo 44999 (vop_time_increment_resolution), so it is easy to calculate what it should be.
marker bit
vop_coded -> yes
vop_rounding_type -> alternating between 1 and 0
intra_dc_vlc_thr = 000
vop_quant 00xxx
vop_fcode_forward  <- could this always be 001? my guess but i am not sure.
ref_select_code - either 01 or 00?? how do we know which?
and now follow the makroblocks....

Edit:
I-Frame VOP header part:
(1/5)x       xx       xc    (0/1)x     x
 0x01xxxx xxxxxxxx xxxx1100  000xxx0x xxxx

vop_coding type - 00 = I-Frame ; 01 = P-frame
modulo time base = 0
marker bit
vop_time_increment <- this is increasing from frame to frame by 3003 (fixed_vop_time_increment) modulo 44999 (vop_time_increment_resolution), so it is easy to calculate what it should be.
marker bit
vop_coded -> yes
vop_rounding_type -> alternating between 1 and 0
intra_dc_vlc_thr = 000
vop_quant 00xxx
vop_fcode_forward  <- could this always be 001? my guess but i am not sure.
ref_select_code - either 01 or 00?? how do we know which?
and now follow the makroblocks....


Summary:
I am surprised that the interlace flag is 0!!
I can restore parts of the VOP-header. Some things I am not sure what they should be.
Especially vop_quant, vop_fcode_forward and ref_select. Somebody with more knowledge about the decoding  of makroblocks could help here maybe?
Maybe I can see patterns when I checked all P-Frames.
« Last Edit: 05/25/2014 11:07 PM by Shanuson »

Offline Lar

  • Fan boy at large
  • Global Moderator
  • Senior Member
  • *****
  • Posts: 10121
  • Saw Gemini live on TV
  • A large LEGO storage facility ... in Michigan
  • Liked: 6963
  • Likes Given: 4754
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1004 on: 05/25/2014 10:32 PM »
For the final work product (if there is such a thing) I suggest a video that repeats the sequence several times interleaved with some text screens that explain who / what / why at a high level.

1) Begin with a brief paragraph about the circumstances and the challenge.
2) Full video first pass, show the original video before restoration.
3) Another paragraph, introduce the NSF forum and crowdsourced effort.
4) A series of before / after shots of individual frames to indicate progress with captions highlighting tools used, round-the-globe contributions, people contributing from their own skillsets....
5) Full video, showing a "true" restoration (no enhancements)
6) Full video, show a side-by-side presentation. in the top left corner of screen (20%), show a graphic animation of the landing process in the main frame bottom right (80%) show the restored video again. For the graphic animation, perhaps  the YouTube video by Oberg N titled "F9R in KSP" is pretty close to what we would need.   Perhaps this can be captioned with the interpretation.
7) A final pass, full screen of the team's video including enhancements such as a frame counter at the bottom right and wronkiew's nice clock in the bottom left.
8) End with credits thanking core people - slow scroll - and every username on the forum - fast scroll.
The full restoration should be edited in with launch video, separation of first stage, supersonic retro burn, and then this landing. SpaceX has much unreleased footage and might be willing to collaborate on the effort.  The film of Wright brother's first flight in 1903 (link below) has spooky parallels with the SpaceX effort.  Even some of the phraseology is directly applicable.  Maybe this should be a separate product, but one or both will be seen by millions (billions?).


Good list!  I support the notion of boxing the original in new larger resolution with the edges used for things. Also nice find on the Wright Brothers.  As a trivia  note, that footage is actually from their flight in 1908. The first flight was far shorter, was linear (no circling back and buzzing the spectators) and didn't have spectators.

http://en.wikipedia.org/wiki/Wright_Flyer:

Quote
The airplane left the rail, but Wilbur pulled up too sharply, stalled, and came down in about three seconds with minor damage.

Repairs after the abortive first flight took three days. When they were ready again on December 17, the wind was averaging more than 20 mph, so the brothers laid the launching rail on level ground, pointed into the wind, near their camp. This time the wind, instead of an inclined launch, helped provide the necessary airspeed for takeoff. Because Wilbur already had the first chance, Orville took his turn at the controls. His first flight lasted 12 seconds for a total distance of 120 ft (36.5 m) shorter than the wingspan of a Boeing 747, as noted by observers in the 2003 commemoration of the first flight.

But none of that matters. This video and its restoration is historic. Everyone involved has something to be quite proud of.
"I think it would be great to be born on Earth and to die on Mars. Just hopefully not at the point of impact." -Elon Musk
"We're a little bit like the dog who caught the bus" - Musk after CRS-8 S1 successfully landed on ASDS OCISLY

Offline Llian Rhydderch

  • Full Member
  • ****
  • Posts: 901
  • Terran Anglosphere
  • Liked: 612
  • Likes Given: 5976
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1005 on: 05/25/2014 10:50 PM »
Agree with mhenderson, meadows.st and mvpel:  how we put together the final/mongo historical document video, with appropriate video meta-text (seqrchable) and on-screen text, time counter added in the matte, etc. are all very important.

I also think we should include the bit-for-bit identical original file, plus attach appropriate source code for the correction input stream to ffm... program (as it has been enhanced for this effort), plus any other supplemental files that might be useful to archive.  (those could be hosted on the spacexlanding wiki, or some other suitable archive location).

One other matter to consider:  I'm hopeful we will be able to put on a Creative Commons license (perhaps share and share alike) on the finished product that would make the end video product widely shareable, including in public encyclopedias like Wikipedia.  I think this effort is sufficiently notable to be it's own Wikipedia article, with sources from several reliable sources that have discussed the effort.

I put something about the license question on the thread two weeks ago, but we were all too heads down on video recovery to think video licensing then.  Here's the link to that post

At any event, getting a clean license on this repaired file, and all the associated work, will be an important consideration that we should discuss/consider also.

Cheers,
 Llian
Re arguments from authority on NSF:  "no one is exempt from error, and errors of authority are usually the worst kind.  Taking your word for things without question is no different than a bracket design not being tested because the designer was an old hand."
"You would actually save yourself time and effort if you were to use evidence and logic to make your points instead of wrapping yourself in the royal mantle of authority.  The approach only works on sheep, not inquisitive, intelligent people."

Offline mvpel

  • Full Member
  • ****
  • Posts: 1124
  • New Hampshire
  • Liked: 1295
  • Likes Given: 1686
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1006 on: 05/25/2014 11:04 PM »
Interlaced == 0?  :o For all iframes?  :o :o But then again...

Since we've seen the clear and unmistakable evidence that it the images are interlaced right there in the macroblocks, it must be that it was captured interlaced at 30 fields per second and then encoded for transmission non-interlaced at 15 frames per second by combining the two fields. This is supported in a way by the presentation time stamp numbers - you point out that they increase by 3003 between each frame and by 60060 (20x3003) between iframes, which given the PTS rate of 90kHz, comes out to 0.03336 seconds per frame, and multiplying that by the NTSC frame rate of 29.97 fps gives you 0.99 - or as close to exactly one second as you can get in this universe.

What's also interesting here is that the presentation timestamp value is claiming to be an NTSC frame rate of 29.97 fps, but real time is advancing at twice that rate as we can see on the clock - the time between iframes is 1.3346 seconds rather than 60060 / 90000 seconds. I'm sure someone familiar with video capture and encoding will be able to enlighten us about this.

As to the question of licensing, this page about "Copyfree" is informative. One could say that SpaceX has released the raw video into the public domain by virtue of their statements and actions, but apparently that's not always sufficient depending on the jurisdiction.

Quote
Provision is identified here for an author assigning copyright to another entity, but not for dedication to the public domain -- simply relinquishing, rather than transfering, copyright. More nuanced legal analyses are generally incomplete and/or contradictory on this subject. Public domain dedication in the United States tends to "work" in practice because the author attempting to dedicate the work to the public domain is not likely to pursue legal action against someone who uses the work as though it were in the public domain, but the question of whether it is actually a public domain work is still largely unanswered. If it is not, the author's heirs may assert copyright protections in the future, or the author might if he or she has a change of heart. Furthermore, any copyright the author of a derivative work might think he or she can claim over that derivative work might be null and void, leading to other unexpected legal problems that are too complex to address here at this time.

So I think you're right, here, Llian - it's going to be important for SpaceX to explicitly designate a CC-BY-SA or something comparable in the near future, especially considering the global character of this effort of creating a "derivative work."
"Ugly programs are like ugly suspension bridges: they're much more liable to collapse than pretty ones, because the way humans (especially engineer-humans) perceive beauty is intimately related to our ability to process and understand complexity. A language that makes it hard to write elegant code makes it hard to write good code." - Eric S. Raymond

Offline Lar

  • Fan boy at large
  • Global Moderator
  • Senior Member
  • *****
  • Posts: 10121
  • Saw Gemini live on TV
  • A large LEGO storage facility ... in Michigan
  • Liked: 6963
  • Likes Given: 4754
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1007 on: 05/25/2014 11:06 PM »
At any event, getting a clean license on this repaired file, and all the associated work, will be an important consideration that we should discuss/consider also.

Very important. This can be considered a derivative work (http://en.wikipedia.org/wiki/Derivative_work ) so the copyright of the original source is important too. Was that ever ascertained?
"I think it would be great to be born on Earth and to die on Mars. Just hopefully not at the point of impact." -Elon Musk
"We're a little bit like the dog who caught the bus" - Musk after CRS-8 S1 successfully landed on ASDS OCISLY

Online Shanuson

  • Full Member
  • **
  • Posts: 282
  • Liked: 197
  • Likes Given: 498
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1008 on: 05/25/2014 11:22 PM »
Ok, finally a rawsplit_part_1_fixed.ts and  rawsplit_part_7_fixed.tsfile that gives no "important" errors with ffmpeg -debug -vstats ...
All PID 3e8 CC's are corrected, all P and I-Frame headers are corrected as far as possible (vop_quant, vop_fcode_forward and ref_select are probably wrong for some Frames)

Cheers
Shanuson

PS, somebody could add those to the online Editor.
PPS, also attached a python script that I use to check for the continuity of the CC of PID 3e8, and for the time_increment value of the VOP header.

Offline michaelni

  • Member
  • Posts: 28
  • Liked: 23
  • Likes Given: 0
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1009 on: 05/26/2014 12:37 AM »
Summary:
I am surprised that the interlace flag is 0!!
I can restore parts of the VOP-header. Some things I am not sure what they should be.
Especially vop_quant, vop_fcode_forward and ref_select. Somebody with more knowledge about the decoding  of makroblocks could help here maybe?
Maybe I can see patterns when I checked all P-Frames.

vop_quant is the initial QP, a wrong value there would lead to artifacts but the image should be recognizeable
vop_fcode is related to the VLC coding of motion vectors its value must be correct or MBs with motion vectors wont decode correctly
larger fcode values allow longer motion vectors so they might be related to more "active" frames. A simple encoder might also simply use the same fcode for all frames
ref_select is used in scalability, it should not be in the header as this should be a non scaleable stream

Offline mhenderson

  • Member
  • Posts: 69
  • USA
  • Liked: 101
  • Likes Given: 18
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1010 on: 05/26/2014 01:01 AM »
@Shanuson -

Thank you.
What do we need to know / be aware of when using this .ts?
How will this affect the legacy mmb work? Will this shift have the same effect as some of the previous changes to the .ts when there were POS changes in iFrames?
What do "part_1" and "part_7" refer to?

Thank you
 

Offline Llian Rhydderch

  • Full Member
  • ****
  • Posts: 901
  • Terran Anglosphere
  • Liked: 612
  • Likes Given: 5976
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1011 on: 05/26/2014 01:22 AM »
At any event, getting a clean license on this repaired file, and all the associated work, will be an important consideration that we should discuss/consider also.

Very important. This can be considered a derivative work (http://en.wikipedia.org/wiki/Derivative_work ) so the copyright of the original source is important too. Was that ever ascertained?

No, I don't think SpaceX ever made that explicit.  Although I doubt they would have much trouble with clarifying that license, given they've put it out to the world for crowdsourced improvement of the video file, as mvpel noted.

I have one contact in the communications/PR group there, and will connect with them to see if we can get SpaceX to formally clarify the original file on the raw.ts file that they made available, and which is serving as the master file for all the great video tech-heads who are doing so much to cleanup the stream and fix errors so we all get the historical view into this video data that this first in the history of human technology deserves:  a successful vertical soft-landing of an orbital booster stage rocket.

Edit:  I have sent the request.  Asked specifically for CC-BY-SA as Lar suggested. 
« Last Edit: 05/26/2014 02:44 AM by Llian Rhydderch »
Re arguments from authority on NSF:  "no one is exempt from error, and errors of authority are usually the worst kind.  Taking your word for things without question is no different than a bracket design not being tested because the designer was an old hand."
"You would actually save yourself time and effort if you were to use evidence and logic to make your points instead of wrapping yourself in the royal mantle of authority.  The approach only works on sheep, not inquisitive, intelligent people."

Offline Lar

  • Fan boy at large
  • Global Moderator
  • Senior Member
  • *****
  • Posts: 10121
  • Saw Gemini live on TV
  • A large LEGO storage facility ... in Michigan
  • Liked: 6963
  • Likes Given: 4754
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1012 on: 05/26/2014 01:29 AM »

I have one contact in the communications/PR group there, and will connect with them to see if we can get SpaceX to formally clarify the original file on the raw.ts file that they made available, and which is serving as the master file for all the great video tech-heads who are doing so much to cleanup the stream and fix errors so we all get the historical view into this video data that this first in the history of human technology deserves:  a successful vertical soft-landing of an orbital booster stage rocket.

That would be awesome. Please ask your contact to see if the original can be released CC-BY-SA ... please avoid NC and ND!!!! Version 4 would be best I think. Here's more on the license.

http://en.wikipedia.org/wiki/Creative_Commons_license

I don't know if a barrage of requests would be useful but maybe asking via another channel or two might be good.
"I think it would be great to be born on Earth and to die on Mars. Just hopefully not at the point of impact." -Elon Musk
"We're a little bit like the dog who caught the bus" - Musk after CRS-8 S1 successfully landed on ASDS OCISLY

Offline michaelni

  • Member
  • Posts: 28
  • Liked: 23
  • Likes Given: 0
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1013 on: 05/26/2014 03:13 AM »
Hi all
ive added some simple bitflip brute forcing support to ffmpeg at github
example: -bitflip_brute 0:28:5:3:19771:20055:2

this will try to flip all combinations of 2 bits between 19771 and 20055 and try to decode this area as 3 MBs starting at 28:5 of frame 0

that is framenum:mb_x:mb_y:mb_count:bitstart:bitend:bitcount

from the 2 experiments i did, it succeeded to find a 1 and a 2 bitflip that i intentionally introduced, in the case of the 2bits it also found 63 alteratives, the 1 flip case was uniquely found though

Offline mvpel

  • Full Member
  • ****
  • Posts: 1124
  • New Hampshire
  • Liked: 1295
  • Likes Given: 1686
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1014 on: 05/26/2014 05:25 AM »
For those of us working on the transport stream headers, it appears that the final TS header before a new frame starts, often followed by a series of null TS packets leading into the 0x4743e83_ header, uses the adaptation field to push the final bytes of the frame out to the end of the TS packet, presumably for alignment purposes. Thus, rather than

0x4703e81_

... you will see:

0x4703e83_

This indicates that there is both an adaptation field and a payload. Immediately after the header, you'll see an adaptation field length that is less than 184, followed by a "00" indicating a "null" adaptation field:

47 03 e8 39 5e 00 ff ff ff ...

This one is 94 bytes. At the end of the 94 bytes, you'll see some payload data that goes to the end of the TS packet.

So this gives us a couple of things to check - we should be able to figure out whether or not the AF length is correct based on where the ff's end, and insure that the AF bit is set in the last packet of the frame if it doesn't end on a clean TS packet boundary.
"Ugly programs are like ugly suspension bridges: they're much more liable to collapse than pretty ones, because the way humans (especially engineer-humans) perceive beauty is intimately related to our ability to process and understand complexity. A language that makes it hard to write elegant code makes it hard to write good code." - Eric S. Raymond

Offline deruch

  • Senior Member
  • *****
  • Posts: 2222
  • California
  • Liked: 1733
  • Likes Given: 3597
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1015 on: 05/26/2014 05:53 AM »
What do "part_1" and "part_7" refer to?

The work on .ts headers is being done on raw_edit8.ts that was split up into 15 parts by Shanuson in a previous post.  The wiki page http://spacexlanding.wikispaces.com/Extraction+of+p-frames explains.
Shouldn't reality posts be in "Advanced concepts"?  --Nomadd

Offline mvpel

  • Full Member
  • ****
  • Posts: 1124
  • New Hampshire
  • Liked: 1295
  • Likes Given: 1686
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1016 on: 05/26/2014 06:01 AM »
I cleaned the transport stream headers on part 131, attached below.

My notes during the cleaning:

Packet 163 at offset 30644 is a VOP start frame, but it needs a bit of attention. Unlike the other start packets, it has an 11-byte adaptation field and the 000001e0 starts 4 bytes later in the ts packet at offset 0x7708. I adjusted the AF length to allow the 0x000001e0 to line up as needed, but I'm not sure why this one is different and what the extra four bytes are for.

The four bytes in question, 0x181803c0, register as padding based on the adaptation field's bitmap, so theoretically should be 0xff, but I'm not sure if it might be something else that's supposed to fall in those four bytes. But it does appear that a reasonable-looking pframe came out.

At offset 0x12d18, this appears to be a null packet, but the continuity counter is off, the count goes from 0x1 to 0x2 across this gap in raw.ts as well:    null packet at 0x24a7ec and at 0x251324  However in raw.ts, this packet at 0x24ea04 reads 0x2e3ffe2f - much closer to 0x471fff1x than a 0x3e8 packet. Setting it to 0x471fff1f at 0x12d18 in fixed_edit8_part_131.ts - looks like it turned out okay.

Offset 0x21004 - a clear null packet with an incorrect CC - 0xd to 0x0 and later 0xe - perhaps an alignment packet inserted by Shanuson?

Offset 0x401ec - the adapation field length 0x77 (119) may be wrong here, but can't tell due to corruption. The last ff is at 0x4021A, which is 42 length, 0x2a, but we then have 0x7F (1 bit) 0xfc (1 bit) then a few more ff's. Leaving it at 0x77 since it doesn't exceed the packet length. The data at this AF length starts with 0x95591f6b.

The pframes look  promising, I've attached one of the better ones for your enjoyment - it's flames lapping out to the left side, and you can see the outline of the legs as the lighting changed from the previous frame.

This still needs work in the MPEG headers, but we should have a solid transport stream.

Edit: (Depressingly, the pframes that emerge from ffmpeg checksum as identical to the pframes out of the original ts file, but I guess you have to actually do the cleaning in order to find that out.)
« Last Edit: 05/26/2014 06:25 AM by mvpel »
"Ugly programs are like ugly suspension bridges: they're much more liable to collapse than pretty ones, because the way humans (especially engineer-humans) perceive beauty is intimately related to our ability to process and understand complexity. A language that makes it hard to write elegant code makes it hard to write good code." - Eric S. Raymond

Offline SwissCheese

  • Full Member
  • *
  • Posts: 164
  • Liked: 249
  • Likes Given: 90
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1017 on: 05/26/2014 07:53 AM »
Correcting the p-frames is often quite tricky because we don't see very well where we have "correct" information. We can also easily put some sequences at a slightly incorrect position. Would it be possible to add a slider enhancing the image contrast? If we could see a little better, it would already help a lot.

Online Shanuson

  • Full Member
  • **
  • Posts: 282
  • Liked: 197
  • Likes Given: 498
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1018 on: 05/26/2014 08:31 AM »
Summary:
I am surprised that the interlace flag is 0!!
I can restore parts of the VOP-header. Some things I am not sure what they should be.
Especially vop_quant, vop_fcode_forward and ref_select. Somebody with more knowledge about the decoding  of makroblocks could help here maybe?
Maybe I can see patterns when I checked all P-Frames.

vop_quant is the initial QP, a wrong value there would lead to artifacts but the image should be recognizeable
vop_fcode is related to the VLC coding of motion vectors its value must be correct or MBs with motion vectors wont decode correctly
larger fcode values allow longer motion vectors so they might be related to more "active" frames. A simple encoder might also simply use the same fcode for all frames
ref_select is used in scalability, it should not be in the header as this should be a non scaleable stream

Thanks for the explanation. I was wondering about the ref_select, I overlooked a ! when i read the ISO. Thank for pointing it out. I will correct my post.
So one has to fix vop_quant first, before one works on a P-Frame? Will the correct value be recognizable from a wrong one?

For fcode i would assume it is 001 for all P-frames at the moment. Will check this for more active and correct parts of the video.


@Shanuson -

Thank you.
a)What do we need to know / be aware of when using this .ts?
b) How will this affect the legacy mmb work? Will this shift have the same effect as some of the previous changes to the .ts  when there were POS changes in iFrames?
c) What do "part_1" and "part_7" refer to?

Thank you

a) that this is most like the most correct raw file you will get. That for parts where I fixed the P-Frame header (same is true for the I-frames work i did before) vop_quant could be wrong. Maybe that is why 0:0:-1 helps sometimes a lot? When all 15 parts are fixed to my satisfaction, I will combine them back together and we have the best possible raw-file.
b) Some will still work. some wont work without shifting some things. Sry.
c) Part of the Stream following I-Frame 1 and 7. They still contain one overlapping TS packet at start and end. 

We should also look at the parts where it already found 19 P-Frames. Tsfix does not find all the right PIDs i think. so maybe there is still some stuff hidden somewhere.

For those of us working on the transport stream headers, it appears that the final TS header before a new frame starts, often followed by a series of null TS packets leading into the 0x4743e83_ header, uses the adaptation field to push the final bytes of the frame out to the end of the TS packet, presumably for alignment purposes.
<snip>

Thanks for pointing it out again and more clearly than I did before. It could be that the last few MB's of a frame are part of that last packet. So identifying this is important for the last row of a file.

Cheers,
Shanuson
 

Offline Microphobe

  • Member
  • Posts: 5
  • Liked: 9
  • Likes Given: 4
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1019 on: 05/26/2014 09:23 AM »
For the final work product (if there is such a thing) I suggest a video that repeats the sequence several times interleaved with some text screens that explain who / what / why at a high level.

1) Begin with a brief paragraph about the circumstances and the challenge.
2) Full video first pass, show the original video before restoration.
3) Another paragraph, introduce the NSF forum and crowdsourced effort.
4) A series of before / after shots of individual frames to indicate progress with captions highlighting tools used, round-the-globe contributions, people contributing from their own skillsets....
5) Full video, showing a "true" restoration (no enhancements)
6) Full video, show a side-by-side presentation. in the top left corner of screen (20%), show a graphic animation of the landing process in the main frame bottom right (80%) show the restored video again. For the graphic animation, perhaps  the YouTube video by Oberg N titled "F9R in KSP" is pretty close to what we would need.   Perhaps this can be captioned with the interpretation.
7) A final pass, full screen of the team's video including enhancements such as a frame counter at the bottom right and wronkiew's nice clock in the bottom left.
8) End with credits thanking core people - slow scroll - and every username on the forum - fast scroll.

Hi, I think you are correct to doubt that there will be a "final" version, my 2c is to plan a "release" sometime prior to the next Falcon 9 mission, but also plan for later releases.

I guess what I am trying to say is, it might be best to assemble video releases in the most automated way possible, because it probably won't be the last time it has to be done.

Disclaimer: I don't really know how much work it involves to assemble a polished video :)

Tags: