Author Topic: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread  (Read 942265 times)

Offline princess

  • Member
  • Posts: 65
  • Liked: 106
  • Likes Given: 25
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1240 on: 06/01/2014 11:30 AM »
(3405,272), (13493,1744), (27000,1824), (53655,2480)

The first values are approximate value, they just have to fall between two MMB's from the previous work, the second is the exact, cumulative number of bits added in the new TS file compared to the old one.

At the TS level everything is byte aligned, so the number of bits inserted and the insertion point will always be a multiple of 8 bits.

Just to clarify - in your example above, are the first numbers bit positions in the old TS file? I can make the output in this format for you, listing everything in bit positions instead of bytes. I'll let you know when I have something, but I'm fairly busy today so I can get to work on it more seriously in the evening.

Offline Quialiss

  • Member
  • Posts: 75
  • Liked: 82
  • Likes Given: 29
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1241 on: 06/01/2014 11:50 AM »
At the TS level everything is byte aligned, so the number of bits inserted and the insertion point will always be a multiple of 8 bits.
I was mostly just clarifying that in my small set, the number of bits inserted is accurate, but the insertion point is not precisely accurate, so you shouldn't expect to match it. 

Just to clarify - in your example above, are the first numbers bit positions in the old TS file? I can make the output in this format for you, listing everything in bit positions instead of bytes. I'll let you know when I have something, but I'm fairly busy today so I can get to work on it more seriously in the evening.

Yes, the bit positions are from the old TS file.  It doesn't matter all that much so long as I know which one it is, it just changes the sign of the offset. 



Offline princess

  • Member
  • Posts: 65
  • Liked: 106
  • Likes Given: 25
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1242 on: 06/01/2014 12:30 PM »
Hi Quialiss,

I made a quick edit to my TS program so that for each data packet it dumps the MP4 frame it's assigned to (identified by its PCR timestamp) and the byte offset in the MP4 stream. I've dumped I-frame 101 (PCR=6262246) for raw_edit8.ts and raw_fixed_final.ts. If you open up the two fixes side by side (either in two editors or with a diff tool like WinDiff or Meld), then you can look at the old file to see the old byte offsets, and then look across to the new file to see what the new byte offsets are.

As an example, the first few lines of the old file are:

PID 0x03e8 PUSI AF[7] Pay13:000001e000008180 MP4 [email protected] bytes 0-160
PID 0x03e8 Pay14:a74b71c963a5402e MP4 [email protected] bytes 160-344
PID 0x03e8 Pay15:955c594e1a752f33 MP4 [email protected] bytes 344-528
Invalid PID 0x0be2 AF[33]
PID 0x03e8 Pay1:e6d894f02f3ce3ef MP4 [email protected] bytes 528-712


The corresponding lines in the new file are:

PID 0x03e8 PUSI AF[7] Pay13:000001e000008180 MP4 [email protected] bytes 0-160
PID 0x03e8 Pay14:a74b71c963a5402e MP4 [email protected] bytes 160-344
PID 0x03e8 Pay15:955c594e1a752f33 MP4 [email protected] bytes 344-528
PID 0x03e8 Pay0:21623132ae67d9e0 MP4 [email protected] bytes 528-712
PID 0x03e8 Pay1:e6d894f02f3ce3ef MP4 [email protected] bytes 712-896


This shows the first change is an insertion of 184 bytes at offset 528 bytes.

I've attached the two text files as a ZIP file, I did them in Linux so they'll have Unix line endings. Sorry it's in byte offsets instead of bits, I can fix that later when I have some time. I'm not sure the offsets that you already found match up to these, so let me know if I've made a mistake anywhere.

Online cartman

  • Full Member
  • ****
  • Posts: 505
  • Greece
  • Liked: 485
  • Likes Given: 2478
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1243 on: 06/01/2014 01:59 PM »
Ok, so i'm a linux guy with web servers and the works, and would like to help. First of all, i would like to help with recovering the video. Is there an updated guide of where to start? should i use arnezami's latest build?
On the wiki there is a section calls "Work in Progress " which has an "ACTIVE" part which contains two links: one to a tutorial one to where we post our results.

There is also a "Reconstruction Tools" section in the wiki. The two top links (one for P-frames one for I-frames) go to the online editors. There you can try to fix the frames.

You don't need my latest addition to the ffmpeg-tool to help for now. What we are developing is for making the online tools more powerful.
Thanks for your answer, somehow i missed it before.

Offline Quialiss

  • Member
  • Posts: 75
  • Liked: 82
  • Likes Given: 29
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1244 on: 06/01/2014 02:02 PM »
Hi princess,

The insertions shown by the log file are vastly larger than the ones I figured out, I think we may have to go deeper!  For reference, the four manually found values for bytes of data inserted are 34, 218, 228, 310.  10 bytes for the smallest change, that might not even be a single macroblock added.  It doesn't look like the level of detail is there in that log.

I don't know nearly enough about mpeg4 encoding, is it possible that the data packets that show as errors were partially decoded, and thus the actual amount of new data inserted is less than shown? 

I'm heading out for the day, I will get back to this later this evening! 
« Last Edit: 06/01/2014 02:04 PM by Quialiss »

Offline princess

  • Member
  • Posts: 65
  • Liked: 106
  • Likes Given: 25
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1245 on: 06/01/2014 03:36 PM »
Hi Quialiss,

An offset of 34 bytes is still within the first TS packet of frame 101 (which contains 160 bytes of MPEG4), and as far as I can see that packet hasn't changed in the two files. You can check it with a hex editor, the offsets you need are:

raw_edit8.ts: Frame 101 starts at TS packet 9279, offset 0x001a9e44 into the file
raw_final_fixed.ts: Frame 101 starts at TS packet 7923, offset 0x0016ba74 into the file

In both cases, the first packet contains the following data:

00000000  47 43 e8 3d 07 10 00 2f  c6 f3 7e 00 00 00 01 e0  |GC.=.../..~.....|
00000010  00 00 81 80 07 21 01 7d  cd ad ff ff 00 00 01 b0  |.....!.}........|
00000020  03 00 00 01 b5 0d 0f 00  00 01 00 00 00 01 20 00  |.............. .|
00000030  c4 8d c0 00 46 56 c0 1e  c0 01 f0 04 9a fc 7c 2e  |....FV........|.|
00000040  ee 2c 08 78 28 c7 00 00  01 b2 65 6d 34 76 20 34  |.,.x(.....em4v 4|
00000050  2e 33 2e 30 2e 30 00 c3  ff 00 00 01 b6 14 70 dc  |.3.0.0........p.|
00000060  0e 61 78 5e 61 fd 94 64  0c 78 aa b4 74 f4 40 e8  |.ax^[email protected]|
00000070  eb 3f 21 5f f5 ec 17 fd  f6 9f ab ba 5b a4 f4 90  |.?!_........[...|
00000080  de 31 dd b1 c5 3d 38 b9  fd 1d 62 0d c8 39 00 4b  |.1...=8...b..9.K|
00000090  2f 3d c9 bb 41 71 84 ff  4e 1f 93 ff de ac 47 39  |/=..Aq..N.....G9|
000000a0  d8 66 0e 21 9d 73 43 20  71 03 0b 22 18 96 5f 69  |.f.!.sC q..".._i|
000000b0  ee f7 26 9a 0a ab 8c fb  1d 73 fa 98              |..&......s..|


If you're on Linux, the following commands show you the TS frame in each case:

dd if=raw_edit8.ts bs=1 skip=1744452 count=188 | hexdump -C
dd if=raw_final_fixed.ts bs=1 skip=1489524 count=188 | hexdump -C

In each case I've just converted the offset into decimal as the dd command doesn't like hex numbers. Please check I'm correct and let me know if I've missed anything, but when I try that on my machine I get identical output.

If this is correct then the numbers you're using don't correspond exactly to the bit offset from the start of the frame - we would then need to look into what exactly the numbers mean in ffmpeg, which unfortunately I don't have much experience with! If we can work out the correspondence between ffmpeg's numbers and the actual byte position in the raw file, we can then calculate the proper offsets.

Offline MTom

  • Full Member
  • ***
  • Posts: 378
  • EU / Hungary
  • Liked: 107
  • Likes Given: 435
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1246 on: 06/01/2014 04:03 PM »
In the mean time, I've seen a few posts by people saying that they have no idea what's going on. That's usually my position (especially with ISS stuff, acronym soup alert!), but now I'm coming from the other side, so I'll make an attempt at an explanation of all the stuff that's going on here. Hopefully, that will allow more people to help fix things. I'll probably make a mistake here and there and/or oversimplify something, so if you spot an error, let me know and I'll fix the text.

<snip>

Wauw!! Great explanation man! :D

This needs to be put on a prominent place on the wiki. For sure.

Thanks for that. A lot.

Regards,

arnezami

Not only the post but the whole thread have to placed on the wiki after finishing: the learning cycles from the beginning has also worth to read if someone really wants to learn about MPEG4.

Btw, this "project" have an other very interesting aspect: the way how it is managed without really managing it.

Explanation: My profession is how to organize companies (actual working at an automobile factory, over 10t employee). The way how here self-organizing works, the dynamic of evolving the roles, aligning and sharing the work between participants, the continuous improvements in the working processes and rapidly integrating into the project is incredible.
As mentioned from others before, the key is the motivation of the participants. But there are many places (and companies) overall where motivated people come together. Such an effective working group comes not automatically as a result. And the good examples remain often hidden. We hear about them (companies, projects, etc) but there aren't insider information to reach how it really works (and built up from the beginning).

I read this thread from the beginning and I enjoyed it very well: it is a great learning effect also for me in my profession. Thanks for the opportunity being here.

And of course: Respect for your huge work (if I have to guess, it could be between 1000 and 2000 working hours until now, maybe more...), and for these results!
« Last Edit: 06/01/2014 04:10 PM by MTom »

Offline saliva_sweet

  • Full Member
  • ****
  • Posts: 552
  • Liked: 422
  • Likes Given: 1325
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1247 on: 06/01/2014 04:30 PM »
we would then need to look into what exactly the numbers mean in ffmpeg

Bitpositions in mmb command are applied after all TS and PES level data is stripped. They are basically bit positions in the mpeg4-img files. For iframes they start with the Visual object layer start code and for P frames the VOP start code. Counting starts from 1 (there is no bitposition zero). Inclusion of a standard TS (no AF) packet in the middle of the frame should then I believe theoretically introduce a 184*8=1472 bit shift in subsequent positions. If there's anything going on involving adaptation fields, then a different number of bytes may be included (or excluded) I believe.

Looks like only one of the four insertions that Quialiss has identified corresponds to such an event. Looks to me there's quite a lot of AF jiggerypokery going on in the new TS update.
« Last Edit: 06/01/2014 04:43 PM by saliva_sweet »

Offline Chris Bergin

Hey guys, here's another shoutout for your work, from Elon! :)



Go to 23 mins 27 seconds! :)

Offline Oersted

  • Member
  • Full Member
  • ****
  • Posts: 874
  • Liked: 467
  • Likes Given: 277
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1249 on: 06/01/2014 07:38 PM »
Hehe, you owe that interviewer a beer, Chris...  :-)

Offline saliva_sweet

  • Full Member
  • ****
  • Posts: 552
  • Liked: 422
  • Likes Given: 1325
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1250 on: 06/01/2014 07:42 PM »
I made a quick edit to my TS program so that for each data packet it dumps the MP4 frame it's assigned to (identified by its PCR timestamp) and the byte offset in the MP4 stream. I've dumped I-frame 101 (PCR=6262246) for raw_edit8.ts and raw_fixed_final.ts. If you open up the two fixes side by side (either in two editors or with a diff tool like WinDiff or Meld), then you can look at the old file to see the old byte offsets, and then look across to the new file to see what the new byte offsets are.

I looked at the logs and I think I see what's going on. Surprisingly ffmpeg does not ignore packets with wrong PIDs but includes them into the stream. The difference between the new and old ts file interpretation comes form adaptation fields that are disabled in the new .ts file. The bytes in them were previously discarded, but are now interpreted as mpeg4 data. Here are the first 4 packets with bad PID and AF from the old TS file from your log:
Invalid PID 0x0be2 AF[33]
Invalid PID 0x035e SCR AF[242] Pay4: PCR: 5405681769 (gap 5399419523)
Invalid PID 0x03b9 PUSI SCR AF[9] PCR: 5586719654 (gap 181037885)
Invalid PID 0x00bf TEI AF[81]

The second AF size is bigger than TS packet and the whole 184 bytes gets discarded in the old version and included in the new one. The AF sizes if we add the AF size byte correspond to the insertions identified by Quialiss (34, 184, 10 and 82 bytes).

Offline Helodriver

  • Full Member
  • ****
  • Posts: 984
  • Liked: 5445
  • Likes Given: 577
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1251 on: 06/01/2014 07:48 PM »
Hehe, you owe that interviewer a beer, Chris...  :-)

He already gave me lifetime L2 membership, so I think were all good.  ;)

Offline mhenderson

  • Member
  • Posts: 69
  • USA
  • Liked: 101
  • Likes Given: 18
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1252 on: 06/01/2014 08:09 PM »
Transcript of Musk Q&A -

Interviewer: Can you talk about the splashdown of the CRS-3 first stage out off in
the Atlantic? What you've learned from the telemetry and the video you might have
gotten back?

Musk: Oh uh, (head cocks back, eyes roll skyward) Yes, as far as the soft landing of
the boost stage.

Interviewer: Yes

Musk: It was interesting when we got the corrupted video back because we
really actually had a really difficult time getting the telemetry back. I'll say a
funny thing. Normally we get the telemetry off of a boat. And we also have a
backup a Navy P3 that was going to go up. And the P3 got iced up and the boats
couldn't go out so actually I sent my plane up. And we had to design and build and
fabricate an antenna that exactly fit in the window of the plane. So we started off
with a pizza dish and were able to <unintelligible> an antenna with a pizza dish and
point it out the window to get the thing.

Interviewer: How about the decoding of that corrupted file? How is that going?

Musk: Yeah, so we got ... the data came through really well but the video was corrupted
because unfortunately when you compress video it's .... it's hard to uncorrupt a
compressed video because you need to figure out the compression on the thing and all
sorts of things.

Interviewer: Right, right, right

Musk: We weren't able to get very far but we put the video online and we sort of
crowdsourced the cleanup of the video. And umm people did a really good job of fixing
it. And I actually created a link to their latest thing. And it was a bunch of people
on the NASASpaceFlight forum that were able to fix the video.

Interviewer: That's really cool

<Next question is asked>

Interviewer: Chris, that's for you.



Offline hutchel

  • Overzealous Enthusiast
  • Full Member
  • *
  • Posts: 104
  • Washington, DC
  • Liked: 5
  • Likes Given: 36
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1253 on: 06/01/2014 09:25 PM »
OK so I used to consider myself good with coding.  I am a complete newbie when I comes to MPEG4, so I tried the online editor and it's still beyond mortal man w/o lots of explanation.

What I am proposing to get more eyes working on this is to focus on getting correct data packets and then the rest of the crowd can start flipping bits in the payload.  What I am hoping for (it may not be possible due to the compression) is to be able to break the payload down to specific parts and display them as such.  Then if the tool can allow us to concentrate on the payload part by part, mortal eyes may have a better chance of flipping bits to start fixing the rest of the picture.  But the key is to get us to the point where we can focus on the parts of the payload as opposed to try to guess at the whole payload as appears to be the case right now.

My other comment is as the online editor gets used by more people - you probably need to moderate inputs before committing them to a master database so we don't inadvertently screw something up and 2. we guard against some malicious event (not that I am expecting that, but stranger things have happened).

Standing by to help, but we have to get this a bit less technical.

Offline SwissCheese

  • Full Member
  • *
  • Posts: 164
  • Liked: 249
  • Likes Given: 90
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1254 on: 06/01/2014 09:52 PM »
I just finished cleaning up the remaining frames in part 11 (201-220). There is an issue with this part: when ffmpeg does not find the end of a p-frame, it just does not use the frame. Usually we can find the end of a frame and paste it at the end, but for frame 218 I can't manage to find it.

Can someone help? Would it be possible to "force" an end of frame MB?

Edit: Could find the end of that frame. But it's not very practical...

Anyway so part 11 is complete!
« Last Edit: 06/01/2014 10:51 PM by SwissCheese »

Offline princess

  • Member
  • Posts: 65
  • Liked: 106
  • Likes Given: 25
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1255 on: 06/01/2014 11:05 PM »
Surprisingly ffmpeg does not ignore packets with wrong PIDs but includes them into the stream. The difference between the new and old ts file interpretation comes form adaptation fields that are disabled in the new .ts file. The bytes in them were previously discarded, but are now interpreted as mpeg4 data.

Firstly, thank you for catching that and working it out. The bit offsets from Quialiss make sense now.

Secondly - OMG, ffmpeg puts invalid PIDs into the MPEG4 stream? Seriously? If that's true it means we will probably need to shift to using the raw_final_fixed_allpids.ts file that I posted earlier in the thread, because the existing raw_final_fixed.ts file has invalid PIDs in the padding sections! These could be corrupting the MPEG4 data!

Link to my post with raw_final_fixed_allpids.ts: http://forum.nasaspaceflight.com/index.php?topic=34597.msg1206016#msg1206016

Calling Shanuson: what's your opinion on this?
« Last Edit: 06/01/2014 11:19 PM by princess »

Offline princess

  • Member
  • Posts: 65
  • Liked: 106
  • Likes Given: 25
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1256 on: 06/01/2014 11:12 PM »
I just finished cleaning up the remaining frames in part 11 (201-220). There is an issue with this part: when ffmpeg does not find the end of a p-frame, it just does not use the frame. Usually we can find the end of a frame and paste it at the end, but for frame 218 I can't manage to find it.

Can someone help?

Frame 218 has a known TS-level corruption at the end, see my post here: http://forum.nasaspaceflight.com/index.php?topic=34597.msg1207719#msg1207719

If ffmpeg is treating invalid packets as data we might need to move to new TS files anyway, so we could use your help identifying the correct bytes in P-frame 218 so we can fix that at the same time too.

Offline SwissCheese

  • Full Member
  • *
  • Posts: 164
  • Liked: 249
  • Likes Given: 90
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1257 on: 06/01/2014 11:29 PM »
I just finished cleaning up the remaining frames in part 11 (201-220). There is an issue with this part: when ffmpeg does not find the end of a p-frame, it just does not use the frame. Usually we can find the end of a frame and paste it at the end, but for frame 218 I can't manage to find it.

Can someone help?

Frame 218 has a known TS-level corruption at the end, see my post here: http://forum.nasaspaceflight.com/index.php?topic=34597.msg1207719#msg1207719

If ffmpeg is treating invalid packets as data we might need to move to new TS files anyway, so we could use your help identifying the correct bytes in P-frame 218 so we can fix that at the same time too.

I have to say I would prefer not to move to another ts file again... :P I prefer to skip bad data with mmb commands than to do everything again.

Offline princess

  • Member
  • Posts: 65
  • Liked: 106
  • Likes Given: 25
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1258 on: 06/01/2014 11:36 PM »
I have to say I would prefer not to move to another ts file again... :P I prefer to skip bad data with mmb commands than to do everything again.

I can quite understand that! I've just checked through raw_final_fixed.ts and it seems that all the invalid packets are between frames. My (limited) understanding of MPEG4 is that once one frame ends it will look for a specific bit pattern to start the next frame, so corruption between frames shouldn't be a problem.

That all means that we probably don't need to shift to new TS files, you'll be happy to know ;)

But yes, some frames (listed in my post) will need specific tweaks at the ends. If the -mmb command can do that easily then that's fine.

Online Shanuson

  • Full Member
  • **
  • Posts: 282
  • Liked: 197
  • Likes Given: 491
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1259 on: 06/01/2014 11:40 PM »
Surprisingly ffmpeg does not ignore packets with wrong PIDs but includes them into the stream. The difference between the new and old ts file interpretation comes form adaptation fields that are disabled in the new .ts file. The bytes in them were previously discarded, but are now interpreted as mpeg4 data.

Firstly, thank you for catching that and working it out. The bit offsets from Quialiss make sense now.

Secondly - OMG, ffmpeg puts invalid PIDs into the MPEG4 stream? Seriously? If that's true it means we will probably need to shift to using the raw_final_fixed_allpids.ts file that I posted earlier in the thread, because the existing raw_final_fixed.ts file has invalid PIDs in the padding sections! These could be corrupting the MPEG4 data!

Link to my post with raw_final_fixed_allpids.ts: http://forum.nasaspaceflight.com/index.php?topic=34597.msg1206016#msg1206016

Calling Shanuson: what's your opinion on this?

I never expected that ffmpeg could do something like that. If you have more than one stream in there, how should it decide to which it belongs? I would understand that it needs the right PID but the AF flag has to be set right too to get all of the data.
I think i also tested that a TS-packet with the wrong PID would get neglected. But I could remember it wrong. It should easy to test though. We could compare the length of the frame files between your version and mine to decide it for example. Or change one header to a wrong PID and see what ffmpeg does with the data.

Also I was maybe a bit to eager to put final in that file name.  Like you wrote, we should at least correct the AF-length where we can or reduce it to 1 if we cant decide what the length is at the end of a frame. That will probably be the last things done on the TS-level side since you fixed the rest of the Ts-headers I were to lazy to fix.

But if the wrong PIDs do not get included into a frame (and i think they don't) this edit would only add maybe a few bytes at the end and thus only be really minor changes to a frame and the mmb - code if at all.

Attached you find the python script I used to split the raw-file into parts.

I have to get back to my real work so I can not really help much from here on out.

Cheers
Shanuson

Tags: