Author Topic: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread  (Read 850235 times)

Offline wronkiew

  • Full Member
  • *
  • Posts: 186
  • 34.502327, -116.971697
  • Liked: 105
  • Likes Given: 116
As soon as the work finishes on recovering the information, I guess some post processing can be done, namely noise filtering/smothering and some morphing to create intermediate frames!

Already working on it. Want to help?

A lot of work still needs to be done on iframes before that can be done.

Everything that I am working on involves transformations on the i- and p- frames. I can fold decoding improvements back into the project as they happen.

Offline Shanuson

  • Full Member
  • **
  • Posts: 258
  • Liked: 171
  • Likes Given: 410
Here are my I-Frames.
Yes for various reasons am i sure that these are the I-Frames, and that they are the correct ones.
But it can be that some data is missing from them or that there is data in them from the next frame.

As for the different Iframe mlindner and i got, i think that is because he is using raw_edit1.ts directly while i put it through tsalign and tsfix first. those i got 269 frames.
here are the pngs from those:
Also as can be seen by the numbering: most of them are 20 frames apart, some are less, indicating where something is still not ok.


Offline moralec

Those look very very nice!

Offline Shanuson

  • Full Member
  • **
  • Posts: 258
  • Liked: 171
  • Likes Given: 410
Here is the fixed.ts where those and the img-files in that zipfolder a few posts back came from:

Edit: And the raw_edit1.zip file with the header of frame169 fixed. I will delete the wrong one.
« Last Edit: 05/12/2014 11:05 PM by Shanuson »

Offline bilbo

  • Member
  • Posts: 94
  • Ground control to Major tom...
  • Liked: 27
  • Likes Given: 51
Has any work been done on the section where the stage tips over?
so far, I have only seen work on the landing, and initial splashdown.
Keep up the good work!

Offline gospacex

  • Senior Member
  • *****
  • Posts: 3028
  • Liked: 535
  • Likes Given: 604
For example, as I posted earlier, there are spikes in noise level where error rate spikes above 30% for a few thousand bytes. If this occurred in a null packet it might look to you like a mildly corrupt data packet, but in reality it is junk.

I was thinking about it and here sequence numbers are a godsend. They are often corrupted too, yes, but it would take a lot of corruption to make long run of them completely unrecognizable!

And as long as you can reasonably guess and restore them, most of the time it conclusively answers the question "is this packet data or null?" - they have separately counting sequences.

Quote
The goal of cleaning the raw stream is to give the folks here the very best mpg4-img files to tweak visually.

Exactly. My worry is that by dropping just one data ts-packet, even corrupted one, we can easily make it orders of magnitude harder to restore the frame.

ts-packet fixing operates on 188-byte blocks, macroblock fixing works on BITS! Spending a minute more on fixing a packet can save hours!

Offline saliva_sweet

  • Full Member
  • ****
  • Posts: 508
  • Liked: 380
  • Likes Given: 1247
Here are my I-Frames.

Wow. That iframe 6 is almost perfect.

Offline mlindner

  • Software Engineer
  • Full Member
  • ****
  • Posts: 1887
  • Space Capitalist
  • Silicon Valley, CA -- previously in Ann Arbor, MI
  • Liked: 465
  • Likes Given: 188
Going to port my iframe8 changes to this new iframe8.
Internal combustion engine in space. It's just a Bad Idea.TM - Robotbeat

Offline saliva_sweet

  • Full Member
  • ****
  • Posts: 508
  • Liked: 380
  • Likes Given: 1247
Going to port my iframe8 changes to this new iframe8.
How do you do that?

Offline mvpel

  • Full Member
  • ****
  • Posts: 1116
  • New Hampshire
  • Liked: 1280
  • Likes Given: 1676
Alpha 3 of my transport stream search and repair tool is finished.

Were you able to find any other sub-byte shifts aside from the one we already know about?
"Ugly programs are like ugly suspension bridges: they're much more liable to collapse than pretty ones, because the way humans (especially engineer-humans) perceive beauty is intimately related to our ability to process and understand complexity. A language that makes it hard to write elegant code makes it hard to write good code." - Eric S. Raymond

Offline theshadow27

  • Member
  • Posts: 28
  • Liked: 27
  • Likes Given: 7
I have tsfix.c matching the patterns from gospacex correctly now, and fixing sequence numbers: http://pastebin.com/dirZVxfS

It seems like there are a ton of sequence number errors, mostly for PID=1000 (which is forced more often than not as all PIDs not 0, 8191, 32, or 64 are forced to 1000).

This does make the sequence number errors in ffmpeg go away, but brings on many new errors, such as:

DTS 4301262034 < 8597009660 out of order
timestamp discontinuity -47730595911, new offset= -47770843290
etc.

The output video looks different, but I would not say better (just playing in VLC).

Offline mvpel

  • Full Member
  • ****
  • Posts: 1116
  • New Hampshire
  • Liked: 1280
  • Likes Given: 1676
The time stamp discontinuities should be able to be fixed without too much heartache since we know this is a monotonic stream. There should be a valid range of values and I suspect the flipped bits would become apparent upon inspection of the prior and subsequent stamps, much like the last four bits of the header.
"Ugly programs are like ugly suspension bridges: they're much more liable to collapse than pretty ones, because the way humans (especially engineer-humans) perceive beauty is intimately related to our ability to process and understand complexity. A language that makes it hard to write elegant code makes it hard to write good code." - Eric S. Raymond

Offline Shanuson

  • Full Member
  • **
  • Posts: 258
  • Liked: 171
  • Likes Given: 410
i fixed manually (by hand) the TS-Headers including the continuity counter throughout that 2nd IFrame (frame13 in raw_edit1.ts or frame14 in fixed.ts) in both those files.
before it had ~9KB in fixed.ts.
After fixing the sequence in fixed.ts It got a bit larger ~10Kb. But there was one 188 KB block of mostly FFs that somehow got included into fixed.ts by tsfix (or less likely tsalign) that could not be accounted for with a continuous continuity counter.
So i also handfixed raw_edit1.ts the same way, which results in the attached png and img-file :D. It is now 16kb large.

Also i am not sure some script can fix all those TS-headers correctly. Sometimes the bytes are completely wrong.

Edit: this is not frame14 at 0x85bf8, but the next one, old IF1, sorry for messing up the frames
« Last Edit: 05/13/2014 01:46 PM by Shanuson »

Offline theshadow27

  • Member
  • Posts: 28
  • Liked: 27
  • Likes Given: 7

To find out how many different TS-PIDs exists, I counted all different "0x47xxxx" appearances in raw_edit1.ts:
This are all which occured 20 times or more:
20 471be8
20 4733e8
21 470fe8
25 4703e9
63 474703
74 474020
81 474000
269 4743e8
4968 471fff
15016 4703e8


I think the first 4 are permutations of the last ones by bitflips.
4743e8 and 4703e8 have the same PID,
When one checks where 474703 is found, one sees that the next byte is E8. So the first 47 is the last byte of a previous packet, and therefore 474703 is not part of a TS-header.
So we end up with only 4 PIDs (13 bit long):
0x0000, 0x0020, 0x03E8,0x1FFF
Each PID has its own continuity counter, which should help deciding which packet is which.
 
With some different flagbits we end with the following TS-headers:
0x4740001W
0x4740201X
0x4703E81Y, 0x4703E83Y and 0x4743E83Y
0x471FFF1Z
with the 4 different counters W,X,Y,Z

For completeness, I counted PIDs by absolute bit position in the raw.ts, here are the top 50:

PID       HEX      COUNT 
1000    x03E8      15592   -- Video
8191    x1FFF      5070     -- Null packet
   0    x0000      84
  32    x0020      75
5096    x13E8      49       -- 1 bit off from 1000
1001    x03E9      48        -- 1 bit off from 1000
 904    x0388      40                -- 2 bit off from 1000
7144    x1BE8      30       -- 2 bit off from 1000
 984    x03D8      29                -- 2 bit off from 1000
1003    x03EB      29       -- 1 bit off from 1000
 616    x0268      28
3048    x0BE8      28
1512    x05E8      27
2000    x07D0      27
 996    x03E4      26
1008    x03F0      26
1002    x03EA      25
4095    x0FFF      24
 500    x01F4      23
1006    x03EE      23
4072    x0FE8      23
8190    x1FFE      23
 808    x0328      19
1004    x03EC      19
 992    x03E0      17
 232    x00E8      16
 488    x01E8      16
 968    x03C8      16
1016    x03F8      16
2024    x07E8      16
6120    x17E8      15
 872    x0368      14
 936    x03A8      14
4000    x0FA0      14
 680    x02A8      13
 744    x02E8      13
1005    x03ED      13
2047    x07FF      13
2536    x09E8      12
8167    x1FE7      11
8189    x1FFD      11
7807    x1E7F      10
 840    x0348      9
5119    x13FF      9
7423    x1CFF      9
8095    x1F9F      9
8127    x1FBF      9
8185    x1FF9      9
 125    x007D      8
 994    x03E2      8


All 1650 are in the attached csv.

I think your 4-PID theory is correct.
« Last Edit: 05/13/2014 01:13 AM by theshadow27 »

Offline mlindner

  • Software Engineer
  • Full Member
  • ****
  • Posts: 1887
  • Space Capitalist
  • Silicon Valley, CA -- previously in Ann Arbor, MI
  • Liked: 465
  • Likes Given: 188
Going to port my iframe8 changes to this new iframe8.
How do you do that?

Apparently the new and the old iframe8 are byte identical. Was hoping some of the issues would be fixed, but apparently not.
Internal combustion engine in space. It's just a Bad Idea.TM - Robotbeat

Offline mvpel

  • Full Member
  • ****
  • Posts: 1116
  • New Hampshire
  • Liked: 1280
  • Likes Given: 1676
I think your 4-PID theory is correct.

And it makes sense, too - it's not like it's a very complicated stream with a bunch of different elementary streams in it. One camera and you're done. The 0x0000 is the program association table which "contains a directory listing of all Program Map Tables," (Wikipedia) then 0x0020 is the first available number in the range of assignable data tables, and finally 1000 is a nice round number in decimal.
"Ugly programs are like ugly suspension bridges: they're much more liable to collapse than pretty ones, because the way humans (especially engineer-humans) perceive beauty is intimately related to our ability to process and understand complexity. A language that makes it hard to write elegant code makes it hard to write good code." - Eric S. Raymond

Offline Adaptation

  • Full Member
  • *
  • Posts: 160
  • Liked: 39
  • Likes Given: 38
Hi guys,

I just noticed that there are no B-frames in the file. Only I-frames and P-frames.

Quote
./ffprobe try1.ts -show_frames > frames.txt

Not sure yet what the implications are yet. Hopefully it will get easier that way.

Regards,

arnezami

That is very fortunate happenstance.  P frames will only have compound errors from any damage from the frames before them in the sequence.  B frames would have compound errors from both before and after them in the sequence. 

Offline Swatch

  • Full Member
  • **
  • Posts: 272
  • Official Aerospace Engineer as of June 13th, 2009
  • Cincinnati
    • ProjectApollo/NASSP: Virtual Systems and Flight Simulation of the Apollo Program
  • Liked: 23
  • Likes Given: 16

To find out how many different TS-PIDs exists, I counted all different "0x47xxxx" appearances in raw_edit1.ts:
...

For completeness, I counted PIDs by absolute bit position in the raw.ts, here are the top 50:
...

I think your 4-PID theory is correct.


Out of curiosity, did anyone do the counting (based on absolute bit position) on the streams that were re-aligned to 188 bits?  I was under the impression that raw.ts was not, so this might throw off your count if you're doing absolute bit position.

How many differenct PID permutations might that produce?

Then it might help for Adaptation's tool to be able to display/color based on the PID permutations.  That might also help identify what type of PID a corrupted packet is supposed to be.

Just a suggestion after digesting all of your work the last few pages.
« Last Edit: 05/13/2014 03:37 AM by Swatch »
Ex-Rocket Scientist in Training, now Rocket Scientist!
M-F trying to make the world of the future a smaller place through expanding horizons...

Offline theshadow27

  • Member
  • Posts: 28
  • Liked: 27
  • Likes Given: 7

To find out how many different TS-PIDs exists, I counted all different "0x47xxxx" appearances in raw_edit1.ts:
...

For completeness, I counted PIDs by absolute bit position in the raw.ts, here are the top 50:
...

I think your 4-PID theory is correct.


Out of curiosity, did anyone do the counting (based on absolute bit position) on the streams that were re-aligned to 188 bits?  I was under the impression that raw.ts was not, so this might throw off your count if you're doing absolute bit position.

How many differenct PID permutations might that produce?

Then it might help for Adaptation's tool to be able to display/color based on the PID permutations.  That might also help identify what type of PID a corrupted packet is supposed to be.

Just a suggestion after digesting all of your work the last few pages.

Sorry, to be clear I always run raw.ts | tsalign before doing anything on it. So the packets are lined up correctly in the stats above.

Edit: the reason I like to start with raw.ts is that it would be nice to have the final solution be generated completely automatically. Maybe then Michael will add a -spacex flag to ffmpeg that will allow raw.ts to play flawlessly ;)
« Last Edit: 05/13/2014 04:48 AM by theshadow27 »

Offline Digitalchromakey

  • Member
  • Posts: 23
  • Phuket
  • Liked: 5
  • Likes Given: 2
Here are my I-Frames.

Wow. That iframe 6 is almost perfect.
I think iframe112-mpg4.img.png is decoded from the ES video data set that was previously used to decode iframe_6 (coerced).

The time stamp in the new  iframe131.mpg4-img.png  frame is 19:35:10:23, whereas the time stamp you can see in iframe_6 (coerced) is 19:35:08:23.

As iframe_8 has a time stamp of 19:35:12:97 - a difference of nearly 5 seconds, it looks like maybe a new I frame has been teased out of the transport stream between the old iframe_6 and iframe_8.
« Last Edit: 05/13/2014 05:18 AM by Digitalchromakey »

Tags: