Author Topic: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread  (Read 958286 times)

Offline SwissCheese

  • Full Member
  • *
  • Posts: 165
  • Liked: 250
  • Likes Given: 94
Hi guys,

I just noticed that there are no B-frames in the file. Only I-frames and P-frames.

Quote
./ffprobe try1.ts -show_frames > frames.txt

Not sure yet what the implications are yet. Hopefully it will get easier that way.

Regards,

arnezami

That is very fortunate happenstance.  P frames will only have compound errors from any damage from the frames before them in the sequence.  B frames would have compound errors from both before and after them in the sequence.

Not really that 'fortunate' - there are very few if any real time encoders of the sort that would be used here that use B-frames. They need to encode in near real time, which means basing frames on frames that haven't yet appeared doesn't really work.  B-frames tends to work best when doing offline encoding. It does give a better compression ratio, but is more computationally expensive.

I am trying to recreate the full movie using the I-frames, a few "nice-looking" P-frames and fill the rest with empty P-frames. How can I convert a png into a P-frame? Or should I create the empty P-frames differently?

For the I-frames, I use for example:
ffmpeg -i frame_131.png -s 704x480 -aspect 22:15 -f image2 -r 44999/3003 frame131.mpg4-img

How does it look with just the iframes? care to join us in the IRC channel?

So I created empty P-frames with an hex editor (just taking a valid P-frame and replacing everything except the 4 first bytes with FF, somehow it seems it worked ;))

I used then the I-frames, the empty P-frames and the following nearly intact P-frames (fixed.ts): 135, 172, 188, 199, 225, 233, 251

And finally created the movie with: ffmpeg -framerate 44999/3003 -i frame%d.mpg4-img -vcodec copy fixed_ts.avi

Not really better than the gif, but it's a beginning ;)

Does someone work on a way to edit the P-frames?

Offline wronkiew

  • Full Member
  • *
  • Posts: 186
  • 34.502327, -116.971697
  • Liked: 105
  • Likes Given: 125
Could pframes be used to help fine tune iframes? If you start with a high quality iframe like 6, and then look for spots where the pframes indicate no change, could you simply carry the block from the good iframe forward (or backward) or does the compression in the iframe make that impractical?

You could in theory carry blocks forward from one i-frame to the next i-frame. However you would need a good decode for that block in every one of the ~19 p-frames in between. I think we are unlikely to see that in practice.

The p-frames also contain block updates, so you could carry those forward to update the following i-frame. That doesn't help much either, because the blocks that are most likely to get updates are the most dynamic. There are lots of updates of the water which is either spinning or full of smoke, spray, or waves. Not so many updates of the rocket body which is most in need of "filling in". Up to iframe 11, at least. This method might be useful for reconstructing some of the water after splashdown.

Offline Chris Bergin

Been talking to SpaceX today about things. Got another awesome from them on this effort, so that's for you guys.

Offline mvpel

  • Full Member
  • ****
  • Posts: 1124
  • New Hampshire
  • Liked: 1295
  • Likes Given: 1686
My thought was to deal with what I presume is low-hanging fruit: blocks with no or virtually no change from one iframe to the next. If there's no updates, does that reduce the pframe coding length enough that it'd be more likely to have dodged corruption?
« Last Edit: 05/13/2014 09:49 PM by mvpel »
"Ugly programs are like ugly suspension bridges: they're much more liable to collapse than pretty ones, because the way humans (especially engineer-humans) perceive beauty is intimately related to our ability to process and understand complexity. A language that makes it hard to write elegant code makes it hard to write good code." - Eric S. Raymond

Offline theshadow27

  • Member
  • Posts: 28
  • Liked: 27
  • Likes Given: 7
My thought was to deal with what I presume is low-hanging fruit: blocks with no or virtually no change from one iframe to the next.

Do P-frame blocks within a given frame propagate like I-frame blocks? Or is it strictly a diff from the same MB?

If we can create fake P-frames with no change, then fill in the data changes one at a time, we might make faster progress. But if they are interdependent you'd need to guess C/L data for each which could get annoying fast (and give people creative license to make the video into whatever)

Offline wronkiew

  • Full Member
  • *
  • Posts: 186
  • 34.502327, -116.971697
  • Liked: 105
  • Likes Given: 125
My thought was to deal with what I presume is low-hanging fruit: blocks with no or virtually no change from one iframe to the next. If there's no updates, does that reduce the pframe coding length enough that it'd be more likely to have dodged corruption?

Heh. It's a rocket landing. With a hover-slam. There are virtually no blocks with no change from one frame to the next.

The pframes seem to be an all-or-nothing deal as far as corruption goes, as you might expect. Some of them are quite good, but a lot of them are really bad.

Offline wronkiew

  • Full Member
  • *
  • Posts: 186
  • 34.502327, -116.971697
  • Liked: 105
  • Likes Given: 125
Do P-frame blocks within a given frame propagate like I-frame blocks? Or is it strictly a diff from the same MB?

If we can create fake P-frames with no change, then fill in the data changes one at a time, we might make faster progress. But if they are interdependent you'd need to guess C/L data for each which could get annoying fast (and give people creative license to make the video into whatever)

No, they do not seem to suffer from the same error propagation problems as the i-frames.

Edit: Actually, some of them have groups of block updates which can propagate errors. It will be much easier to fix, though, because there are fewer opportunities for errors to get compounded.
« Last Edit: 05/13/2014 10:04 PM by wronkiew »

Offline SwissCheese

  • Full Member
  • *
  • Posts: 165
  • Liked: 250
  • Likes Given: 94
Actually we can edit the P-frames the same as the I-frames, the generated PNG image then looks good, but I cannot manage to write them back as P-frames, also when using something like this:

ffmpeg -s:0 704:480 -i frameXXX.mpg4-img -f image2 frameXXX_modified.mpg4-img

I hoped it would just copy the file, but it automatically converts the data into an I-frame it seems.

I also tried to use -f mp4 or to add -vf '[out]select=eq(pict_type\,P)', with no more success.

Does someone know how to force ffmpeg to write a P-frame?

Edit:

I finally managed to do it with:
ffmpeg -s:0 704:480 -i frameXXX.mpg4-img -vcodec copy -an -f image2 frameXXX_mod.mpg4-img

Edit(2):

It copies the P-frame but does not handle the -mmb ...
« Last Edit: 05/14/2014 12:06 PM by SwissCheese »

Offline moralec

Hi All,

Just a little update on the Wiki.

The current table with iFrames is a bit of a mess, as it has frames extracted from three different sources (coerced.ts from arnezami, try1.ts from mlindner and -until minutes ago- edit5.ts from Shanuson).

The most recent version (edit5.ts) is going to have the complete set of  15 workable IFrames. In order to organize things a bit, I created a separate page on the wiki to focus on these, hoping  these will be the definitive iframes. You can access from the main page, or directly here http://spacexlanding.wikispaces.com/Progress+on+edit5.ts . The previous table still has all the edits on frames from coerced.ts  and try1.

As of today, there is only one iframe in the new page (the one of frame #14 in the video) as its the only one that is completely fixed and ready to be edited. If anyone has some time available, please work on that one.  As other iframes become available (Shanuson is working on some), they will start appearing there. The online editor 2 will also have an updated version of those iframes for those that are using that tool.

And important note regarding backward compatibility. The new set of iframes includes previously unseen frames, but also some that are  equivalent to those we have already fixed. However, due to changes in iframe sizes, positions of MBs will change. This means that for existing iframes, previous -mmb codes will be only be partially applicable.  Keep this in mind as some .mmb codes will have to get reworked.

Thanks everyone for the good work!
« Last Edit: 05/14/2014 02:06 PM by moralec »

Offline theshadow27

  • Member
  • Posts: 28
  • Liked: 27
  • Likes Given: 7
The online editor 2 will also have an updated version of those iframes for those that are using that tool.

Just to add on this: Big thanks to Iain for implementing some pretty cool new features in http://spacex.slapbet.org

- MB Errors (from the log) are now shown next to the MB X:Y display above the image when an MB is clicked. This includes DC clipping, dquant, ibpc, ac-tex, and stuffing (though I'm not even sure what they all mean ;) ).

- Total number of MB with errors are show on the right. It would be expected that a good -mbm would reduce the total number of MB with errors by more than one. This also allows a quick check to see which of two similar MBM strings is "better"

- Checkbox to enable highlighting MB with errors. This provides invaluable visual assistance in identifying where problems start. It would have been almost impossible to get this clear of a picture just looking at the log. It is interesting to watch how blocks with problems don't always look like it, and how visually defective blocks don't have any errors. This might drastically simplify debugging mbms.

So if you are using Online Editor V2 and haven't refreshed the page in a while, give it a F5. Great work Iain!
« Last Edit: 05/14/2014 04:19 PM by theshadow27 »

So I was building the latest FFMpeg for windows from here https://github.com/michaelni/FFmpeg/tree/spacexdebug1

I managed to actually get it working (yay for me) but when I was testing it I found that the output for the same image with no -mmb applied was different to the previous version of ffmpeg I was using.

OLD: http://i.imgur.com/CeHOIrB.png
NEW: http://i.imgur.com/sMInZkY.png

The output seems a bit cleaner but I don't know why it's different, if anyone knows why this is, or more importantly if it's actually expected and OK, then I'll deploy it to the Online Editor to include the latest functionality.

Offline arnezami

  • Full Member
  • **
  • Posts: 284
  • Liked: 262
  • Likes Given: 346
So I was building the latest FFMpeg for windows from here https://github.com/michaelni/FFmpeg/tree/spacexdebug1

I managed to actually get it working (yay for me) but when I was testing it I found that the output for the same image with no -mmb applied was different to the previous version of ffmpeg I was using.

OLD: http://i.imgur.com/CeHOIrB.png
NEW: http://i.imgur.com/sMInZkY.png

The output seems a bit cleaner but I don't know why it's different, if anyone knows why this is, or more importantly if it's actually expected and OK, then I'll deploy it to the Online Editor to include the latest functionality.
I'm not sure what version you were on but I also noticed some positive changes a few days ago when I updated. I think somebody did some bugfixes/improvements. Very similar to yours.
« Last Edit: 05/14/2014 08:12 PM by arnezami »

Cool, I put the latest ffmpeg on the image handler the editor uses. If someone can summarise what the changes to the editor itself should be (I think it's an additional parameter fr the macroblock operation but I don't know what it's for) then I can add it :)

Offline mvpel

  • Full Member
  • ****
  • Posts: 1124
  • New Hampshire
  • Liked: 1295
  • Likes Given: 1686
The output seems a bit cleaner but I don't know why it's different, if anyone knows why this is, or more importantly if it's actually expected and OK, then I'll deploy it to the Online Editor to include the latest functionality.

Iain, the new version of that frame you're seeing was extracted from a cleaned-up version of the transport stream, where errors in the transport stream packet identifiers, flag bits, and sequence numbers were corrected in order to allow more 184-byte pieces of the MPEG data to be incorporated. If you review from about page 38, that's where Shanuson posts the first pass at the extracted images from the error-corrected transport stream, and prior pages are where we were discussing how to get from point A to pont B (or point 0x03E8) with the protocol.

So yes, it's expected and OK, and you can go ahead and deploy.
"Ugly programs are like ugly suspension bridges: they're much more liable to collapse than pretty ones, because the way humans (especially engineer-humans) perceive beauty is intimately related to our ability to process and understand complexity. A language that makes it hard to write elegant code makes it hard to write good code." - Eric S. Raymond

Only I ran the new ffmpeg on the old images not the new ones :/

Anyway it's deployed, if it's wrong (I can't tell) I can revert easily enough.

Offline Joffan

  • NSF Irregular
  • Full Member
  • ****
  • Posts: 1376
  • Liked: 428
  • Likes Given: 1162
Only I ran the new ffmpeg on the old images not the new ones :/

Anyway it's deployed, if it's wrong (I can't tell) I can revert easily enough.

Hi Iain
Interface request for http://spacexlanding.wikispaces.com/Online+Editor : can you set it so that when the user clicks on a block in the image, the relevant part of the log is shown? and maybe fill in the existing pos value in the edit window (if it's a previously unspecified block).
Max Q for humanity becoming spacefaring

Which bits of the log do you need? There's already quite a bit there above the image.

As for auto adding pos, currently, because of the way it's built, that would end up adding a bunch of effectively useless parts to the mmb (i.e. if you just click around and don't actually do anything) There might be a way round this, I'll have a think tomorrow when I get back from work =)

Offline SwissCheese

  • Full Member
  • *
  • Posts: 165
  • Liked: 250
  • Likes Given: 94
@Michael Niedermayer:

I've been investigating the possibility of automatically setting all L1,L2,L3,L4,C1,C2 values for all macroblocks for a frame. Basicly inputting an mmb and letting a script detect "bad lines" in the picture. Moving from the right-bottom corner to the upper-top.

I'm starting to believe (using some extra logging and some manual experimentation) that this might actually be possible. If it does this would improve the quality of the frames quite dramatically and would save a lot of manual work!

However there is one thing that I find difficult to do myself. And I think you might be able to help us. Instead of using this kind of setting:

15:14:-1:0:-40:0:0:0:20

where I replace the current macroblock with a blank one and change the DC values.

But what I really want to do is this:

15:14:57217:0:-40:0:0:0:20

In other words: I want to keep the macroblock with its (structured) data as much as I can (at the very least the blocks in it which are set to 0 above) but change only the DC "starting" values. So not blanken it, but modifying it so these DC-values make sure propogation to other blocks works the same as now with -1 (therefore fixing the color and intensity of following blocks) but I also keep the current blocks inside the current macroblock.

Is something like that possible?

Regards,

arnezami
Hi guys,

I've created a new feature that implements the above.

Simply put you can now do something like this:

15:14:57217:0:-40:0:0:0:20

This will keep the macroblock (so not nullify it) bit it will change the DC values. With this you can really fix the lum/chrom issues in the current I-frames. Should have a big effect. I hope.

[edit] Forgot: it now also logs the 6 DC values for each macroblock. Which can come in handy in combination with this new feature.

Also you can do this now:

15:14:57217:0:0:0:0:0:0:0
15:14:57217:0:0:0:0:0:0:63

What this does is change direction from where a DC prediction are coming/inherited from: left or top. For each 1-bit in this number it tells it to get the DC-prediction from the top for a specific DC (4 x lum, 2 x chrom). A 0-bit means left. The number consists of 6 bit (hence max is 63). So in effect a 0 is all left, a 63 is all top. I don't know yet whether this feature is going to be very useful but it can't hurt.

I've attached the 3 source files that were changed. If somebody can check if it doesn't conflict with the existing branch (and maybe test it) and then push it to github, that would be great. Its these files btw:

libavcodec/mpegvideo.h
libavcodec/mpeg4video.h
libavcodec/h263dec.c

[edit] Fixed a bug: when using -2 it would get stuck in that mode. zip re-uploaded.

I need some sleep now ;)

Regards,

arnezami

PS. I was also trying to make a reverse-macroblock finder of sorts. So instead of telling its starting bitposition you would tell it its ending bitposition (which would usually be a starting bitposition of a mb you already have). But I ran into a lot of issues. Even when you try all possible DC-directions it still won't find the right starting position. Even when I fake the right DC values left to the block. I'm not sure why yet. Probably something I don't understand yet. This part of the code is in there but its commented out. It was also extremely error-log-heavy...

Iain, is it possible to add this in the online editor interface? It is quite useful :)
« Last Edit: 05/14/2014 10:21 PM by SwissCheese »

Yup, I'll look at that tomorrow too.

Offline Joffan

  • NSF Irregular
  • Full Member
  • ****
  • Posts: 1376
  • Liked: 428
  • Likes Given: 1162
Which bits of the log do you need? There's already quite a bit there above the image.

As for auto adding pos, currently, because of the way it's built, that would end up adding a bunch of effectively useless parts to the mmb (i.e. if you just click around and don't actually do anything) There might be a way round this, I'll have a think tomorrow when I get back from work =)

I meant the scrolling image interpretation log below the image - can you position that to show the relevant block as maybe the third line in the window? And actually now you point out the line above the image (which I had been looking for and never found for some reason), that has the current position of the block anyway so it's fairly easy to cut and paste across.
Max Q for humanity becoming spacefaring

Tags: