Author Topic: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread  (Read 1201461 times)

Offline wronkiew

  • Full Member
  • *
  • Posts: 186
  • 34.502327, -116.971697
  • Liked: 105
  • Likes Given: 125
Could pframes be used to help fine tune iframes? If you start with a high quality iframe like 6, and then look for spots where the pframes indicate no change, could you simply carry the block from the good iframe forward (or backward) or does the compression in the iframe make that impractical?

You could in theory carry blocks forward from one i-frame to the next i-frame. However you would need a good decode for that block in every one of the ~19 p-frames in between. I think we are unlikely to see that in practice.

The p-frames also contain block updates, so you could carry those forward to update the following i-frame. That doesn't help much either, because the blocks that are most likely to get updates are the most dynamic. There are lots of updates of the water which is either spinning or full of smoke, spray, or waves. Not so many updates of the rocket body which is most in need of "filling in". Up to iframe 11, at least. This method might be useful for reconstructing some of the water after splashdown.

Offline Chris Bergin

Been talking to SpaceX today about things. Got another awesome from them on this effort, so that's for you guys.
Support NSF via L2 -- Help improve NSF -- Site Rules/Feedback/Updates
**Not a L2 member? Whitelist this forum in your adblocker to support the site and ensure full functionality.**

Offline mvpel

  • Full Member
  • ****
  • Posts: 1125
  • New Hampshire
  • Liked: 1303
  • Likes Given: 1685
My thought was to deal with what I presume is low-hanging fruit: blocks with no or virtually no change from one iframe to the next. If there's no updates, does that reduce the pframe coding length enough that it'd be more likely to have dodged corruption?
« Last Edit: 05/13/2014 09:49 pm by mvpel »
"Ugly programs are like ugly suspension bridges: they're much more liable to collapse than pretty ones, because the way humans (especially engineer-humans) perceive beauty is intimately related to our ability to process and understand complexity. A language that makes it hard to write elegant code makes it hard to write good code." - Eric S. Raymond

Offline theshadow27

  • Member
  • Posts: 28
  • Liked: 27
  • Likes Given: 7
My thought was to deal with what I presume is low-hanging fruit: blocks with no or virtually no change from one iframe to the next.

Do P-frame blocks within a given frame propagate like I-frame blocks? Or is it strictly a diff from the same MB?

If we can create fake P-frames with no change, then fill in the data changes one at a time, we might make faster progress. But if they are interdependent you'd need to guess C/L data for each which could get annoying fast (and give people creative license to make the video into whatever)

Offline wronkiew

  • Full Member
  • *
  • Posts: 186
  • 34.502327, -116.971697
  • Liked: 105
  • Likes Given: 125
My thought was to deal with what I presume is low-hanging fruit: blocks with no or virtually no change from one iframe to the next. If there's no updates, does that reduce the pframe coding length enough that it'd be more likely to have dodged corruption?

Heh. It's a rocket landing. With a hover-slam. There are virtually no blocks with no change from one frame to the next.

The pframes seem to be an all-or-nothing deal as far as corruption goes, as you might expect. Some of them are quite good, but a lot of them are really bad.

Offline wronkiew

  • Full Member
  • *
  • Posts: 186
  • 34.502327, -116.971697
  • Liked: 105
  • Likes Given: 125
Do P-frame blocks within a given frame propagate like I-frame blocks? Or is it strictly a diff from the same MB?

If we can create fake P-frames with no change, then fill in the data changes one at a time, we might make faster progress. But if they are interdependent you'd need to guess C/L data for each which could get annoying fast (and give people creative license to make the video into whatever)

No, they do not seem to suffer from the same error propagation problems as the i-frames.

Edit: Actually, some of them have groups of block updates which can propagate errors. It will be much easier to fix, though, because there are fewer opportunities for errors to get compounded.
« Last Edit: 05/13/2014 10:04 pm by wronkiew »

Offline SwissCheese

  • Full Member
  • *
  • Posts: 166
  • Liked: 256
  • Likes Given: 107
Actually we can edit the P-frames the same as the I-frames, the generated PNG image then looks good, but I cannot manage to write them back as P-frames, also when using something like this:

ffmpeg -s:0 704:480 -i frameXXX.mpg4-img -f image2 frameXXX_modified.mpg4-img

I hoped it would just copy the file, but it automatically converts the data into an I-frame it seems.

I also tried to use -f mp4 or to add -vf '[out]select=eq(pict_type\,P)', with no more success.

Does someone know how to force ffmpeg to write a P-frame?

Edit:

I finally managed to do it with:
ffmpeg -s:0 704:480 -i frameXXX.mpg4-img -vcodec copy -an -f image2 frameXXX_mod.mpg4-img

Edit(2):

It copies the P-frame but does not handle the -mmb ...
« Last Edit: 05/14/2014 12:06 pm by SwissCheese »

Offline moralec

Hi All,

Just a little update on the Wiki.

The current table with iFrames is a bit of a mess, as it has frames extracted from three different sources (coerced.ts from arnezami, try1.ts from mlindner and -until minutes ago- edit5.ts from Shanuson).

The most recent version (edit5.ts) is going to have the complete set of  15 workable IFrames. In order to organize things a bit, I created a separate page on the wiki to focus on these, hoping  these will be the definitive iframes. You can access from the main page, or directly here http://spacexlanding.wikispaces.com/Progress+on+edit5.ts . The previous table still has all the edits on frames from coerced.ts  and try1.

As of today, there is only one iframe in the new page (the one of frame #14 in the video) as its the only one that is completely fixed and ready to be edited. If anyone has some time available, please work on that one.  As other iframes become available (Shanuson is working on some), they will start appearing there. The online editor 2 will also have an updated version of those iframes for those that are using that tool.

And important note regarding backward compatibility. The new set of iframes includes previously unseen frames, but also some that are  equivalent to those we have already fixed. However, due to changes in iframe sizes, positions of MBs will change. This means that for existing iframes, previous -mmb codes will be only be partially applicable.  Keep this in mind as some .mmb codes will have to get reworked.

Thanks everyone for the good work!
« Last Edit: 05/14/2014 02:06 pm by moralec »

Offline theshadow27

  • Member
  • Posts: 28
  • Liked: 27
  • Likes Given: 7
The online editor 2 will also have an updated version of those iframes for those that are using that tool.

Just to add on this: Big thanks to Iain for implementing some pretty cool new features in http://spacex.slapbet.org

- MB Errors (from the log) are now shown next to the MB X:Y display above the image when an MB is clicked. This includes DC clipping, dquant, ibpc, ac-tex, and stuffing (though I'm not even sure what they all mean ;) ).

- Total number of MB with errors are show on the right. It would be expected that a good -mbm would reduce the total number of MB with errors by more than one. This also allows a quick check to see which of two similar MBM strings is "better"

- Checkbox to enable highlighting MB with errors. This provides invaluable visual assistance in identifying where problems start. It would have been almost impossible to get this clear of a picture just looking at the log. It is interesting to watch how blocks with problems don't always look like it, and how visually defective blocks don't have any errors. This might drastically simplify debugging mbms.

So if you are using Online Editor V2 and haven't refreshed the page in a while, give it a F5. Great work Iain!
« Last Edit: 05/14/2014 04:19 pm by theshadow27 »

So I was building the latest FFMpeg for windows from here https://github.com/michaelni/FFmpeg/tree/spacexdebug1

I managed to actually get it working (yay for me) but when I was testing it I found that the output for the same image with no -mmb applied was different to the previous version of ffmpeg I was using.

OLD: http://i.imgur.com/CeHOIrB.png
NEW: http://i.imgur.com/sMInZkY.png

The output seems a bit cleaner but I don't know why it's different, if anyone knows why this is, or more importantly if it's actually expected and OK, then I'll deploy it to the Online Editor to include the latest functionality.

Offline arnezami

  • Full Member
  • **
  • Posts: 285
  • Liked: 267
  • Likes Given: 378
So I was building the latest FFMpeg for windows from here https://github.com/michaelni/FFmpeg/tree/spacexdebug1

I managed to actually get it working (yay for me) but when I was testing it I found that the output for the same image with no -mmb applied was different to the previous version of ffmpeg I was using.

OLD: http://i.imgur.com/CeHOIrB.png
NEW: http://i.imgur.com/sMInZkY.png

The output seems a bit cleaner but I don't know why it's different, if anyone knows why this is, or more importantly if it's actually expected and OK, then I'll deploy it to the Online Editor to include the latest functionality.
I'm not sure what version you were on but I also noticed some positive changes a few days ago when I updated. I think somebody did some bugfixes/improvements. Very similar to yours.
« Last Edit: 05/14/2014 08:12 pm by arnezami »

Cool, I put the latest ffmpeg on the image handler the editor uses. If someone can summarise what the changes to the editor itself should be (I think it's an additional parameter fr the macroblock operation but I don't know what it's for) then I can add it :)

Offline mvpel

  • Full Member
  • ****
  • Posts: 1125
  • New Hampshire
  • Liked: 1303
  • Likes Given: 1685
The output seems a bit cleaner but I don't know why it's different, if anyone knows why this is, or more importantly if it's actually expected and OK, then I'll deploy it to the Online Editor to include the latest functionality.

Iain, the new version of that frame you're seeing was extracted from a cleaned-up version of the transport stream, where errors in the transport stream packet identifiers, flag bits, and sequence numbers were corrected in order to allow more 184-byte pieces of the MPEG data to be incorporated. If you review from about page 38, that's where Shanuson posts the first pass at the extracted images from the error-corrected transport stream, and prior pages are where we were discussing how to get from point A to pont B (or point 0x03E8) with the protocol.

So yes, it's expected and OK, and you can go ahead and deploy.
"Ugly programs are like ugly suspension bridges: they're much more liable to collapse than pretty ones, because the way humans (especially engineer-humans) perceive beauty is intimately related to our ability to process and understand complexity. A language that makes it hard to write elegant code makes it hard to write good code." - Eric S. Raymond

Only I ran the new ffmpeg on the old images not the new ones :/

Anyway it's deployed, if it's wrong (I can't tell) I can revert easily enough.

Offline Joffan

Only I ran the new ffmpeg on the old images not the new ones :/

Anyway it's deployed, if it's wrong (I can't tell) I can revert easily enough.

Hi Iain
Interface request for http://spacexlanding.wikispaces.com/Online+Editor : can you set it so that when the user clicks on a block in the image, the relevant part of the log is shown? and maybe fill in the existing pos value in the edit window (if it's a previously unspecified block).
Getting through max-Q for humanity becoming fully spacefaring

Which bits of the log do you need? There's already quite a bit there above the image.

As for auto adding pos, currently, because of the way it's built, that would end up adding a bunch of effectively useless parts to the mmb (i.e. if you just click around and don't actually do anything) There might be a way round this, I'll have a think tomorrow when I get back from work =)

Offline SwissCheese

  • Full Member
  • *
  • Posts: 166
  • Liked: 256
  • Likes Given: 107
@Michael Niedermayer:

I've been investigating the possibility of automatically setting all L1,L2,L3,L4,C1,C2 values for all macroblocks for a frame. Basicly inputting an mmb and letting a script detect "bad lines" in the picture. Moving from the right-bottom corner to the upper-top.

I'm starting to believe (using some extra logging and some manual experimentation) that this might actually be possible. If it does this would improve the quality of the frames quite dramatically and would save a lot of manual work!

However there is one thing that I find difficult to do myself. And I think you might be able to help us. Instead of using this kind of setting:

15:14:-1:0:-40:0:0:0:20

where I replace the current macroblock with a blank one and change the DC values.

But what I really want to do is this:

15:14:57217:0:-40:0:0:0:20

In other words: I want to keep the macroblock with its (structured) data as much as I can (at the very least the blocks in it which are set to 0 above) but change only the DC "starting" values. So not blanken it, but modifying it so these DC-values make sure propogation to other blocks works the same as now with -1 (therefore fixing the color and intensity of following blocks) but I also keep the current blocks inside the current macroblock.

Is something like that possible?

Regards,

arnezami
Hi guys,

I've created a new feature that implements the above.

Simply put you can now do something like this:

15:14:57217:0:-40:0:0:0:20

This will keep the macroblock (so not nullify it) bit it will change the DC values. With this you can really fix the lum/chrom issues in the current I-frames. Should have a big effect. I hope.

[edit] Forgot: it now also logs the 6 DC values for each macroblock. Which can come in handy in combination with this new feature.

Also you can do this now:

15:14:57217:0:0:0:0:0:0:0
15:14:57217:0:0:0:0:0:0:63

What this does is change direction from where a DC prediction are coming/inherited from: left or top. For each 1-bit in this number it tells it to get the DC-prediction from the top for a specific DC (4 x lum, 2 x chrom). A 0-bit means left. The number consists of 6 bit (hence max is 63). So in effect a 0 is all left, a 63 is all top. I don't know yet whether this feature is going to be very useful but it can't hurt.

I've attached the 3 source files that were changed. If somebody can check if it doesn't conflict with the existing branch (and maybe test it) and then push it to github, that would be great. Its these files btw:

libavcodec/mpegvideo.h
libavcodec/mpeg4video.h
libavcodec/h263dec.c

[edit] Fixed a bug: when using -2 it would get stuck in that mode. zip re-uploaded.

I need some sleep now ;)

Regards,

arnezami

PS. I was also trying to make a reverse-macroblock finder of sorts. So instead of telling its starting bitposition you would tell it its ending bitposition (which would usually be a starting bitposition of a mb you already have). But I ran into a lot of issues. Even when you try all possible DC-directions it still won't find the right starting position. Even when I fake the right DC values left to the block. I'm not sure why yet. Probably something I don't understand yet. This part of the code is in there but its commented out. It was also extremely error-log-heavy...

Iain, is it possible to add this in the online editor interface? It is quite useful :)
« Last Edit: 05/14/2014 10:21 pm by SwissCheese »

Yup, I'll look at that tomorrow too.

Offline Joffan

Which bits of the log do you need? There's already quite a bit there above the image.

As for auto adding pos, currently, because of the way it's built, that would end up adding a bunch of effectively useless parts to the mmb (i.e. if you just click around and don't actually do anything) There might be a way round this, I'll have a think tomorrow when I get back from work =)

I meant the scrolling image interpretation log below the image - can you position that to show the relevant block as maybe the third line in the window? And actually now you point out the line above the image (which I had been looking for and never found for some reason), that has the current position of the block anyway so it's fairly easy to cut and paste across.
Getting through max-Q for humanity becoming fully spacefaring

Offline theshadow27

  • Member
  • Posts: 28
  • Liked: 27
  • Likes Given: 7
Which bits of the log do you need? There's already quite a bit there above the image.

As for auto adding pos, currently, because of the way it's built, that would end up adding a bunch of effectively useless parts to the mmb (i.e. if you just click around and don't actually do anything) There might be a way round this, I'll have a think tomorrow when I get back from work =)

I meant the scrolling image interpretation log below the image - can you position that to show the relevant block as maybe the third line in the window? And actually now you point out the line above the image (which I had been looking for and never found for some reason), that has the current position of the block anyway so it's fairly easy to cut and paste across.

I suspect you hadn't seen it because it was added yesterday :) I think this change (the extra info at the top) is better than scrolling the log, right? Or do you still see utility in that? This might actually be relevant with the new dc value logging (see edit2).


IainCole: Since the MB already auto-populates the mmb field, I think he's referring to the byte offset of the MB into the "Invert Bits" box. In your code, that's var pos = parseInt(match[4]); near line 714. It could just fill in a new invert bits with a mask of 0.

Edit: I don't know about utility, but it would also be fun to have 8 checkboxes rather than typing in the hex. It would make it easier for non-software people to visualize. Maybe a shift left/right/invert button. I can code it if you're busy

Edit2: I didn't think that it was actually possible to scroll a textarea in js but apparently it is http://features.sheep.art.pl/AutoscrolledTextarea , though due to interop issues it's non-trivial. The trick would be to append one line at a time, recording the length of the string when you hit a mb_pos_info and storing it with the rest of the MB data. 
« Last Edit: 05/15/2014 04:40 am by theshadow27 »

Tags:
 

Advertisement NovaTech
Advertisement Northrop Grumman
Advertisement
Advertisement Margaritaville Beach Resort South Padre Island
Advertisement Brady Kenniston
Advertisement NextSpaceflight
Advertisement Nathan Barker Photography
1