Author Topic: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread  (Read 947494 times)

Offline Shanuson

  • Full Member
  • **
  • Posts: 282
  • Liked: 197
  • Likes Given: 497
Here is also the fixed_edit8.ts for those of you using windows :D

Offline moralec


Looks amazing. But what .ts are you using? Im getting something slightly differnt on edit8

Edit5. The last one in the online editor. It's a useful tool. It was the same in edit5 and try1 so I assumed it would be the same in edit8 too. I'll check edit8.

Edit: I got the same thing from edit8. The one posted here: http://forum.nasaspaceflight.com/index.php?topic=34597.msg1198706#msg1198706

I'm getting a bulish version on the online editor... no idea why  :o


Offline Oersted

  • Member
  • Full Member
  • ****
  • Posts: 879
  • Liked: 471
  • Likes Given: 279
Am I the only one thinking some of the frames currently look very Turneresque?!

Turneresque indeed, with a bit of Mondrian thrown in, unfortunately!

Offline saliva_sweet

  • Full Member
  • ****
  • Posts: 552
  • Liked: 423
  • Likes Given: 1328
I'm getting a bulish version on the online editor... no idea why  :o

I'm attaching the mmb as text. For some reason it's not pasting correctly.

Offline moralec

Sweet! That was it!  ;)

A few updates to the UI of the editor today, so refresh if you haven't recently :)

Offline arnezami

  • Full Member
  • **
  • Posts: 282
  • Liked: 262
  • Likes Given: 345
Hi guys,

Here is the source for reverse searching of macroblocks.  8)

Only this file was changed:

libavcodec/h263dec.c

Please test first and then push to github.

How it works:

Lets say you know that a correct macroblock starts here:

0:0-1,19:06:22832

and you want to find 18:06 but you don't know its bitposition. So you do this:

0,0:0:-1,18:06:--22832:0,19:06:22832

With this you give the instruction: when reaching mb 18:06 start with the position 22832 (which is the starting bitposition of 19:06) and try to find a possible starting bitposition before 22832. What it will do is first set bitposition of 18:06 to 22831, then decode from there and if the decoding of the macroblock does not end at 22832 it will start from 22830 etc. If it does at some point end at 22832 it will keep that as the solution. You can see the final position in the log.

If you don't think it's the real solution you can tell it to skip the one or more candidates:

0,0:0:-1,18:06:--22832:1,19:06:22832

This will skip the first hit and will take the second hit as the solution.

Now after experimenting with this I saw there can be quite a few false positives: for one example I found 11 possible solutions. So you will still have to determine yourselves which one is the most likely starting position (probably the one with the lowest bitposition number). In practice the real starting position is the one where the macroblock before that (in this case 17:06) ends again. So I think you can see where this is going ;)

There is a max of 2048 tries at the moment btw. If it doesn't find a solution it will set the starting position at the postion you told it to start searching from (22832 in this case).

Sometimes it crashes after finding no solution. I'm not sure why yet. So handle with care.

Important note: do not post a reverse searching command in your solution online! If you find a bitposition using this technique then replace the search-command with a normal command (eg 18:06:22712). Otherwise you will create a LOT of unnecessary log to others.

Regards,

arnezami

PS. This technique will cause quite a bit of error-log because you are trying to start from a lot of wrong bitpositions. Never do two searches at the same time!
« Last Edit: 05/16/2014 06:18 AM by arnezami »

Offline SwissCheese

  • Full Member
  • *
  • Posts: 164
  • Liked: 249
  • Likes Given: 90
Still trying to get something out of the p-frames...

We can visualize quite well the macroblock types with -debug vis_mb_type and the motion vectors with -vismv 7.  It's nice but does not help so much as it is only a visual information. Would it be possible to write the macroblock type and motion vector values besides the macroblock dc values? I think it would help quite a lot.

edit: found that we can write out the MB types with -debug mb_type

edit2: wrote some code lines to do it :), will post tomorrow
« Last Edit: 05/17/2014 12:59 AM by SwissCheese »

Offline SwissCheese

  • Full Member
  • *
  • Posts: 164
  • Liked: 249
  • Likes Given: 90
The updated version displaying the MB type and the motion vectors is in attachment, it was based on the latest version of Arnezami. I hope it does not introduce any bug ;)

I also tried to rebuild frames +/- "as the encoder does" with some basic matlab code, the problem is that many P-frames are highly corrupted... but it worked not too badly for frames 169 (I-frame), 171 and 172.

I don't know if it's a good idea to go this way, but I was stuck trying to repair P-frames: as soon as I touched them (tried to correct them with matlab), they would not be used during the video encoding (even when simply reading and rewriting a frame, without changing anything).

Btw some P-frames like 171 or 176 have issues in their first data bytes or at the end of the header (bytes 4/5/6), I don't really know, and are not converted to PNG, a copy/paste of these bytes from a valid frame solves the issue.

Offline arnezami

  • Full Member
  • **
  • Posts: 282
  • Liked: 262
  • Likes Given: 345
Hi guys,

I noticed a problem in the online editor. It is removing zero's in the DC sections. This goes wrong if you want to do something like this: 16:05:65003:0:0:-5:0:0:0. Now it makes 16:05:65003:::-5,0 out of that. Which is wrong.

Only if all 6 DC values (L1:L2:L3:L4:C1:C2) are zero should it remove the 6 zero's (L1:L2:L3:L4:C1:C2). Otherwise there should always be 6 numbers there. The direction-number after that is optional.

Also I think it is not updated to the latest commit where the short version 16:05:65003::15 of changing the directions was introduced.

Regards,

arenzami

Offline arnezami

  • Full Member
  • **
  • Posts: 282
  • Liked: 262
  • Likes Given: 345
The updated version displaying the MB type and the motion vectors is in attachment, it was based on the latest version of Arnezami. I hope it does not introduce any bug ;)

I also tried to rebuild frames +/- "as the encoder does" with some basic matlab code, the problem is that many P-frames are highly corrupted... but it worked not too badly for frames 169 (I-frame), 171 and 172.

I don't know if it's a good idea to go this way, but I was stuck trying to repair P-frames: as soon as I touched them (tried to correct them with matlab), they would not be used during the video encoding (even when simply reading and rewriting a frame, without changing anything).

Btw some P-frames like 171 or 176 have issues in their first data bytes or at the end of the header (bytes 4/5/6), I don't really know, and are not converted to PNG, a copy/paste of these bytes from a valid frame solves the issue.
Cool! And interesting. The engine is just igniting! :)

Regarding the P-frames: I think it would be a good idea to extend the -mmb syntax with a framenumber.

So you would have something like this:

-mmb FRAME168:16:04:37259,23:02:73229=FRAME169:01:01:2342,34:02:-1  etc.

Something like that.

That way you can give it either the complete .ts or a .ts that starts with an I-frame until the P-frame you are actually working on. It would run through all the frames changing their macroblocks. And the P-frame you're interested in will inherit all the fixes (and errors of course) that have preceded it.

The only big problem is that the mmb will get quite big. We would need some kind of script that assembles all the mmb's of the frames and calls ffmpeg with that long mmb.

Good idea?

Regards,

arnezami
« Last Edit: 05/17/2014 01:21 PM by arnezami »

Offline Avron

  • Canadian Member
  • Senior Member
  • *****
  • Posts: 4924
  • Liked: 150
  • Likes Given: 152
...I was stuck trying to repair P-frames:...



Amazing, quite amazing to see P- frames. I would hope Elon send you all an invite to tour the factory for the work effort he triggered, I am so sure he will do something great to say thanks.

Offline mvpel

  • Full Member
  • ****
  • Posts: 1124
  • New Hampshire
  • Liked: 1295
  • Likes Given: 1686
The only big problem is that the mmb will get quite big. We would need some kind of script that assembles all the mmb's of the frames and calls ffmpeg with that long mmb.

In condor_submit, you just specify the "-append" argument multiple times to add more parameters. Seems like the same idea would work here, and allow a bit more flexibility in the context of this idea.
"Ugly programs are like ugly suspension bridges: they're much more liable to collapse than pretty ones, because the way humans (especially engineer-humans) perceive beauty is intimately related to our ability to process and understand complexity. A language that makes it hard to write elegant code makes it hard to write good code." - Eric S. Raymond

Offline arnezami

  • Full Member
  • **
  • Posts: 282
  • Liked: 262
  • Likes Given: 345
The updated version displaying the MB type and the motion vectors is in attachment, it was based on the latest version of Arnezami. I hope it does not introduce any bug ;)

I also tried to rebuild frames +/- "as the encoder does" with some basic matlab code, the problem is that many P-frames are highly corrupted... but it worked not too badly for frames 169 (I-frame), 171 and 172.

I don't know if it's a good idea to go this way, but I was stuck trying to repair P-frames: as soon as I touched them (tried to correct them with matlab), they would not be used during the video encoding (even when simply reading and rewriting a frame, without changing anything).

Btw some P-frames like 171 or 176 have issues in their first data bytes or at the end of the header (bytes 4/5/6), I don't really know, and are not converted to PNG, a copy/paste of these bytes from a valid frame solves the issue.
Cool! And interesting. The engine is just igniting! :)

Regarding the P-frames: I think it would be a good idea to extend the -mmb syntax with a framenumber.

So you would have something like this:

-mmb FRAME168:16:04:37259,23:02:73229=FRAME169:01:01:2342,34:02:-1  etc.

Something like that.

That way you can give it either the complete .ts or a .ts that starts with an I-frame until the P-frame you are actually working on. It would run through all the frames changing their macroblocks. And the P-frame you're interested in will inherit all the fixes (and errors of course) that have preceded it.

The only big problem is that the mmb will get quite big. We would need some kind of script that assembles all the mmb's of the frames and calls ffmpeg with that long mmb.

Good idea?

Regards,

arnezami
Hi guys,

I've extended the mmb-syntax so it can now handle multiple frames at the time. See the above quote.

The syntax is:

Quote
-mmb FRAME0:16:04:37259,23:02:73229=FRAME1:01:01:2342,34:02:-1

Note the '=' sign!

The old syntax still works on files with one frame in it.

I've also incorporated SwissCheese' logging (and changed back the && into & ;))

I haven't actually tested it with a file with multiple frames in it. But it *should* work. Let me know!

[edit] Important: the frame numbers should be the frame numbers in your file. If you use a file with say 3 frames in it their frame numbers should be 0, 1 and 2. So not 168, 169, 170 in that case. Should I add the frame number in the log too? Not sure.

Regards,

arnezami
« Last Edit: 05/17/2014 01:49 PM by arnezami »

Online Chris Bergin

Coming up on the 150,000 view mark for this thread/effort. Noting it as there's a small active team of people really working hard on this, but also a HUGE amount of people watching that work. Always nice to have a large appreciative audience, both inside and outside the space industry.

You'd be astonished how many views are from IP addresses in a place called Hawthorne, California! ;D

Offline moralec

The updated version displaying the MB type and the motion vectors is in attachment, it was based on the latest version of Arnezami. I hope it does not introduce any bug ;)

I also tried to rebuild frames +/- "as the encoder does" with some basic matlab code, the problem is that many P-frames are highly corrupted... but it worked not too badly for frames 169 (I-frame), 171 and 172.

I don't know if it's a good idea to go this way, but I was stuck trying to repair P-frames: as soon as I touched them (tried to correct them with matlab), they would not be used during the video encoding (even when simply reading and rewriting a frame, without changing anything).

Btw some P-frames like 171 or 176 have issues in their first data bytes or at the end of the header (bytes 4/5/6), I don't really know, and are not converted to PNG, a copy/paste of these bytes from a valid frame solves the issue.
Cool! And interesting. The engine is just igniting! :)

Inspiring work SwissCheese! this is Amazing! Yet another piece of the puzzle (engine ignition) has been uncovered.

The idea of using matlab sounds good to me. I think we should accept that some of the frames of the video are beyond recovery.  Considering this, a good approach could be to deconstruct the video frame by frame recovering as many usable frames as possible, and then reconstruct it back again keeping only the good frames, to then interpolate (like the guys on the curiosity landing did) the missing ones in order to obtain a decent dynamic video.

My guess is the p-frames immediately after a fixed iframe will be the best looking, as only some macro blocks should have changed..... but as the sequence continues moving forward, the subsequent frames will start to accumulate errors and they will look worse and worse. That is, naturally, until the video gets to the next iframe that has been manually fixed.

For how many frames did you apply the process?  Could you the same technique for the frames after of after 72 or 209 (which could show leg deployment and touchdown)? do you thing my hyphotesis of constant deterioration will hold?

Edit: On other news, I tried to apply your very long mmb code for frame 92 (the frame formally known as iframe # 5) to the newest version (the one obtained from edit5.ts) and unfortunately is not working anymore. AS you where the main author, maybe you will be able to figure out what has changed. I'm guessing a small adjustment should be sufficient to get that frame visible again! It could also be a problem on how the code was pasted in the forum: I believe this happened to Saliva_sweet
« Last Edit: 05/17/2014 02:34 PM by moralec »

Offline billh

  • Full Member
  • ***
  • Posts: 357
  • Houston
  • Liked: 244
  • Likes Given: 140
The updated version displaying the MB type and the motion vectors is in attachment, it was based on the latest version of Arnezami. I hope it does not introduce any bug ;)

I also tried to rebuild frames +/- "as the encoder does" with some basic matlab code, the problem is that many P-frames are highly corrupted... but it worked not too badly for frames 169 (I-frame), 171 and 172.

I don't know if it's a good idea to go this way, but I was stuck trying to repair P-frames: as soon as I touched them (tried to correct them with matlab), they would not be used during the video encoding (even when simply reading and rewriting a frame, without changing anything).

Btw some P-frames like 171 or 176 have issues in their first data bytes or at the end of the header (bytes 4/5/6), I don't really know, and are not converted to PNG, a copy/paste of these bytes from a valid frame solves the issue.
Cool! And interesting. The engine is just igniting! :)

I don't think that's the engine igniting. It looks more like the plume is just beginning to interact with the ocean surface.

Offline edfishel

  • Full Member
  • *
  • Posts: 150
  • Liked: 19
  • Likes Given: 66
For the small group of people who are working really, really hard on this...thank you!  It's beyond my comprehension.  But I thoroughly appreciate the work you are doing.  Chris Bergin is so right.  You are doing wonderful work on behalf of all of us.

Offline SwissCheese

  • Full Member
  • *
  • Posts: 164
  • Liked: 249
  • Likes Given: 90
The updated version displaying the MB type and the motion vectors is in attachment, it was based on the latest version of Arnezami. I hope it does not introduce any bug ;)

I also tried to rebuild frames +/- "as the encoder does" with some basic matlab code, the problem is that many P-frames are highly corrupted... but it worked not too badly for frames 169 (I-frame), 171 and 172.

I don't know if it's a good idea to go this way, but I was stuck trying to repair P-frames: as soon as I touched them (tried to correct them with matlab), they would not be used during the video encoding (even when simply reading and rewriting a frame, without changing anything).

Btw some P-frames like 171 or 176 have issues in their first data bytes or at the end of the header (bytes 4/5/6), I don't really know, and are not converted to PNG, a copy/paste of these bytes from a valid frame solves the issue.
Cool! And interesting. The engine is just igniting! :)

Regarding the P-frames: I think it would be a good idea to extend the -mmb syntax with a framenumber.

So you would have something like this:

-mmb FRAME168:16:04:37259,23:02:73229=FRAME169:01:01:2342,34:02:-1  etc.

Something like that.

That way you can give it either the complete .ts or a .ts that starts with an I-frame until the P-frame you are actually working on. It would run through all the frames changing their macroblocks. And the P-frame you're interested in will inherit all the fixes (and errors of course) that have preceded it.

The only big problem is that the mmb will get quite big. We would need some kind of script that assembles all the mmb's of the frames and calls ffmpeg with that long mmb.

Good idea?

Regards,

arnezami
Hi guys,

I've extended the mmb-syntax so it can now handle multiple frames at the time. See the above quote.

The syntax is:

Quote
-mmb FRAME0:16:04:37259,23:02:73229=FRAME1:01:01:2342,34:02:-1

Note the '=' sign!

The old syntax still works on files with one frame in it.

I've also incorporated SwissCheese' logging (and changed back the && into & ;))

I haven't actually tested it with a file with multiple frames in it. But it *should* work. Let me know!

[edit] Important: the frame numbers should be the frame numbers in your file. If you use a file with say 3 frames in it their frame numbers should be 0, 1 and 2. So not 168, 169, 170 in that case. Should I add the frame number in the log too? Not sure.

Regards,

arnezami

I don't exactly understand how to use it, could you make a small .zip package with an example? (text file with command lines and used .ts or .mp4-img files)

(sorry for the &  ::)  :P)
« Last Edit: 05/17/2014 02:39 PM by SwissCheese »

Offline SwissCheese

  • Full Member
  • *
  • Posts: 164
  • Liked: 249
  • Likes Given: 90
The updated version displaying the MB type and the motion vectors is in attachment, it was based on the latest version of Arnezami. I hope it does not introduce any bug ;)

I also tried to rebuild frames +/- "as the encoder does" with some basic matlab code, the problem is that many P-frames are highly corrupted... but it worked not too badly for frames 169 (I-frame), 171 and 172.

I don't know if it's a good idea to go this way, but I was stuck trying to repair P-frames: as soon as I touched them (tried to correct them with matlab), they would not be used during the video encoding (even when simply reading and rewriting a frame, without changing anything).

Btw some P-frames like 171 or 176 have issues in their first data bytes or at the end of the header (bytes 4/5/6), I don't really know, and are not converted to PNG, a copy/paste of these bytes from a valid frame solves the issue.
Cool! And interesting. The engine is just igniting! :)

Inspiring work SwissCheese! this is Amazing! Yet another piece of the puzzle (engine ignition) has been uncovered.

The idea of using matlab sounds good to me. I think we should accept that some of the frames of the video are beyond recovery.  Considering this, a good approach could be to be to deconstruct the video frame by frame recovering as many usable frames as possible, and then reconstruct it back again keeping only the good frames, to then interpolate (like the guys on the curiosity landing did) in order to obtain a decent dynamic video.

My guess is the p-frames immediately after a fixed iframe will be the best looking, as only some macro blocks should have changed..... but as the sequence continues moving forward, the subsequent frames will start to accumulate errors and they will look worse and worse. That is, naturally, until the video gets to the next iframe that has been manually fixed.

For how many frames did you apply the process?  Could you the same technique for the frames after of after 72 or 209 (which could show leg deployment and touchdown)? do you thing my hyphotesis of constant deterioration will hold?

Edit: On other news, I tried to apply your very long mmb code for frame 92 (the frame formally known as iframe # 5) to the newest version (the one obtained from edit5.ts) and unfortunately is not working anymore. AS you where the main author, maybe you will be able to figure out what has changed. I'm guessing a small adjustment should be sufficient to get that frame visible again! It could also be a problem on how the code was pasted in the forum: I believe this happened to Saliva_sweet

Yes it is exactly the problem, it works just after the I-frame and as long as we have good quality P-frames, but if a serie of P-frames is missing/corrupted it does not give great results anymore... Also seen in frame 176: some MB have somehow a wrong type (coded as stand-alone MB, when they are clearly difference MB...) and other have really low values (very dark) for no reason that I understand.
« Last Edit: 05/17/2014 02:35 PM by SwissCheese »

Tags: