Author Topic: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread  (Read 1201788 times)

The latest TS files have been added to the online editor

Offline michaelni

  • Member
  • Posts: 28
  • Liked: 23
  • Likes Given: 0
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1141 on: 05/29/2014 06:54 pm »
1) There are a lot of possible combination to try. If an error correction code was used then that information could possibly allow us to discard some of them (e.g. if a single-error correcting code was used at the bit level, then any single-bit flips will have already been fixed and don't need to be tried, and there may be a more limited number of combinations of flipped bits to check).

You miss that a FEC code capable to fix n bit errors, does fix them for the transmitted packet/message, while the transmitted packet is made up of data and parity symbols in a systematic code or at least more symbols than there are data symbols in the more generic case. Thus for example a 1-3 bit correcting FEC code could fix 3 bits in the transmitted packet but there might be just 1 or eve none of these in the data part. We only have the data part of the packets, not the parity part. Also if in that same example there are 4 bit flips, the FEC decoder might actualy (depending on what type of code and implementation it is) add more errors.

Offline arnezami

  • Full Member
  • **
  • Posts: 285
  • Liked: 267
  • Likes Given: 378
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1142 on: 05/29/2014 06:55 pm »
arnezami, could you post a patch? I was about to start doing the exact same thing :). Hopefully the additional output will allow a kind of guided bit flipping, where we can quickly discard flips that yield obvious nonsense.

@Lourens: I will try but there are still quite a few problems with this version I noticed. For example you can see that bits are missing. Thats not good ;). It's also very hacky atm. Will keep you updated.
Okay, I'll go and have a look at the FFmpeg source myself as well, will post when I have something.
Hi Lourens,

In that case this will probably help. This is my hacky version based on an older version of h263dec.c Mostly mpeg4videodec.c is changed though. Watch for "BITLOG".

For everyone: this is not meant to be merged directly with any repository! It's just experimental stuff.

Regards,

arnezami
« Last Edit: 05/29/2014 06:56 pm by arnezami »

Offline michaelni

  • Member
  • Posts: 28
  • Liked: 23
  • Likes Given: 0
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1143 on: 05/29/2014 06:58 pm »
Maybe. Currently the mmb-command work in the "scope" after the VOP-quant etc have been decoded. Maybe/probably we are too late there. So I'm not sure if it's possible using the mmb-commands. But maybe using a different command. @michaelni: do you know how to do this?

mmb should be able to flip bits in  mpeg4-es headers, but i didnt try

Offline arnezami

  • Full Member
  • **
  • Posts: 285
  • Liked: 267
  • Likes Given: 378
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1144 on: 05/29/2014 07:32 pm »
@arnesami - should we adopt a numbering convention of PART.FRAME like 10.01 - 10.20? This allows for new pFrames to be found without changing all frames after.
Sure. That could work. It's a bit more safe I guess.

Unless we know for sure each I-frame has no more than 19 P-frames. Because from what I see it really looks like that is the case.

I don't have a strong preference. I like simpler numbers if possible.
« Last Edit: 05/29/2014 07:33 pm by arnezami »

Offline mhenderson

  • Member
  • Posts: 71
  • USA
  • Liked: 101
  • Likes Given: 18
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1145 on: 05/29/2014 07:40 pm »
The latest TS files have been added to the online editor

If possible, could you add the new .ts to the older spacex.slapbet.org version of the editor?

The "iFrame only" online tool is MUCH faster to work with. I believe we are back to square one on the iFrame mmps.

Thank you.

Offline MTom

  • Full Member
  • ****
  • Posts: 573
  • EU / Hungary
  • Liked: 340
  • Likes Given: 993
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1146 on: 05/29/2014 07:46 pm »
This is why we needed a youtube version ;D

Elon Musk ‏@elonmusk  4m 
 More
 Amazing repair job of Falcon 9 ocean landing vid by @NASASpaceflight forum. Now shows leg deploy http://www.youtube

Totally distracted me during Soyuz docking. I hope he doesn't do that again! (;D Joke!)

Again, that's all deserved for what you guys are doing. It's direct praise from one of the most important people on the planet right now. You're a motivated bunch, but I'm sure that helps while you dig through all these bytes of data :)


... and there is the update on the SpaceX website showing the video from yesterday:  :D

Quote
UPDATE (5/28/2014)

Members of NASASpaceFlight.com's forum are continuing to make great progress with the frame by frame repair of the Falcon 9 first stage soft splashdown in the Atlantic Ocean.

http://www.spacex.com/news/2014/04/29/first-stage-landing-video

The latest TS files have been added to the online editor

If possible, could you add the new .ts to the older spacex.slapbet.org version of the editor?

The "iFrame only" online tool is MUCH faster to work with. I believe we are back to square one on the iFrame mmps.

Thank you.

If someone who knows how to do it can generate the mp4-img files for the iframes then I can add them to the old editor.

Offline SwissCheese

  • Full Member
  • *
  • Posts: 166
  • Liked: 256
  • Likes Given: 107
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1148 on: 05/29/2014 08:49 pm »
The latest TS files have been added to the online editor

If possible, could you add the new .ts to the older spacex.slapbet.org version of the editor?

The "iFrame only" online tool is MUCH faster to work with. I believe we are back to square one on the iFrame mmps.

Thank you.

Many frames are the same as with previous version, and the ones that changed are quite easily fixed by finding a few bit offsets between the previous and the current version.

I will try to prepare the wiki tomorrow, today neither Moralec nor I can do it.

In any case if we want to have a full video soon, I think we should concentrate on parts 7, 8, 13 and 15 (maybe also 3, 4 and 5, and finish part 9, see my previous post), for the other ones we can always use what was done before.
« Last Edit: 05/29/2014 09:05 pm by SwissCheese »

Offline arnezami

  • Full Member
  • **
  • Posts: 285
  • Liked: 267
  • Likes Given: 378
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1149 on: 05/29/2014 08:50 pm »
The latest TS files have been added to the online editor

If possible, could you add the new .ts to the older spacex.slapbet.org version of the editor?

The "iFrame only" online tool is MUCH faster to work with. I believe we are back to square one on the iFrame mmps.

Thank you.

If someone who knows how to do it can generate the mp4-img files for the iframes then I can add them to the old editor.

You can do this:

Quote
./ffmpeg -i fixed_edit8_part_169.ts -vcodec copy -an -f image2 fixed_edit_part_169_%d.mpg4-img

This assumes you have fixed_edit8_part_169.ts and produces .mp4-img files. In this case you want the first one "fixed_edit_part_169_1.mpg4-img" since it's the I-frame. You can throw away the other mpg4-img files, since these are the P-frames.

And you have to do this for each of the 15 ts-parts.
« Last Edit: 05/29/2014 08:51 pm by arnezami »

Offline mhenderson

  • Member
  • Posts: 71
  • USA
  • Liked: 101
  • Likes Given: 18
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1150 on: 05/29/2014 09:13 pm »
The latest TS files have been added to the online editor

If possible, could you add the new .ts to the older spacex.slapbet.org version of the editor?

The "iFrame only" online tool is MUCH faster to work with. I believe we are back to square one on the iFrame mmps.

Thank you.

If someone who knows how to do it can generate the mp4-img files for the iframes then I can add them to the old editor.

You can do this:

Quote
./ffmpeg -i fixed_edit8_part_169.ts -vcodec copy -an -f image2 fixed_edit_part_169_%d.mpg4-img

This assumes you have fixed_edit8_part_169.ts and produces .mp4-img files. In this case you want the first one "fixed_edit_part_169_1.mpg4-img" since it's the I-frame. You can throw away the other mpg4-img files, since these are the P-frames.

And you have to do this for each of the 15 ts-parts.

You may ignore my request.

I found a quick fix that makes the newer online editor much more responsive ... null out the pFrames by adding this to the mmb for the full sequence. It quickly redraws the iFrame.

=FRAME1:0:0:-1=FRAME2:0:0:-1=FRAME3:0:0:-1=FRAME4:0:0:-1
=FRAME5:0:0:-1=FRAME6:0:0:-1=FRAME7:0:0:-1=FRAME8:0:0:-1
=FRAME9:0:0:-1=FRAME10:0:0:-1=FRAME11:0:0:-1=FRAME12:0:0:-1
=FRAME13:0:0:-1=FRAME14:0:0:-1=FRAME15:0:0:-1=FRAME16:0:0:-1
=FRAME17:0:0:-1=FRAME18:0:0:-1=FRAME19:0:0:-1

Sorry, I was thinking inside the box. ;)

Mike

The latest TS files have been added to the online editor

If possible, could you add the new .ts to the older spacex.slapbet.org version of the editor?

The "iFrame only" online tool is MUCH faster to work with. I believe we are back to square one on the iFrame mmps.

Thank you.

If someone who knows how to do it can generate the mp4-img files for the iframes then I can add them to the old editor.

You can do this:

Quote
./ffmpeg -i fixed_edit8_part_169.ts -vcodec copy -an -f image2 fixed_edit_part_169_%d.mpg4-img

This assumes you have fixed_edit8_part_169.ts and produces .mp4-img files. In this case you want the first one "fixed_edit_part_169_1.mpg4-img" since it's the I-frame. You can throw away the other mpg4-img files, since these are the P-frames.

And you have to do this for each of the 15 ts-parts.

You may ignore my request.

I found a quick fix that makes the newer online editor much more responsive ... null out the pFrames by adding this to the mmb for the full sequence. It quickly redraws the iFrame.

=FRAME1:0:0:-1=FRAME2:0:0:-1=FRAME3:0:0:-1=FRAME4:0:0:-1
=FRAME5:0:0:-1=FRAME6:0:0:-1=FRAME7:0:0:-1=FRAME8:0:0:-1
=FRAME9:0:0:-1=FRAME10:0:0:-1=FRAME11:0:0:-1=FRAME12:0:0:-1
=FRAME13:0:0:-1=FRAME14:0:0:-1=FRAME15:0:0:-1=FRAME16:0:0:-1
=FRAME17:0:0:-1=FRAME18:0:0:-1=FRAME19:0:0:-1

Sorry, I was thinking inside the box. ;)

Mike

I have added them anyway, thanks arnezami for the instructions :)

Offline moralec

arnezami, could you post a patch? I was about to start doing the exact same thing :). Hopefully the additional output will allow a kind of guided bit flipping, where we can quickly discard flips that yield obvious nonsense.

Yes we can do that. I'll put the current pages on archive and create new empty ones. :)

@moralec: I think you meant to reply on this post by SwissCheese (quoted below) right?  (instead of Laurens post)
@moralec/SwissCheese: would it be a good idea to number the I-frames to 1,21,41,61 etc? And the P-frames in between? Since we expect each I-frame to contain exactly 19 P-frames right?

Yes. My Bad.... I´m struggling from a mobile device.

I think thats a good idea. I'm going to build the new pages to upload results based on that numbering. :)
However, we need to keep in mind the correspondences in order to import the previous improvements.


Offline Lourens

  • Full Member
  • *
  • Posts: 156
  • The Netherlands
  • Liked: 206
  • Likes Given: 304
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1153 on: 05/29/2014 10:46 pm »
1) There are a lot of possible combination to try. If an error correction code was used then that information could possibly allow us to discard some of them (e.g. if a single-error correcting code was used at the bit level, then any single-bit flips will have already been fixed and don't need to be tried, and there may be a more limited number of combinations of flipped bits to check).

You miss that a FEC code capable to fix n bit errors, does fix them for the transmitted packet/message, while the transmitted packet is made up of data and parity symbols in a systematic code or at least more symbols than there are data symbols in the more generic case. Thus for example a 1-3 bit correcting FEC code could fix 3 bits in the transmitted packet but there might be just 1 or eve none of these in the data part. We only have the data part of the packets, not the parity part. Also if in that same example there are 4 bit flips, the FEC decoder might actualy (depending on what type of code and implementation it is) add more errors.

I'm not sure I understand completely what you wrote here, but I think you're right that it doesn't help as much as I thought it would.

arnezami, could you post a patch? I was about to start doing the exact same thing :). Hopefully the additional output will allow a kind of guided bit flipping, where we can quickly discard flips that yield obvious nonsense.

@Lourens: I will try but there are still quite a few problems with this version I noticed. For example you can see that bits are missing. Thats not good ;). It's also very hacky atm. Will keep you updated.
Okay, I'll go and have a look at the FFmpeg source myself as well, will post when I have something.
Hi Lourens,

In that case this will probably help. This is my hacky version based on an older version of h263dec.c Mostly mpeg4videodec.c is changed though. Watch for "BITLOG".

For everyone: this is not meant to be merged directly with any repository! It's just experimental stuff.

Regards,

arnezami

That does help, thanks! I've modded the MB pos/size line a bit in my version to show, it now looks like this:

[mpeg4 @ 0x1c2bba0] MB pos/size: 0 000:40:03:8960 39 ac_pred: 1 aic: 0 aic_dir: 0 dquant: 0 qscale: 4 dc: 142 143 140 141 - 135 124

but that doesn't actually help much. I've put your additions into my version as well, and they are much more useful. Now to link it up to my automated bit flipper and see if it yields something. That's for tomorrow though.

Offline Oersted

  • Member
  • Senior Member
  • *****
  • Posts: 2897
  • Liked: 4098
  • Likes Given: 2773
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1154 on: 05/29/2014 11:37 pm »
1) Thread currently at >252,000 reads which puts it at 15th most read SpaceX thread of all time and 13th most read SpaceX conversation[1]. See original link for images of the overall ranking.

Never in the history of space flight was a thread read by so many who understood so little...  ;-)

- I'm one of them, btw! - But I think we all share the joy of seeing consummate professionals cooperating in an amateur endeavour, coaxing intelligible images from a seemingly hopelessly mangled blur of bits and bytes. To me it replicates the fascination of listening to ground control during a Saturn-Apollo lift-off; obscure abbreviations tersely announced...
« Last Edit: 05/29/2014 11:38 pm by Oersted »

Offline moralec

The latest TS files have been added to the online editor

Following this update the wiki now has 16 new (empty) pages for you all to  upload your progress on each frame.
I followed the same structure as last time (one main page, and 15 secondary pages). Please bookmark http://spacexlanding.wikispaces.com/raw_final_fixedMMB :)

The old pages are still available *look under OLD/ARCHIVED in the homepage*. They have all the MMB codes we were able to obtain from the previous ts files. Many of them should work (specially from those sections that already had 19 p-frames in the previous version).  However, please take into account that frame numbers have changed as new p-frames where discovered. A good way to orient yourself is using the 15 I-frames that remain the same.

Good luck! And for those anxiously waiting for the Dragon II event tonight, remember that working on p-frames is a fantastic way to kill time. 

Offline arnezami

  • Full Member
  • **
  • Posts: 285
  • Liked: 267
  • Likes Given: 378
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1156 on: 05/30/2014 03:06 am »
The latest TS files have been added to the online editor

Following this update the wiki now has 16 new (empty) pages for you all to  upload your progress on each frame.
I followed the same structure as last time (one main page, and 15 secondary pages). Please bookmark http://spacexlanding.wikispaces.com/raw_final_fixedMMB :)

The old pages are still available *look under OLD/ARCHIVED in the homepage*. They have all the MMB codes we were able to obtain from the previous ts files. Many of them should work (specially from those sections that already had 19 p-frames in the previous version).  However, please take into account that frame numbers have changed as new p-frames where discovered. A good way to orient yourself is using the 15 I-frames that remain the same.

Good luck! And for those anxiously waiting for the Dragon II event tonight, remember that working on p-frames is a fantastic way to kill time.

I filled in this page: http://spacexlanding.wikispaces.com/Frames+Part+10

I created all new pictures for them. Most of them were the same as before but the following were (clearly) different and/or I had to change:  (these are the new numbers)

- 187: changed mmb
- 191: looks different, I did not change mmb, not sure if needed
- 193: changed mmb
- 194: needs work, quite different

I haven't tried these in a video, so please check!

Regards,

arnezami
« Last Edit: 05/30/2014 03:07 am by arnezami »

Offline mvpel

  • Full Member
  • ****
  • Posts: 1125
  • New Hampshire
  • Liked: 1303
  • Likes Given: 1685
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1157 on: 05/30/2014 03:28 am »
I filled in this page: http://spacexlanding.wikispaces.com/Frames+Part+10

I've been meaning to ask... what's the story behind the big grey rectangles in frames 186-188? Can you tell if it's an MPEG error or an NTSC video camera error?
"Ugly programs are like ugly suspension bridges: they're much more liable to collapse than pretty ones, because the way humans (especially engineer-humans) perceive beauty is intimately related to our ability to process and understand complexity. A language that makes it hard to write elegant code makes it hard to write good code." - Eric S. Raymond

Offline Mariusuiram

  • Full Member
  • *
  • Posts: 125
  • Liked: 129
  • Likes Given: 128
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1158 on: 05/30/2014 03:33 am »

Never in the history of space flight was a thread read by so many who understood so little...  ;-)


Its like magic. I check it every morning and skim through. My eyes glaze over for 90% as I look for a new gif or image. Then I feel guilty just looking for the pictures and I go back to give a concerted effort to understand the conversation. And then I just feel dense, old, and confused so I go back to looking at the pictures  ;)

Offline wronkiew

  • Full Member
  • *
  • Posts: 186
  • 34.502327, -116.971697
  • Liked: 105
  • Likes Given: 125
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1159 on: 05/30/2014 04:29 am »
I filled in this page: http://spacexlanding.wikispaces.com/Frames+Part+10

I've been meaning to ask... what's the story behind the big grey rectangles in frames 186-188? Can you tell if it's an MPEG error or an NTSC video camera error?

MPEG error. I was able to fix some of them in 185 by random bit flipping.

Tags:
 

Advertisement NovaTech
Advertisement Northrop Grumman
Advertisement
Advertisement Margaritaville Beach Resort South Padre Island
Advertisement Brady Kenniston
Advertisement NextSpaceflight
Advertisement Nathan Barker Photography
0