Author Topic: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread  (Read 851756 times)

Offline princess

  • Member
  • Posts: 65
  • Liked: 106
  • Likes Given: 25
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1380 on: 06/05/2014 12:07 PM »
But I bet it *would* be helpful if someone would compile a list of the adjusted start/end positions of the FEC coding blocks referenced to the current .ts and including the discovered bit shifts, etc, like princess did

I can calculate a mapping between file offsets in raw.ts and the current TS file if necessary, would this be useful?

Also, if people want to calculate statistics on the bitflip errors then I have TS files that have more fixes than the current TS file - my current versions have correct contents for all PIDs except the data PID, i.e. all padding is correct, the Program Association Tables and Program Map Tables are all correct and the "padding" AFs in the data are correct too. One of my side projects is trying to make a TS file that's "what the rocket transmitted", so I can make that available to people who are doing statistical anaylsis on the error distribution. Let me know what you need and I'll make it available.

The current TS file starts at the first I-frame, but the rocket transmitted a few P-frames of data before that. So as part of my "what the rocket transmitted" work I'm also fixing those P-frames up to the same standard - we won't be able to use them in videos (as there's no I-frame to relate them to), but the work could help in statistical analysis of errors.

Finally, as we're now able to map between TS file offsets and MPEG4 bit positions correctly (thanks to those who discovered ffmpeg's behaviour here), I am planning on making a tool that can apply the XOR parts of -mmb commands directly to the TS file. As far as I can see those should apply fairly easily but the other -mmb commands work at a higher level and can't map directly back to bitflips. However, if we can use even part of the -mmb commands then we can work out some of the bitflips within the MPEG4 stream.

Offline princess

  • Member
  • Posts: 65
  • Liked: 106
  • Likes Given: 25
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1381 on: 06/05/2014 12:12 PM »
TS fixers:  How horrified would you be to know that macroblock 0:0 starts at 67 on some of the p frames instead of the normal 66?  I've noticed nothing visually unusual about the frames that start at 67, but...

I wouldn't be surprised - in one of the marathon fixing sessions I was missing a Program Map Table packet that should have appeared by a certain point. I eventually found it shifted right by one bit, so I just overwrote the entire thing with the correct PMT. I didn't look into the contents of the data packets, but it wouldn't be a big shock to hear there are further bitshifts in there.

I believe that a bitshift shouldn't affect the MPEG4 stream much as it's bit-aligned for much of the time. From reading the MPEG4 spec it seems as though sometimes during the headers it aligns to a byte, but once we're into macroblocks then the stream isn't byte aligned and a bitshift won't do much damage. If this is wrong please let me know!

Offline mhenderson

  • Member
  • Posts: 69
  • USA
  • Liked: 101
  • Likes Given: 18
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1382 on: 06/05/2014 02:11 PM »
We got the tail end, it sure would be nice to see the rest. I am NOT complaining, this puzzle has enough pieces already.  Getting ten seconds of good transmission might **still be** nice, though.

Collecting several comments over the last 24 hours:

@Quialiss said "The few seconds before 'our' video are from earlier in the descent, I'd recognize that dust spot anywhere.  No timestamps to tell when that was in relation to the start of the recovered video, though...   "  (Referring to the video SpaceX showed yesterday at the Atlantic Council conference)

@ajmartin said "Oh, and I noticed something else that seems interesting: the values of two of the header fields, PCR and vop_time_increment, consistently indicate that the video recording started 61 seconds before the start of the file.  (Presumably the rocket was too far away for a useful signal to be received during that time.)"

@Princess said "The current TS file starts at the first I-frame, but the rocket transmitted a few P-frames of data before that. So as part of my "what the rocket transmitted" work I'm also fixing those P-frames up to the same standard - we won't be able to use them in videos (as there's no I-frame to relate them to), but the work could help in statistical analysis of errors."

edit: changed "have been" to "still be" ... @ChrisBergin, can you request 10 seconds of .ts from earlier in the flight?
« Last Edit: 06/05/2014 02:16 PM by mhenderson »

Offline wronkiew

  • Full Member
  • *
  • Posts: 186
  • 34.502327, -116.971697
  • Liked: 105
  • Likes Given: 116

Offline mvpel

  • Full Member
  • ****
  • Posts: 1116
  • New Hampshire
  • Liked: 1280
  • Likes Given: 1676
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1384 on: 06/05/2014 03:10 PM »
I'm pretty sure SpaceX has much more raw .ts to supply.  The released portion just contains what they're most interested in recovering.

Hundreds of frames of looking down towards the cloud deck would be pretty boring to watch, I expect. But that might be an interesting data set to try to apply the ideas that AJMartin proposes with respect to algorithmically identifying bit flips and shifts based on the FEC coding patterns, since that section of the transport stream and the frame data could very well be considerably more uniform than this section.
"Ugly programs are like ugly suspension bridges: they're much more liable to collapse than pretty ones, because the way humans (especially engineer-humans) perceive beauty is intimately related to our ability to process and understand complexity. A language that makes it hard to write elegant code makes it hard to write good code." - Eric S. Raymond

Offline jpfulton314

  • Life Member
  • Member
  • Posts: 35
  • Alamogordo, New Mexico, USA
  • Liked: 17
  • Likes Given: 52
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1385 on: 06/05/2014 04:09 PM »
This story made the pages of Slashdot!  See: http://beta.slashdot.org/story/202953

It makes me proud to be a member here.

Offline SwissCheese

  • Full Member
  • *
  • Posts: 164
  • Liked: 249
  • Likes Given: 80
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1386 on: 06/05/2014 04:17 PM »
My computer just had a major breakdown  :'(, I probably won't be able to help much in the next days...

Offline Quialiss

  • Member
  • Posts: 75
  • Liked: 82
  • Likes Given: 29
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1387 on: 06/05/2014 05:06 PM »
My computer just had a major breakdown  :'(, I probably won't be able to help much in the next days...

Noooooooooo!

... Maybe by the time you get back I'll be able to recover p frames well enough you don't have to fix all my 1-2 off errors in alignment.   ;D

Offline cscott

  • Senior Member
  • *****
  • Posts: 2638
  • Liked: 1843
  • Likes Given: 658
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1388 on: 06/05/2014 05:14 PM »
We got the tail end, it sure would be nice to see the rest. I am NOT complaining, this puzzle has enough pieces already.  Getting ten seconds of good transmission might **still be** nice, though.

I'm pretty sure SpaceX has much more raw .ts to supply.  The released portion just contains what they're most interested in recovering.

see:
http://www.reddit.com/r/spacex/comments/24bsn2/first_stage_landing_video/ch6eqte?context=3

I'm guessing that telemetry and video were interleaved, and they've given us the bitrange where they could confidently separate the telemetry bits from the video bits.  The in-house repair guy might have been trusted with some more "maybe video maybe telemetry" bits (might even be ITAR restrictions in play).   The more corrupted frames at the start and end of transmission are the ones where separating out the telemetry would be most difficult.

I'd suppose that Gwynne's earlier video is from the main tracking/telemetry sites before the first stage dipped over the horizon.  I agree that a segment of equivalent .ts might be interesting to us (and boring to the public at large!).

Offline mhenderson

  • Member
  • Posts: 69
  • USA
  • Liked: 101
  • Likes Given: 18
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1389 on: 06/05/2014 09:04 PM »
@wronkiew - The slowmo and clock fix are great. An interesting byproduct is how clearly the slowmo helps (me at least) see where the low hanging fruit is -- big improvement with narrowly focused attention. Thank you.

Chris Bergin has requested .ts data from earlier in the flight. Fingers crossed.

Offline princess

  • Member
  • Posts: 65
  • Liked: 106
  • Likes Given: 25
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1390 on: 06/05/2014 10:32 PM »
Chris Bergin has requested .ts data from earlier in the flight. Fingers crossed.

I'll jump on that and do fixups as soon as we have it!

Offline jdnz82

@wronkiew - The slowmo and clock fix are great. An interesting byproduct is how clearly the slowmo helps (me at least) see where the low hanging fruit is -- big improvement with narrowly focused attention. Thank you.

Chris Bergin has requested .ts data from earlier in the flight. Fingers crossed.

Excellent! good to hear ( i asked this question on the other thread :) )

Offline Chris Bergin

Going to be difficult with the request as they are all back into launch nerves mode with the upcoming Static Fire of the F9 that had a tantrum on the pad.

Need the SpaceXers reading this thread, and I know there are a LOT of you, to be proactive and help us with that clip and the requests noted. I'm happy to be the go-between.

Offline mhenderson

  • Member
  • Posts: 69
  • USA
  • Liked: 101
  • Likes Given: 18
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1393 on: 06/06/2014 11:14 AM »
the F9 that had a tantrum on the pad.

Tell her she gets to go swimming and will be home again soon.

Offline Quialiss

  • Member
  • Posts: 75
  • Liked: 82
  • Likes Given: 29
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1394 on: 06/06/2014 01:28 PM »
Request for the web editor:

Allow switching between frames when the 'Isolate frame' box is checked.  Saving the isolated p frames and flipping between them to get alignment correct is the only p frame related thing I still have to do in an outside editor, getting rid of that step would make finalizing alignments so much less painful. 

Offline mhenderson

  • Member
  • Posts: 69
  • USA
  • Liked: 101
  • Likes Given: 18
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1395 on: 06/06/2014 02:50 PM »
Since 281 is the final iFrame and is currently washed out, it would be best to enhance the contrast so the video ends on a nicer image.

I see the online editor now has a "suggest improvement to MMB" button.  I like that a lot, submissions should be controlled and referreed by a benevolent dictator.

This is a step in the right direction, but I know my limits ... perhaps someone more skilled at this can take it over the finish line.

Frame 15(281)
X:3545:01,X:2003:A0,0:0:559,40:1:-1,0:2:-2:25:25:25:25:0:0,0:10:-2:60:60:60:60:0:0:40,
1:10:53276,32:10:-1::40,33:10:57352,0:13:69419:0:0:0:1:0:0:63,
28:15:-2:-70:-70:-70:-70:0:0:63,29:15:83102,8:29:-1

« Last Edit: 06/06/2014 03:28 PM by mhenderson »

Offline Chris Bergin

I didn't even say anything! :)

Yet.............just now:

Offline Chris Bergin

Oh you know what that means....SpaceXers are following the spacexlandingrestoration channel too. Great work guys!

(Very nice of SpaceX to add the @nasaspaceflight and thread link! :))

Offline spaceboy89

  • Member
  • Posts: 60
  • Guildford, UK
  • Liked: 6
  • Likes Given: 1
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1398 on: 06/06/2014 04:24 PM »
SpaceX posted the same on Facebook as well :)


Offline wronkiew

  • Full Member
  • *
  • Posts: 186
  • 34.502327, -116.971697
  • Liked: 105
  • Likes Given: 116
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1399 on: 06/06/2014 04:26 PM »
Apparently the Wikispaces API doesn't allow file uploads.  >:(

The auto-generated MMBs are uploaded here, in case you need to track down a bug:

http://adama.nocdirect.com/~wronkiew/spx_crs3/mmbs/

IainCole, can you change the youtube uploader to grab http://adama.nocdirect.com/~wronkiew/spx_crs3/landing_clockfix_3hr.mp4 ? I can then move the trigger to the 3hr script.

Tags: