Author Topic: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread  (Read 854694 times)

Offline Quialiss

  • Member
  • Posts: 75
  • Liked: 82
  • Likes Given: 29
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1260 on: 06/02/2014 12:46 AM »
Surprisingly ffmpeg does not ignore packets with wrong PIDs but includes them into the stream. The difference between the new and old ts file interpretation comes form adaptation fields that are disabled in the new .ts file. The bytes in them were previously discarded, but are now interpreted as mpeg4 data.

Firstly, thank you for catching that and working it out. The bit offsets from Quialiss make sense now.

Wizards, both of you! 

Princess, can you provide the logs with the informative errors for Part 7, Part 8 and Part 12 so I can fix up those iframes too?

Offline mvpel

  • Full Member
  • ****
  • Posts: 1116
  • New Hampshire
  • Liked: 1280
  • Likes Given: 1679
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1261 on: 06/02/2014 01:14 AM »
I never expected that ffmpeg could do something like that. If you have more than one stream in there, how should it decide to which it belongs? I would understand that it needs the right PID but the AF flag has to be set right too to get all of the data.

While I was going through spme of the parts I started to get a feeling that it wasn't quite ignoring everything that it should have, considering that my before- and after-cleaning images were coming out the same length when I knew that I had corrected TS headers that were supposed to be part of 0x3e8 but had had a bit or two flipped to change the PID. Hopefully one of us can make some time to test that - my money's on Princess. :D At the time I just thought I'd made a late-night mistake in extracting the images, but maybe not. This ill-at-ease sensation is probably why I started correcting every byte of padding packets to 0xff, even though their contents are supposed to be ignored. Maybe overkill, but it does restore the transport stream more closely to its original state in any case, even if it has no bearing on the frames.
"Ugly programs are like ugly suspension bridges: they're much more liable to collapse than pretty ones, because the way humans (especially engineer-humans) perceive beauty is intimately related to our ability to process and understand complexity. A language that makes it hard to write elegant code makes it hard to write good code." - Eric S. Raymond

Offline lgjy98d

  • Member
  • Posts: 10
  • Liked: 6
  • Likes Given: 0
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1262 on: 06/02/2014 02:25 AM »
I doesn't seem like rocket science, but it does seem like a lot of work specially on the online tool side (we don't want you to hate us Iaincole). Ideas?

VCSes - version control systems - are really good at moralec's database problem. One idea is to shoehorn this into a VCS by using a VCS as storage.

Here's one way to do that: force MMB lists into a VCS friendly format, by
a) breaking up the MMBs into one per line [VCSes tend to work best on whole lines at a time] and
b) sort the MMBs into a standard sort order, most likely the raster order most folks are already used to.  [EDIT: I guess this isn't strictly necessary, but may make IainCole's life easier]
c) Then you need to always check in and out of the VCS via something that understands the format. IainCole's website, or some other website, for example, would be the only entity with access to the VCS. This website would be responsible for parsing raw MMBs, and providing the MMBs back in a ffmpeg-friendly format.
d) Ideally this file format would have comments to allow for notes and simple alternatives. The VCS would be responsible for full 'branches' [git/mercurial is good for this].

And then a flat file format might come in handy in other ways.

There are other ways: you could use raw RCS diffs to manage things; you could find a diff program that's really good at big long lines (most aren't); you can code up the delta checking by hand; you can simply store all the MMBs ever in groups, normalized in the database, for users to sort out; etc.

This was pretty much exactly my suggestion here: http://forum.nasaspaceflight.com/index.php?topic=34597.msg1207365#msg1207365

The added bonus being that the online editor is already using that file to load MMBs.

People can either submit MMBs by pull request, or a few key people can be made collaborators on that git repo to add them in based off submissions to the wiki.

Edit: There's obviously some administrative overhead with making sure that the wiki / json file are in sync, but short of writing a new system to do the job of both things, I think it's the simplest option.

I consider pull-request-for-best-mmbs approach to be too complex for majority or video recovery effort participants. Wiki was simple but not simple enough. I've gone 1 step further, introduced Google Spreadsheets. You can compare the original and spreadsheet version of the page about 4th part:
 * http://spacexlanding.wikispaces.com/Frames+Part+4
 * http://spacexlanding.wikispaces.com/Frames+Part+4+Spreadsheet

To unify best MMB storage and facilitate structural data retrieval I'd setup 15 Google Spreadsheets (one per part) and for convenience linked them in Spreadsheet column to Progress table of http://spacexlanding.wikispaces.com/raw_final_fixedMMB.

There is an idea to make Online Editor to pull "Best MMBs" data direcly from Spreadsheet tables, and idea is laid out in more details at: https://github.com/IainCole/SpaceXVideoApp2/pull/1.

Offline arnezami

  • Full Member
  • **
  • Posts: 282
  • Liked: 262
  • Likes Given: 341
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1263 on: 06/02/2014 03:05 AM »
OK so I used to consider myself good with coding.  I am a complete newbie when I comes to MPEG4, so I tried the online editor and it's still beyond mortal man w/o lots of explanation.

What I am proposing to get more eyes working on this is to focus on getting correct data packets and then the rest of the crowd can start flipping bits in the payload.  What I am hoping for (it may not be possible due to the compression) is to be able to break the payload down to specific parts and display them as such.  Then if the tool can allow us to concentrate on the payload part by part, mortal eyes may have a better chance of flipping bits to start fixing the rest of the picture.  But the key is to get us to the point where we can focus on the parts of the payload as opposed to try to guess at the whole payload as appears to be the case right now.

My other comment is as the online editor gets used by more people - you probably need to moderate inputs before committing them to a master database so we don't inadvertently screw something up and 2. we guard against some malicious event (not that I am expecting that, but stranger things have happened).

Standing by to help, but we have to get this a bit less technical.
Hi hutchel,

I am assuming you are not the only one who is feeling this "steep learning curve". So I really want to ask you about your opinion (I might be too "deep into this" and not see it). @all_potential_helpers: If you had a similar feeling like hutchel did then the following is also meant for you.

1 - Do you think we need better tutorials for using the editor? What would you consider better?
2 - Do you think the online editor should be more intuitive to use? What is not intuitive right now?
3 - Do you need more feedback flipping single bits or changing bitpositions? Do you need to see the actual bits? What kind of feedback apart from the resulting frame?
4 - Do you need to have a better explanation about MPEG4? Do you want to see a concrete example based on the actual video data?
5 - Do you want simpler and more concrete tasks to do? It's now divided into roughly 300 parts. Should we divide it more? Or should it be divided differently?

Can you please at least sort the above numbers in order of priority? And then explain why you think some are more important than others? And maybe then try to answer (some of) the questions? Thanks!

Could you also tell what you read (tutorial/wiki page wise) before using the online editor? Because maybe we can do something about the (more prominent) placement of the explanations already available on the wiki/forum.

I really like to know. And hopefully you can help us here.

Thanks in advance.

Regards,

arnezami
« Last Edit: 06/02/2014 03:07 AM by arnezami »

Offline Quialiss

  • Member
  • Posts: 75
  • Liked: 82
  • Likes Given: 29
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1264 on: 06/02/2014 04:51 AM »
1 - Do you think we need better tutorials for using the editor? What would you consider better?
2 - Do you think the online editor should be more intuitive to use? What is not intuitive right now?
3 - Do you need more feedback flipping single bits or changing bitpositions? Do you need to see the actual bits? What kind of feedback apart from the resulting frame?
4 - Do you need to have a better explanation about MPEG4? Do you want to see a concrete example based on the actual video data?
5 - Do you want simpler and more concrete tasks to do? It's now divided into roughly 300 parts. Should we divide it more? Or should it be divided differently?

Short version: 3 and to a lesser extent 1.  2 4 5 are completely off my radar for concerns, but I'm not your ideal person because I haven't had a problem with the learning curve.


Lets see... I started contributing about one week ago with exactly zero knowledge of video recovery except what I'd picked up by following the efforts here, as I'd been following what was going on in the recovery effort since it started.  I couldn't see a way to contribute until the web editor went online.  That dropped the barrier to entry monumentally, and I've been fine with the learning curve from that point, so I'm not exactly the perfect person to answer the questions, but it's a POV from someone new to the recovery effort with no relevant prior experience. 

I started by looking at the work done by other people and trying to figure out how they'd done it... moving the blocks around and nulling out others was self explanatory, but I couldn't figure out how in the world people found the correct start positions for blocks in the i frames when they're wrong, and honestly I'm still not sure about that part and if it's more than trial and error.  So this is the one thing that I definitely would like some more information on.  If I recall correctly I fiddled around for a day or three trying things out before I started getting results that I thought were good enough to share. 

The p frame tutorial is great!  The first thing I did was start with p frames, and they were surprisingly easy to figure out, much easier than repairing i frames... of course, by pure chance I chose a set to start on that was only slightly damaged, but I still find it easier to work on p frames.

I went back to the i frames after that and still couldn't figure out how to get more data out of them... so I figured I'd work on all the luminance and chroma errors that plague most of the iframes.  I tried to start by just looking at the corrections people had made, didn't get terribly good results, so I went digging into the posts about macroblock structure and sub blocks, and how values propagate through the frame.  I had to go back several times and reread and pick up on new things as I was working, but the existing information on the wiki was sufficient to understand how to improve frames that way. 

These would be the two resources I kept going back to for reference.  I STILL don't understand why the propagation of values changes so dramatically sometimes, but I've learned to work with it.  (If I understand correctly the direction a block inherits from is dynamic.. I just don't quite understand why it switches seemingly arbitrarily due to luminance changes)
http://spacexlanding.wikispaces.com/MMB+Syntax 
http://forum.nasaspaceflight.com/index.php?topic=34597.msg1194911#msg1194911 - macroblock and subblock structure

And I didn't exactly reference this, but I have a scratchpad where I keep things like 26:10  dust spot, 21:14 left leg base, 15-16:11 left leg tip, which help in aligning features.  Knowing where the static features in the frames should be helps a lot when working with p frames, and the coordinates help much more than inexact circles on an image. 
http://forum.nasaspaceflight.com/index.php?topic=34597.msg1196634#msg1196634 - locations of the dust on the camera. 

Offline Quialiss

  • Member
  • Posts: 75
  • Liked: 82
  • Likes Given: 29
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1265 on: 06/02/2014 05:11 AM »
And to follow the wall of text, a bit of eye candy and showing some of the repair process.  I saved the image I was working on periodically to check chroma values at a higher saturation and to make sure I'm not missing too much.  I decided to keep the intermediary images and animate the process. 

There's no new data in the frame, just a lot of luminance and chroma adjustments.  You can see the left-right, top-down progression and then a final pass on chroma issues, there's around ~10 changes in each frame.

Online saliva_sweet

  • Full Member
  • ****
  • Posts: 508
  • Liked: 380
  • Likes Given: 1252
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1266 on: 06/02/2014 07:22 AM »
I started by looking at the work done by other people and trying to figure out how they'd done it... moving the blocks around and nulling out others was self explanatory, but I couldn't figure out how in the world people found the correct start positions for blocks in the i frames when they're wrong, and honestly I'm still not sure about that part and if it's more than trial and error.  So this is the one thing that I definitely would like some more information on.

For me it's trial and error really. But you really need ffmpeg to be set up locally to do it. What I've been doing is  running a script that tries out a range of starting positions (usually you can eyeball a region where there's a reasonably good chance to hit upon good data, e.g. lots of green blocks usually means that the search will come up empty). All the resulting images are dumped to a folder and can later be quickly flipped through in an image viewer to see if any positions worked. Same goes for flipping bits, flip a range of bits and look at all results(I try both single flips and the triple flip pattern identified previously in this thread 1000000000000011), but you need to be confident in MB start positions, so it really only works immediately after good data.

Offline arnezami

  • Full Member
  • **
  • Posts: 282
  • Liked: 262
  • Likes Given: 341
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1267 on: 06/02/2014 08:00 AM »
And to follow the wall of text, a bit of eye candy and showing some of the repair process.  I saved the image I was working on periodically to check chroma values at a higher saturation and to make sure I'm not missing too much.  I decided to keep the intermediary images and animate the process. 

There's no new data in the frame, just a lot of luminance and chroma adjustments.  You can see the left-right, top-down progression and then a final pass on chroma issues, there's around ~10 changes in each frame.
I love your visualization of the luma/chroma fixing! It shows its power :)

It's very cool that after doing the extracting, positioning and now the repairing, the incredible result of the work of all people who worked on it, finally hits you.  8)
« Last Edit: 06/02/2014 10:05 AM by arnezami »

Offline Geron

  • Full Member
  • **
  • Posts: 214
  • Liked: 55
  • Likes Given: 6
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1268 on: 06/02/2014 08:26 AM »
You guys are awesome, strong work!!

Offline arnezami

  • Full Member
  • **
  • Posts: 282
  • Liked: 262
  • Likes Given: 341
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1269 on: 06/02/2014 09:11 AM »
Stunning!


Online moralec

Amazing work guys! Both the recent frame fixes and the redraw of the clock look amazing.

Regarding, the questions arenzami posted, I would like to invite all to join  the discussion. There are probably a lot of  newcomers, interested in helping out, and the better we incorporate them to our work the best final video we will get.

I assume that some of the people that still visit this forum actually tried to help, but failed and quit trying due to the technical difficulties associated with it. If any of you are reading this, your input will be greatly appreciated.

Offline hutchel

  • Overzealous Enthusiast
  • Full Member
  • *
  • Posts: 104
  • Washington, DC
  • Liked: 5
  • Likes Given: 36
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1271 on: 06/02/2014 11:37 AM »


1 - Do you think we need better tutorials for using the editor? What would you consider better?
2 - Do you think the online editor should be more intuitive to use? What is not intuitive right now?
3 - Do you need more feedback flipping single bits or changing bitpositions? Do you need to see the actual bits? What kind of feedback apart from the resulting frame?
4 - Do you need to have a better explanation about MPEG4? Do you want to see a concrete example based on the actual video data?
5 - Do you want simpler and more concrete tasks to do? It's now divided into roughly 300 parts. Should we divide it more? Or should it be divided differently?

I guess I was impatient and only experimented a little bit.  So let me see if I can address 1-5
1/2/3.   The online editor seems to be straight forward - although a bit level display might help when we get to the payload work.  I suspect the issue is I haven't really taken the time to fully understand the embedded format - that may be the crux of the issue - I think I was asking for the editor to completely decode the format and display the decoded parts for us to address individually.  I think that way helpers can focus on the values of the fields as opposed to having to decode on the fly.
4.  This may help also - How do the good blocks look and an explanation of how one impacts others.
5.  Discrete tasks are good, but again, I think the biggest barrier to entry is the steep learning curve .... I suspect most potential helpers have an hour or 2 here and there they can give to the effort.  If it takes 10 or more hours of investigation and trial or error to get anything, then most will remain on the sidelines.

Maybe a concise enough tutorial for each set of tasks.  i.e. fix abc in the following blocks.  Here's the mechanics (and maybe the theory) of how to fix abc.....

I recognize the data is compressed, what I keep struggling with is why all the padding that you are finding in the stream?  That doesn't make sense.   If I was compressing, the padding is the easiest to get rid of.

This is going to sound like a no nothing, but are we to the point in the effort to make a non-compressed stream?  It might be easier to work on.  It appears you are doing that a little at a time to tweak it, but the results appear to be recompressed and inserted back into the compressed stream.

Just some idle thoughts from someone who would like to help, but can only give you an hour here or there....

I will say I am VERY impressed with those that have been willing to give so much more to this effort than I.

edited to fix quotes
« Last Edit: 06/02/2014 11:38 AM by hutchel »

Offline mvpel

  • Full Member
  • ****
  • Posts: 1116
  • New Hampshire
  • Liked: 1280
  • Likes Given: 1679
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1272 on: 06/02/2014 12:10 PM »
I recognize the data is compressed, what I keep struggling with is why all the padding that you are finding in the stream?

The uplink has a constant bit rate, so it will send a fixed number of bits in one second regardless. But the video stream produces about 15 frames per second and no more, and if all those frames are compressed well enough to wind up using less than the number of bits that are available in the uplink in that one second... it's a "stream," so it can't just stop transmitting or else the receiver would think something broke - every available transport stream packet has to be filled with something, even if all the video data has already been sent. That's where the padding comes in.
« Last Edit: 06/02/2014 12:14 PM by mvpel »
"Ugly programs are like ugly suspension bridges: they're much more liable to collapse than pretty ones, because the way humans (especially engineer-humans) perceive beauty is intimately related to our ability to process and understand complexity. A language that makes it hard to write elegant code makes it hard to write good code." - Eric S. Raymond

Offline mhenderson

  • Member
  • Posts: 69
  • USA
  • Liked: 101
  • Likes Given: 18
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1273 on: 06/02/2014 12:10 PM »
Feature request: Excel has an "audit..., trace dependents" feature. Choose a cell and see which cells are dependent upon it (a chain of formula references). There is also a "trace precedents" feature to see which cells contribute to the current cell.

Excel shows related cells visually with arrows. Nice, but a simple list would suffice.

Similar functions would help here.

Typing on mobile, hope I am clear.

Offline mvpel

  • Full Member
  • ****
  • Posts: 1116
  • New Hampshire
  • Liked: 1280
  • Likes Given: 1679
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1274 on: 06/02/2014 12:16 PM »
Stunning!

I think there's going to be some SpaceX people weeping for joy this morning when they see that. Just... wow.
"Ugly programs are like ugly suspension bridges: they're much more liable to collapse than pretty ones, because the way humans (especially engineer-humans) perceive beauty is intimately related to our ability to process and understand complexity. A language that makes it hard to write elegant code makes it hard to write good code." - Eric S. Raymond

Online Chris Bergin

Stunning!



Amazing!

I'd note that you guys need to think about the next milestone point, then we should think about another updated video and we'll get that out there like the previous.

We're not going to have Elon tweeting every time, but when we get to the point we've done all we can, we can show the milestones.

When we do get to that "we've done all we can" point, then we should consider an article on site. Heck, we have Elon quotes now! :) I'll need a LOT of help with to translate the technical to something readable by Joe Public, quote some of the team here and use the Elon quotes, etc.

Also, it was suggested to me we could also have a highly technical overview as a press release style write up. I'm cool with all of that, obviously.

Something to keep in mind.

PS Per the next milestone video. What we absolutely should do is "where we are" before the ORBCOMM launch, as that launch is probably going to have a good video beamed back. Regardless, this is the historic first, so there's no value lost on this effort if ORBCOMM sends a good video back.
« Last Edit: 06/02/2014 12:33 PM by Chris Bergin »

Offline RanulfC

  • Senior Member
  • *****
  • Posts: 4416
  • Heus tu Omnis! Vigilate Hoc!
  • Liked: 771
  • Likes Given: 32
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1276 on: 06/02/2014 01:09 PM »
Amazing!

I'd note that you guys need to think about the next milestone point, then we should think about another updated video and we'll get that out there like the previous.

We're not going to have Elon tweeting every time, but when we get to the point we've done all we can, we can show the milestones.

When we do get to that "we've done all we can" point, then we should consider an article on site. Heck, we have Elon quotes now! :) I'll need a LOT of help with to translate the technical to something readable by Joe Public, quote some of the team here and use the Elon quotes, etc.

Also, it was suggested to me we could also have a highly technical overview as a press release style write up. I'm cool with all of that, obviously.

Something to keep in mind.

PS Per the next milestone video. What we absolutely should do is "where we are" before the ORBCOMM launch, as that launch is probably going to have a good video beamed back. Regardless, this is the historic first, so there's no value lost on this effort if ORBCOMM sends a good video back.

Just another little "tidbit" to keep in the back of your minds is this:

If the history of "space" related aerospace is anything like the "aero" side then at some point "space-ports" and "space-bases" will take on names derived from supporters and pioneers who helped things along.

So, the "question" then becomes do we push for "NASASpaceflightForums" Spaceport or just name it "Chris Bergin" Space Force Base and name all the streets after the L2 members?

Remember, it's never to Early to start thinking about the Future! And how to mess with it's inhabitants :)

Randy
From The Amazing Catstronaut on the Black Arrow LV:
British physics, old chap. It's undignified to belch flames and effluvia all over the pad, what. A true gentlemen's orbital conveyance lifts itself into the air unostentatiously, with the minimum of spectacle and a modicum of grace. Not like our American cousins' launch vehicles, eh?

Offline Lee Jay

  • Elite Veteran
  • Global Moderator
  • Senior Member
  • *****
  • Posts: 6652
  • Liked: 936
  • Likes Given: 138
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1277 on: 06/02/2014 01:27 PM »
PS Per the next milestone video. What we absolutely should do is "where we are" before the ORBCOMM launch, as that launch is probably going to have a good video beamed back. Regardless, this is the historic first, so there's no value lost on this effort if ORBCOMM sends a good video back.

I think the Orbcomm video, even if it's solid, will be in darkness.  So, this video will remain unique even if the Orbcomm video is good.

Offline rocketguy101

  • Member
  • Full Member
  • ****
  • Posts: 827
    • Strib's Rocket Page
  • Liked: 201
  • Likes Given: 595
Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
« Reply #1278 on: 06/02/2014 01:44 PM »
Stunning!

I think there's going to be some SpaceX people weeping for joy this morning when they see that. Just... wow.
Trust me, there are non SpaceX people jumping for joy when they log on in the mornings to see advances made overnight!!! 8)
David

Online Galactic Penguin SST

PS Per the next milestone video. What we absolutely should do is "where we are" before the ORBCOMM launch, as that launch is probably going to have a good video beamed back. Regardless, this is the historic first, so there's no value lost on this effort if ORBCOMM sends a good video back.

I think the Orbcomm video, even if it's solid, will be in darkness.  So, this video will remain unique even if the Orbcomm video is good.

It probably won't if the launch slips by "only" 2 days, which I think might be possible...... ;)
Chinese spaceflight is a cosmic riddle wrapped in a galactic mystery inside an orbital enigma... - (not) Winston Churchill

Tags: