Author Topic: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread  (Read 962543 times)

Offline arnezami

  • Full Member
  • **
  • Posts: 284
  • Liked: 262
  • Likes Given: 346
Don't think that's simple. You'd have to store the id of the block below and to the right of the block you are placing. Also every block AFAIK _HAS_ to reference either the top or the left, so you could only change the block to reference the other direction which would probably mess things up worse.
No what I mean is that you change its "transformation" data so that the result is that it is the "opposite" of what it receives from its left and top. That way no matter what block is placed right or bottom the will have data from left and top that is effectively the same as if that block is placed at 0,0.

I'm not sure changing the "transformation" data this way is possible (I guess its a mathematical problem). But I did'nt mean to solve this by making pointers to blocks that are right or bottom.

Another way of doing this is to say to a block: don't "listen" to your left block, instead "listen" to the left "block" of 0,0 (initial state I guess). I believe these things are called "predictions" in mepg-4.

We created a way of doing some of the work through a mini web app. You can find it here: http://spacexlanding.wikispaces.com/Online+Editor
Hi,

What would be really useful is when you click somewhere on the generated pixel it would put the x and y of the corresponding block in your clipboard (or show it somewhere on the page so yuu can copy-paste it from there.

Even more useful would be if you could parse the log on the serverside and when you click on the image you get x,y and bitposition in your clipboard! Or maybe simply the line from the log corresponding with the x and y you just clicked.

Just an idea.

Regards,

arnezami

I updated the image handler to do this (http://dz0bwiwndcjbh.cloudfront.net/info/<frame>?mmb=<mmb>) but I haven't got time to put it into the web app atm, I'll do that later unless someone else wants to

I've now updated the app to allow the selection of MB, I "think" it's right but let me know if it isn't

Online Chris Bergin

And a big high five and a welcome to you Iain!

Note from the SpaceX side (paraphrased) says that a request has been sent to provide a 10 second shot from the camera while the vehicle was still on the pad. This should go up on the page where the videos were posted on the SpaceX site, hopefully soon.

I'm told the pad shot is the most error free telem they have, but it's recognized the need for a shot later in flight that has the muddy streaks for reference. That is being inquired about, per the note I've been sent.

The main issue is everyone is busy with the upcoming flights, as can be expected.

We have a lot of SpaceX folk watching this thread and we'll get some additional eyeballs via an official PAO approach today. It's noted that people are impressed with the work going on here, so well done everyone!

PS Per the PAO approach, I think this process deserves a news article. It's fascinating enough and of interest to the readership. It may also shake the tree for a few more video experts, but we're clearly doing well on that score already. Problem with that is I pretty much know next to nothing about what you guys are doing from a tech standpoint, so if one of you guys is confident at being able to summarize, send me a message, as I can write around it and subedit. No rush on that.
« Last Edit: 05/07/2014 01:05 PM by Chris Bergin »

Offline Okie_Steve

  • Full Member
  • **
  • Posts: 270
  • Oklahoma
  • Liked: 72
  • Likes Given: 107
Chris,

If SpaceX is willing to provide a bit of information about the stream format it would be very helpful to know the location of any ECC/CRC errors and the packet size and any type of cross-interleave etc. used in the telemetry stream to help identify "possibly corrupt"and "probably correct" portions of the data.

Online Chris Bergin

Chris,

If SpaceX is willing to provide a bit of information about the stream format it would be very helpful to know the location of any ECC/CRC errors and the packet size and any type of cross-interleave etc. used in the telemetry stream to help identify "possibly corrupt"and "probably correct" portions of the data.

I'll ask....as much as you have already asked by posting it (such people watching the thread).

Offline Untribium

  • Member
  • Posts: 32
  • Switzerland
  • Liked: 32
  • Likes Given: 78
Amazing work! A few days ago I was worried this wouldn't go anywhere, especially considering that we're most likely (hopefully) going to get new footage in a few days :)
I put a little bit of work into iframe 7. Basically, I started with the line by mlindner and modified it slightly to fix the color issue around the legs.
X:18494:01, 19:05:-1, 20:06:20716,15:12:-1, 22:12:45309, 22:15:-1, 23:15:63096, 26:15:-1, 30:15:64161

Instead of taking the first "nice" block after some garbage, I take one further to the right and if necessary down, so that all the blocks that influence this block are non-garbage. Unfortunately, in this case the first nice block contains part of the left leg, so there's some detail lost...any ideas on how to fix the colors while keeping the good blocks?
Oh, could someone quickly explain what the first part of the line does (X:18494:01)? I read through all the posts, but I didn't find an answer (may have missed it...)

Edit: Found an improvement right after posting...
« Last Edit: 05/07/2014 03:18 PM by Untribium »

Offline wronkiew

  • Full Member
  • *
  • Posts: 186
  • 34.502327, -116.971697
  • Liked: 105
  • Likes Given: 125
All right, I have to give up for the night. This is my best run for iframe 12:

X:598:1,0:0:550,1:0:-1,14:09:29489,0:10:31664,0:11:34522,0:12:37439,
0:13:40079,0:14:43312:0:15:46476,0:16:49752,0:17:53744,0:18:57020,
0:19:59654,5:19:-1,06:19:60089,0:20:62314,0:21:64864,0:22:67531,
0:23:69892,0:24:72120,0:25:73937,0:26:75964,0:27:77748,0:28:79454,
0:29:86580

This one is a bit better for iframe 12, still not great:

X:552:1,X:553:1,0:0:550,1:0:-1,14:09:29489,0:10:31664,0:11:34522,
0:12:37439,0:13:40079,0:14:43312:0:15:46476,0:16:49752,0:17:53744,
0:18:57020,0:19:59654,5:19:-1,06:19:60089,0:20:62314,0:21:64864,
0:22:67531,0:23:69892,0:24:72120,0:25:73937,0:26:75964,0:27:77748,
0:28:79454,0:29:86580

Offline Smokablecheese

  • Member
  • Posts: 1
  • Liked: 0
  • Likes Given: 0
Just a small improvement on IFrame 9, I made a modification to Swisscheeses last edit


mmb 20:04:-1:0:0:-5:0:0:0,24:04:16023,1:15:-1,3:16:79127,41:19:-1,8:20:96220,15:04:16023

Offline moralec

Amazing work everyone.
Remember we have a wiki space for the project (http://spacexlanding.wikispaces.com/) for you to add your versions.

Offline Shanuson

  • Full Member
  • **
  • Posts: 297
  • Liked: 206
  • Likes Given: 618
How are the MB's ordered in the file?

Is it
a) left->right, top->bottom
b) zigzag like (see fig 7-4-c in the ISO file (attached))
c) something else?

I ask because AFAIUI we need the startbit of every MB in an Iframe so that if one MB is somehow broken the decoder still knows where the next begins.
It would be nice to know in which order one should work on the MB's.

Cheers,
Shanuson

Offline wronkiew

  • Full Member
  • *
  • Posts: 186
  • 34.502327, -116.971697
  • Liked: 105
  • Likes Given: 125
How are the MB's ordered in the file?

Is it
a) left->right, top->bottom
b) zigzag like (see fig 7-4-c in the ISO file (attached))
c) something else?

I ask because AFAIUI we need the startbit of every MB in an Iframe so that if one MB is somehow broken the decoder still knows where the next begins.
It would be nice to know in which order one should work on the MB's.

Cheers,
Shanuson

Left to right, top to bottom. Not like the attached diagram. However, blocks can affect others to the right and below.

Offline mlindner

  • Software Engineer
  • Senior Member
  • *****
  • Posts: 2040
  • Space Capitalist
  • Silicon Valley, CA -- previously in Ann Arbor, MI
  • Liked: 665
  • Likes Given: 253
How are the MB's ordered in the file?

Is it
a) left->right, top->bottom
b) zigzag like (see fig 7-4-c in the ISO file (attached))
c) something else?

I ask because AFAIUI we need the startbit of every MB in an Iframe so that if one MB is somehow broken the decoder still knows where the next begins.
It would be nice to know in which order one should work on the MB's.

Cheers,
Shanuson

First block is top left corner, and the blocks continue going left to right until they hit end of row, then go to next row and repeat.

Every block has 6 sub-blocks, 4 luma blocks (brightness) and 2 chroma blocks (color). The 4 luma blocks are the 4 squares in a block you often see. Both color blocks cover the entire block and are two different color planes. https://en.wikipedia.org/wiki/YUV There are 4 Y blocks and 1 U and 1 V block. Each one of those sub blocks references either the block above it or the block to the left of it (they all don't have to reference the same block). Which block they reference is determined on-the-fly (not hardcoded) so if a block is screwed up then it could cause blocks that would reference it, to not reference it or vice versa.
Internal combustion engine in space. It's just a Bad Idea.TM - Robotbeat

Offline R7

  • Propulsophile
  • Senior Member
  • *****
  • Posts: 2738
    • Don't worry.. we can still be fans of OSC and SNC
  • Liked: 954
  • Likes Given: 662
The zigzag image is how DCT (discrete cosine transformation) coefficients of a 8x8 block are ordered in the data stream. A macroblock has 6 of these blocks, four for 16x16 luminance information and two for 8x8 color information.

The compression trickery comes from the fact that often most of those coefficients are zero so they are written using variable length encoding, like almost anything in the protocol. The downside royal PITA is that single bit error can make the decoder go haywire, not only read wrong values but read too many or too few bits.

It's a wonder the pictures are as good as they are.

AD·ASTRA·ASTRORVM·GRATIA

Offline John_L

How would one go about re-encoding the video with the replacement iframes, what's the command line for ffmpeg?

Offline aero

  • Senior Member
  • *****
  • Posts: 2996
  • 92129
  • Liked: 797
  • Likes Given: 282
How soon will we see a .gif of the improved frames?
Retired, working interesting problems

Offline Swatch

  • Full Member
  • **
  • Posts: 273
  • Official Aerospace Engineer as of June 13th, 2009
  • Cincinnati
    • ProjectApollo/NASSP: Virtual Systems and Flight Simulation of the Apollo Program
  • Liked: 28
  • Likes Given: 16
This has been some amazing work to watch develop fellas, keep up the good work!
Ex-Rocket Scientist in Training, now Rocket Scientist!
M-F trying to make the world of the future a smaller place through expanding horizons...

Offline wronkiew

  • Full Member
  • *
  • Posts: 186
  • 34.502327, -116.971697
  • Liked: 105
  • Likes Given: 125
How would one go about re-encoding the video with the replacement iframes, what's the command line for ffmpeg?

This is a good point, and it got me thinking. If everything goes as planned, SpaceX will launch again in a few days, and I think they're planning to attempt recovery again. This could upstage the (arguably more historic) first landing video. We need to put together a video with what we've got today, tomorrow at the lastest. If necessary, I can hand-assemble something from the iframes and pieces of the pframes. It wouldn't be as pretty as what arnezami and mlindner can do, so I'll wait until I hear from them.

Matt

Offline mlindner

  • Software Engineer
  • Senior Member
  • *****
  • Posts: 2040
  • Space Capitalist
  • Silicon Valley, CA -- previously in Ann Arbor, MI
  • Liked: 665
  • Likes Given: 253
How would one go about re-encoding the video with the replacement iframes, what's the command line for ffmpeg?

This is a good point, and it got me thinking. If everything goes as planned, SpaceX will launch again in a few days, and I think they're planning to attempt recovery again. This could upstage the (arguably more historic) first landing video. We need to put together a video with what we've got today, tomorrow at the lastest. If necessary, I can hand-assemble something from the iframes and pieces of the pframes. It wouldn't be as pretty as what arnezami and mlindner can do, so I'll wait until I hear from them.

Matt

It's actually a lot easier than that. If no one else does it by Friday Eastern time then I'll be modifying ffmpeg to add a frame specifier to the mmb options. Then we simply pile all the mmb options into one long entry and run ffmpeg with it. viola, produced video. This would also allow us to try tweaking some p-frames if we so wish.

Producing a video now with what we have won't be very attractive yet. We need to fine tune a lot of the frames still. I'm doing that now for iframe8, which should hopefully look really really good. This means doing things like whenever you use -1 and replace the block, fine tune all the luma and chroma settings for it to have the least effect on surrounding blocks.
« Last Edit: 05/07/2014 05:51 PM by mlindner »
Internal combustion engine in space. It's just a Bad Idea.TM - Robotbeat

Offline John_L

It's actually a lot easier than that. If no one else does it by Friday Eastern time then I'll be modifying ffmpeg to add a frame specifier to the mmb options. Then we simply pile all the mmb options into one long entry and run ffmpeg with it. viola, produced video. This would also allow us to try tweaking some p-frames if we so wish.

Producing a video now with what we have won't be very attractive yet. We need to fine tune a lot of the frames still. I'm doing that now for iframe8, which should hopefully look really really good. This means doing things like whenever you use -1 and replace the block, fine tune all the luma and chroma settings for it to have the least effect on surrounding blocks.

So you're saying at this point, ffmpeg doesn't have the ability to re-encode with replacement iframes?

Offline AncientU

  • Senior Member
  • *****
  • Posts: 6243
  • Liked: 4078
  • Likes Given: 5597
This has been some amazing work to watch develop fellas, keep up the good work!
Yes, truly incredible effort!
So cool to see the first ever soft landing emerge frame by frame.
"If we shared everything [we are working on] people would think we are insane!"
-- SpaceX friend of mlindner

Tags: