NASASpaceFlight.com Forum

SpaceX Vehicles and Missions => SpaceX Reusability => Topic started by: Chris Bergin on 04/30/2014 07:43 PM

Title: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 04/30/2014 07:43 PM
Certainly needs cleaning up! Video experts around the world requested to help SpaceX with this.

http://www.youtube.com/watch?v=7m8H8OlJ3o8

Best repair so far:
https://www.youtube.com/watch?v=er66BActC4E

Background:

CRS-3 Falcon 9 first stage to sport legs and attempt soft splashdown
http://www.nasaspaceflight.com/2014/02/crs-3-falcon-9-first-stage-sport-legs-attempt-soft-splashdown/

SpaceX outlines CRS-3 landing legs plan toward first stage recovery ambitions
http://www.nasaspaceflight.com/2014/02/spacex-crs-3-landing-legs-plan-first-stage-recovery-ambitions/

SpaceX Falcon 9 successfully launches CRS-3 Dragon
http://www.nasaspaceflight.com/2014/04/spacex-crs-3-dragon-new-milestones/

Rockets that return home – SpaceX pushing the boundaries
http://www.nasaspaceflight.com/2014/04/rockets-return-home-spacex-pushing-boundaries/

--

UPDATE:

The video updates, in order, are available on this channel:
https://www.youtube.com/channel/UCyZDgyJBYz3OXD3JbDJzNww

Here's the post-competition technical feature article - by the team's Lourens Veen!

http://www.nasaspaceflight.com/2014/06/recovering-falcon-9-ocean-landing-video-done/
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 04/30/2014 07:53 PM
Guy on twitter (not sure if he's on here, but in response to me on twitter....)

Ryan Ziolko ‏@MediumFidelity  6m
@nasaspaceflight It's going to be tricky, the file contains large sections like this: pic.twitter.com/E5Csw1sjXy
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Lars_J on 04/30/2014 07:53 PM
The original data files are available for download from SpaceX here:
http://www.spacex.com/news/2014/04/29/first-stage-landing-video

 
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: StephenB on 04/30/2014 07:59 PM
If the data is just not there (like those ffff's), then no recovery is possible, right?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jarnis on 04/30/2014 08:04 PM
If the data is just not there (like those ffff's), then no recovery is possible, right?

Well, some data could be transferred from known good images from the same camera as far as the fixed part of the image (rocket body) goes. Beyond that, it would require some imagination and frame-by-frame touching up. It would be partially fiction at that point...

If there only were some additional fully intact frames, a bunch of damaged ones between them could be interpolated but that just isn't there I think.

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: swlee on 04/30/2014 08:07 PM
On the SpaceX web page, they preface the videos with:
"Below is a look at the original video data from the first stage landing, recovered from the Falcon 9 onboard camera and shot right before splashdown".

Does this imply that the camera itself was recovered (and, by extension, at least some part of the first stage)?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Lars_J on 04/30/2014 08:10 PM
No, it is the *data* that has been recovered. (although they did find some pieces)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: billh on 04/30/2014 08:17 PM
No, it is the *data* that has been recovered. (although they did find some pieces)

Looks like they only found "some pieces" of the data, too.  :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: adrianwyard on 04/30/2014 08:18 PM
It's interesting to me that this demonstrates the value of using analog rather than digital signal transmission in some circumstances. Even an extremely noisy analog signal would have been easier to reconstruct than this one, with its macroblocking/i-frame image-segment tracking weirdness. It can be done, but would be easiest if you had access to the exact compression code used on the camera. If SpaceX are smart they'll release that info.

Before now I've been disappointed to see the Wireless Video System used for EVAs had quite a lot of static and dropouts (given that we live in an HD digital world) but in retrospect perhaps that is the best signal to send.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Lee Jay on 04/30/2014 08:21 PM
My recommendation for cleaning this up is to repeat the test.  :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: MTom on 04/30/2014 08:21 PM
If the data is just not there (like those ffff's), then no recovery is possible, right?

Well, some data could be transferred from known good images from the same camera as far as the fixed part of the image (rocket body) goes. Beyond that, it would require some imagination and frame-by-frame touching up. It would be partially fiction at that point...

If there only were some additional fully intact frames, a bunch of damaged ones between them could be interpolated but that just isn't there I think.

The top of many frames are more or less intact. Maybe they could be a good starting point.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: meekGee on 04/30/2014 08:32 PM
Which means, IMO, they should rotate the image 90 degrees between successive key frames, as far as the compression is concerned.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Lars_J on 04/30/2014 08:34 PM
Which means, IMO, they should rotate the image 90 degrees between successive key frames, as far as the compression is concerned.


So... Basically use an completely new codec?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 04/30/2014 08:44 PM
Hi guys,

Today I have done a low-level analysis of the raw.ts file. Most notably the "Transport Stream Headers" (see http://en.wikipedia.org/wiki/MPEG_transport_stream )

While it is bad, its not as bad as is being suggested above. The raw file consists of 23,838 ts-packets (containing 188 bytes each). Each one of these has a 4-byte header. These should always begin with the hexadecimal value "47". Of these 23,838 headers the following are correct:

4703e8: 15,014 times
471fff: 4,966 times
4743e8: 263 times
474000: 81 times
474020: 74 times

The actual video-packets have the code "4703e8" and as you can see there are quite a lot of them! And they contain a LOT of data.

The code "471fff" identify the so called null-packets and these contain a lot of FF's. But this is totally normal. Since they are "used for fixed bandwidth padding".

The other correct packets are basicly overhead. I will not go into them. But they are essential for the video working. There are a lot more than shown above btw.

The problem lies in the remaining bad headers (still several thousand): they get ignored by the video-player which causes it to act strangely. Also: a lot of bits (not just in the header but also in the content) are flipped. But my hope is that this is not the biggest issue: mpeg-4 is quite resilient.

I'm trying to figure out if I can "repair" most of the headers so the video player can at least decide what to do with the data in it (now it will just ignore it because they have the wrong PID, invalidating valid packets in the meantime).

This however is no simple task. :P

Fingers crossed.

Regards,

arnezami

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 04/30/2014 09:13 PM
Good luck Arnezami! If you have any success, also PM me as I'll hook you up with the SpaceX people (the e-mail address they provide is just an "inbox pile" address, I can get you direct).
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: meadows.st on 04/30/2014 09:30 PM
Hi guys,

Today I have done a low-level analysis of the raw.ts file. Most notably the "Transport Stream Headers" (see http://en.wikipedia.org/wiki/MPEG_transport_stream )

While it is bad, its not as bad as is being suggested above. The raw file consists of 23,838 ts-packets (containing 188 bytes each). Each one of these has a 4-byte header. These should always begin with the hexadecimal value "47". Of these 23,838 headers the following are correct:

4703e8: 15,014 times
471fff: 4,966 times
4743e8: 263 times
474000: 81 times
474020: 74 times

The actual video-packets have the code "4703e8" and as you can see there are quite a lot of them! And they contain a LOT of data.

The code "471fff" identify the so called null-packets and these contain a lot of FF's. But this is totally normal. Since they are "used for fixed bandwidth padding".

The other correct packets are basicly overhead. I will not go into them. But they are essential for the video working. There are a lot more than shown above btw.

The problem lies in the remaining bad headers (still several thousand): they get ignored by the video-player which causes it to act strangely. Also: a lot of bits (not just in the header but also in the content) are flipped. But my hope is that this is not the biggest issue: mpeg-4 is quite resilient.

I'm trying to figure out if I can "repair" most of the headers so the video player can at least decide what to do with the data in it (now it will just ignore it because they have the wrong PID, invalidating valid packets in the meantime).

This however is no simple task. :P

Fingers crossed.

Regards,

arnezami

Great work @arnezami. If there is a way to "crowdsource" the crowdsource (e.g. by breaking the effort into small brute force chunks that the rest of us can do and that you could reassemble), I volunteer.  I don't have the expertise to do the manual parsing but am happy to do some scripting or even manual editing if instructed on what needs to be done.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: MP99 on 04/30/2014 09:40 PM
While it is bad, its not as bad as is being suggested above. The raw file consists of 23,838 ts-packets (containing 188 bytes each). Each one of these has a 4-byte header. These should always begin with the hexadecimal value "47". Of these 23,838 headers the following are correct:

4703e8: 15,014 times
471fff: 4,966 times
4743e8: 263 times
474000: 81 times
474020: 74 times

The actual video-packets have the code "4703e8" and as you can see there are quite a lot of them! And they contain a LOT of data.

The code "471fff" identify the so called null-packets and these contain a lot of FF's. But this is totally normal. Since they are "used for fixed bandwidth padding".

It seems a real shame that they didn't use the ~25% of unwanted bandwidth for parity / CRC / redundancy. Ah well, roll on flight #10.

Anyway - best of luck with this.

Cheers, Martin
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: QuantumG on 04/30/2014 10:10 PM
So.... I did a little digging on that video stream.

Everything I've seen online so far, including what SpaceX put out, just looks like they used ffmpeg to pull out the frames.. which is what I did too. However, I also did ffmpeg -i raw.ts raw.mpg and then compared the bytes in the mpg to the ts.

It skips the first 847k of the ts, but after that it hums along. Mostly the data comes across in 184 byte chunks, with an occasional truncated or dropped chunk. There's some really big gaps though.

If you know how to extract I frames manually from a stream, you could probably easily pull at least one out of those big gaps.

The top 5 are:

8864 bytes at 0x0011d470
11120 bytes at 0x00170664
15952 bytes at 0x000d6260
23716 bytes at 0x000df508
23904 bytes at 0x001502dc

I wrote a little code to attempt it but I only saw one extra I frame and couldn't get it to convert to an image. Now I'm out of time.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IRobot on 04/30/2014 10:37 PM
The first thing I would do to improve video quality on future flights would be to encode a monochrome image. Color is useless for this type of image.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SpaceX_MS on 04/30/2014 10:45 PM
Great thread. If one of you guys can help, it will be worth your time and effort. This is an Elon call to arms.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Lar on 04/30/2014 11:04 PM
Great thread. If one of you guys can help, it will be worth your time and effort. This is an Elon call to arms.

Can you get the specs on the camera used including the software load (and it sounds like, especially the codec) ??
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: docmordrid on 05/01/2014 01:38 AM
Yes! We need confing and codec specs like GOP length and other mpeg-(4?) parameters.

And I agree: using monochromatic analog video would solve a LOT of problems where signal strength and  interference is an issue. Newer isn't always better.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: ugordan on 05/01/2014 01:52 AM
The first thing I would do to improve video quality on future flights would be to encode a monochrome image. Color is useless for this type of image.

MPEG already operates on luminance/chrominance components and most of the bits allocated are for luminance (i.e. monochrome) component. I'm not sure what you're suggesting, it wouldn't do much for improving video quality and color actually helps a lot when you have objects with similar brightness in the scene.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: neilh on 05/01/2014 02:34 AM
A "SpaceX team member" (same account they used for an AMA last year) posted the following comment

http://www.reddit.com/r/spacex/comments/2bsn2/first_stage_landing_video/ch6f8io
Quote
[–]spacexdevtty [+1] 8 points 2 hours ago (12|4)
Hey all, SpaceX team here. Just wanted to answer some frequently asked questions we've been getting:
Q: Why is the video so bad? A: This was recorded over a very lossy RF link.
Q: Why release the video? A: We've had some success here (and with a little outside help as well, see http://aeroquartet.com/[1] ) manually stitching the video back together, but it's time consuming work. We don't expect the video to be 100% recoverable but we're hoping folks out there with more MPEG expertise than we have can provide assistance recovering more of the video.
Q: What codec? Settings? A: MPEG 4 Part 2, P/I 15, 15fps, NTSC, fixed bitrate
Q: How did you create repair1.ts? A: This was a joint effort between us and an outside consultant. First we took a pass on the data to align every MPEG packet on a 188 byte boundary and set the packet start byte to 0x47. Then we identified blocks within keyframes that contain bit errors, and then manually flipping bits in those corrupt blocks to see if it recovers more of the image.
Q: What is a TS file? A: http://lmgtfy.com/?q=MPEG+TS[2]
Feel free to post more questions here, we'll try and respond. We really appreciate all the help and great ideas! Thank you!!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IRobot on 05/01/2014 07:53 AM
The first thing I would do to improve video quality on future flights would be to encode a monochrome image. Color is useless for this type of image.

MPEG already operates on luminance/chrominance components and most of the bits allocated are for luminance (i.e. monochrome) component. I'm not sure what you're suggesting, it wouldn't do much for improving video quality and color actually helps a lot when you have objects with similar brightness in the scene.
I think we would all be happy with objects with similar brightness, if we could see them. I wouldn't mind if the whole rocket looked of the same brightness as long as I could see the flame, legs and ocean.

So color is useless here.

Also slight color noise introduces a lot of information between frames.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Adaptation on 05/01/2014 08:11 AM
Quote
First we took a pass on the data to align every MPEG packet on a 188 byte boundary and set the packet start byte to 0x47.

I'm not seeing that.  For instance location 26EC is divisible by 188 (0XBC) but its value is 4F not 47.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: docmordrid on 05/01/2014 08:13 AM
Color video, via chroma subsampling, also reduces apparent resolution. An example is NTSC where colors are spread across 2 horizontal lumance pixels. You end up with ˝ the horizontal resolution and full vertical resolution.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/01/2014 09:27 AM
Quote
First we took a pass on the data to align every MPEG packet on a 188 byte boundary and set the packet start byte to 0x47.

I'm not seeing that.  For instance location 26EC is divisible by 188 (0XBC) but its value is 4F not 47.
I can concur. In the repair1.ts file the sync bytes have not been "fixed" to 47 (hex). Maybe they uploaded the wrong file? Not that this does much: the rest of the header is usually broken as well. Also, in the raw.ts there are 5 places where I had to add exactly 56 bytes in order for the headers to align on the 188 bytes grid.

Anyway. Back to trying to get a little more life out of this video.  ;)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: ugordan on 05/01/2014 11:24 AM
I think we would all be happy with objects with similar brightness, if we could see them. I wouldn't mind if the whole rocket looked of the same brightness as long as I could see the flame, legs and ocean.

So color is useless here.

You're not getting me. When everything is the same brightness, you have no idea what you're looking at, where one object ends and the next one begins. This is where color is important, to distinguish between different materials.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: ugordan on 05/01/2014 11:26 AM
Color video, via chroma subsampling, also reduces apparent resolution. An example is NTSC where colors are spread across 2 horizontal lumance pixels. You end up with ˝ the horizontal resolution and full vertical resolution.

Chroma subsampling is precisely why I said luminance takes the most bits for encoding, because the 2 chroma channels are reduced in resolution by a factor of 4 or more. Luminance is not subsampled, unless you tell the encoder to do that for some reason.

The upshot of all this is that having color as additional information in the stream really doesn't have as big an impact as some would like to believe. It's not like it's 3 full red/green/blue frames encoded and 3x the bandwidth of a single monochrome image.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: input~2 on 05/01/2014 12:24 PM
This frame at 14 sec. seems interesting
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: AJA on 05/01/2014 12:51 PM
Ok..let me throw this out there: Would having the telemetry bitstream as opposed to only the video bitstream help? I'm thinking on the lines of whether additional error correction coding etc. would introduce errors in the TSH, or in the error correction code of the videostream. Somewhat like a frame-shift mutation, or an insertion/deletion mutation.


Color video, via chroma subsampling, also reduces apparent resolution. An example is NTSC where colors are spread across 2 horizontal lumance pixels. You end up with ˝ the horizontal resolution and full vertical resolution.

Chroma subsampling is precisely why I said luminance takes the most bits for encoding, because the 2 chroma channels are reduced in resolution by a factor of 4 or more. Luminance is not subsampled, unless you tell the encoder to do that for some reason.

The upshot of all this is that having color as additional information in the stream really doesn't have as big an impact as some would like to believe. It's not like it's 3 full red/green/blue frames encoded and 3x the bandwidth of a single monochrome image.


How about a monochrome image through a red-filter? Reduced data (although not a 3x reduction as you point out), without reduction in discerning boundaries. After all, differential luminance in one channel is what contributes to discernibility in the composite. The ocean would be dark, while the surf kicked up (being white) would still give you a sense of "sea level". You'd probably be able to write some compression code that wouldn't bother transmitting pixels below a certain black value anyway. Plus, we know the colour of the ocean, and the colour of Falcon's legs to a good degree... so as long as you took the channel with the highest variance, you'd be able to reconstruct a fairly decent "false colour" image..?


I still think they should put a hundred memory cards inside small, sealed, brightly painted empty plastic boxes (only a data cable coming out), and stick it with strong, but water soluble adhesive to the inner wall of the first stage-second stage interface. Then offer a monetary reward to the local Florida fishing community for whoever returns most boxes :P


Or simply have a drone fly much closer to the stage than would be allowed for manned airborne assets.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Sohl on 05/01/2014 12:58 PM
There's a lot of interesting ideas presented here to get better video data, but the next scheduled flight is probably too soon to make any changes on the vehicle side.  Perhaps they can do better with aircraft or ship assets to be in a better position for signal acquisition (especially if weather cooperates).  Later on, it should not matter as much as they get close to and hopefully fully succeeding at a land landing.

But let's hope for calm seas on this next flight!   ;)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mmeijeri on 05/01/2014 12:59 PM
I still think they should put a hundred memory cards inside small, sealed, brightly painted empty plastic boxes (only a data cable coming out), and stick it with strong, but water soluble adhesive to the inner wall of the first stage-second stage interface. Then offer a monetary reward to the local Florida fishing community for whoever returns most boxes :P

They had what they called "Talon pods" on earlier flights where they tried to recover v1.0 first stages. I haven't heard anything about that this time.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: ugordan on 05/01/2014 01:42 PM
How about a monochrome image through a red-filter? Reduced data (although not a 3x reduction as you point out), without reduction in discerning boundaries.

But why bother? Assuming 4x4 subsampled chrominance channels, color video only has 12.5% more bits to encode than the same video where only luminance is encoded. Seeing as how bad the transmission losses were, that wouldn't have made any difference whatsoever.

Again, why go around making up solutions that would only be relevant for a couple of seconds before splashdown? The end goal is recovering the stage, not getting better video of a landing stage that ends up being lost anyway. A better approach is to just record the stream onboard for later playback and even that would not be needed for land boostback where the stage doesn't go over the horizon to primary range tracking.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IRobot on 05/01/2014 02:18 PM
How about a monochrome image through a red-filter? Reduced data (although not a 3x reduction as you point out), without reduction in discerning boundaries.

But why bother? Assuming 4x4 subsampled chrominance channels, color video only has 12.5% more bits to encode than the same video where only luminance is encoded. Seeing as how bad the transmission losses were, that wouldn't have made any difference whatsoever.
@AJA why red filter? Maybe a luminance filter with IR cut, but can't understand the reason to use the red filter.

12.5% is actually a lot. When receiving such error-prone signal, it makes a lot of difference. It also reduces transmission power requirements (for the same frame rate), therefore more power available for transmission, therefore better S/N ratio.

Also true monochrome cameras are up to 3x more sensitive, meaning less (camera) noise to start with.

Again, why go around making up solutions that would only be relevant for a couple of seconds before splashdown?
Because the way up is quite well documented. The way down is not.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: ugordan on 05/01/2014 02:29 PM
12.5% is actually a lot. When receiving such error-prone signal, it makes a lot of difference. It also reduces transmission power requirements (for the same frame rate), therefore more power available for transmission, therefore better S/N ratio.

Maybe. *If* that's the only telemetry sent. Who's to say the video wasn't multiplexed along all the other, high rate vehicle telemetry so the 12.5% for video is more of a noise in the total bandwidth budget?

Also true monochrome cameras are up to 3x more sensitive, meaning less (camera) noise to start with.

Seriously? At the codec quality settings they're using, the camera dirt that's deposited on the way down you're worried about camera noise?

Again, why go around making up solutions that would only be relevant for a couple of seconds before splashdown?
Because the way up is quite well documented. The way down is not.

Oh, I'm sure they have it quite well-documented. Just not in a format your typical rocket pr0n enthusiast likes. It's in the form of vehicle telemetry. That's gold, anything else is just gravy.

It still doesn't change my argument that any such solutions are just too much trouble for the amount of use they'll have eventually.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: rst on 05/01/2014 02:36 PM
Just a note:  for those who may not have noticed, there's a parallel discussion of the video and what may or may not be visible on particular frames on the CRS-3/SpX-3 discussion thread in the missions section:

http://forum.nasaspaceflight.com/index.php?topic=31513.1650
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: hrissan on 05/01/2014 02:41 PM
12.5% is actually a lot. When receiving such error-prone signal, it makes a lot of difference. It also reduces transmission power requirements (for the same frame rate), therefore more power available for transmission, therefore better S/N ratio.

Maybe. *If* that's the only telemetry sent. Who's to say the video wasn't multiplexed along all the other, high rate vehicle telemetry so the 12.5% for video is more of a noise in the total bandwidth budget?

Also true monochrome cameras are up to 3x more sensitive, meaning less (camera) noise to start with.

Seriously? At the codec quality settings they're using, the camera dirt that's deposited on the way down you're worried about camera noise?

Again, why go around making up solutions that would only be relevant for a couple of seconds before splashdown?
Because the way up is quite well documented. The way down is not.

Oh, I'm sure they have it quite well-documented. Just not in a format your typical rocket pr0n enthusiast likes. It's in the form of vehicle telemetry. That's gold, anything else is just gravy.

It still doesn't change my argument that any such solutions are just too much trouble for the amount of use they'll have eventually.
Worked on sort of "telemetry", we divided data into what's important (10%, in this case sensors data) and what's less important (90%, in this case video feed). Both were CRC-ed, but the first got retransmitted if corrupted or lost, the second was not. Sort of TCP and UDP.

On worsening channel the retransmissions of sensor data occupied more and more bits until no bits were available to video.

I guess SpaceX does the same, so if we see SOME video, it means ALL sensor data was received without gaps.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IRobot on 05/01/2014 02:47 PM
Oh, I'm sure they have it quite well-documented. Just not in a format your typical rocket pr0n enthusiast likes. It's in the form of vehicle telemetry. That's gold, anything else is just gravy.
If you want to go that way, we don't need a video. And yet we have one.

It still doesn't change my argument that any such solutions are just too much trouble for the amount of use they'll have eventually.
What trouble? Switch the camera for a mono version? Or reconfigure the camera to transmit in gray scale? You make it sound like mono cameras are troublesome, but in fact they are EXACTLY the same! Usually manufacturers supply the same camera in mono and color version.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: ugordan on 05/01/2014 03:00 PM
Oh, I'm sure they have it quite well-documented. Just not in a format your typical rocket pr0n enthusiast likes. It's in the form of vehicle telemetry. That's gold, anything else is just gravy.
If you want to go that way, we don't need a video. And yet we have one.

Exactly. We don't *need* a video. Having video and needing video are separate things.

It still doesn't change my argument that any such solutions are just too much trouble for the amount of use they'll have eventually.
What trouble? Switch the camera for a mono version? Or reconfigure the camera to transmit in gray scale? You make it sound like mono cameras are troublesome, but in fact they are EXACTLY the same! Usually manufacturers supply the same camera in mono and color version.

Did you read what I wrote about monochrome video bandwidth impacts and your point about camera noise? I'm not sure why you're thinking that switching to grayscale video would simply sort out all issues with transmission, which is ultimately the only issue we had here. It's not color. It's not camera noise. It's packet loss.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Lars_J on 05/01/2014 03:27 PM
Ugordan is right. Switching to monochrome would not save much bandwidth. (if any) Don't argue if you don't understand how chroma (color) is sub-sampled compared to luminosity (brightness).
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IRobot on 05/01/2014 04:05 PM
Ugordan is right. Switching to monochrome would not save much bandwidth. (if any) Don't argue if you don't understand how chroma (color) is sub-sampled compared to luminosity (brightness).
I do understand better than you think. I've designed scientific grade CCD cameras :)

And I work with Colorimeters and Spectrometers. Part of my day's work is dealing with YCbCr, XYZ, xyY CIELab, etc.

My point is that any useless information should be discarded to save bandwidth. 12.5% might be enough to save a frame.

EDIT: the used codec can have color subsampling of 4:2:2 or 4:4:4, meaning at best a 2:1 relation between luminance and chroma. So it is more likely that the color information is 33% of the video bandwidth, not considering compression effects of similarities between frames. Where did you get the 12.5%?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: R7 on 05/01/2014 04:52 PM
EDIT: the used codec can have color subsampling of 4:2:2 or 4:4:4, meaning at best a 2:1 relation between luminance and chroma. So it is more likely that the color information is 33% of the video bandwidth, not considering compression effects of similarities between frames. Where did you get the 12.5%?

Wouldn't 4:2:2 mean 1:1 ratio between luma and chroma bandwidths? (=8 luma samples and 4 Cb + 4 Cr samples)

The 12.5% is assuming (non-mpeg4 standard?) 4x4 subsampling, kind of "4:1:0:0:0".

Btw anyone know what kind of error correction methods the RF-link had? Some sort of Reed-Solomon encapsulation layer?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: ugordan on 05/01/2014 05:09 PM
I just checked the *.ts SpaceX posted, the format is 4:2:0 so that would bring the total number of bits to encode to 50% more than monochromatic video so I stand corrected. I must have been thinking of older codecs like Indeo that did allow as high as 4x4 subsampling of chroma.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: rickyramjet on 05/01/2014 06:02 PM
The problem is the noisy signal, not the codec.  The reason for the noisy signal is signal strength, pure and simple, worsened by the distance from the rocket and the bad weather.  The easiest solution is to install a more powerful transmitter.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Adaptation on 05/01/2014 06:04 PM
Quote
First we took a pass on the data to align every MPEG packet on a 188 byte boundary and set the packet start byte to 0x47.

I'm not seeing that.  For instance location 26EC is divisible by 188 (0XBC) but its value is 4F not 47.
I can concur. In the repair1.ts file the sync bytes have not been "fixed" to 47 (hex). Maybe they uploaded the wrong file? Not that this does much: the rest of the header is usually broken as well. Also, in the raw.ts there are 5 places where I had to add exactly 56 bytes in order for the headers to align on the 188 bytes grid.

Anyway. Back to trying to get a little more life out of this video.  ;)

Well some bits in the header should be able to be restored as well. 

Here is a prototype for the two header bytes.
1000 0111  (this is the G or 47)
0Y0Y YYYY
YYYY YYYY
00X1 XXXX

Where 1's and 0's are values that should be set regardless of contents of packets.  X's contain data that may be determined by analysing headers before and after this packet.  Y's contain data that could possibly be determined by analyzing data within the packet, knowing the identifiers associated with the codec and several invalid values could be excluded.

Without doing much analysis we can make some bitwise filters for the second and fourth bytes to fix five bits. 

0101 1111  to be and with the second byte (this sets the packet to not be ignored and not to give it special priority)
0011 1111  to be and with the fourth byte  (this declares the stream to be unencrypted)
0001 0000  to be ored with the fourth byte  (this sets packet contains payload to true, which is possibly too big of an assumption)

http://en.wikipedia.org/wiki/MPEG_transport_stream#Packet
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IRobot on 05/01/2014 06:11 PM
The problem is the noisy signal, not the codec.  The reason for the noisy signal is signal strength, pure and simple, worsened by the distance from the rocket and the bad weather.  The easiest solution is to install a more powerful transmitter.
More or less. If you use a codec that reduces the information to 50% (for example by not using color), and keeping a frame rate of 15fps, you can actually send the same frame twice (or send the same transmission packet twice). But as that would require a deep software change, the other option would be to double the number of frames, 30fps. Then differences between frames would be reduced to half, increasing compression ratio.

I'm no codec expert, so there are probably better solutions on how to use bandwidth to reduce transmission noise (data corruption). Still, a change in the codec is probably a lot easier than replacing the transmitter. A more powerful transmitter also uses more energy.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Lars_J on 05/01/2014 06:49 PM
Someone on Youtube did a decent effort of cleaning it up:

I'm not sure how accurate it is, and I don't think the legs were extended in the beginning of the clip? Nonetheless it looks neat.

https://www.youtube.com/watch?v=CnyrleS782g
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IRobot on 05/01/2014 06:56 PM
That guy got the rocket image from the best frame and overimposed it on all frames... it offers a visual cue, but that's all.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Adaptation on 05/01/2014 07:01 PM
the other option would be to double the number of frames, 30fps. Then differences between frames would be reduced to half, increasing compression ratio.

Doubling the framerate would do little to solve the problem on a modern codec.  You would need a full frame codec like MJPEG for that to really work.  You could reduce the threshold for sending a keyframe or have keyframes sent twice.  As they are using a fixed bit rate stream there may be room for some of these tricks. 

The best thing they could do is know better where the rocket will come down and have adequate downlink capability there. 

Higher transmit power is nice but it only gets you so far, double the power and you get 1/4 more range, you can only double the power so many times before the strategy gets out of hand.  But using for instance a 28 dbi directional antenna on the receiver gives you the same result as multiplying your transmit power 500 times.  The only problem is you have to be able to point it very accurately, if you're off by just 5° you are only going to get 250x the receive power but it steeply drops from there a few more degrees and its the same as sticking blinders up on your receiver. 

AFIK this launch did not aggressively attempt landing at a precise spot because more velocity was given to the dragon to assure the highest possible margins for mission success. 
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: AJA on 05/01/2014 07:02 PM
@AJA why red filter? Maybe a luminance filter with IR cut, but can't understand the reason to use the red filter.

Falcon's white. The legs too.. have a white border (scroll to find input~2's YT screenshot ITT). The ocean's blue. Now that means that the RGB luminances of Falcon are (say) r1, g1, b1, whereas the ocean's is ~0, ~g2(bodies of water do look greenish sometimes don't they? Plus, plankton?), b2. The difference |b2-b1|, is I would assume the smallest of the three pairs and doesn't really help in terms of establishing where the legs end and water begins (in the image data). |g2-g1| would probably be more than the blue channel differences, but the largest, by far would be |r2-0| (I'm assuming that the ocean's black in the red channel..or very close to it atleast). So this allows you to differentiate.

While it may seem useless, and like a really poor version of a grasshopper video if you're not able to tell if Falcon is moving up and down in response to the waves... I'm counting on the fact that as waves break, the surf is going to be white...and will be visible in the image as well.

They may still want to cut out IR.. because the water might be radiating, and once the engine lights, it'll probably saturate the sensor.

I don't think this'd require much modification at all. Unless they're using some special space qualified camera, with a custom chip, a custom form factor etc.... can't you get a black and white camera and stick a red filter in front of it? If they keep the same data payload, they can trade two channels for more dynamic resolution...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: MP99 on 05/01/2014 07:21 PM
12.5% is actually a lot. When receiving such error-prone signal, it makes a lot of difference. It also reduces transmission power requirements (for the same frame rate), therefore more power available for transmission, therefore better S/N ratio.

Maybe. *If* that's the only telemetry sent. Who's to say the video wasn't multiplexed along all the other, high rate vehicle telemetry so the 12.5% for video is more of a noise in the total bandwidth budget?

Also true monochrome cameras are up to 3x more sensitive, meaning less (camera) noise to start with.

Seriously? At the codec quality settings they're using, the camera dirt that's deposited on the way down you're worried about camera noise?

Again, why go around making up solutions that would only be relevant for a couple of seconds before splashdown?
Because the way up is quite well documented. The way down is not.

Oh, I'm sure they have it quite well-documented. Just not in a format your typical rocket pr0n enthusiast likes. It's in the form of vehicle telemetry. That's gold, anything else is just gravy.

It still doesn't change my argument that any such solutions are just too much trouble for the amount of use they'll have eventually.
Worked on sort of "telemetry", we divided data into what's important (10%, in this case sensors data) and what's less important (90%, in this case video feed). Both were CRC-ed, but the first got retransmitted if corrupted or lost, the second was not. Sort of TCP and UDP.

On worsening channel the retransmissions of sensor data occupied more and more bits until no bits were available to video.

I guess SpaceX does the same, so if we see SOME video, it means ALL sensor data was received without gaps.

You guys are worried about how many bits the chroma component takes when some of analysis says they included substantial fill-in packets to bump up the data rate to make it a fixed rate transmission??

If they had infinite time to work on the transmission system it would have been nice to optimise it with lots of redundancy data instead of fill-in "0xffffffff" packets. (Though maybe those "ffff"s make it easier to re-synchronise the stream once major errors start to bite??)

But, I suspect this is more of an off-the-shelf system that was stymied by weather conditions on the day.

Next launch / splashdown should have an easier time of it.

cheers, Martin
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: eeergo on 05/01/2014 07:56 PM
Someone on Youtube did a decent effort of cleaning it up:

I'm not sure how accurate it is, and I don't think the legs were extended in the beginning of the clip? Nonetheless it looks neat.

https://www.youtube.com/watch?v=CnyrleS782g

The overlay this person made is actually quite misleading: there are some misplaced pixels at 0:14-0:15 from the engine exhaust that appear as yellow artifacts to the left of the image - in the original video they were not so apparent since there was a lot of noise, but here they take the context away and get quite distracting. Also, it makes the splashdown and subsequent tipping over of the stage very confusing to watch, since the legs should be submerged.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Lars_J on 05/01/2014 08:07 PM
Yes, the overlay works better for the first part of the video.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: michaelni on 05/02/2014 03:08 AM
The video seems simple profile level 3 mpeg4 video in mpeg TS.
It seems none of the error resilience features of mpeg4 have been used when encoding it. Which is a pitty, had slices been used then the decoder could resume decoding of a frame at the next slice start, had data partitioning been used then the more important low resolution information and motion vectors would have been coded first in each slice making errors less likely to damage them. And had rvlcs been used then slices could have been decoded from both the start and end again, limiting the impact of bit errors.

I know nothing about how the video was generated or how it was transmitted, but if there was some FEC in there then then it should be possible in principle to re-run FEC decoding after manual fixing up all mpeg-TS and mpeg4-ES headers. And as such manual fixing would decrease the errors, FEC would then have fewer errors to deal with and might in a few rare cases be able to fix a few more errors.
Also if some kind of CRCs have been used, CRCs can also be used to correct bit errors as long as the number of errors is sufficiently small, which each CRC would need to correct, the exact number that could be corrected that way depends on the packet size and  crc polynomial being used.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/02/2014 04:52 AM
Yeah. It's a real challenge. Pretty stuck here.

But I still got a few ideas I want to try...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SVBarnard on 05/02/2014 06:12 AM
can someone please explain to me why spacex still hasn't released the footage they got from their airplane? Why are they being so secretive about it? I mean seeing is believing so why not just release the footage and prove to everyone in the world they really accomplished such an unprecedented feat?

I mean they did actually record it from their airplane right?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: luinil on 05/02/2014 06:15 AM
the airplane might not have been close enough to take a video.

Remember the weather was pretty heavy, NASA renounced to send their plane.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Lars_J on 05/02/2014 06:16 AM
can someone please explain to me why spacex still hasn't released the footage they got from their airplane? Why are they being so secretive about it? I mean seeing is believing so why not just release the footage and prove to everyone in the world they really accomplished such an unprecedented feat?

I mean they did actually record it from their airplane right?

I suspect we will see more footage when SpaceX releases their usual mission highlights video.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/02/2014 09:03 AM
Over at the reddit thread (http://www.reddit.com/r/spacex/comments/24bsn2/first_stage_landing_video/) somebody posted his very useful analysis. I will quote it here:

Quote
Argh. I have some MPEG-2/MPEG-4 experience, and had a go at fixing the file too. Sadly, it's no better than the version SpaceX posted, in terms of useable frames and recognizable objects in the frame. I think there's just too much lost data (and the I-frames made up too much of the missing data) to get much back. But I'd love to be wrong!

The order and detail of my efforts was:

 * Identify discontinuities in the file -- where 0x47 sync bytes jumped offset. Unlike the SpaceX file (which apparently removed data), I added 0xFFs to pad the data to align the next identifiable packet to a 188 byte multiple.

 * Identify the different PIDs in the file. There were the usual suspects: 0x0000 (PAT: Program Allocation Table), 0x0020 (PMT: Program Map Table referenced by the PAT), 0x03e8 (PES video stream, referenced by the PMT), and 0x1fff (padding packets to align the transport stream bit clock with the Program Clock Reference).

 * Identify video PES streams by start code and ID (0x000001e0). I created a template packet that looked like this (hex): 47 43 e8 3_ 07 10 00 __ __ __ __ __ 00 00 01 E0 00 00 81 80 07 I tested each packet against this template, computing the Hamming distance and printing out packets below 23 bits difference. This showed me all the likely PES headers, which I could examine and repair. The five "__" bytes contain the Program Clock Reference value for that point in the transport stream. The PCR advances at 90kHz, with a 300-count fraction/extension that advances at 27MHz. In this file, the PCR advances at multiples of 6006 (decimal) or 66.733 milliseconds. I managed to recover 276 PCR values, and the time stamps span 318 frames (at 6006 PCR counts per frame) or 21.221 seconds. For PCR values that were not multiples of 6006, but were clearly close or off by one or two flipped bits, I flipped them back (hex file editor) to get the interval right.

 * I also normalized the Presentation Time Stamp (PTS) values, which appear to lag the PCR of the packet by exactly 10000 counts (111.111 milliseconds). I used the PCR and PTS intervals to cross-check values (looking for multiples of 6006 in the PCR and a difference of +10000 between the PCR and PTS) and fixed anything that was out of sorts.

 * I identified all transport stream packets as being either PAT, PMT, video, or padding, and coerced bit errors in the header to have the correct sync byte (0x47) and flags (turning off the transport stream error indicator, transport priority, and scrambling bits. I used the continuity counter value to tell whether successive packets might actually be the same PID (usually 0x3e8, video). Based on all this information, I coerced PIDs to the most likely value, and turned any obviously junk packets into padding (null) packets. I fixed up the continuity counter on the packets of each PID so they were incrementing without gaps.

 * I then dug into the MPEG-4 visual object sequence and visual object structures. I saw damage on some of the bits that described the dimensions of the image frame, which were causing VLC and ffmpeg some consternation. So I normalized those.

At this point, I ran out of time. I have a lot of other projects I should be working on. So I share all this in hopes others can build on this, or point out a fatal flaw in my approach and improve on it.

Here's the file I've hand-edited to correct all the PCR and PTS timing issues I could find:

edited.ts (http://www.sharebrained.com/spacex/edited.ts)

And here's that file run through a script to coerce the transport stream headers into more sensible values (which might or might not actually be an improvement):

coerced.ts (http://www.sharebrained.com/spacex/coerced.ts)

Good luck! And I hope SpaceX gets better weather and position for their RF link on the next launch. Being a software radio nerd, I can appreciate the challenges they face catching several megabits of data from an object they don't want to be too close to when it comes down.

    - Jared
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: michaelni on 05/02/2014 03:35 PM
here are two videos which where transcoded with a modified FFmpeg.
The modifications force the decoder to continue decoding frames even once errors where detected, this should show more parts of the frames at the expense of also significnatly more random damaged and ugly looking blocks.
Also motion compensation and error concealment where disabled, the way neither correct nor incorrect parts of previous frames would leak in the current frame.
Overall the video certainly is less pleasing to watch that way but it should contain more information for analysis and be easier to make sense of if someone wants to analize it frame by frame

The first is based on raw.ts, the second is based on the coerced.ts file from Jarred which was posted here
https://ffmpeg.org/~michael/space-x-video/raw.ts-nomc-ignoreerr.mkv
and
https://ffmpeg.org/~michael/space-x-video/Jared-coerced.ts-nomc-ignoreerr.mkv
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: michaelni on 05/02/2014 03:59 PM
and here are teh 3 videos, raw.ts and the 2 from jared decoded with ffmpeg with error concealment tuned a bit (that is favoring predicting from the previous frame with MV 0,0), which might look slightly more pleasing than the original, but shouldnt contain more information

https://ffmpeg.org/~michael/space-x-video/raw.ts-tunedec.mkv
https://ffmpeg.org/~michael/space-x-video/Jared-edited.ts-tunedec.mkv
https://ffmpeg.org/~michael/space-x-video/Jared-coerced.ts-tunedec.mkv
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/02/2014 06:36 PM
here are two videos which where transcoded with a modified FFmpeg.
The modifications force the decoder to continue decoding frames even once errors where detected, this should show more parts of the frames at the expense of also significnatly more random damaged and ugly looking blocks.
Also motion compensation and error concealment where disabled, the way neither correct nor incorrect parts of previous frames would leak in the current frame.
Overall the video certainly is less pleasing to watch that way but it should contain more information for analysis and be easier to make sense of if someone wants to analize it frame by frame

The first is based on raw.ts, the second is based on the coerced.ts file from Jarred which was posted here
https://ffmpeg.org/~michael/space-x-video/raw.ts-nomc-ignoreerr.mkv
and
https://ffmpeg.org/~michael/space-x-video/Jared-coerced.ts-nomc-ignoreerr.mkv

Wooow! I didn't realize this at first. But we have Michael Niedermayer (http://www.ffmpeg.org/consulting.html) in the house!  8)

One of the creators of ffmpeg itself. Gulp.

Hope we can work together to figure this out. I like your approach though. Currently I'm focusing on the I-frames and altering the decoder to give me more information about what is wrong with certain (macro)blocks. But I've seen the decoder source code for the first time today, while you may know it by heart.

Ok guys. I think we're in good hands now ;)

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/02/2014 06:44 PM
Interesting work guys! Keep up the good work!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Helodriver on 05/02/2014 06:52 PM
From that cleanup it appears the video definitely ends before the engine quits and definitely before the vehicle begins to topple. Still no definitive answer as to what the "8 seconds" entailed. Nice work on the video.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Marams on 05/02/2014 08:02 PM
here are two videos which where transcoded with a modified FFmpeg.
The modifications force the decoder to continue decoding frames even once errors where detected, this should show more parts of the frames at the expense of also significnatly more random damaged and ugly looking blocks.
Also motion compensation and error concealment where disabled, the way neither correct nor incorrect parts of previous frames would leak in the current frame.
Overall the video certainly is less pleasing to watch that way but it should contain more information for analysis and be easier to make sense of if someone wants to analize it frame by frame

The first is based on raw.ts, the second is based on the coerced.ts file from Jarred which was posted here
https://ffmpeg.org/~michael/space-x-video/raw.ts-nomc-ignoreerr.mkv
and
https://ffmpeg.org/~michael/space-x-video/Jared-coerced.ts-nomc-ignoreerr.mkv

Nice work. The following frames from the raw.ts where quite interesting because they clearly show the time stamp in the lower left corner. There are a lot frames with clearly visible last numbers of time stamps.

MECO was around 19:28:19
landing video started at 19:35:03 - 1s or so
landing video ended at 19:35:19

In every frame there is 19:35 in the lower left corner with the exact same pixels. For crypto-analysis a known plain text would help for sure :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: michaelni on 05/02/2014 08:08 PM
Hope we can work together to figure this out. I like your approach though. Currently I'm focusing on the I-frames and altering the decoder to give me more information about what is wrong with certain (macro)blocks. But I've seen the decoder source code for the first time today, while you may know it by heart.

If we assume that a frame has been damaged by a flipped bit, then in general the decoder will not detect it when reading that bit but rather will either show artifacts and continue without a detected error or more likely get desynced and detect some kind or inconsistency like a out of range value or incorrect marker bit at a later point. For example if theres an error detected in MB at location XxY the actual bit error that caused it could be a few macroblocks before it,
MBs are 16x16 pixel in mpeg4 and are stored in raster scan order left to right and top to bottom.
mpeg4 (A)SP uses variable length coding (huffman coding with a few unused codes), so bit errors can throw things out of sync (vlc not matching up between encoder and decoder or symbols not matching up, that is for example a MB type could end up being decoded as a motion vector after a bit error). If detected inconsistencies are ignored then the decoder can end up falling into sync again by chance and continue to decode the bitstream somewhat reasonably. After such resync various values like motion vectors, quantization parameters or values used for AC/DC prediction could/would be wrong though causing some artifacts but could/should still contain some human recognizeable features of the original.

I suspect if the number of flipped bits per frame is small enough and nothing is missing of a frame, someone with alot of time could find the bits by trial and error and by starting from the location in raster scan order where artifacts are first vissible. error concealment would have to be disabled in the used decoder though so that one can see where the artifacts start.
The error concealment code in ffmpeg by default conceals several MBs prior to a detected error.
Error concealment can be disabled with -ec 0

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/02/2014 08:16 PM
Nice work. The following frames from the raw.ts where quite interesting because they clearly show the time stamp in the lower left corner. There are a lot frames with clearly visible last numbers of time stamps.

MECO was around 19:28:19
landing video started at 19:35:03 - 1s or so
landing video ended at 19:35:19

In every frame there is 19:35 in the lower left corner with the exact same pixels. For crypto-analysis a known plain text would help for sure :)

Great find!!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Michael Bloxham on 05/02/2014 08:23 PM
Looks like the stage is still going reasonably fast when it hits the water, and plunges deep into the water?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mmeijeri on 05/02/2014 08:25 PM
https://ffmpeg.org/~michael/space-x-video/raw.ts-tunedec.mkv
https://ffmpeg.org/~michael/space-x-video/Jared-edited.ts-tunedec.mkv
https://ffmpeg.org/~michael/space-x-video/Jared-coerced.ts-tunedec.mkv

Won't play for me with either Windows Media Player or DivX. What software can I use to play it?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: YetAnotherLurker on 05/02/2014 08:30 PM
Won't play for me with either Windows Media Player or DivX. What software can I use to play it?

VLC Media Player
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/02/2014 08:31 PM
https://ffmpeg.org/~michael/space-x-video/raw.ts-tunedec.mkv
https://ffmpeg.org/~michael/space-x-video/Jared-edited.ts-tunedec.mkv
https://ffmpeg.org/~michael/space-x-video/Jared-coerced.ts-tunedec.mkv

Won't play for me with either Windows Media Player or DivX. What software can I use to play it?

Something based on ffmpeg or libav. http://mpv.io/installation/ will work
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mheney on 05/02/2014 09:34 PM
Xine on a linux box works as well.  I believe you can download a version of xine for Windows ...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: MTom on 05/02/2014 10:47 PM
Nice work. The following frames from the raw.ts where quite interesting because they clearly show the time stamp in the lower left corner. There are a lot frames with clearly visible last numbers of time stamps.

MECO was around 19:28:19
landing video started at 19:35:03 - 1s or so
landing video ended at 19:35:19

In every frame there is 19:35 in the lower left corner with the exact same pixels. For crypto-analysis a known plain text would help for sure :)

Great find!!

And this is placed on the bottom of the frames.
This sounds me (without any knowledge of video) as a good point: could it be a good reference point for the correction? (The top of the frames have a good quality on the raw video, after identifiing the bottom lines could it be like a "picture frame" for the work?)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jcc on 05/03/2014 01:17 AM
Xine on a linux box works as well.  I believe you can download a version of xine for Windows ...

I just downloaded the latest RealPlayer update "RealPlayer Cloud" on win7, and it plays mkv files.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: AS-503 on 05/03/2014 01:24 AM
All Windows based OS can benefit greatly from installing a complete CODEC package like K-Lite (mega). This will not only allow playback of virtually everything, but also enable thumbnails and super-charge Windows Media Player to playback almost all video and audio file extensions.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: MarsInMyLifetime on 05/03/2014 02:29 AM
FWIW, I was able to watch these mkv files using J River Media Center which seems to have (or get) the right codec.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IslandPlaya on 05/03/2014 02:52 AM
Just download VLC player. It saves downloading all them dodgy codec packs.
If VLC can't play it then forget it...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: MP99 on 05/03/2014 08:13 AM
Just download VLC player. It saves downloading all them dodgy codec packs.
If VLC can't play it then forget it...

Concur.

Cheers, Martin
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: grythumn on 05/03/2014 11:38 AM
For windows, Media Player Classic Homecinema or VLC are the standard applications. I usually have better luck with MPC for hardware acceleration (not important for this particular case.)

-R C
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Digitalchromakey on 05/03/2014 02:54 PM
here are two videos which where transcoded with a modified FFmpeg.
The modifications force the decoder to continue decoding frames even once errors where detected, this should show more parts of the frames at the expense of also significnatly more random damaged and ugly looking blocks.
Also motion compensation and error concealment where disabled, the way neither correct nor incorrect parts of previous frames would leak in the current frame.
Overall the video certainly is less pleasing to watch that way but it should contain more information for analysis and be easier to make sense of if someone wants to analize it frame by frame

The first is based on raw.ts, the second is based on the coerced.ts file from Jarred which was posted here
https://ffmpeg.org/~michael/space-x-video/raw.ts-nomc-ignoreerr.mkv
and
https://ffmpeg.org/~michael/space-x-video/Jared-coerced.ts-nomc-ignoreerr.mkv
The second transport stream above which is based on Jared's coerced ts with motion vectors and error concealment turned off, does make very interesting viewing on a frame by frame basis. There are several fixed features in each frame that we know are spatially invariant on a frame by frame basis - the legs, the first stage edges. the real time read out display and to an extent the rocket plume.

Looking on a frame by frame basis you can clearly see the edges for the legs, often as (a) cohesive block(s), but located in clearly the wrong frame position.

It should be possible, but incredible tedious for someone wit the right tools to manually reverse engineer a transport stream, using these measurable, known fixed special errors as frame by frame variable special offsets and moving all the associated data (including motion vectors, etc) within these clearly defined mislocated blocks over to their known correct spacial position in a reverse engineered transport stream.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Carreidas 160 on 05/03/2014 02:58 PM
I'm neither a video encoding/decoding expert nor qualified rocket engineer but...

Is there a way to clean up a noisy video signal using genetic algorithms?

I.e. randomly keep switching bits (turn 0s into 1s) in parts of the signal that are known to be corrupt, and see whether these variations clean up the picture or not? And subsequently lock the bits that improve quality?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/03/2014 09:17 PM
Hi guys,

Litte update.

I have been making some progress. I can now tell the codec to skip bad macroblocks (16x16 pixel blocks). And I can tell it when to start the next good macroblock and where its data source begins (on a bit-level).

As a proof of concept I have taken the coerced.ts and from that basicly recreated the I-frame SpaceX released earlier. See the attached image. (btw I haven't spend time to repair the lower part of the image)

The difference is that with this I can "fix" the output of an I-frame without actually fixing the data itself. I "just" have to say what blocks are "bad" and where the new "good" block begins (bit wise). The latter is also very time consuming right now. But I have some ideas that maybe everybody can help. Maybe.

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: dorkmo on 05/03/2014 09:37 PM
awesome stuff.

to comment on the content of what can be seen so far,
it looks to me like the legs both came out a little bit. then only the left leg progressed, then stopped. then the right leg extended fully. then finally the left leg extended fully.

i wonder if thats typical of whats been seen in testing
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Adaptation on 05/04/2014 12:01 AM
Hi guys,

Litte update.

I have been making some progress. I can now tell the codec to skip bad macroblocks (16x16 pixel blocks). And I can tell it when to start the next good macroblock and where its data source begins (on a bit-level).

As a proof of concept I have taken the coerced.ts and from that basicly recreated the I-frame SpaceX released earlier. See the attached image. (btw I haven't spend time to repair the lower part of the image)

The difference is that with this I can "fix" the output of an I-frame without actually fixing the data itself. I "just" have to say what blocks are "bad" and where the new "good" block begins (bit wise). The latter is also very time consuming right now. But I have some ideas that maybe everybody can help. Maybe.

Regards,

arnezami

That is really freakin good work.  Would you mind sharing your latest .ts and a little more on how you are determining which blocks are good and bad.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/04/2014 04:02 AM
Hi guys,

Litte update.

I have been making some progress. I can now tell the codec to skip bad macroblocks (16x16 pixel blocks). And I can tell it when to start the next good macroblock and where its data source begins (on a bit-level).

As a proof of concept I have taken the coerced.ts and from that basicly recreated the I-frame SpaceX released earlier. See the attached image. (btw I haven't spend time to repair the lower part of the image)

The difference is that with this I can "fix" the output of an I-frame without actually fixing the data itself. I "just" have to say what blocks are "bad" and where the new "good" block begins (bit wise). The latter is also very time consuming right now. But I have some ideas that maybe everybody can help. Maybe.

Regards,

arnezami

That is really freakin good work.  Would you mind sharing your latest .ts and a little more on how you are determining which blocks are good and bad.

Hi Adaptation,

I have not changed any .ts lately. The coerced.ts file from Jared is pretty good for now.

What I'm doing is changing the behavior of the decoder itself: libavcodecs/h263dec.c (http://git.videolan.org/?p=ffmpeg.git;a=blob;f=libavcodec/h263dec.c;h=523fca9e7f9406211c519f0380a722fd6b0268fa;hb=HEAD) to be more exact. Thats the code that loops through the macroblocks. The code that actually decodes the bits themselves is also very important: libavcodec/ituh263dec.c (http://git.videolan.org/?p=ffmpeg.git;a=blob;f=libavcodec/ituh263dec.c;h=d9170df8266f0274adee5c19d758574f39e0a540;hb=HEAD). Although I don't change it.

There are essentially two ways I'm determining whether a macroblock is bad: either ffmpeg gives an error like "Error in MB: " or I see it visually. That's basicly it.

What is a good block and more importantly where its starts in the datastream is much harder to derermine. This is because these blocks have a variable length and nowhere does it say how long they are (it usually varies from 30 to ~150 bits, but sometimes its a couple hundred bits). This is for I-frames btw: the frames that are basically independent pictures in the stream and require no other frames to be decoded. There are 13 I-frames in the file btw. We have seen only two partials, so there is a lot to be discovered still.

Anyway, this is the real challenge: figuring out the starting positions of the good macroblocks after a few bad ones. And maybe we can all collaborate here. Maybe. But for now what I have tried to do is simply let the codec iterate through lot of starting positions for a block and decode from there, and if there is no error it assumes it is ok. Semi-automated trial and error essentially.

This however doesn't always work: sometimes there is no error but clearly the block (visually) isn't correct. But when trying all kinds of random starting positions another block (later in the stream) will become visible (albeit in the wrong position as already discussed in this thread). Since I log all the bit-positions of all the macroblocks I can now determine what the starting position (bit-wise) is of that block. Then I "tell" (aka hack) the codec to read that block earlier when it is supposed to be shown.

But for now I hack this in the code and I want to make this a little bit more user friendly.

Hope that answers your questions.

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/04/2014 08:53 AM
Hi guys,

Here is an overview of the I-frames in the file:

#Position in fileSize in bytesFrame numbermp4-img linkimage linkCurrent state
0~254956n/an/anot extracted yet, transport stream issue?
18478801667333iframe_1.zip (http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=581910)0% (gray)
211466121103953iframe_2.zip (http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=581911)iframe_2.png (http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=581924)10%
31207712892757iframe_3.zip (http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=581912)0% (gray)
414462841653673iframe_4.zip (http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=581913)0% (gray)
517448281709493iframe_5.zip (http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=581914)0% (gray)
6205427629622113iframe_6.zip (http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=581915)0% (gray)
7264196415064150iframe_7.zip (http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=581946)iframe_7.png (http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=581934)15%
8294088420428169iframe_8.zip (http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=581947)iframe_8.png (http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=581936)45% - 60%
9323942815730188iframe_9.zip (http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=581918)iframe_9.png (http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=581942)15%
10353797210009207iframe_10.zip (http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=581919)iframe_10.png (http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=581944)10%
11384065214113227iframe_11.zip (http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=581920)0% (gray)
12413581212169246iframe_12.zip (http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=581921)0% (gray)

Some explanation about this table:

The column "Position in file" means the position (in bytes) in the coerced.ts file. So where it's data (according to ffmpeg) begins. "Size in bytes" is the size according to ffmpeg. "Frame number" is the frame number according to ffmpeg. All based on the current coerced.ts file. Note that iframe_0 hasn't been extracted yet and several iframes don't give any output at the moment (just blank grey).

For each I-Frame I've also attached an .mp4-img file and a .png picture produced by the standard ffmpeg.

The mp4-img files were produced like this: (tip from michaelni)

Quote
./ffmpeg -i coerced.ts -vcodec copy -an -f image2 frame%d.mpg4-img

In his words:

Quote
This would split the mpeg-ts into 1 file per frame, each containing just the mpeg4-es data, no mpeg-ts anymore

This is going to be useful so we can target 1 I-frame at the time. I'm currently working on a slightly altered ffmpeg version (and some scripts) with which you can manipulate multiple macroblock positions, and semi-automatically iterate over many bit-positions in order to find a "good" block. That way anyone can help recover these images. But I still have a lot more work to do before that happens ;).

Regards,

arnezami

[edit] You need to add the option "-s:0 704x480" to ffmpeg if you want it to accept an .mp4-img file as input.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/04/2014 04:51 PM
Hi guys,

I've created a way so that everybody (with some basic linux skills) can help restoring these I-frames.

[edit]Attached is one file: install.txt. It contains instructions on how to install this adapted version of ffmpeg.

Here is the last part of the install.txt:

Quote
In your current directory there is now a new ffmpeg file
You can now use this new ffmpeg with additional options, namely:

1) "-debug mb_pos_size" : this enables logging of the x, y, bitposition and bitsize of the decoded macroblocks
2) "-mmb x1:y1:bitpos1,x2:y2:bitpos2" : with this you can overrule the bitposition of multiple macroblocks (so they start at the correct data)
3) "-err_detect ignore_err" : this makes sure decoding doesn't stop if there are errors (mainly due to the bad alignment of data) (this change was done by Michael Niedermayer)

Example usage:
./ffmpeg -mmb 07:14:17233,19:14:57914 -debug mb_pos_size -err_detect ignore_err -s:0 704:480 -i iframe_8.mpg4-img -f image2 img-%03d.png

In the above example we do serveral things:

- Since the macroblock at x:7 y:14 is defect we overrule its datapoint with a block at 17233. The data of this block is chosen because it contains almost no data (only 30 bits) which ensures that when put here it won't affect neighbouring blocks too much. It's a "silent" block if you will, which doesn't create too much noise.
- We have found out (after experimenting and examining the log) that a specific part of the left leg of the stage begins at bitposition 57914. Since we know where to place it (x:19 y:14) we assign this block to that datapoiint.
- We use "iframe_8.mpg4-img" as the source (see the forum, there is a list of all I-frames, http://forum.nasaspaceflight.com/index.php?topic=34597.msg1193316#msg1193316) but therefore we need to add the option "-s:0 704:480", otherwise ffmpeg cant determine the size of the video
- the option "-f image2 img-%03d.png" makes sure that the I-frame is being exported as a lossless png (the %03d is not needed when there is only one frame in the source)
- In the resulting .png you can see that the I-frame looks pretty good already!

Good luck and have fun!

Regards,
arnezami

PS. If you have any questions (or it doens't work somehow) please let me know.
PPS. Because the forum doesn't let me upload a tar.gz file I had to double compress it with zip. Sorry.

[edit] New install.txt uploaded because I used an odd version of ffmpeg which could screw up your own.
[edit] Even newer install.txt since we now have it on git! Thanks mlindner!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/04/2014 08:02 PM
Hi Guys,

iframe2 is starting to become alive!  ;D

No legs deployed!

btw I did a trick to make this work, because of that everything is sort of shifted up.

Have fun!

arnezami

@mlindner: I have responded to you in a pm.

[edit] almost forgot, I did this: "-mmb 40:03:11229,42:03:11229,42:05:19620"
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/04/2014 09:35 PM
Hi Guys,

iframe2 is starting to become alive!  ;D

No legs deployed!

btw I did a trick to make this work, because of that everything is sort of shifted up.

Have fun!

arnezami

@mlindner: I have responded to you in a pm.

[edit] almost forgot, I did this: "-mmb 40:03:11229,42:03:11229,42:05:19620"

Awesome!

What does your code do when it skips a block?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/04/2014 09:54 PM
Awesome!

What does your code do when it skips a block?
Blocks are not really skipped as such, they are on-the-fly given a different point where they retrieve their data. So if a blocks' data would normally be corrupt I just tell it to look for different data (from a block earlier or later). And with a later block I point it its proper data point again. Therefore avoiding the corrupt data of one block affecting other blocks.

My newest strategy is to turn use detect_err = ignore, and look for blocks that seem to contain visual data (rocket structure and such). And then (by counting x and y on my screen) determining the x and y of that block, look at the log and see the bit position. Then set block x:0 y:0 to that bitposition! That works great for finding real data parts. :)

Btw. Attached is a tantalizing very sketchy version of iframe11...

Can anyone confirm this is AFTER landing? Because SpaceX might be interested in this. Be careful though looks can be deceiving with this much noise. But do I see an intact fuselage? And a leg? Or is it artefact?

Ooh and there is the timestamp :P

Hihi this is fun! :)


[edit] Forgot it again. I did: "-mmb 0:0:10020" here.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/04/2014 11:09 PM
Awesome!

What does your code do when it skips a block?
Blocks are not really skipped as such, they are on-the-fly given a different point where they retrieve their data. So if a blocks' data would normally be corrupt I just tell it to look for different data (from a block earlier or later). And with a later block I point it its proper data point again. Therefore avoiding the corrupt data of one block affecting other blocks.

My newest strategy is to turn use detect_err = ignore, and look for blocks that seem to contain visual data (rocket structure and such). And then (by counting x and y on my screen) determining the x and y of that block, look at the log and see the bit position. Then set block x:0 y:0 to that bitposition! That works great for finding real data parts. :)

Btw. Attached is a tantalizing very sketchy version of iframe11...

Can anyone confirm this is AFTER landing? Because SpaceX might be interested in this. Be careful though looks can be deceiving with this much noise. But do I see an intact fuselage? And a leg? Or is it artefact?

Ooh and there is the timestamp :P

Hihi this is fun! :)


[edit] Forgot it again. I did: "-mmb 0:0:10020" here.

Arnezami, now that we have (and are getting more) partially good I-frames, how do we apply these same changes to the video so that the following frames can reference them properly?

Also I don't understand your description of what you do with blocks. So when it uses "different data" is it using data from other blocks or possibly just finds the correct location of its own data?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: michaelni on 05/04/2014 11:39 PM
The mp4-img files were produced like this: (tip from michaelni)

Quote
./ffmpeg -i coerced.ts -vcodec copy -an -f image2 frame%d.mpg4-img

Its possible to extract a few more frames at the start with
./ffmpeg -i coerced.ts -vcodec copy -an -copyinkf  -f image2 frame%d.mpg4-img
They are not keyframes though
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: michaelni on 05/05/2014 01:45 AM
Ive put arnezamis changes on github:
(with some simplifications to the implementation, and very basic docs from mlindner)

should be easier to download that way

https://github.com/michaelni/FFmpeg/tree/spacexdebug1
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/05/2014 02:04 AM
Ive put arnezamis changes on github:
(with some simplifications to the implementation, and very basic docs from mlindner)

should be easier to download that way

https://github.com/michaelni/FFmpeg/tree/spacexdebug1

Just a correction, the docs are from arnezami. I only put them up in an initial github tree which michaelni merged changes from.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/05/2014 02:56 AM
@mlindner: I can confirm the steps your new install.txt work. I've added it to my original post and removed the .zip file.

Thanks mlindner!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: grythumn on 05/05/2014 03:31 AM
Couple of notes:

1) This compiles (slowly) and runs fine under cygwin, so windows people can get in on the fun too.
2) When you replace a bad MMB with a small one, it keeps going from that point in the bitstream. You need to restart the stream of the next good MMB at its original point in the bitstream or the picture looks much worse and you get ghosts.

Original  error:
[mpeg4 @ 0x8001af20] MB pos/size: 0 19:04:15480 73
[mpeg4 @ 0x8001af20] MB pos/size: 0 20:04:15553 209
[mpeg4 @ 0x8001af20] ac-tex damaged at 21 4
[mpeg4 @ 0x8001af20] MB pos/size: -1 21:04:15762 73
[mpeg4 @ 0x8001af20] Error at MB: 201
[mpeg4 @ 0x8001af20] illegal dc vlc
[mpeg4 @ 0x8001af20] MB pos/size: -1 22:04:15835 10
[mpeg4 @ 0x8001af20] Error at MB: 202
[mpeg4 @ 0x8001af20] I cbpc damaged at 23 4
[mpeg4 @ 0x8001af20] MB pos/size: -1 23:04:15845 6
[mpeg4 @ 0x8001af20] Error at MB: 203
[mpeg4 @ 0x8001af20] MB pos/size: 0 24:04:15851 78
[mpeg4 @ 0x8001af20] MB pos/size: 0 25:04:15929 96

 ./ffmpeg -mmb 21:04:3656 -debug mb_pos_size -err_detect ignore_err -s:0 704:480 -i iframe_9.mpg4-img -f image2 img-%03d.png
[mpeg4 @ 0x80022240] MB pos/size: 0 19:04:15480 73
[mpeg4 @ 0x80022240] MB pos/size: 0 20:04:15553 209
[mpeg4 @ 0x80022240] MB pos/size: 0 21:04:10340 22
[mpeg4 @ 0x80022240] MB pos/size: 0 22:04:10362 22
[mpeg4 @ 0x80022240] MB pos/size: 0 23:04:10384 23
[mpeg4 @ 0x80022240] MB pos/size: 0 24:04:10407 102
[mpeg4 @ 0x80022240] MB pos/size: 0 25:04:10509 226


 ./ffmpeg -mmb 21:04:3656,24:04:15851 -debug mb_pos_size -err_detect ignore_err -s:0 704:480 -i iframe_9.mpg4-img -f image2 img-%03d.png
[mpeg4 @ 0x80022460] MB pos/size: 0 19:04:15480 73
[mpeg4 @ 0x80022460] MB pos/size: 0 20:04:15553 209
[mpeg4 @ 0x80022460] MB pos/size: 0 21:04:3656 22
[mpeg4 @ 0x80022460] MB pos/size: 0 22:04:3678 22
[mpeg4 @ 0x80022460] MB pos/size: 0 23:04:3700 22
[mpeg4 @ 0x80022460] MB pos/size: 0 24:04:15851 78
[mpeg4 @ 0x80022460] MB pos/size: 0 25:04:15929 96
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/05/2014 03:42 AM
Couple of notes:

1) This compiles (slowly) and runs fine under cygwin, so windows people can get in on the fun too.
2) When you replace a bad MMB with a small one, it keeps going from that point in the bitstream. You need to restart the stream of the next good MMB at its original point in the bitstream or the picture looks much worse and you get ghosts.

Original  error:
[mpeg4 @ 0x8001af20] MB pos/size: 0 19:04:15480 73
[mpeg4 @ 0x8001af20] MB pos/size: 0 20:04:15553 209
[mpeg4 @ 0x8001af20] ac-tex damaged at 21 4
[mpeg4 @ 0x8001af20] MB pos/size: -1 21:04:15762 73
[mpeg4 @ 0x8001af20] Error at MB: 201
[mpeg4 @ 0x8001af20] illegal dc vlc
[mpeg4 @ 0x8001af20] MB pos/size: -1 22:04:15835 10
[mpeg4 @ 0x8001af20] Error at MB: 202
[mpeg4 @ 0x8001af20] I cbpc damaged at 23 4
[mpeg4 @ 0x8001af20] MB pos/size: -1 23:04:15845 6
[mpeg4 @ 0x8001af20] Error at MB: 203
[mpeg4 @ 0x8001af20] MB pos/size: 0 24:04:15851 78
[mpeg4 @ 0x8001af20] MB pos/size: 0 25:04:15929 96

 ./ffmpeg -mmb 21:04:3656 -debug mb_pos_size -err_detect ignore_err -s:0 704:480 -i iframe_9.mpg4-img -f image2 img-%03d.png
[mpeg4 @ 0x80022240] MB pos/size: 0 19:04:15480 73
[mpeg4 @ 0x80022240] MB pos/size: 0 20:04:15553 209
[mpeg4 @ 0x80022240] MB pos/size: 0 21:04:10340 22
[mpeg4 @ 0x80022240] MB pos/size: 0 22:04:10362 22
[mpeg4 @ 0x80022240] MB pos/size: 0 23:04:10384 23
[mpeg4 @ 0x80022240] MB pos/size: 0 24:04:10407 102
[mpeg4 @ 0x80022240] MB pos/size: 0 25:04:10509 226


 ./ffmpeg -mmb 21:04:3656,24:04:15851 -debug mb_pos_size -err_detect ignore_err -s:0 704:480 -i iframe_9.mpg4-img -f image2 img-%03d.png
[mpeg4 @ 0x80022460] MB pos/size: 0 19:04:15480 73
[mpeg4 @ 0x80022460] MB pos/size: 0 20:04:15553 209
[mpeg4 @ 0x80022460] MB pos/size: 0 21:04:3656 22
[mpeg4 @ 0x80022460] MB pos/size: 0 22:04:3678 22
[mpeg4 @ 0x80022460] MB pos/size: 0 23:04:3700 22
[mpeg4 @ 0x80022460] MB pos/size: 0 24:04:15851 78
[mpeg4 @ 0x80022460] MB pos/size: 0 25:04:15929 96

Yeah took me a bit to realize that too. It's somewhat intentional as the errors may come in a string and also because a given block is dependent on the previous block for its start location. So if you replace a block with a different block then the start location of the next block is obviously wrong so its useful to let it continue along and wrap back to the same data to possibly "hit" the data at a better bit position.

Edit: attached my best try at iframe 8 thus far here. mmb setting 08:14:17233,11:14:56840 Also we need to get these iframes as perfect as possible as every other following frame references them for their own data. Afterwards we might get a pretty good looking video.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/05/2014 04:43 AM
It seems like the errors aren't all that common, but they're pretty uniformly spread around and it appears that SpaceX didn't encode the mpeg4 with lots of the settings that could detect and correct for errors that are built into the mpeg4 format.

Mpeg4 is a self-referential format meaning nearly every piece of data depends on some other piece of data and errors will propagate through the system as they accumulate until you have garbage referencing garbage. Those are all fixed when you hit an I-frame (what we're working on fixing), but there's very few I-frames, only 13, in the whole video that everything else references. When we fix these errors, they will still accumulate but they'll accumulate slower so things may become more recognizable.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/05/2014 04:53 AM
Hi guys,

Nice to see its starting to work for you too. :)

I'm going to explain a bit more how this all  works and how you can use it. Because I feel there is a need for it.

Attached are 4 illustrations representing 4 scenarios. I think these are going to be useful in understanding how it all works and what you can do with this tool.

1. All is perfect.

In the first illustration you see that all is perfect. The top part represents the bits from the stream. You can see that there are "end markers" in the stream. These basicly tell the decoder that the data for a block has ended. (btw I'm not entirely sure how this actually works, but this I think is close to it). At each end-marker the decoder happily switches to the next block of 16x16 pixels. Note that the length of the data for each 16x16 block is not the same.

The lower part represents the 16x16 pixel blocks. They have identifying numbers. And as you can see they have the colors (representing its content) of the correct bits-stream-parts. So far so good.

2. Defect in bitstream

In the second illustration you see there is a defect in the bitstream (yellow). A bitsteam error can simply mean the content of a 16x16 pixel block doesn't seem quite right (without affecting other blocks too much). However lets assume that this time its worse: the defective bits are now interpreted by the decoder as an end-marker! Ouch. This is big trouble because firstly block 2 is broken off too early and will contain some nasty data (yellow in lower part) which can severely affect all blocks to its right and bottom). But secondly block 3 starts with completely wrong data! It is now on the complete wrong path since all bits are interpreted wrongly and the chance that some of these bits are (wrongly) interpreted as end-markers increases too. This is actually happening in this illustration/scenario. And at some point it even reads a sequence of bits that don't make sense at all. The decoder stops in such a situation. This leaves blocks 5-9 grey. This is the reason only the top parts of the I-frames were visible and the lower parts were grey.

In order to fix this you could try to restore the bitstream itself. But this is very hard to do and extremely time consuming (fixing compressed data is no joke!). The worse is: you don't know when the bad bits stop. We need to know where the good bits start again and try to avoid being dependent on having to do tedious and time consuming work in order to make any progress.

3. Reassigning the bitpointer twice

The problem now becomes: how do you "skip" a block? This seems impossible to do. There is no nice index of the bitstream (as fas as I know).

The first thing to do is make sure is not to halt the decoder when it encounters an error. This way the decoder sometimes rediscovers actual end-marks. And when it does it usually will keep finding more valid end-marks. Of course these will be read by the wrong blocks (aka you see them appearing in the wrong position in the picture). But still useful.

So we now know some valid end-marks (which also mark the start of block-data of course). In order to make use of them we should on-the-fly reassign the bitstream pointer of a block to such an end/start-marker. But first we need to make sure that block 2 doesn't read the bad bits.

So we do two reassignments:

1) Just before the decoder starts with decoding block 2 we reassign its bitpointer to (for example) a block in the past. In this case we set its bitpointer to the bits of block 1 (see third illustration). This is mainly to prevent that really bad visual effects come into our pixels (maybe there is a way to simply "do nothing" with a block, which may be better. I don't know, maybe others do).

2) Just before block 3 starts we assign its bitstream to the end-mark of block 2 (aka start-mark of block 3. Btw  we assume we found/guessed this somehow. This prevents us from reading the bad bits. The bad data is "skipped".

In the lower part of illustration 3 you can see how the content will look like if we do the above. Only one block is "broken" (contains data from a different block that is) and all the other blocks get their own intended data.

4. Reassigning the bitpointer once

A useful technique I discovered was that if you know the bitposition of certain good block-data (using a combination of your eyes which make out the x,y coordinate and the log which contains the corresponding bitpointer at that time) you can simply set the bitpointer of block 0 to that bitposition. I do this is in illustration 4. Then you can easily see how much good data there is and where something is wrong again. If you do this randomly (assigning block 0 to random bitpositions) you can easely detect good data parts in the stream. Which is what we need. So with this you can create a "data-map" of good and bad data. And that will eventually allow you to carefully "skip" the bad data.

Hopefully that helps. My fingers hurt :P

Regards and have a good day,

arnezami

[edit] grammar
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: dorkmo on 05/05/2014 05:18 AM
awesome stuff.

to comment on the content of what can be seen so far,
it looks to me like the legs both came out a little bit. then only the left leg progressed, then stopped. then the right leg extended fully. then finally the left leg extended fully.

i wonder if thats typical of whats been seen in testing

Are you serious? I don't see how you could have possibly seen such detail from any version of the video so far. SpaceX has the actual flight data of all these systems and they say the legs deployed correctly as hoped. I'm gonna go with that.

watching the coerced video, it looks like the entire deployment occured durring the clock time of 00:06 to 00:10. theres only a sliver of blocks in a handful of frames and the image jumps around from left to right a lot.

also it looks like around 15 seconds you stop seeing any signs of flame

and to me it looks like maybe at 17 you can see a splash up of water from impact or a leaning/settling of the rocket to the side, which is up in the video
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/05/2014 05:22 AM
<snip>

arnezami I've been talking to michaelni a lot today in learning how to do this process and I think your concept of "end markers" is off. Every block has a pre-defined size which is determined by its format. When a block is mis-interpreted then the size automatically becomes wrong and the block "eats" extra bits from the next block or misses bits from its block shoving them into the next block. Occasionally these errors cancel each other out (because there is a small finite number of block types and sizes) and the streaming is "resynced" to the proper block ordering so you get a string of correct blocks. You often don't see this resync because after its resynced the blocks are still referencing all the previous corrupted blocks. This is why after you fix one block suddenly lots more blocks become readable IMO.

If michaelni could comment though that would probably clear more things up.

Edit: The key difference that matters from your explanation is that changing the start position of a block can _increase_ or _decrease_ how far away the stop position is for that block because of how the block is interpreted.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: grythumn on 05/05/2014 05:29 AM
Best image so far of Frame 9.

-Bob

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IRobot on 05/05/2014 05:35 AM
I wonder if we reach a point from where some data alignment becomes impossible to correctly to determine. In that case, would it be possible to generate a couple of hundred of thousands of combinations, generate a result image for each, put them on a web server and allow the community to evaluate them visually?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/05/2014 05:39 AM
I wonder if we reach a point from where some data alignment becomes impossible to correctly to determine. In that case, would it be possible to generate a couple of hundred of thousands of combinations, generate a result image for each, put them on a web server and allow the community to evaluate them visually?

I've actually been doing exactly that myself. You can throw out some settings because there are errors, but lots are "valid" data wise but are still garbage image-wise. I just have a text editor open with the setting in question, a terminal open so i can hit up arrow key (last command) and enter to run it, and have it open an image viewer with the new image. So i have a feedback time of about 5 seconds per value change. I've been thinking of turning it into a script that others could run.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/05/2014 05:41 AM
Here's all my best iframe8's this far.

Edit: And just a note, I've only been working on the middle section thus far which is why the bottom section stays garbage. Also, the settings used are in the filenames, drop the iframe8- section and change the -'s to :'s.

Edit2: Working on these videos has wanted to make me go find the guy who wrote their rocket video encoder and shake them. Why the hell do they interlace their video and then re-encode the interlacing? It increases the bandwidth for their videos a crap ton by doing that which allows for higher error rate. Further, they didn't add all the stuff built into the codec that anyone streaming video uses to ensure error correction. It's almost like the cameras were slapped on as an afterthought purely for PR. It was #1 on my list if I ever got an internship at SpaceX would be to look into their codebase, look up who did the commits on this code and go bug them about it.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/05/2014 06:18 AM
<snip>

arnezami I've been talking to michaelni a lot today in learning how to do this process and I think your concept of "end markers" is off. Every block has a pre-defined size which is determined by its format. When a block is mis-interpreted then the size automatically becomes wrong and the block "eats" extra bits from the next block or misses bits from its block shoving them into the next block. Occasionally these errors cancel each other out (because there is a small finite number of block types and sizes) and the streaming is "resynced" to the proper block ordering so you get a string of correct blocks. You often don't see this resync because after its resynced the blocks are still referencing all the previous corrupted blocks. This is why after you fix one block suddenly lots more blocks become readable IMO.

If michaelni could comment though that would probably clear more things up.

Edit: The key difference that matters from your explanation is that changing the start position of a block can _increase_ or _decrease_ how far away the stop position is for that block because of how the block is interpreted.
Very interesting! If that's true than that would probably help greatly in (semi-)automating this process! Because I had almost given up on that. So I'm very interested in an explanation of how this actually works.

Regards,

arnezami

[edit] When I look at my log though I almost do not see any of the same length of bitsizes per block. So I don't think macroblocks have any sort of fixed sizes.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/05/2014 06:36 AM
<snip>

arnezami I've been talking to michaelni a lot today in learning how to do this process and I think your concept of "end markers" is off. Every block has a pre-defined size which is determined by its format. When a block is mis-interpreted then the size automatically becomes wrong and the block "eats" extra bits from the next block or misses bits from its block shoving them into the next block. Occasionally these errors cancel each other out (because there is a small finite number of block types and sizes) and the streaming is "resynced" to the proper block ordering so you get a string of correct blocks. You often don't see this resync because after its resynced the blocks are still referencing all the previous corrupted blocks. This is why after you fix one block suddenly lots more blocks become readable IMO.

If michaelni could comment though that would probably clear more things up.

Edit: The key difference that matters from your explanation is that changing the start position of a block can _increase_ or _decrease_ how far away the stop position is for that block because of how the block is interpreted.
Very interesting! If that's true than that would probably help greatly in (semi-)automating this process! Because I had almost given up on that. So I'm very interested in an explanation of how this actually works.

Regards,

arnezami

[edit] When I look at my log though I almost do not see any of the same length of bitsizes per block. So I don't think macroblocks have any sort of fixed sizes.

They're content-based sizes I think. Maybe I'm wrong. When michaelni gets up he can answer.

Just as a note, in my (or michaelni's repo) version of your code, re-compile while uncommenting the "//#define TRACE" variable in libavcodec/get_bits.h. It prints a bunch of info about the contents of every block as it decodes it (run with your debug option on).
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/05/2014 07:07 AM
I need some "truth" data. Do we have any camera views from any other video that shows what the lower portion of the image (the rocket itself) looks like at all? I'm trying to figure out something remarkable about it so I can tell one block from another.

I need to order a bunch of puzzle pieces. Each has numbers on the bottom that need to be in order, they are perfectly square, varying flat white and there are either too few pieces or too many pieces and I need to either make some new ones or throw some away.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Digitalchromakey on 05/05/2014 07:09 AM
<snip>

arnezami I've been talking to michaelni a lot today in learning how to do this process and I think your concept of "end markers" is off. Every block has a pre-defined size which is determined by its format. When a block is mis-interpreted then the size automatically becomes wrong and the block "eats" extra bits from the next block or misses bits from its block shoving them into the next block. Occasionally these errors cancel each other out (because there is a small finite number of block types and sizes) and the streaming is "resynced" to the proper block ordering so you get a string of correct blocks. You often don't see this resync because after its resynced the blocks are still referencing all the previous corrupted blocks. This is why after you fix one block suddenly lots more blocks become readable IMO.

If michaelni could comment though that would probably clear more things up.

Edit: The key difference that matters from your explanation is that changing the start position of a block can _increase_ or _decrease_ how far away the stop position is for that block because of how the block is interpreted.
Very interesting! If that's true than that would probably help greatly in (semi-)automating this process! Because I had almost given up on that. So I'm very interested in an explanation of how this actually works.

Regards,

arnezami

[edit] When I look at my log though I almost do not see any of the same length of bitsizes per block. So I don't think macroblocks have any sort of fixed sizes.
The MPEG 4 spec has the algorithm that allows you to parse the elementary visual bit stream, which can be of varying bit length, however the sequence_end_code x000001B7 is always byte aligned with defined stuffing bit codes preceding the end code depending on the number of bits required.

The sequence start codes are also similarly byte aligned.

Quote
Bits to be stuffed Stuffing Codeword
1 0
2 01
3 011
4 0111
5 01111
6 011111
7 0111111
8 01111111
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/05/2014 07:17 AM
<snip>

arnezami I've been talking to michaelni a lot today in learning how to do this process and I think your concept of "end markers" is off. Every block has a pre-defined size which is determined by its format. When a block is mis-interpreted then the size automatically becomes wrong and the block "eats" extra bits from the next block or misses bits from its block shoving them into the next block. Occasionally these errors cancel each other out (because there is a small finite number of block types and sizes) and the streaming is "resynced" to the proper block ordering so you get a string of correct blocks. You often don't see this resync because after its resynced the blocks are still referencing all the previous corrupted blocks. This is why after you fix one block suddenly lots more blocks become readable IMO.

If michaelni could comment though that would probably clear more things up.

Edit: The key difference that matters from your explanation is that changing the start position of a block can _increase_ or _decrease_ how far away the stop position is for that block because of how the block is interpreted.
Very interesting! If that's true than that would probably help greatly in (semi-)automating this process! Because I had almost given up on that. So I'm very interested in an explanation of how this actually works.

Regards,

arnezami

[edit] When I look at my log though I almost do not see any of the same length of bitsizes per block. So I don't think macroblocks have any sort of fixed sizes.
The MPEG 4 spec has the algorithm that allows you to parse the elementary visual bit stream, which can be of varying bit length, however the sequence_end_code x000001B7 is always byte aligned with defined stuffing bit codes preceding the end code depending on the number of bits required.

The sequence start codes are also similarly byte aligned.

Quote
Bits to be stuffed Stuffing Codeword
1 0
2 01
3 011
4 0111
5 01111
6 011111
7 0111111
8 01111111

A link to the spec section in question?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Syrinx on 05/05/2014 07:26 AM
I'm anxious to assist any way that I can.  I've got the custom ffmpeg built and installed (from the github link).  I'm a good engineer and programmer but I know next to nothing about MPEG or graphics in general.

I guess I'm confused about what I'm supposed to do now.  What do I give ffmpeg and what should I expect to get out?  Do I need to download the video or iframes from somewhere?  If so, I don't want to repair what you guys have already repaired.  How are you guys staying in sync?  Is there any coordination going on?

Would it be more productive for me to wait until I can assist in a more brute force fashion?  By that I mean using my eyeballs rather than my engineering skills.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/05/2014 07:31 AM
I'm anxious to assist any way that I can.  I've got the custom ffmpeg built and installed (from the github link).  I'm a good engineer and programmer but I know next to nothing about MPEG or graphics in general.

I guess I'm confused about what I'm supposed to do now.  What do I give ffmpeg and what should I expect to get out?  Do I need to download the video or iframes from somewhere?  If so, I don't want to repair what you guys have already repaired.  How are you guys staying in sync?  Is there any coordination going on?

Would it be more productive for me to wait until I can assist in a more brute force fashion?  By that I mean using my eyeballs rather than my engineering skills.

Download the iframes linked previously. Then use the command that was listed to generate .png from that. Then modify the -mmb (the hard part) in various ways to make the image better. Format is listed in previous post as well: X, Y, block_start. Start by using an mmb that one of us mentioned and the given iframe and make sure you get the same image, then start manipulating those values to start teaching yourself what its doing.

./ffmpeg -mmb mmb_stuff_here -debug mb_pos_size -err_detect ignore_err -s:0 704:480 -i path/to/iframe/file/here -f image2 img.png
To see the original iframe just remove the -mmb option.

No coordination other than a couple of PMs between me and arnezami and this forum. I was also talking directly to michaelni on IRC in the #ffmpeg channel on freenode, but that was mostly to ask questions to learn about the format. Just assume that if no one's mentioned their work on a frame then no one has started on it yet. I'm personally stuck on iframe8 right now as I can't figure out the mess at the bottom. It's 3:40 AM here so I'll be heading to bed shortly, but I'll be resuming when I get up. No work or school tomorrow/today (Monday).

If enough people want to coordinate to work on it by tomorrow then I'll probably set up an IRC channel and we can coordinate/teach each other there live.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Syrinx on 05/05/2014 07:58 AM
OK, thanks for the info.

The iframes can be downloaded from reply #89 in this thread.

I downloaded iframe 11, ran this:

ffmpeg -mmb 0:0:10020 -debug mb_pos_size -err_detect ignore_err -i iframes/iframe_11.mpg4-img -f image2 img-11.png

and got the same image that had been posted earlier.

Since there are only 12 iframes, does that mean there are only 12 images in the entire video?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jarnis on 05/05/2014 08:04 AM
I need some "truth" data. Do we have any camera views from any other video that shows what the lower portion of the image (the rocket itself) looks like at all? I'm trying to figure out something remarkable about it so I can tell one block from another.

I need to order a bunch of puzzle pieces. Each has numbers on the bottom that need to be in order, they are perfectly square, varying flat white and there are either too few pieces or too many pieces and I need to either make some new ones or throw some away.

There are images from the rocketcam in this thread

http://forum.nasaspaceflight.com/index.php?topic=34502.105

However, I think these are actually taken from 2nd stage (higher up in the stack, slightly different angle) and a camera on the 1st stage was used only for this landing footage, so the image is not entirely same. You'd still get some idea as to what the rocket body looks like.

I do agree that it would probably help if SpaceX would release at least a couple of frames from the 1st stage camera from higher up. Reportedly they have good video of the braking burns but chose not to release it, perhaps to keep their secret sauce secret.

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jakusb on 05/05/2014 08:07 AM
I need some "truth" data. Do we have any camera views from any other video that shows what the lower portion of the image (the rocket itself) looks like at all? I'm trying to figure out something remarkable about it so I can tell one block from another.

I need to order a bunch of puzzle pieces. Each has numbers on the bottom that need to be in order, they are perfectly square, varying flat white and there are either too few pieces or too many pieces and I need to either make some new ones or throw some away.

There are images from the rocketcam in this thread

http://forum.nasaspaceflight.com/index.php?topic=34502.105

However, I think these are actually taken from 2nd stage (higher up in the stack, slightly different angle) and a camera on the 1st stage was used only for this landing footage, so the image is not entirely same. You'd still get some idea as to what the rocket body looks like.

I do agree that it would probably help if SpaceX would release at least a couple of frames from the 1st stage camera from higher up. Reportedly they have good video of the braking burns but chose not to release it, perhaps to keep their secret sauce secret.

Maybe something for Chris to pitch in? :)

Edit/Lar: I PMed Chris. If you want to make sure Chris sees it, PMing him is a better approach than just mentioning his name in a post. He's godlike but not omniescent :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/05/2014 08:18 AM
I think I got a bit more of iframe 8, hard to tell what the correct alignment is.

ffmpeg -mmb 07:14:17233,14:14:57111,08:15:17233,09:15:63546,19:16:17233,21:16:73568,08:21:17233,00:22:111664 -debug mb_pos_size -err_detect ignore_err -s:0 704:480 -i iframe_8.mpg4-img -f image2 img-%03d.png
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Digitalchromakey on 05/05/2014 08:27 AM
<snip>

arnezami I've been talking to michaelni a lot today in learning how to do this process and I think your concept of "end markers" is off. Every block has a pre-defined size which is determined by its format. When a block is mis-interpreted then the size automatically becomes wrong and the block "eats" extra bits from the next block or misses bits from its block shoving them into the next block. Occasionally these errors cancel each other out (because there is a small finite number of block types and sizes) and the streaming is "resynced" to the proper block ordering so you get a string of correct blocks. You often don't see this resync because after its resynced the blocks are still referencing all the previous corrupted blocks. This is why after you fix one block suddenly lots more blocks become readable IMO.

If michaelni could comment though that would probably clear more things up.

Edit: The key difference that matters from your explanation is that changing the start position of a block can _increase_ or _decrease_ how far away the stop position is for that block because of how the block is interpreted.
Very interesting! If that's true than that would probably help greatly in (semi-)automating this process! Because I had almost given up on that. So I'm very interested in an explanation of how this actually works.

Regards,

arnezami

[edit] When I look at my log though I almost do not see any of the same length of bitsizes per block. So I don't think macroblocks have any sort of fixed sizes.
The MPEG 4 spec has the algorithm that allows you to parse the elementary visual bit stream, which can be of varying bit length, however the sequence_end_code x000001B7 is always byte aligned with defined stuffing bit codes preceding the end code depending on the number of bits required.

The sequence start codes are also similarly byte aligned.

Quote
Bits to be stuffed Stuffing Codeword
1 0
2 01
3 011
4 0111
5 01111
6 011111
7 0111111
8 01111111

A link to the spec section in question?
ISO/IEC 144696-2 Section 6 Video Bitstream Syntax and Semantics.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: grythumn on 05/05/2014 08:30 AM
Some more notes:

The macro blocks are 16x16. Sometimes you want to replace a block that isn't throwing an error because it messes up the image, and that helps work out what number to mess with.

Work sequentially, if possible... changing stuff early in the picture can cause green blocks to reappear in what was junky, but clear, picture.

Some green blocks just won't go away.

All I've gotten from frame 4 is the top of the timestamp, at position 119316:

-Bob
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Syrinx on 05/05/2014 09:07 AM
I think I understand what's going on now.

There are 44 * 30 macroblocks in the picture.  ffmpeg is dumping the x-y locations of the macroblocks, and the byte offset and byte size of each macroblock.

E.g.,

[mpeg4 @ 0x60007a520] MB pos/size: 0 30:00:12347 104
[mpeg4 @ 0x60007a520] MB pos/size: 0 31:00:12451 53
[mpeg4 @ 0x60007a520] MB pos/size: 0 32:00:12504 91
[mpeg4 @ 0x60007a520] MB pos/size: 0 33:00:12595 116
[mpeg4 @ 0x60007a520] MB pos/size: 0 34:00:12711 77
[mpeg4 @ 0x60007a520] MB pos/size: 0 35:00:12788 133
[mpeg4 @ 0x60007a520] MB pos/size: 0 36:00:12921 73
[mpeg4 @ 0x60007a520] MB pos/size: 0 37:00:12994 95
[mpeg4 @ 0x60007a520] MB pos/size: 0 38:00:13089 110
[mpeg4 @ 0x60007a520] ac-tex damaged at 39 0
[mpeg4 @ 0x60007a520] MB pos/size: -1 39:00:13199 7


The fields are:

[mpeg4 @ 0x60007a520] MB pos/size: return-code x:y:offset size]

You guys are manually looking at the picture, picking out a familiar macroblock and placing it at the correct x-y location using the -mmb override.  When a macroblock is placed at the correct x-y location, it might also make other macroblocks snap in to place.  Then you repeat for another macroblock.

A return code of -1 means ffmpeg couldn't decode the macroblock.  0 means success, but it doesn't necessarily mean the macroblock is in the correct x-y location.

Is that all correct?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/05/2014 09:52 AM
I need some "truth" data. Do we have any camera views from any other video that shows what the lower portion of the image (the rocket itself) looks like at all? I'm trying to figure out something remarkable about it so I can tell one block from another.

I need to order a bunch of puzzle pieces. Each has numbers on the bottom that need to be in order, they are perfectly square, varying flat white and there are either too few pieces or too many pieces and I need to either make some new ones or throw some away.

There are images from the rocketcam in this thread

http://forum.nasaspaceflight.com/index.php?topic=34502.105

However, I think these are actually taken from 2nd stage (higher up in the stack, slightly different angle) and a camera on the 1st stage was used only for this landing footage, so the image is not entirely same. You'd still get some idea as to what the rocket body looks like.

I do agree that it would probably help if SpaceX would release at least a couple of frames from the 1st stage camera from higher up. Reportedly they have good video of the braking burns but chose not to release it, perhaps to keep their secret sauce secret.

Na, I need an image from the lower camera. Preferably during retroburn or something, but any time works.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/05/2014 10:00 AM
I think I understand what's going on now.

There are 44 * 30 macroblocks in the picture.  ffmpeg is dumping the x-y locations of the macroblocks, and the byte offset and byte size of each macroblock.

E.g.,

[mpeg4 @ 0x60007a520] MB pos/size: 0 30:00:12347 104
[mpeg4 @ 0x60007a520] MB pos/size: 0 31:00:12451 53
[mpeg4 @ 0x60007a520] MB pos/size: 0 32:00:12504 91
[mpeg4 @ 0x60007a520] MB pos/size: 0 33:00:12595 116
[mpeg4 @ 0x60007a520] MB pos/size: 0 34:00:12711 77
[mpeg4 @ 0x60007a520] MB pos/size: 0 35:00:12788 133
[mpeg4 @ 0x60007a520] MB pos/size: 0 36:00:12921 73
[mpeg4 @ 0x60007a520] MB pos/size: 0 37:00:12994 95
[mpeg4 @ 0x60007a520] MB pos/size: 0 38:00:13089 110
[mpeg4 @ 0x60007a520] ac-tex damaged at 39 0
[mpeg4 @ 0x60007a520] MB pos/size: -1 39:00:13199 7


The fields are:

[mpeg4 @ 0x60007a520] MB pos/size: return-code x:y:offset size]

You guys are manually looking at the picture, picking out a familiar macroblock and placing it at the correct x-y location using the -mmb override.  When a macroblock is placed at the correct x-y location, it might also make other macroblocks snap in to place.  Then you repeat for another macroblock.

A return code of -1 means ffmpeg couldn't decode the macroblock.  0 means success, but it doesn't necessarily mean the macroblock is in the correct x-y location.

Is that all correct?

We're not placing macroblocks at the correct locations as is replacing corrupted macroblocks with blocks from _elsewhere_ in the image. Every macroblock in the image is a reference+changes to either the block directly above it or the block directly to the left (this choice is made when the video stream is created). If you replace a macroblock with another block that has "roughly" the same "+changes" as the actual block then the created block will look roughly like the original and blocks that reference it will look roughly how they are supposed to.

In some cases where things are too messed up we look for a macroblock that has changes that we know should be at a specific location (the legs) and forcibly put it there. That doesn't fix any previous blocks, but as you mention it often makes other blocks that follow it snap into place. Ideally after you put that block in place you should work backwards (right to left, bottom to top) toward the other bad blocks trying to replace those as well while maintaining the same "block address" of the block you fixed.

Everything else is correct.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/05/2014 10:03 AM
Since there are only 12 iframes, does that mean there are only 12 images in the entire video?

There are 12 iframes which are complete images that reference no previous images. Every other frame is a reference to one of those frames. To be specific, any frames in between two iframes reference the previous iframe, other frames that also reference that iframe or pieces of the frame reference other pieces of the same frame. Welcome to the world of high-compression video codecs. :P Errors propagate like hell.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/05/2014 10:26 AM
Na, I need an image from the lower camera. Preferably during retroburn or something, but any time works.

This picture of iframe2 contains data at the (nearly) bottom:

 http://forum.nasaspaceflight.com/index.php?topic=34597.msg1193527#msg1193527 (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1193527#msg1193527)

Note that the clock is the lowest block. So this is what you want I think. It's just that everything in this picture is moved up because I "fast-forwarded" at some point (essentially skipped like 4 rows). So especially the left is what you want. And the stage is probable symetrical. So could be useful.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/05/2014 10:37 AM
Na, I need an image from the lower camera. Preferably during retroburn or something, but any time works.

Maybe you can check the Cassiope launch highlights movie (at 2:40) where they show the reentry burn: http://forum.nasaspaceflight.com/index.php?topic=32946.msg1109241#msg1109241
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: fatdeeman on 05/05/2014 12:53 PM
Absolutely in awe of what's being achieved here. I had heard before that videos with errors still contained a lot of usable information but I had no idea it could be retrieved to this extent. I had visions of having to reconstruct entire frames by moving blocks around like a puzzle and guessing if they were in the right place.

Absolutely fantastic work being done here!

One more thing, Elon mentioned fan sourcing helped restore video footage before. Does any one know of any examples?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Asmegin on 05/05/2014 01:33 PM
Absolutely in awe of what's being achieved here. I had heard before that videos with errors still contained a lot of usable information but I had no idea it could be retrieved to this extent. I had visions of having to reconstruct entire frames by moving blocks around like a puzzle and guessing if they were in the right place.

Absolutely fantastic work being done here!

One more thing, Elon mentioned fan sourcing helped restore video footage before. Does any one know of any examples?

+1 to being amazed. I wish I could help, but I literally have no idea what you guys are talking about :D

As for the video, it's possible he was referencing the Mars Curiosity Landing video which was amazing after someone took 4 weeks to fix it: https://www.youtube.com/watch?v=Esj5juUzhpU
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: fatdeeman on 05/05/2014 01:46 PM
Absolutely in awe of what's being achieved here. I had heard before that videos with errors still contained a lot of usable information but I had no idea it could be retrieved to this extent. I had visions of having to reconstruct entire frames by moving blocks around like a puzzle and guessing if they were in the right place.

Absolutely fantastic work being done here!

One more thing, Elon mentioned fan sourcing helped restore video footage before. Does any one know of any examples?

+1 to being amazed. I wish I could help, but I literally have no idea what you guys are talking about :D

As for the video, it's possible he was referencing the Mars Curiosity Landing video which was amazing after someone took 4 weeks to fix it: https://www.youtube.com/watch?v=Esj5juUzhpU

Ah yes I was watching this the other day funnily enough but it didn't click! The amount of detail in this one compared to the version NASA released is incredible!

I couldn't do what these guys are doing either! A lot of the things being said make sense to me but as far as doing it myself I wouldn't have a clue! I'd be happy if there was anything slightly less perplexing like cheap labour!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: llanitedave on 05/05/2014 04:51 PM
Wow, all that and sound too!  Beautiful!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jeff Lerner on 05/05/2014 05:21 PM
I also want to say how amazed I am at the effort and progress being made with this video.

Having said that, I thought I'd toss in my $0.02 as a retired IT project manager of 38 years. I often  found that the  developers working for me would get so caught up developing their code that they has no time to see the bigger picture...that's where I came in as PM..

So,having said that, I'm wondering if anyone is aware of any other efforts on any other space related sites, photo or video sites or for that matter, even at SpaceX, to answer Elon's call to arms to fix up this video...I'm just thinking that it would be sad to have all this great effort wasted because some where else, someone else has already done this work.....

...and thats maybe where Chris can help out as he has some contacts at SpaceX and can inquire as to whether they are aware of any other progress being made...if little else, Chris could at least make SpaceX know that people on this site are working on a solution and might require some info from SpaceX.....
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Syrinx on 05/05/2014 05:46 PM
We're not placing macroblocks at the correct locations as is replacing corrupted macroblocks with blocks from _elsewhere_ in the image. Every macroblock in the image is a reference+changes to either the block directly above it or the block directly to the left (this choice is made when the video stream is created). If you replace a macroblock with another block that has "roughly" the same "+changes" as the actual block then the created block will look roughly like the original and blocks that reference it will look roughly how they are supposed to.

In some cases where things are too messed up we look for a macroblock that has changes that we know should be at a specific location (the legs) and forcibly put it there. That doesn't fix any previous blocks, but as you mention it often makes other blocks that follow it snap into place. Ideally after you put that block in place you should work backwards (right to left, bottom to top) toward the other bad blocks trying to replace those as well while maintaining the same "block address" of the block you fixed.

Everything else is correct.

Is it really just a matter of finding a suitable stand-in for corrupt macroblocks?  Maybe the process is not so bad then!

But I was looking at iframe 11.  Macroblock 0:0 is valid (not corrupt) as reported by the debug info, but somebody replaced it anyway (with 5:0 I think) and it greatly improved the image.

(Maybe I am mistaken.  Maybe iframe 11 0:0 is corrupt.  I would check but I'm not on my home computer right now.)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: swervin on 05/05/2014 05:48 PM
Perhaps the folks who assisted in the cleaned up Curiosity video could be contacted for 'lessons learned' on that video? ... Or even assist on this one if they were interested.

I thought it would be cool if the 'code' was broken on how to clean this up, if that was made available, a la SETI @ home, to use the computing power of a bunch of computers to clean it up faster. Then again, maybe not much computing power is req'd.

This coming from someone who has no idea how to do this ... Haha

Cheers,
Splinter
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: grythumn on 05/05/2014 05:52 PM
I also want to say how amazed I am at the effort and progress being made with this video.

Having said that, I thought I'd toss in my $0.02 as a retired IT project manager of 38 years. I often  found that the  developers working for me would get so caught up developing their code that they has no time to see the bigger picture...that's where I came in as PM..

So,having said that, I'm wondering if anyone is aware of any other efforts on any other space related sites, photo or video sites or for that matter, even at SpaceX, to answer Elon's call to arms to fix up this video...I'm just thinking that it would be sad to have all this great effort wasted because some where else, someone else has already done this work.....

Aside from the youtube video, only attempt I've seen is:

temp bana/dtZOQ

That's a photoshop, not actual data.

-Bob
EDIT: Apparently I can't link to it. Try an imaging site that begins with an i and ends with an r. Popular on facebook and such.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Lars_J on 05/05/2014 05:52 PM
Perhaps the folks who assisted in the cleaned up Curiosity video could be contacted for 'lessons learned' on that video? ... Or even assist on this one if they were interested.

The Curiosity video is a completely different scenario. No cleanup of re-interpretation of missing data was needed - and if it was, not anywhere near this level.

The real achievement of the Curiosity video was the smooth frame interpolation from still frames to create the illusion of a full frame rate video.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/05/2014 05:56 PM
I'm truly amazed on the outstanding work that is being done in recovering the I-frames. Kudos to arnezami for developing the approach, and for everyone else that has been involved. Even if only those 12 images are recovered, I'm sure they are going to be really valuable for Space X.

Just a question: Is anybody collecting/monitoring the progress on this task? As several of you are working on parallel, I believe a monitoring tool would be useful in order to lower the risk of having several individuals doing the same thing. 

To start, may I suggest on keeping a table like the one originally describing the I-Frames (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1193316#msg1193316) in order for everyone to know who is working on each one?


I'll be glad to take that responsibility if you, the guys actually working on the images,  find it useful...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: neoforce on 05/05/2014 05:57 PM
Perhaps the folks who assisted in the cleaned up Curiosity video could be contacted for 'lessons learned' on that video? ... Or even assist on this one if they were interested.

The Curiosity video is a completely different scenario. No cleanup of re-interpretation of missing data was needed - and if it was, not anywhere near this level.

The real achievement of the Curiosity video was the smooth frame interpolation from still frames to create the illusion of a full frame rate video.

Some additional details on the differences... 

In the Falcon 9 video, we have compression that lost blocks due to transmission fall out.  In the Curiosity video, it was by design 4 frames per second.  All of the data for each of those 4 frames per second were recovered. So what he did was "create" an additional 26 frames for each second of footage to get it to 30 frames per second.

It is a very different problem.  His was enhancing, adding full frames between existing frames of complete information.  In this thread, here they are trying to repair data within each frame of the video.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/05/2014 06:00 PM
<snip>

arnezami I've been talking to michaelni a lot today in learning how to do this process and I think your concept of "end markers" is off. Every block has a pre-defined size which is determined by its format. When a block is mis-interpreted then the size automatically becomes wrong and the block "eats" extra bits from the next block or misses bits from its block shoving them into the next block. Occasionally these errors cancel each other out (because there is a small finite number of block types and sizes) and the streaming is "resynced" to the proper block ordering so you get a string of correct blocks. You often don't see this resync because after its resynced the blocks are still referencing all the previous corrupted blocks. This is why after you fix one block suddenly lots more blocks become readable IMO.

If michaelni could comment though that would probably clear more things up.

Edit: The key difference that matters from your explanation is that changing the start position of a block can _increase_ or _decrease_ how far away the stop position is for that block because of how the block is interpreted.
Hi,

I think I found most the answer to this question.

In the MPEG 4-2 specification ( I googled for "Video Bitstream Syntax and Semantics" and found a doc file called "MPEG-2_Video_IS.doc") its says in section 6.2.6 that blocks end with "End of block"-bits (10b I believe). Later on in section 8.2 it says that macroblocks will end with the "End of block"-symbol of its last block. We have 4 blocks in a macroblock I believe. See the attachments.

So I think my intuition of the macroblocks ending with stop-bits ("end markers") is pretty spot on, albeit with a little nuance that the stop-bits are in the blocks not in the macroblock. The document also gives a lot of info on bits that may be recognizable somehow. Not sure yet how we can use this for semi-automation.

Regards,

arnezami

[edit] I'm sorry this is of course MPEG-2. Duh. This probably does not apply to MPEG-4.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: vanderhilst on 05/05/2014 06:12 PM
Hi all, I'm really enjoying to learn from the inspiring progress that is made by all you guys,
as well as by the efforts of SpaceX to create this video in the first place!!

As an experienced programmer in Mathematica, I wasn't able to chip in up to this point, but it seems to me that you have come to a point where automated calls to ffmpeg may be beneficial to the process of recovering the I-frames. I'm willing to jump in, and have two questions:

1) Did anybody listen to Jeff Lerner's comments, reply #132?

2) Is it sensible to use this way to have ffmpeg run under windows? http://www.wikihow.com/Install-FFmpeg-on-Windows

Thanks and enjoy the revealing!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/05/2014 06:19 PM
I was toying with the idea of creating a web app so that people could do this visually and would make it easier to distribute the workload. I think it would be a good idea but I also think that by the time it's done people will have managed to get most of the way there. I don't know if something like that would have any re-use value save for something like this happening again.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/05/2014 06:25 PM
Aha!

I think I found a way to detect the beginning of macroblocks! :)

They seem to have a fixed 11-bit macroblock_escape code (0000000100b) followed by an 1-11 bit macroblock_address_increment! (very useful for detecting where the macroblock should be). If this is actually true then finding the start of macroblocks might actually become quite easy. And its even followed by a macroblock_type of 1-9 bits. That's a lot of bits to be non-random.

See attachments.

Regards,

arnezami

[edit] I'm sorry this is of course MPEG-2. Duh. This probably does not apply to MPEG-4.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: laszlo on 05/05/2014 06:25 PM
can someone please explain to me why spacex still hasn't released the footage they got from their airplane? Why are they being so secretive about it? I mean seeing is believing so why not just release the footage and prove to everyone in the world they really accomplished such an unprecedented feat?

I mean they did actually record it from their airplane right?

The left landing gear didn't deploy and they didn't want the embarrassment of showing it skidding off the runway. Oh, wait. Wrong landing. My bad.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Syrinx on 05/05/2014 06:26 PM
2) Is it sensible to use this way to have ffmpeg run under windows? http://www.wikihow.com/Install-FFmpeg-on-Windows

First, you need to install the custom ffmpeg version that's been made available in this thread on github.  You probably already knew that, but I wanted to make it explicit.

I didn't try the install mechanism presented at that link.  I installed mine on top of Cygwin.  It look about an hour and I had no problems, but of course you also need to install Cygwin if you don't have it.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: grythumn on 05/05/2014 06:56 PM
2) Is it sensible to use this way to have ffmpeg run under windows? http://www.wikihow.com/Install-FFmpeg-on-Windows

This might work without a full cygwin install (at least cygcheck thinks I got all the DLLs). But I recommend it anyway, or at least some sort of improved shell.

EDIT: This is a command line utility. Please only use if you know how to use such.

-Bob
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: vanderhilst on 05/05/2014 07:10 PM
Yay! Thank you both, Syrinx and Bob!

The ffmpeg as downloadable in the post by Bob above, combined with the method in the wikihow link I posted a few posts earlier, actually allow me to get a correct response to a command line command "ffmpeg -version" under 64-bit Windows 7.

I will proceed to try and run the other commands explained before to recreate the results posted by others.
If successful, I will try to see whether automation will do anything useful. I will start with I-frame 11, if any.

edit: for clarity: I did NOT install Cygwin.  :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: cosmonautdjp on 05/05/2014 07:23 PM
I asked Doug Ellison, the JPL guy who did the MSL descent video, if he had any tips.

@danieljphilo:@doug_ellison They're trying to improve the SpaceX first stage landing video at http://nasaspaceflight.com  Any tips? http://forum.nasaspaceflight.com/index.php?topic=34597.135 …

@doug_ellison:@danieljphilo Nope. Beyond my skills.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jarnis on 05/05/2014 07:29 PM
I asked Doug Ellison, the JPL guy who did the MSL descent video, if he had any tips.

@danieljphilo:@doug_ellison They're trying to improve the SpaceX first stage landing video at http://nasaspaceflight.com  Any tips? http://forum.nasaspaceflight.com/index.php?topic=34597.135 …

@doug_ellison:@danieljphilo Nope. Beyond my skills.

Not surprised. MSL video was about doing high quality interpolation from fairly good keyframes.

Here the keyframes are in tatters and you really need to do painstaking manual reconstruction...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/05/2014 07:31 PM
I asked Doug Ellison, the JPL guy who did the MSL descent video, if he had any tips.

@danieljphilo:@doug_ellison They're trying to improve the SpaceX first stage landing video at http://nasaspaceflight.com  Any tips? http://forum.nasaspaceflight.com/index.php?topic=34597.135 …

@doug_ellison:@danieljphilo Nope. Beyond my skills.

Top man. Met him in real life and really happy he ended up at JPL.

Keep up the good work here. It's all over my head, but clearly a hardcore process. Very impressive!

Remember, any big breakthrough, send me a message and I'll fast track you to someone at SpaceX. There's probably a free ticket to ride on Dragon in it for a win! ;)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Syrinx on 05/05/2014 07:41 PM
We're not placing macroblocks at the correct locations as is replacing corrupted macroblocks with blocks from _elsewhere_ in the image. Every macroblock in the image is a reference+changes to either the block directly above it or the block directly to the left (this choice is made when the video stream is created). If you replace a macroblock with another block that has "roughly" the same "+changes" as the actual block then the created block will look roughly like the original and blocks that reference it will look roughly how they are supposed to.

In some cases where things are too messed up we look for a macroblock that has changes that we know should be at a specific location (the legs) and forcibly put it there. That doesn't fix any previous blocks, but as you mention it often makes other blocks that follow it snap into place. Ideally after you put that block in place you should work backwards (right to left, bottom to top) toward the other bad blocks trying to replace those as well while maintaining the same "block address" of the block you fixed.

Everything else is correct.

Is it really just a matter of finding a suitable stand-in for corrupt macroblocks?  Maybe the process is not so bad then!

But I was looking at iframe 11.  Macroblock 0:0 is valid (not corrupt) as reported by the debug info, but somebody replaced it anyway (with 5:0 I think) and it greatly improved the image.

(Maybe I am mistaken.  Maybe iframe 11 0:0 is corrupt.  I would check but I'm not on my home computer right now.)

OK, I double-checked and macroblock 0:0 is indeed valid:

[mpeg4 @ 0x80062980] MB pos/size: 0 00:00:10020 128
[mpeg4 @ 0x80062980] MB pos/size: -1 01:00:10148 102
[mpeg4 @ 0x80062980] Error at MB: 1
[mpeg4 @ 0x80062980] MB pos/size: 0 02:00:10250 109
[mpeg4 @ 0x80062980] I cbpy damaged at 3 0
[mpeg4 @ 0x80062980] MB pos/size: -1 03:00:10359 4
[mpeg4 @ 0x80062980] Error at MB: 3
[mpeg4 @ 0x80062980] I cbpc damaged at 4 0
[mpeg4 @ 0x80062980] MB pos/size: -1 04:00:10363 6
[mpeg4 @ 0x80062980] Error at MB: 4
[mpeg4 @ 0x80062980] MB pos/size: 0 05:00:10369 102
[mpeg4 @ 0x80062980] MB pos/size: 0 06:00:10471 64
[mpeg4 @ 0x80062980] MB pos/size: 0 07:00:10535 92
[mpeg4 @ 0x80062980] MB pos/size: 0 08:00:10627 52
[mpeg4 @ 0x80062980] MB pos/size: 0 09:00:10679 64


So should I "simply" find suitable stand-ins for corrupt macroblocks?  Should I replace (some) valid macroblocks?  Both?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/05/2014 08:01 PM
OK, I double-checked and macroblock 0:0 is indeed valid:

[mpeg4 @ 0x80062980] MB pos/size: 0 00:00:10020 128
[mpeg4 @ 0x80062980] MB pos/size: -1 01:00:10148 102
[mpeg4 @ 0x80062980] Error at MB: 1
[mpeg4 @ 0x80062980] MB pos/size: 0 02:00:10250 109
[mpeg4 @ 0x80062980] I cbpy damaged at 3 0
[mpeg4 @ 0x80062980] MB pos/size: -1 03:00:10359 4
[mpeg4 @ 0x80062980] Error at MB: 3
[mpeg4 @ 0x80062980] I cbpc damaged at 4 0
[mpeg4 @ 0x80062980] MB pos/size: -1 04:00:10363 6
[mpeg4 @ 0x80062980] Error at MB: 4
[mpeg4 @ 0x80062980] MB pos/size: 0 05:00:10369 102
[mpeg4 @ 0x80062980] MB pos/size: 0 06:00:10471 64
[mpeg4 @ 0x80062980] MB pos/size: 0 07:00:10535 92
[mpeg4 @ 0x80062980] MB pos/size: 0 08:00:10627 52
[mpeg4 @ 0x80062980] MB pos/size: 0 09:00:10679 64


So should I "simply" find suitable stand-ins for corrupt macroblocks?  Should I replace (some) valid macroblocks?  Both?

First. I'm not sure you have already but if not please read/look at this fairly thourough explanation of how it all works:

http://forum.nasaspaceflight.com/index.php?topic=34597.msg1193685#msg1193685 (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1193685#msg1193685)

Second. I did intentionally replace 0,0 with a different block. But this is just for experimentation.

The reason why 0,0 has advantages is that there are no blocks above or left of it. How mpeg-4 works is that a block "inherits" the traits of the blocks left and above it. So if you're at 0,0 there is no corruption from other blocks before that.

You have to understand that in mpeg-4 everything is chained somehow: vertical, horizontal and time-wise. But its all chained in one direction: left to right, top to bottom, previous to next. This really saves in the number of bytes it takes to store/transmit a video but it also makes it very hard to reconstruct it when something is only slightly off, because a lot of other things depend on it being right: errors propagate rapidly. This is the difficulty we are trying to overcome.

But when you start a block at 0,0 you eliminate most of all this domino-effect troubles. You lose the information what you would have gotten from a top and left block when you started at your normal x and y, but its just to see which blocks are good or bad. Which is much easier to determine from 0,0. So I just used 0,0 to prove this point. Don't worry about it. The first block from 0,0 might be correct and completely ok. But I didn't care for that in this particular experiment: I wanted a starting block with no left and upper blocks! And there is only one block in the game that has that: 0,0.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/05/2014 08:04 PM
We're not placing macroblocks at the correct locations as is replacing corrupted macroblocks with blocks from _elsewhere_ in the image. Every macroblock in the image is a reference+changes to either the block directly above it or the block directly to the left (this choice is made when the video stream is created). If you replace a macroblock with another block that has "roughly" the same "+changes" as the actual block then the created block will look roughly like the original and blocks that reference it will look roughly how they are supposed to.

In some cases where things are too messed up we look for a macroblock that has changes that we know should be at a specific location (the legs) and forcibly put it there. That doesn't fix any previous blocks, but as you mention it often makes other blocks that follow it snap into place. Ideally after you put that block in place you should work backwards (right to left, bottom to top) toward the other bad blocks trying to replace those as well while maintaining the same "block address" of the block you fixed.

Everything else is correct.

Is it really just a matter of finding a suitable stand-in for corrupt macroblocks?  Maybe the process is not so bad then!

But I was looking at iframe 11.  Macroblock 0:0 is valid (not corrupt) as reported by the debug info, but somebody replaced it anyway (with 5:0 I think) and it greatly improved the image.

(Maybe I am mistaken.  Maybe iframe 11 0:0 is corrupt.  I would check but I'm not on my home computer right now.)

"Suitable" is still rather difficult though. I personally don't have a good grasp of "suitable" other than trial and error.

I haven't looked at the image in question, but often a block is corrupted such that its still has valid data. Corruption is not binary. That means a 1 bit change could just slightly tweak the color for example and would still be completely valid. I've been looking into making some more modifications to allow for overriding from the command line the contents of a block so we can try to fix the block rather than replace it.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: didacticus on 05/05/2014 08:55 PM
Very interesting reading all the discussion here! Just wanted to say I appreciate the effort you are all putting in to retrieve this video. Good luck!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: LucR on 05/05/2014 09:34 PM
I think I found a way to detect the beginning of macroblocks! :)
They seem to have a fixed 11-bit macroblock_escape code (0000000100b) followed by an 1-11 bit macroblock_address_increment!

Sorry, the macroblock_escape won't help for decoding I-frames, because:
Quote from: MPEG-2_Video_IS.doc section 6.3.17 Macroblock
macroblock_escape -- The macroblock_escape is a fixed bit-string ‘0000 0001 000’ which is used when the difference between macroblock_address and previous_macroblock_address is greater than 33. It causes the value of macroblock_address_increment to be 33 greater than the value that will be decoded by subsequent macroblock_escape and the macroblock_address_increment codewords.
In an I-frame all macroblocks are typically coded (i.e. not skipped, "There shall be no skipped macroblocks in I-pictures except [...]" at the end of section 6.3.7) so there will be no occurrences of macroblock_escape.

However, in I-frames the immediately following macroblock_address_increment code should encode a value of 1 which are coded as the bitstream bits 0000 0101 01 (from Table B-1 --- Variable length codes for macroblock_address_increment).
This is still 10 bits of constant data, hope that helps some.

If there is no spatial scalability, Table B-2 applies for the immediately following macroblock_type code, which means that it is either 1 (Intra) or 01 (Intra, Quant); another 1 or 2 constant bits.

I haven't looked at the actual bitstreams (yet); are slices start codes not used? This would make it easier to restart part-way through a frame.

By the way, for background: I have some amount of MPEG experience but it's not at the macroblock level. However, I do know how to read the bitstream syntax specs.

Edit: Added note about macroblock_type.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/05/2014 09:39 PM
In figuring out good methods, here's how I'm doing it.

I use the CLI line: ./ffmpeg -mmb `cat mmb.txt` -debug mb_pos_size -err_detect ignore_err -s:0 704:480 -i iframes/iframe_8.mpg4-img -f image2 img-001.png && open img-001.png

I use ` ` to pipe the text document into the CLI so I can rapidly change it and the open command (Mac OS X specific) to open the image.

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/05/2014 09:43 PM
I think I found a way to detect the beginning of macroblocks! :)
They seem to have a fixed 11-bit macroblock_escape code (0000000100b) followed by an 1-11 bit macroblock_address_increment!

Sorry, the macroblock_escape won't help for decoding I-frames, because:
Quote from: MPEG-2_Video_IS.doc section 6.3.17 Macroblock
macroblock_escape -- The macroblock_escape is a fixed bit-string ‘0000 0001 000’ which is used when the difference between macroblock_address and previous_macroblock_address is greater than 33. It causes the value of macroblock_address_increment to be 33 greater than the value that will be decoded by subsequent macroblock_escape and the macroblock_address_increment codewords.
In an I-frame all macroblocks are typically coded (i.e. not skipped, "There shall be no skipped macroblocks in I-pictures except [...]" at the end of section 6.3.7) so there will be no occurrences of macroblock_escape.

However, in I-frames the immediately following macroblock_address_increment code should encode a value of 1 which are coded as the bitstream bits 0000 0101 01 (from Table B-1 --- Variable length codes for macroblock_address_increment).
This is still 10 bits of constant data, hope that helps.

I haven't looked at the actual bitstreams (yet); are slices start codes not used? This would make it easier to restart part-way through a frame.

By the way, for background: I have some amount of MPEG experience but it's not at the macroblock level. However, I do know how to read the bitstream syntax specs.

Thanks for the tips. I don't know MPEG as well as you so if you could investigate and find that out and see if that works that'd be awesome!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/05/2014 09:47 PM
I am also really impressed by the work done here.

Attached you find a version of the ISO/IEC 14496 which defines the MPEG 4-2 standard.
(Edit: all these sections and tables and stuff mentioned in LucR's posting are found in there)
If I read it correctly there are no fixed end-codes of a block or marcoblock, but I only jump through the document.
Llke a good specification document it is really hard to read, at least for me and my knowledge of not too many such documents ;)
Maybe it helps to understand more of the problem.

Good Luck with finding the right bits :D

Shanuson
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/05/2014 10:03 PM
mlindner was asking for a non-corrupted image, so this one comes from the reentry burn of Cassiope launch

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: LucR on 05/05/2014 10:08 PM
If I read it correctly there are no fixed end-codes of a block or marcoblock, but I only jump through the document.
AFAIK this is correct, the "End of block" code varies, it can be either 10 (Table B-14) or 0110 (Table B-15), as described in 7.2.2 Other coefficients. In either case the bitstring is probably too short to provide for meaningful recovery.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Adaptation on 05/05/2014 10:37 PM
I updated my Transport stream tool.  It's probably not going to help much at this point but it does have a hex editor and powerful search feature with a bit level wildcard filter, ability to return results with a specified tolerated number of single bit errors and the results are easy to visualize and locate. 

To do:
Disable controls until a ts file is opened.
Make the search capable of finding results that are not byte aligned. 
Add a replace feature with a wildcard filter.
Enable bit level insertion and deletion ability. (would bit shift everything after that point in the file)


I think its a good companion tool for this transport stream editor. 
http://www.videohelp.com/tools/TS-Packet-Editor

-Update, version 3 is out-
http://forum.nasaspaceflight.com/index.php?topic=34597.msg1197014#msg1197014
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: TrueBlueWitt on 05/05/2014 10:48 PM
I feel like we're missing an important piece of this puzzle.

Was the video data stream was on it's own channel, or there was other data being transmitted on the same channel with some other controls in place to keep the data segregated?

If it was on it's own channel, having an analogue(time domain) copy of the original transmission, perhaps even with time code synced to the video file provided, there might be a lot more that could be done to extract the information.  Having the the time domain data stream it would also be much easier to tell where and when data was dropping out.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: djellison on 05/05/2014 10:52 PM
I asked Doug Ellison, the JPL guy who did the MSL descent video, if he had any tips

While I did some MSL MARDI vid stuff - I was never involved in any of the gorgeous interpolated and rotoscoped versions that some really creative people came up with. 

Impressed people are making some progress with the F9 1.1 feed - that's some seriously junky data!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: lgjy98d on 05/06/2014 12:13 AM
Just a question: Is anybody collecting/monitoring the progress on this task? As several of you are working on parallel, I believe a monitoring tool would be useful in order to lower the risk of having several individuals doing the same thing. 

To start, may I suggest on keeping a table like the one originally describing the I-Frames (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1193316#msg1193316) in order for everyone to know who is working on each one?

There is wiki at http://spacexlanding.wikispaces.com/ with current state of affairs.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: apollolanding on 05/06/2014 12:45 AM
Just when I thought I'd seen it all from the NSF crowd, this thread shows up.  Amazing, amazing work!  I don't understand 99% of what you guys are sharing, but I find the labor and results fascinating nonetheless!  Bravo Zulu to you all!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/06/2014 12:56 AM
mlindner was asking for a non-corrupted image, so this one comes from the reentry burn of Cassiope launch

That's a different camera unfortunately.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/06/2014 12:59 AM
I updated my Transport stream tool.  It's probably not going to help much at this point but it does have a hex editor and powerful search feature with a bit level wildcard filter, ability to return results with a specified tolerated number of single bit errors and the results are easy to visualize and locate. 

To do:
Disable controls until a ts file is opened.
Make the search capable of finding results that are not byte aligned. 
Add a replace feature with a wildcard filter.
Enable bit level insertion and deletion ability. (would bit shift everything after that point in the file)


I think its a good companion tool for this transport stream editor. 
http://www.videohelp.com/tools/TS-Packet-Editor

Do you have something that's not for windows?

Also, a tool that could search by bits rather than hex would be even more helpful. Hardly anything in these video blocks appears to be byte aligned so hex searching doesn't help much.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: michaelni on 05/06/2014 01:01 AM
Ive added 2 features to https://github.com/michaelni/FFmpeg/tree/spacexdebug1
first, you can now use syntax like
-mmb 40:03:-1
to fill the image with "blank" blocks starting from 40:03 and until the next specified command
and
-mmb X:20009:1B
to xor 0x1B at bit position 20009 of the frame, that is this allows to flip some bits without the need to edit the file
(atm this is limited to max 8bit per command)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/06/2014 01:02 AM
I feel like we're missing an important piece of this puzzle.

Was the video data stream was on it's own channel, or there was other data being transmitted on the same channel with some other controls in place to keep the data segregated?

If it was on it's own channel, having an analogue(time domain) copy of the original transmission, perhaps even with time code synced to the video file provided, there might be a lot more that could be done to extract the information.  Having the the time domain data stream it would also be much easier to tell where and when data was dropping out.

Someone was mentioning over on reddit they wish they had gotten an SDR (software defined radio) recording of the signal then we could dive into the signal processing layer to recover the signal.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/06/2014 01:05 AM
Ive added 2 features to https://github.com/michaelni/FFmpeg/tree/spacexdebug1
first, you can now use syntax like
-mmb 40:03:-1
to fill the image with "blank" blocks starting from 40:03 and until the next specified command
and
-mmb X:20009:1B
to xor 0x1B at bit position 20009 of the frame, that is this allows to flip some bits without the need to edit the file
(atm this is limited to max 8bit per command)

Oh awesome that helps a lot. I was trying to edit things via a hexeditor but was making little progress because I couldn't find the alignments of the data.

Quick question. When you say "xor 0x1B at bit position" does that mean it takes the next 8 bits starting at 20009 and xor's them with the 0x1B value?
Edit: He said "yes."
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/06/2014 01:23 AM
Just a question: Is anybody collecting/monitoring the progress on this task? As several of you are working on parallel, I believe a monitoring tool would be useful in order to lower the risk of having several individuals doing the same thing. 

To start, may I suggest on keeping a table like the one originally describing the I-Frames (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1193316#msg1193316) in order for everyone to know who is working on each one?

There is wiki at http://spacexlanding.wikispaces.com/ with current state of affairs.

Thanks for setting that up.

Just set up an IRC channel here on freenode for us. Use your favorite IRC client or use https://webchat.freenode.net/

Channel is #nsf-spacex-video
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: michaelni on 05/06/2014 02:03 AM
Ive added 2 features to https://github.com/michaelni/FFmpeg/tree/spacexdebug1
first, you can now use syntax like
-mmb 40:03:-1
to fill the image with "blank" blocks starting from 40:03 and until the next specified command
and
-mmb X:20009:1B
to xor 0x1B at bit position 20009 of the frame, that is this allows to flip some bits without the need to edit the file
(atm this is limited to max 8bit per command)

Added another feature, now you can adjust DC values of the blank blocks
each MB has 6 blocks 4 luma and 2 chroma
example to adjust the 4th luma DC of (blank) MB 40:3 by +100
-mmb 40:03:-1:0:0:0:100:0:0
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Adaptation on 05/06/2014 02:04 AM
Do you have something that's not for windows?

Also, a tool that could search by bits rather than hex would be even more helpful. Hardly anything in these video blocks appears to be byte aligned so hex searching doesn't help much.

Pretty sure mine would work in wine, if it dosent let me know and I can fix it.  Not certain about the other one. 

My tool does have a bit shift, with auto search selected you just press the bit shift button 7 times and it will make the result pop out at their respective offset. 

React OS in a vm is a good alternative should wine fail you.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: knotnic on 05/06/2014 03:10 AM
The video seems simple profile level 3 mpeg4 video in mpeg TS.
It seems none of the error resilience features of mpeg4 have been used when encoding it. Which is a pitty, had slices been used then the decoder could resume decoding of a frame at the next slice start, had data partitioning been used then the more important low resolution information and motion vectors would have been coded first in each slice making errors less likely to damage them. And had rvlcs been used then slices could have been decoded from both the start and end again, limiting the impact of bit errors.

I know nothing about how the video was generated or how it was transmitted, but if there was some FEC in there then then it should be possible in principle to re-run FEC decoding after manual fixing up all mpeg-TS and mpeg4-ES headers. And as such manual fixing would decrease the errors, FEC would then have fewer errors to deal with and might in a few rare cases be able to fix a few more errors.

Also if some kind of CRCs have been used, CRCs can also be used to correct bit errors as long as the number of errors is sufficiently small, which each CRC would need to correct, the exact number that could be corrected that way depends on the packet size and  crc polynomial being used.

My [outside the industry, and not a video expert either] guess is that the the reason they didn't enable them here is that the mpeg data has been extracted from a telemetry stream, and what we are seeing is after their own decryption/forward error correction/CRC was performed.  For minimizing data rate or other reasons perhaps they decided not to double up error-correction, since it was already being done.  I suspect they are probably required to encrypt the data stream for ITAR reasons and therefore couldn't release pre-error-corrected raw telemetry even if they wanted to.


And by the way-- amazing effort so far, by all involved. 
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/06/2014 03:42 AM
Ive added 2 features to https://github.com/michaelni/FFmpeg/tree/spacexdebug1
first, you can now use syntax like
-mmb 40:03:-1
to fill the image with "blank" blocks starting from 40:03 and until the next specified command
and
-mmb X:20009:1B
to xor 0x1B at bit position 20009 of the frame, that is this allows to flip some bits without the need to edit the file
(atm this is limited to max 8bit per command)

Added another feature, now you can adjust DC values of the blank blocks
each MB has 6 blocks 4 luma and 2 chroma
example to adjust the 4th luma DC of (blank) MB 40:3 by +100
-mmb 40:03:-1:0:0:0:100:0:0

Can you explain more how to use this feature and what it does and why its useful? We're getting really far down in the trenches here.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: michaelni on 05/06/2014 04:03 AM
Added another feature, now you can adjust DC values of the blank blocks
each MB has 6 blocks 4 luma and 2 chroma
example to adjust the 4th luma DC of (blank) MB 40:3 by +100
-mmb 40:03:-1:0:0:0:100:0:0

Can you explain more how to use this feature and what it does and why its useful? We're getting really far down in the trenches here.

Some of the decoded pictures show rectangular areas extending to the right and down with wrong color/brightness, adjusting the DC value of the MB in the left top corner of such rectangle would allow to workaround the artifact
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/06/2014 04:08 AM
Using michaelni's changes I have made the best version to date I think of the upper portion of iframe8. Really works amazingly well. Now to start into the mess at the bottom.

-mmb 08:14:-1:15:-15:1:14:-2:1,09:14:56683

Edit: fixed to better version
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/06/2014 05:16 AM
Hi guys,

Great stuff all! Like the wiki and the new "empty"-block commands!

Btw I updated the install.txt so you can download the new version:

http://forum.nasaspaceflight.com/index.php?topic=34597.msg1193442#msg1193442

Regards and have fun, :)

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/06/2014 07:08 AM
I think I got a bit more of iframe 8, hard to tell what the correct alignment is.

ffmpeg -mmb 07:14:17233,14:14:57111,08:15:17233,09:15:63546,19:16:17233,21:16:73568,08:21:17233,00:22:111664 -debug mb_pos_size -err_detect ignore_err -s:0 704:480 -i iframe_8.mpg4-img -f image2 img-%03d.png

Working from yours combining with mine I have this so far.

-mmb 08:14:-1:15:-15:1:14:-2:1,09:14:56683,21:16:73568,08:21:17233,00:22:111664,01:28:139511,05:28:143051

I need more information on what's at the bottom so going to work on one of the other iframes now. I'm moving on to working on iframe7.

Edit: I think part of the problem is that we're seeing remnants of that darn dirty water that they splashed all over the rocket... So we have these dark and light splotches.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/06/2014 10:01 AM
Hack job on iframe7. I can do better, but probably am calling quits for tonight.

-mmb X:18494:01,19:05:-1,19:06:20616,15:12:-1,17:12:44357
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: gospacex on 05/06/2014 11:28 AM
In order to fix this you could try to restore the bitstream itself. But this is very hard to do and extremely time consuming (fixing compressed data is no joke!). The worse is: you don't know when the bad bits stop.

If data errors are random, if garbled bits are not occurring in bursts, but randomly, then fixing garbled block entails inverting one bit and looking whether image got better in the block and after it. If it got worse, you inverted a "good" bit. If it got better, you inverted a corrupted one.

Thankfully, there aren't than many bits in, say, 32 bytes - only 256.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/06/2014 02:29 PM
In order to fix this you could try to restore the bitstream itself. But this is very hard to do and extremely time consuming (fixing compressed data is no joke!). The worse is: you don't know when the bad bits stop.

If data errors are random, if garbled bits are not occurring in bursts, but randomly, then fixing garbled block entails inverting one bit and looking whether image got better in the block and after it. If it got worse, you inverted a "good" bit. If it got better, you inverted a corrupted one.

Thankfully, there aren't than many bits in, say, 32 bytes - only 256.

Nope. Errors tend to come in bursts I've seen, which makes sense. Also, inverting a bad bit can make the entire image worse because a following bad bit could have been partially correcting for the changes the previous change did.

Also, its better to talk about bits here because, to save space, almost nothing is byte aligned. Everything is packed together.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: gospacex on 05/06/2014 05:15 PM
In order to fix this you could try to restore the bitstream itself. But this is very hard to do and extremely time consuming (fixing compressed data is no joke!). The worse is: you don't know when the bad bits stop.

If data errors are random, if garbled bits are not occurring in bursts, but randomly, then fixing garbled block entails inverting one bit and looking whether image got better in the block and after it. If it got worse, you inverted a "good" bit. If it got better, you inverted a corrupted one.

Thankfully, there aren't than many bits in, say, 32 bytes - only 256.

Nope. Errors tend to come in bursts I've seen, which makes sense.

It makes sense if no bit interleaving encoding was employed on the data link.

I hoped it *has been* used, making error bit flips spread instead of bunching up. But maybe SpaceX didn't go for a more expensive hardware with those capabilities...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: lgjy98d on 05/06/2014 07:45 PM
I've started experimenting with I-Frame 12 and doing some automated analysis.

better.png was produced with -mmb 0:0:-1,1:0:583

Legend for -annotated images:
* Red circles annotate the MBs that disappeared from base.png after the modification
* Green circles annotate the MBs that appeared in the better.png after -mmb modification
* Crossed MBs are ones that considered corrupted by ffmpeg

I'd be glad to get feedack if anyone sees any additional information from the annotated images.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jakusb on 05/06/2014 07:56 PM
I've started experimenting with I-Frame 12 and doing some automated analysis.

better.png was produced with -mmb 0:0:-1,1:0:583

Legend for -annotated images:
* Red circles annotate the MBs that disappeared from base.png after the modification
* Green circles annotate the MBs that appeared in the better.png after -mmb modification
* Crossed MBs are ones that considered corrupted by ffmpeg

I'd be glad to get feedack if anyone sees any additional information from the annotated images.

Just curious, but why is the time stamp so clear? And does it help to know it should be much further to lower left corner?
It is the one big constant in a frames and pretty predictable what it should be.

Ps: btw, amazing what work is being put into this. Kudos! :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/06/2014 08:32 PM
Just curious, but why is the time stamp so clear? And does it help to know it should be much further to lower left corner?

Luck. Yes.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/06/2014 08:37 PM
Hi Guys,

My 2 CC. A little improvement on Frame 11:

mmb=0:0:10008

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/06/2014 08:37 PM
Hack job on iframe7. I can do better, but probably am calling quits for tonight.

-mmb X:18494:01,19:05:-1,19:06:20616,15:12:-1,17:12:44357

I could improve a bit the image with the command:

-mmb X:18494:01,20:05:-1,02:06:18794,15:12:-1,19:12:45121,19:15:-1,20:15:62858,21:15:-1,29:15:64161

I could compile ffmpeg with windows using MinGW (had to add some packages: yasm, gettext, glib, pkg-config), the only thing that does not work is to write the png, so I had to write a jpg.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: John_L on 05/06/2014 08:45 PM
Not sure if anyone's  interested or not but I wrote an MS-DOS batch file to automate the process a bit if anyone's interested in running multiple MMB values:

set a=1
for /f "tokens=1 delims=" %%i in (file.txt) do call :part2 %%i
goto :fin
:part2
ffmpeg -mmb %%i -debug mb_pos_size -err_detect ignore_err -s:0 704:480 -i iframe_1.mpg4-img -f image2 img-%a%.png
set /a a+=1
goto :eof
:fin

copy the above text into a text file and save as "name.bat" where name is the name you want to give it, mine's simply named "doit.bat"

then create a text file with just the mmb values... i.e.:

0:0:-1,0:24:19233
0:0:-1,0:24:19234
0:0:-1,0:24:19235

... or whatever values you want after the -mmb in the ffmpeg command line.  Save this out as "file.txt".

Then, when you type "doit" from the command line the batch file will read in the first value from file.txt and plug it into the ffmpeg command and execute it, the resultant output files will be named "img-x.png" where x is a number.  Don't forget to edit which iframe you're working on as well.

I also installed a program called ImageMagick-6.8.9-Q16.  It has a display program that can be called from the command line.  If you install it, you can copy imdisplay.exe to the directory that contains ffmpeg and the batch files (and iframe of course), and you can edit the batch file to include "imdisplay img-%a%.png" after the ffmpeg command line in the call and it will display them as well.  Like so:

set a=1
for /f "tokens=1 delims=" %%i in (file.txt) do call :part2 %%i
goto :fin
:part2
ffmpeg -mmb %%i -debug mb_pos_size -err_detect ignore_err -s:0 704:480 -i iframe_1.mpg4-img -f image2 img-%a%.png
imdisplay img-%a%.png
set /a a+=1
goto :eof
:fin


Not sure if anyone wants this or not, but thought I'd throw it out there if anyone wishes to use it.

John.

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/06/2014 09:10 PM
I was also able to recover the timestamp on frame 6  :)

mmb=0:0:61341
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: grythumn on 05/06/2014 09:25 PM
There was a request for updated cygwin (windows) binary. I split the DLLs out from the executable; if you grabbed the last package, you don't need the DLLs again.

-Bob
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/06/2014 09:32 PM
Awesome, thanks for the update :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/06/2014 09:50 PM
We created a way of doing some of the work through a mini web app. You can find it here: http://spacexlanding.wikispaces.com/Online+Editor
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/06/2014 09:53 PM
We created a way of doing some of the work through a mini web app. You can find it here: http://spacexlanding.wikispaces.com/Online+Editor

Which is really cool and awesome. People don't have to compile anything now.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/06/2014 09:57 PM
We created a way of doing some of the work through a mini web app. You can find it here: http://spacexlanding.wikispaces.com/Online+Editor

Which is really cool and awesome. People don't have to compile anything now.

Working like a charm! I got this with iframe 12 and 0:0:1001:


 
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/06/2014 10:11 PM
We created a way of doing some of the work through a mini web app. You can find it here: http://spacexlanding.wikispaces.com/Online+Editor

Which is really cool and awesome. People don't have to compile anything now.

Working like a charm! I got this with iframe 12 and 0:0:1001:
Great work!

Ok. I will say this now. This is a picture of an intact falcon 9 first stage slightly embedded into water. You can't see the full legs because the tips are under water. You can see small parts of them: only the top. They look "thinner" and shorter that way.

That' s my take on it now. I really do think we have given SpaceX visual proof they did it. :) I am sure they are enjoying this :) :)

And it would only make sense that they would give out video until right after spashdown, where they know (from telemetry) that everything was fine.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/06/2014 10:12 PM
iframe_11 with -mmb 0:0:-1,0:13:32118,0:25:-1,1:25:82578,7:29:-1

(edited)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/07/2014 12:00 AM
Hi people.

I got a bit of sea from frame 6 and moved the legs and the clock from previous post.

edit: forgot the main bit -mmb 0:0:583,00:01:6129,17:14:64694,2:25:202509
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/07/2014 12:19 AM
I tried to recenter iframe 12 from previous solution, difficult...

-mmb 0:0:31664,40:03:-1,41:03:43222,39:6:-1,40:06:53586,39:8:-1,0:9:59654,36:9:-1,0:10:62314,
36:10:-1,40:10:64640,40:12:-1,0:13:69892,0:15:-1,1:15:74005,39:16:-1,0:17:77748,39:17:-1,0:18:79454

(edit: see http://forum.nasaspaceflight.com/index.php?topic=34597.msg1194745#msg1194745 for a better version)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: grythumn on 05/07/2014 12:30 AM
We created a way of doing some of the work through a mini web app. You can find it here: http://spacexlanding.wikispaces.com/Online+Editor

Very cool. You may want to add the debug log output to the page somehow... you're kind of shooting blind without it. Apologies if it is there and I missed it...

Hope the hosting doesn't cost you an arm and a leg...

-Bob
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: grythumn on 05/07/2014 01:02 AM
Improvement on Frame 9. You can sort of see part of the rocket body now, but if I try to shift it left it goes all green blocks on me again.

The null MMBs help a lot.

20:04:-1:0:0:-5:0:0:0,26:04:16025,21:07:-1,22:07:27346,05:15:-1,09:15:73077,
12:15:-1,20:15:73574,24:15:-1,32:15:74745,1:16:-1,4:16:77775,
8:16:-1,9:16:78463,26:16:-1,27:16:81473,28:16:-1,29:16:82604,31:16:82767,
36:16:-1,37:16:83144

-Bob
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mmeijeri on 05/07/2014 01:28 AM
It is a bit early for this, but I notice there are many slightly discoloured blocks in the otherwise seemingly correctly restored areas. Could these be single bit errors in the lowest order DCT coefficients? If so, we may eventually be able to correct them by hand. Maybe we can even semi-automate the process.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/07/2014 04:54 AM
Improvement on Frame 9. You can sort of see part of the rocket body now, but if I try to shift it left it goes all green blocks on me again.

The null MMBs help a lot.

20:04:-1:0:0:-5:0:0:0,26:04:16025,21:07:-1,22:07:27346,05:15:-1,09:15:73077,
12:15:-1,20:15:73574,24:15:-1,32:15:74745,1:16:-1,4:16:77775,
8:16:-1,9:16:78463,26:16:-1,27:16:81473,28:16:-1,29:16:82604,31:16:82767,
36:16:-1,37:16:83144

-Bob

ffmpeg -mmb 20:04:-1:0:0:-5:0:0:0,26:04:16025,21:07:-1,22:07:27346,01:12:-1,02:12:50994,
03:15:-1,04:15:72834,05:15:-1,09:15:73077,10:15:-1,11:15:73260,12:15:-1,21:15:73843,
24:15:-1,4:16:77775,8:16:-1,16:16:79528,26:16:-1,27:16:81473,28:16:-1,29:16:82604,
31:16:82767,36:16:-1,37:16:83144 -debug mb_pos_size -err_detect ignore_err -s:0 704:480 -i iframe_9.mpg4-img -f image2 img-%03d.png

Matt
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: John_L on 05/07/2014 05:29 AM
I think I've made some headway with iframe 2.

-mmb 40:3:1424,12:19:18210,20:6:2006,25:8:18680,2:20:35080


Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/07/2014 05:47 AM
I think I got a bit more of iframe 8, hard to tell what the correct alignment is.

ffmpeg -mmb 07:14:17233,14:14:57111,08:15:17233,09:15:63546,19:16:17233,21:16:73568,08:21:17233,00:22:111664 -debug mb_pos_size -err_detect ignore_err -s:0 704:480 -i iframe_8.mpg4-img -f image2 img-%03d.png

Working from yours combining with mine I have this so far.

-mmb 08:14:-1:15:-15:1:14:-2:1,09:14:56683,21:16:73568,08:21:17233,00:22:111664,01:28:139511,05:28:143051

I need more information on what's at the bottom so going to work on one of the other iframes now. I'm moving on to working on iframe7.

Edit: I think part of the problem is that we're seeing remnants of that darn dirty water that they splashed all over the rocket... So we have these dark and light splotches.

More of frame 8:

08:14:-1:15:-15:1:14:-2:1,09:14:56683,21:16:73568,08:21:17233,00:22:111664,
42:24:-1,05:25:127302,06:25:-1,07:25:127642,08:25:-1,09:25:127920,10:25:-1,
18:25:128500,20:25:-1,22:25:128837,25:25:-1,29:25:129306,10:27:-1,19:27:136208,
21:27:-1,24:27:136771,01:28:139511,05:28:143051

Frame 12 calls me.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/07/2014 06:04 AM
Frame 12 calls me.

Yes. I think iframe12 contains the "money shot". :)

If you look at the version from SwissCheese you only see the pistons that extend the legs. You can't see the legs themselves anymore. They have become very "thin". This is perfectly consistent with the great simulations done by meadows.st ( here (http://forum.nasaspaceflight.com/index.php?topic=34065.msg1165692#msg1165692) ).

I've attached both the improved iframe 12 and the simulation of a submerged first stage. Look closely at the the legs in our frame and then look at the drawing from meadows.st. It fits.

This is a first stage in the water! I'm telling you :) :)

You guys all rock!

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: aero on 05/07/2014 06:16 AM
Quote
[edit] Is that a piece of the leg broken off right-above the right leg? Is that the piece of the leg they found? That would be cool  8)

My guess is that it is a splotch on the lens from the dirty water bath at launch. Maybe it is from some other source but it shows up in the same spot in a lot of the frames so I think it is on the lens.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/07/2014 06:27 AM
Quote
[edit] Is that a piece of the leg broken off right-above the right leg? Is that the piece of the leg they found? That would be cool  8)

My guess is that it is a splotch on the lens from the dirty water bath at launch. Maybe it is from some other source but it shows up in the same spot in a lot of the frames so I think it is on the lens.
You're right.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/07/2014 06:34 AM
Good morning chaps. This is such an impressive effort - and a big welcome to the new people who have helped push this forward.

We've got SpaceX people watching this thread, but I think we're close to asking PAO to make Elon and the top brass aware of the progress. Will send out some e-mails today.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/07/2014 06:44 AM
Good morning chaps. This is such an impressive effort - and a big welcome to the new people who have helped push this forward.

We've got SpaceX people watching this thread, but I think we're close to asking PAO to make Elon and the top brass aware of the progress. Will send out some e-mails today.

It would be great if you could get someone, anyone, to give us any screenshot from this camera of the stage from when they had signal from it, like just after separation. It would help greatly to know the dark/light patterns on the lower part of the image.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/07/2014 06:49 AM
One more note, to people working on the images. It's very important that the _positions_ of the blocks be in the correct locations. I notice some people just chopping off the top part of the image. That really needs to be avoided. Compare to iframe8 for example and get the legs into the correct positions at the least. If they're not in the right positions they're useless as iframes.

Great work though everyone!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/07/2014 07:00 AM
Good morning chaps. This is such an impressive effort - and a big welcome to the new people who have helped push this forward.

We've got SpaceX people watching this thread, but I think we're close to asking PAO to make Elon and the top brass aware of the progress. Will send out some e-mails today.

It would be great if you could get someone, anyone, to give us any screenshot from this camera of the stage from when they had signal from it, like just after separation. It would help greatly to know the dark/light patterns on the lower part of the image.

That's what I see in iframe 12. The water is buggy but the bottom half of the frame is really clear.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/07/2014 07:48 AM
We created a way of doing some of the work through a mini web app. You can find it here: http://spacexlanding.wikispaces.com/Online+Editor
Hi,

What would be really useful is when you click somewhere on the generated pixel it would put the x and y of the corresponding block in your clipboard (or show it somewhere on the page so yuu can copy-paste it from there.

Even more useful would be if you could parse the log on the serverside and when you click on the image you get x,y and bitposition in your clipboard! Or maybe simply the line from the log corresponding with the x and y you just clicked.

Just an idea.

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/07/2014 07:51 AM
All right, I have to give up for the night. This is my best run for iframe 12:

X:598:1,0:0:550,1:0:-1,14:09:29489,0:10:31664,0:11:34522,0:12:37439,
0:13:40079,0:14:43312:0:15:46476,0:16:49752,0:17:53744,0:18:57020,
0:19:59654,5:19:-1,06:19:60089,0:20:62314,0:21:64864,0:22:67531,
0:23:69892,0:24:72120,0:25:73937,0:26:75964,0:27:77748,0:28:79454,
0:29:86580
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/07/2014 07:53 AM
Good morning chaps. This is such an impressive effort - and a big welcome to the new people who have helped push this forward.

We've got SpaceX people watching this thread, but I think we're close to asking PAO to make Elon and the top brass aware of the progress. Will send out some e-mails today.

Send them this link too:

http://spacexlanding.wikispaces.com/ (http://spacexlanding.wikispaces.com/)

Good overview of the findings in this thread.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/07/2014 09:27 AM
iframe 10 with:

-mmb 2:5:-1,0:7:18695,23:13:37251,24:13:-1,7:17:47677
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/07/2014 09:53 AM
We created a way of doing some of the work through a mini web app. You can find it here: http://spacexlanding.wikispaces.com/Online+Editor
Hi,

What would be really useful is when you click somewhere on the generated pixel it would put the x and y of the corresponding block in your clipboard (or show it somewhere on the page so yuu can copy-paste it from there.

Even more useful would be if you could parse the log on the serverside and when you click on the image you get x,y and bitposition in your clipboard! Or maybe simply the line from the log corresponding with the x and y you just clicked.

Just an idea.

Regards,

arnezami

I updated the image handler to do this (http://dz0bwiwndcjbh.cloudfront.net/info/<frame>?mmb=<mmb>) but I haven't got time to put it into the web app atm, I'll do that later unless someone else wants to
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/07/2014 09:54 AM
Ive added 2 features to https://github.com/michaelni/FFmpeg/tree/spacexdebug1
first, you can now use syntax like
-mmb 40:03:-1
to fill the image with "blank" blocks starting from 40:03 and until the next specified command
and
-mmb X:20009:1B
to xor 0x1B at bit position 20009 of the frame, that is this allows to flip some bits without the need to edit the file
(atm this is limited to max 8bit per command)
Hi Michael,

I've been looking at a lot of frames that were "fixed" with the blank blocks. I think they are useful. But what would be better is a block that cancels/negates all information coming from the left and upper block.

In other words a block that if you would place one of its kind above (and one its kind to the left) of a block with content it would be as if that block with content would be at 0,0: no disturbance at all. Of course this would also remove all good information from the left and top but I think that might in many cases to better than the blank-block. Maybe choose to cancel only the top information or only the left information.

Do you think this can be done?

So like this:

-1 : let info through from left and top
-2 : let info through from left, cancel from top
-3 : cancel info from left, let it through from top
-4 : cancel info from left and top

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/07/2014 10:16 AM
And a nice iframe 9  :)

-mmb 20:04:-1:0:0:-5:0:0:0,24:04:16025,1:15:-1,3:16:79137,42:19:-1,8:20:96260
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/07/2014 10:52 AM
Ive added 2 features to https://github.com/michaelni/FFmpeg/tree/spacexdebug1
first, you can now use syntax like
-mmb 40:03:-1
to fill the image with "blank" blocks starting from 40:03 and until the next specified command
and
-mmb X:20009:1B
to xor 0x1B at bit position 20009 of the frame, that is this allows to flip some bits without the need to edit the file
(atm this is limited to max 8bit per command)
Hi Michael,

I've been looking at a lot of frames that were "fixed" with the blank blocks. I think they are useful. But what would be better is a block that cancels/negates all information coming from the left and upper block.

In other words a block that if you would place one of its kind above (and one its kind to the left) of a block with content it would be as if that block with content would be at 0,0: no disturbance at all. Of course this would also remove all good information from the left and top but I think that might in many cases to better than the blank-block. Maybe choose to cancel only the top information or only the left information.

Do you think this can be done?

So like this:

-1 : let info through from left and top
-2 : let info through from left, cancel from top
-3 : cancel info from left, let it through from top
-4 : cancel info from left and top

Regards,

arnezami

Don't think that's simple. You'd have to store the id of the block below and to the right of the block you are placing. Also every block AFAIK _HAS_ to reference either the top or the left, so you could only change the block to reference the other direction which would probably mess things up worse.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/07/2014 11:12 AM
iframe 2 at correct position:

-mmb 40:3:-1,16:10:18210,4:28:-1,40:28:76955
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/07/2014 11:17 AM
I created a new set of iframes from michaelni's try1.ts file he linked on IRC. mpg4-img and pngs are in the zip. These supposedly contain more information because they are larger, whether its recoverable information or not is another question. mmb's from other iframes may require adapting to work

http://ffmpeg.org/~michael/space-x-video/try1.ts
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/07/2014 11:20 AM
Don't think that's simple. You'd have to store the id of the block below and to the right of the block you are placing. Also every block AFAIK _HAS_ to reference either the top or the left, so you could only change the block to reference the other direction which would probably mess things up worse.
No what I mean is that you change its "transformation" data so that the result is that it is the "opposite" of what it receives from its left and top. That way no matter what block is placed right or bottom the will have data from left and top that is effectively the same as if that block is placed at 0,0.

I'm not sure changing the "transformation" data this way is possible (I guess its a mathematical problem). But I did'nt mean to solve this by making pointers to blocks that are right or bottom.

Another way of doing this is to say to a block: don't "listen" to your left block, instead "listen" to the left "block" of 0,0 (initial state I guess). I believe these things are called "predictions" in mepg-4.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/07/2014 11:56 AM
We created a way of doing some of the work through a mini web app. You can find it here: http://spacexlanding.wikispaces.com/Online+Editor
Hi,

What would be really useful is when you click somewhere on the generated pixel it would put the x and y of the corresponding block in your clipboard (or show it somewhere on the page so yuu can copy-paste it from there.

Even more useful would be if you could parse the log on the serverside and when you click on the image you get x,y and bitposition in your clipboard! Or maybe simply the line from the log corresponding with the x and y you just clicked.

Just an idea.

Regards,

arnezami

I updated the image handler to do this (http://dz0bwiwndcjbh.cloudfront.net/info/<frame>?mmb=<mmb>) but I haven't got time to put it into the web app atm, I'll do that later unless someone else wants to

I've now updated the app to allow the selection of MB, I "think" it's right but let me know if it isn't
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/07/2014 12:56 PM
And a big high five and a welcome to you Iain!

Note from the SpaceX side (paraphrased) says that a request has been sent to provide a 10 second shot from the camera while the vehicle was still on the pad. This should go up on the page where the videos were posted on the SpaceX site, hopefully soon.

I'm told the pad shot is the most error free telem they have, but it's recognized the need for a shot later in flight that has the muddy streaks for reference. That is being inquired about, per the note I've been sent.

The main issue is everyone is busy with the upcoming flights, as can be expected.

We have a lot of SpaceX folk watching this thread and we'll get some additional eyeballs via an official PAO approach today. It's noted that people are impressed with the work going on here, so well done everyone!

PS Per the PAO approach, I think this process deserves a news article. It's fascinating enough and of interest to the readership. It may also shake the tree for a few more video experts, but we're clearly doing well on that score already. Problem with that is I pretty much know next to nothing about what you guys are doing from a tech standpoint, so if one of you guys is confident at being able to summarize, send me a message, as I can write around it and subedit. No rush on that.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Okie_Steve on 05/07/2014 01:49 PM
Chris,

If SpaceX is willing to provide a bit of information about the stream format it would be very helpful to know the location of any ECC/CRC errors and the packet size and any type of cross-interleave etc. used in the telemetry stream to help identify "possibly corrupt"and "probably correct" portions of the data.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/07/2014 02:16 PM
Chris,

If SpaceX is willing to provide a bit of information about the stream format it would be very helpful to know the location of any ECC/CRC errors and the packet size and any type of cross-interleave etc. used in the telemetry stream to help identify "possibly corrupt"and "probably correct" portions of the data.

I'll ask....as much as you have already asked by posting it (such people watching the thread).
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Untribium on 05/07/2014 03:07 PM
Amazing work! A few days ago I was worried this wouldn't go anywhere, especially considering that we're most likely (hopefully) going to get new footage in a few days :)
I put a little bit of work into iframe 7. Basically, I started with the line by mlindner and modified it slightly to fix the color issue around the legs.
X:18494:01, 19:05:-1, 20:06:20716,15:12:-1, 22:12:45309, 22:15:-1, 23:15:63096, 26:15:-1, 30:15:64161
(http://dz0bwiwndcjbh.cloudfront.net/7?v=1&mmb=X:18494:01,%2019:05:-1,%2020:06:20716,15:12:-1,%2022:12:45309,%2022:15:-1,%2023:15:63096,%2026:15:-1,%2030:15:64161)
Instead of taking the first "nice" block after some garbage, I take one further to the right and if necessary down, so that all the blocks that influence this block are non-garbage. Unfortunately, in this case the first nice block contains part of the left leg, so there's some detail lost...any ideas on how to fix the colors while keeping the good blocks?
Oh, could someone quickly explain what the first part of the line does (X:18494:01)? I read through all the posts, but I didn't find an answer (may have missed it...)

Edit: Found an improvement right after posting...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/07/2014 03:32 PM
All right, I have to give up for the night. This is my best run for iframe 12:

X:598:1,0:0:550,1:0:-1,14:09:29489,0:10:31664,0:11:34522,0:12:37439,
0:13:40079,0:14:43312:0:15:46476,0:16:49752,0:17:53744,0:18:57020,
0:19:59654,5:19:-1,06:19:60089,0:20:62314,0:21:64864,0:22:67531,
0:23:69892,0:24:72120,0:25:73937,0:26:75964,0:27:77748,0:28:79454,
0:29:86580

This one is a bit better for iframe 12, still not great:

X:552:1,X:553:1,0:0:550,1:0:-1,14:09:29489,0:10:31664,0:11:34522,
0:12:37439,0:13:40079,0:14:43312:0:15:46476,0:16:49752,0:17:53744,
0:18:57020,0:19:59654,5:19:-1,06:19:60089,0:20:62314,0:21:64864,
0:22:67531,0:23:69892,0:24:72120,0:25:73937,0:26:75964,0:27:77748,
0:28:79454,0:29:86580
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Smokablecheese on 05/07/2014 03:37 PM
Just a small improvement on IFrame 9, I made a modification to Swisscheeses last edit


mmb 20:04:-1:0:0:-5:0:0:0,24:04:16023,1:15:-1,3:16:79127,41:19:-1,8:20:96220,15:04:16023
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/07/2014 03:45 PM
Amazing work everyone.
Remember we have a wiki space for the project (http://spacexlanding.wikispaces.com/) for you to add your versions.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/07/2014 03:52 PM
How are the MB's ordered in the file?

Is it
a) left->right, top->bottom
b) zigzag like (see fig 7-4-c in the ISO file (attached))
c) something else?

I ask because AFAIUI we need the startbit of every MB in an Iframe so that if one MB is somehow broken the decoder still knows where the next begins.
It would be nice to know in which order one should work on the MB's.

Cheers,
Shanuson
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/07/2014 04:07 PM
How are the MB's ordered in the file?

Is it
a) left->right, top->bottom
b) zigzag like (see fig 7-4-c in the ISO file (attached))
c) something else?

I ask because AFAIUI we need the startbit of every MB in an Iframe so that if one MB is somehow broken the decoder still knows where the next begins.
It would be nice to know in which order one should work on the MB's.

Cheers,
Shanuson

Left to right, top to bottom. Not like the attached diagram. However, blocks can affect others to the right and below.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/07/2014 04:20 PM
How are the MB's ordered in the file?

Is it
a) left->right, top->bottom
b) zigzag like (see fig 7-4-c in the ISO file (attached))
c) something else?

I ask because AFAIUI we need the startbit of every MB in an Iframe so that if one MB is somehow broken the decoder still knows where the next begins.
It would be nice to know in which order one should work on the MB's.

Cheers,
Shanuson

First block is top left corner, and the blocks continue going left to right until they hit end of row, then go to next row and repeat.

Every block has 6 sub-blocks, 4 luma blocks (brightness) and 2 chroma blocks (color). The 4 luma blocks are the 4 squares in a block you often see. Both color blocks cover the entire block and are two different color planes. https://en.wikipedia.org/wiki/YUV There are 4 Y blocks and 1 U and 1 V block. Each one of those sub blocks references either the block above it or the block to the left of it (they all don't have to reference the same block). Which block they reference is determined on-the-fly (not hardcoded) so if a block is screwed up then it could cause blocks that would reference it, to not reference it or vice versa.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: R7 on 05/07/2014 04:34 PM
The zigzag image is how DCT (discrete cosine transformation (http://en.wikipedia.org/wiki/Discrete_cosine_transform)) coefficients of a 8x8 block are ordered in the data stream. A macroblock has 6 of these blocks, four for 16x16 luminance information and two for 8x8 color information.

The compression trickery comes from the fact that often most of those coefficients are zero so they are written using variable length encoding, like almost anything in the protocol. The downside royal PITA is that single bit error can make the decoder go haywire, not only read wrong values but read too many or too few bits.

It's a wonder the pictures are as good as they are.

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: John_L on 05/07/2014 05:05 PM
How would one go about re-encoding the video with the replacement iframes, what's the command line for ffmpeg?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: aero on 05/07/2014 05:35 PM
How soon will we see a .gif of the improved frames?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Swatch on 05/07/2014 05:37 PM
This has been some amazing work to watch develop fellas, keep up the good work!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/07/2014 05:43 PM
How would one go about re-encoding the video with the replacement iframes, what's the command line for ffmpeg?

This is a good point, and it got me thinking. If everything goes as planned, SpaceX will launch again in a few days, and I think they're planning to attempt recovery again. This could upstage the (arguably more historic) first landing video. We need to put together a video with what we've got today, tomorrow at the lastest. If necessary, I can hand-assemble something from the iframes and pieces of the pframes. It wouldn't be as pretty as what arnezami and mlindner can do, so I'll wait until I hear from them.

Matt
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/07/2014 05:48 PM
How would one go about re-encoding the video with the replacement iframes, what's the command line for ffmpeg?

This is a good point, and it got me thinking. If everything goes as planned, SpaceX will launch again in a few days, and I think they're planning to attempt recovery again. This could upstage the (arguably more historic) first landing video. We need to put together a video with what we've got today, tomorrow at the lastest. If necessary, I can hand-assemble something from the iframes and pieces of the pframes. It wouldn't be as pretty as what arnezami and mlindner can do, so I'll wait until I hear from them.

Matt

It's actually a lot easier than that. If no one else does it by Friday Eastern time then I'll be modifying ffmpeg to add a frame specifier to the mmb options. Then we simply pile all the mmb options into one long entry and run ffmpeg with it. viola, produced video. This would also allow us to try tweaking some p-frames if we so wish.

Producing a video now with what we have won't be very attractive yet. We need to fine tune a lot of the frames still. I'm doing that now for iframe8, which should hopefully look really really good. This means doing things like whenever you use -1 and replace the block, fine tune all the luma and chroma settings for it to have the least effect on surrounding blocks.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: John_L on 05/07/2014 06:03 PM
It's actually a lot easier than that. If no one else does it by Friday Eastern time then I'll be modifying ffmpeg to add a frame specifier to the mmb options. Then we simply pile all the mmb options into one long entry and run ffmpeg with it. viola, produced video. This would also allow us to try tweaking some p-frames if we so wish.

Producing a video now with what we have won't be very attractive yet. We need to fine tune a lot of the frames still. I'm doing that now for iframe8, which should hopefully look really really good. This means doing things like whenever you use -1 and replace the block, fine tune all the luma and chroma settings for it to have the least effect on surrounding blocks.

So you're saying at this point, ffmpeg doesn't have the ability to re-encode with replacement iframes?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: AncientU on 05/07/2014 06:07 PM
This has been some amazing work to watch develop fellas, keep up the good work!
Yes, truly incredible effort!
So cool to see the first ever soft landing emerge frame by frame.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/07/2014 06:14 PM
It's actually a lot easier than that. If no one else does it by Friday Eastern time then I'll be modifying ffmpeg to add a frame specifier to the mmb options. Then we simply pile all the mmb options into one long entry and run ffmpeg with it. viola, produced video. This would also allow us to try tweaking some p-frames if we so wish.

Producing a video now with what we have won't be very attractive yet. We need to fine tune a lot of the frames still. I'm doing that now for iframe8, which should hopefully look really really good. This means doing things like whenever you use -1 and replace the block, fine tune all the luma and chroma settings for it to have the least effect on surrounding blocks.

So you're saying at this point, ffmpeg doesn't have the ability to re-encode with replacement iframes?

You might be able to manually have it use those iframes and re-encode it that way. I'm not sure how to use ffmpeg to do so. We haven't been producing replacement iframes anyway, only modifications to iframes that need to be applied every time.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Sohl on 05/07/2014 06:40 PM
Excellent work people!  I agree the historical significance of this video makes restoration efforts worthwhile even if the Orbcom flight provides a much cleaner landing video.  I'd probably enjoy this bit-hacking myself if I wasn't up to my neck in my day-job programming work!   :-[
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: John_L on 05/07/2014 06:44 PM
You might be able to manually have it use those iframes and re-encode it that way. I'm not sure how to use ffmpeg to do so. We haven't been producing replacement iframes anyway, only modifications to iframes that need to be applied every time.

Right, well, I'm nowhere near an expert at mpeg encoding, but it was my assumption that once the iframes were done, we'd re-encode the other frames using the iFrames as a reference, or would reinserting the fixed iframes automatically  fix the other frames (because they're now referencing the fixed iframes), which would mean we would just need a way to reinsert the iframes into the video.  Luckily, we have michaelni onboard helping out... nice to have the curator of ffmpeg onboard... ;)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: grythumn on 05/07/2014 06:45 PM
Frame 9, cleaned up some single-bit errors up top.

20:04:-1,24:04:16023,X:16410:01,16:5:-1,17:05:18692,X:19903:1,X:27244:1,
1:15:-1,3:16:79127,41:19:-1,8:20:96220,15:04:16023

-Bob
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/07/2014 06:54 PM
Frame 9, cleaned up some single-bit errors up top.

20:04:-1,24:04:16023,X:16410:01,16:5:-1,17:05:18692,X:19903:1,X:27244:1,
1:15:-1,3:16:79127,41:19:-1,8:20:96220,15:04:16023

-Bob

Nice work! The mixed up part of the right leg is likely also a bit error.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: grythumn on 05/07/2014 07:00 PM
Technique: Find suspect macroblock. Isolate bit range. Plug them into the for loop. Put the string ",X,$i:1," in the middle of your MMB string, where it goes from a bit range point point of view. Run the loop, wait a bit, flip through the resulting pictures. Can usually narrow it down to 4 or 5. If nothing works, try changing the macroblock to the left, or nulling out the macro block... if it's a multibit error, it's a lot more permutations to try to figure out.

For example, my last run:
for i in {32260..32407} ; do ./ffmpeg -debug mb_pos_size -err_detect ignore_err -s:0 704:480 -mmb 20:04:-1,24:04:16023,X:16410:01,16:5:-1,17:05:18692,X:19903:1, X:27244:1,X:$i:1,1:15:-1,3:16:79127,41:19:-1, 8:20:96220,15:04:16023 -i iframe_9.mpg4-img -f image2 img-%03d-$i.png 2> /dev/null ; done

-Bob
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/07/2014 07:04 PM
Technique: Find suspect macroblock. Isolate bit range. Plug them into the for loop. Put the string ",X,$i:1," in the middle of your MMB string, where it goes from a bit range point point of view. Run the loop, wait a bit, flip through the resulting pictures. Can usually narrow it down to 4 or 5. If nothing works, try changing the macroblock to the left, or nulling out the macro block... if it's a multibit error, it's a lot more permutations to try to figure out.

For example, my last run:
for i in {32260..32407} ; do ./ffmpeg -debug mb_pos_size -err_detect ignore_err -s:0 704:480 -mmb 20:04:-1,24:04:16023,X:16410:01,16:5:-1,17:05:18692,X:19903:1, X:27244:1,X:$i:1,1:15:-1,3:16:79127,41:19:-1, 8:20:96220,15:04:16023 -i iframe_9.mpg4-img -f image2 img-%03d-$i.png 2> /dev/null ; done

-Bob

If you start with an error-free image, you can also eliminate any bit-flips that introduce errors by grepping through the output. I have a script that does just that.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: dorkmo on 05/07/2014 07:09 PM
i made a super basic spreadsheet that just concatenates the x:y:bitpos. i was just guna use it for me but heres the link. https://docs.google.com/spreadsheets/d/1sP5lyTbC3ICyca03ZJlWwWaeIn_H6MSeqcp2ACT1HOQ/edit?usp=sharing
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/07/2014 07:18 PM
The University of Wisconsin has a very large HTCondor pool which would probably be very useful to run through this processing. I'll reach out and see if there's anyone over there who can leverage their tens of thousands of available CPU cores to crank out macroblock iterations.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/07/2014 07:54 PM
Technique: Find suspect macroblock. Isolate bit range. Plug them into the for loop. Put the string ",X,$i:1," in the middle of your MMB string, where it goes from a bit range point point of view. Run the loop, wait a bit, flip through the resulting pictures. Can usually narrow it down to 4 or 5. If nothing works, try changing the macroblock to the left, or nulling out the macro block... if it's a multibit error, it's a lot more permutations to try to figure out.

For example, my last run:
for i in {32260..32407} ; do ./ffmpeg -debug mb_pos_size -err_detect ignore_err -s:0 704:480 -mmb 20:04:-1,24:04:16023,X:16410:01,16:5:-1,17:05:18692,X:19903:1, X:27244:1,X:$i:1,1:15:-1,3:16:79127,41:19:-1, 8:20:96220,15:04:16023 -i iframe_9.mpg4-img -f image2 img-%03d-$i.png 2> /dev/null ; done

-Bob

Oh that's an awesome idea. Hadn't thought of doing it for bit flipping. I was already doing that for moving start locations of blocks around.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: CraigLieb on 05/07/2014 08:00 PM
Not really able to follow the technical process being executed, but I have been putting the most recent IFrame images posted into a powerpoint (using it like a flip book). One thing that jumps out is some artifacts that are at the top center and offset to the left of each of the frames. Maybe they can be used like a key to lock in some image locations like the time stamp at the bottom on currently hard interpret frames.

I have placed a few of the frames top strips next to each other here so you can see what I mean.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/07/2014 08:01 PM
Not really able to follow the technical process being executed, but I have been putting the most recent IFrame images posted into a powerpoint (using it like a flip book). One thing that jumps out is some artifacts that are at the top center and offset to the left of each of the frames. They may be like a key that can be used to lock in some image locations like the time stamp at the bottom. I have captured a few of the frames just top strips next to each other.

Yay for dust and dirt on the lens.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/07/2014 08:03 PM
Hi guys,

I would like to explain what we are actually doing. Because some of you might not completely grasp it. And I think I can best explain it by an illustration. Otherwise it gets too technical too quickly. I will do it in terms of rockets. ;)

--

Imagine a book. A thick book. It's a book containing very detailed technical instructions on how to build a new type of rocket.  It contains 13 chapters. One for each rocket part. Each chapter consists of an introductory paragraph explaining what the part is, how it should look like and how it works. None of the following paragraphs can be understood if this paragraph has not been read. The following paragraps are highly dependent on eachother: if you haven't read them in sequence they will be meaningless since they always refer heavely to the previous one to understand.

All sentences are written in a foreign language. And all punctuation is removed. Also all captitals are made lower case. The letters in the words are coded: each letter can only be decoded if you know the previous letter. For example this code: when reading a letter add the letternumber (a=1, b=2 etc) of the previous letter to this one to get the real letter. All whitespaces are removed. So each paragraph is just a string of seemingly random letters.

Ok. Now you're thinking: "This is insane. Who would read such a book?"

But this book was never intended to be read by a human being! It was supposed to be read by a machine that would also automatically build the rocket parts. Here is the real problem though: the printing machine that printed the book had a major error: it sometimes printed the wrong letter! So the book has thousands of random errors in it! Unsuprisingly when the book is now given to the rocket-part-building-machine it says: "Error". And that's basicly it.

Ok it turns out that the rocket-part-building-machine made a few incomplete and defective drawings of parts before giving the error. Mostly useless. So it seems that this is impossible to fix. Trying to figure out which letters are randomized by hand is insanely hard and time consuming.

What we did though was the following: instead of fixing the book we changed the machine so it would (1) give us better errors and (2) allow us to overrule certain instructions. We fed the machine with the first paragraph of a chapter and it started to draw the part. When we thought it went wrong we gave it the instruction to skip that portion of the drawing. Because apparently it had read a wrongly printed letter.

Of course we did not know how far it had to skip. So we guessed several times and hoped it would start drawing something sane again. And each time it was drawing something that looked sensible we knew it was reading uncompromised letters. We got an understanding which parts of the paragraph were wrongly printed and then mostly avoided them. By doing this many times and collaborating with several people we have now been able to let the machine draw several almost complete and fairly accurate parts. We do not know yet if it is going to be possible to let the machine read the other paragraphs in the chapters (they are much harder). But getting these drawings is already very satisfying.

--

I hope this illustration helps you (who might not understand the deeply technical stuff) to understand what the problem really is. How insane it is. And I think it's a quite good representation of what is actually the case.

Regards,

arnezami

---

PS. Below is a translation from the illustration into the real problem.

- Book: Video

- 13 chapters: 13 parts of the video

- First paragraph of each chapter: I-frame (independent picture)

- Other paragraphs: P- and B-frames (all dependent on each other)

- Foreign language: the language of video decoding (things like "discrete cosine transform", "motion vectors", "chrominance and luminance blocks" etc)

- Removal of punctuation and capitals: there are essentially no clear "markers" in the bitstream: no marker to detect the beginning of a horizonal line in the picture.

- Words: macroblocks (16 x 16 pixel blocks)

- Letter coding and removal of whitespaces: the coding of the bits is such that you need every previous bit to know what the current one means withing a word. The words are also variable in length. There is essentially no clear marker between each macroblock.

- Printing errors: random bits that were flipped due to bad reception

- Rocket-part-building-machine: the video decoder (the open source ffmpeg in our case)

- Drawings: the pictures the video decoder makes after trying to decode a frame
 
The transportstream is not mentioned. But effectively many pages of the book were missing. We mostly fixed that: they were actually "hidden" and we discovered most of them.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: AncientU on 05/07/2014 08:07 PM
Thanks, that is perfect even for those of us who don't build rockets!
Ever thought of writing users' manuals?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Adaptation on 05/07/2014 08:11 PM
 

The University of Wisconsin has a very large HTCondor pool which would probably be very useful to run through this processing. I'll reach out and see if there's anyone over there who can leverage their tens of thousands of available CPU cores to crank out macroblock iterations.

This could work, they could run it through an edge detection filter to find ones with good time stamp in the proper location and the dust artifacts on the lens then bias the rest of the results against having strong vertical or horizontal lines.

A little bit of evolutinary code could reduce the power needed drastically, simply did this change make it better or worse.  If it was better use it as the starting point for the next set of changes. 
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/07/2014 08:17 PM
The University of Wisconsin has a very large HTCondor pool which would probably be very useful to run through this processing. I'll reach out and see if there's anyone over there who can leverage their tens of thousands of available CPU cores to crank out macroblock iterations.

This could work, they could run it through an edge detection filter to find ones with good time stamp in the proper location then biase the rest of the results against having strong verticle or horisontal lines.

We could then load "candidate images" in a server and crowdsource it. (like tomnod)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/07/2014 08:17 PM
Hi guys,

I would like to explain what we are actually doing. Because some of you might not completely grasp it. And I think I can best be explained it by an illustration. Otherwise it gets too technical too quickly. I will do it in terms of rockets. ;)

--

Imagine a book. A thick book. It's a book containing very detailed technical instructions on how to build a new type of rocket.  It contains 13 chapters. One for each rocket part. Each chapter consists of an introductory paragraph explaining what the part is, how it should look like and how it works. None of the following paragraphs can be understood if this paragraph has not been read. The following paragraps are highly dependent on eachother: if you haven't read them in sequence they will be meaningless since they always refer heavely to the previous one to understand.

All sentences are written in a foreign language. And all punctuation is removed. Also all captitals are made lower case. The letters in the words are coded: each letter can only be decoded if you know the previous letter. For example this code: when reading a letter add the letternumber (a=1, b=2 etc) of the previous letter from this one to get the real letter. All whitespaces are removed. So each paragraph is just a string of seemingly random letters.

Ok. Now you're thinking: "This is insane. Who would read such a book?"

But this book was never intended to be read by a human being! It was supposed to be read by a machine that would also automatically build the rocket parts. Here is the real problem though: the printing machine that printed the book had a major error: it sometimes printed the wrong letter! So the book has thousands of random errors in it! Unsuprisingly when the book is now given to the rocket-part-building-machine it says: "Error". And that's basicly it.

Ok it turnes out that the rocket-part-building-machine made a few incomplete and defective drawings of parts before giving the error. Mostly useless. So it seems that this is impossible to fix. Trying to figure out which letters are randomized by hand is insanely hard and time consuming.

What we did though was the following: instead of fixing the book we changed the machine so it would (1) give us better errors and (2) allow us to overrule certain instructions. We fed the machine with the first paragraph of a chapter and it started to draw the part. When we thought it went wrong we gave it the instruction to skip that portion of the drawing. Because apparently it had read a wrongly printed letter.

Of course we did not know how far it had to skip. So we guessed several times and hoped it would start drawing something sane again. And each time it was drawing something that looked sensible we knew it was reading uncompromised letters. We got an understanding which parts of the paragraph were wrongly printed and then mostly avoided them. By doing this many times and collaborating with several people we have now been able to let the machine draw several almost complete and fairly accurate parts. We do not know yet if it is going to be possible to let the machine read the other paragraphs in the chapters (they are much harder). But getting these drawings is already very satisfying.

--

I hope this illustration helps you (who might not understand the deeply technical stuff) to understand what the problem really is. How insane it is. And I think it's a quite good representation of what is actually the case.

Regards,

arnezami

---

PS. Below is a translation from the illustration into the real problem.

- Book: Video

- 13 chapters: 13 parts of the video

- First paragraph of each chapter: I-frame (independent picture)

- Other paragraphs: P- and B-frames (all dependent on each other)

- Foreign language: the language of video decoding (things like "discrete cosine transform", "motion vectors", "chrominance and luminance blocks" etc)

- Removal of punctuation and capitals: there are essentially no clear "markers" in the bitstream: no marker to detect the beginning of a horizonal line in the picture.

- Words: macroblocks (16 x 16 pixel blocks)

- Letter coding and removal of whitespaces: the coding of the bits is such that you need every previous bit to know what the current one means withing a word. The words are also variable in length. There is essentially no clear marker between each macroblock.

- Printing errors: random bits that were flipped due to bad reception

- Rocket-part-building-machine: the video decoder (the open source ffmpeg in our case)

- Drawings: the pictures the video decoder makes after trying to decode a frame
 
The transportstream is not mentioned. But effectively many pages of the book were missing. We mostly fixed that: they were actually "hidden" and we discovered most of them.


Such a really good post! :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: John_L on 05/07/2014 08:19 PM
And to continue the analogy... a new book will be written this Saturday....lol
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/07/2014 08:33 PM
Technique: Find suspect macroblock. Isolate bit range. Plug them into the for loop. Put the string ",X,$i:1," in the middle of your MMB string, where it goes from a bit range point point of view. Run the loop, wait a bit, flip through the resulting pictures. Can usually narrow it down to 4 or 5. If nothing works, try changing the macroblock to the left, or nulling out the macro block... if it's a multibit error, it's a lot more permutations to try to figure out.

For example, my last run:
for i in {32260..32407} ; do ./ffmpeg -debug mb_pos_size -err_detect ignore_err -s:0 704:480 -mmb 20:04:-1,24:04:16023,X:16410:01,16:5:-1,17:05:18692,X:19903:1, X:27244:1,X:$i:1,1:15:-1,3:16:79127,41:19:-1, 8:20:96220,15:04:16023 -i iframe_9.mpg4-img -f image2 img-%03d-$i.png 2> /dev/null ; done

-Bob

One more note. X doesn't behave quite as you think. The bitmask is inverted. X:pos:1 actually flips the bit 8 positions away from pos. It reads the next 8 bits starting from pos with pos being the MSB of those 8 bits. If you want to flip just the target bit, do X:pos:80.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: R7 on 05/07/2014 08:39 PM
And to continue the analogy... a new book will be written this Saturday....lol

Let's hope they have changed the page layout to contain additional information to help detect sentence boundaries, errors and such. This macroblock necromancy goes only so far.  :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/07/2014 08:40 PM
And to continue the analogy... a new book will be written this Saturday....lol

Let's hope they have changed the page layout to contain additional information to help detect sentence boundaries, errors and such. This macroblock necromancy goes only so far.  :)

Should let one of us hop on their plane with an SDR with a patchfeed into their radios. :P
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: starsilk on 05/07/2014 08:50 PM
And to continue the analogy... a new book will be written this Saturday....lol

Let's hope they have changed the page layout to contain additional information to help detect sentence boundaries, errors and such. This macroblock necromancy goes only so far.  :)

Should let one of us hop on their plane with an SDR with a patchfeed into their radios. :P

three important things that should be communicated back to SpaceX:

1) turn on the error correction features in their camera/encoder (probably too late for Saturday's launch.. but maybe)

2) turn off interlace

3) make an analog recording of the telemetry stream..
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/07/2014 08:54 PM
3) make an analog recording of the telemetry stream..

Well a digital recording of the analog waveform, but yes.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/07/2014 09:01 PM
3) make an analog recording of the telemetry stream..

Would this be analog enough?  ;D
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: TrueBlueWitt on 05/07/2014 09:06 PM
And to continue the analogy... a new book will be written this Saturday....lol

Let's hope they have changed the page layout to contain additional information to help detect sentence boundaries, errors and such. This macroblock necromancy goes only so far.  :)

Should let one of us hop on their plane with an SDR with a patchfeed into their radios. :P

three important things that should be communicated back to SpaceX:

1) turn on the error correction features in their camera/encoder (probably too late for Saturday's launch.. but maybe)

2) turn off interlace

3) make an analog recording of the telemetry stream..


(2)
Why is anything still recorded using Interlaced processing(OK.. 1080i.. but still)???? 
And my personal favorite.. TV commercials that are recorded in interlaced SD.. 

(3)
High resolution(24bit with high sample rate) "Analog" copy of the data stream with time reference to assess where S/N and coherence are bad would be very helpful. I think I mentioned that many pages ago.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: starsilk on 05/07/2014 09:09 PM
3) make an analog recording of the telemetry stream..

Would this be analog enough?  ;D

nah. if we're going to record something important like this, it needs to at least be a metal tape. they can afford the extra cost  ;)

Edit: just make sure to push the 'metal' button on the tape recorder first.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/07/2014 09:10 PM
Why is anything still recorded using Interlaced processing? ???

(http://www.graphicdesignbasics.com/imagesvr_ce/graphicdesign/uploadedfiles/2009/04/dr-mccoy.jpg)
"Dammit, Jim, I'm a rocket scientist, not a video technician!"
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Turix on 05/07/2014 09:13 PM
I had a go at frame 1, I managed to find most of the timestamp and a bit of the top of the rocket (notice the darker feature bottom left, which appears to match some of the other frames - though I'm not sure how well its aligned).

Done using: 41:00:-1, 33:17:66620, 03:28:112199
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/07/2014 09:30 PM
Someone needs to take a crack at iframe3. Don't know if we can get anything out of that or not.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/07/2014 09:51 PM
Here's a bunch of ocean from iframe4 (the new try1 version). Unfortunately unaligned. No legs to be seen. I think this is the last iframe with legs stowed. They appear to be deployed on iframe5.

-mmb 5:1:4200,0:3:10968,8:3:17911,34:3:23414,19:8:39488,1:9:41549,18:9:42693

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: grythumn on 05/07/2014 10:22 PM
One more note. X doesn't behave quite as you think. The bitmask is inverted. X:pos:1 actually flips the bit 8 positions away from pos. It reads the next 8 bits starting from pos with pos being the MSB of those 8 bits. If you want to flip just the target bit, do X:pos:80.

Ah, boundary conditions/endianness/alignment, how do I love thee. I start doing this manually, 01,02,04,08,10,20,40,80, increment by 8, then decided to automate it without looking at the source. :)

Empirically, it worked fairly well as a semi-targeted brute force search. The correct one should work better. :) I'm playing with a #define TRACE build, but don't understand quite what I'm doing when twiddling bits yet to narrow the search range smaller than the whole macroblock.

-Bob
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/07/2014 10:35 PM
Ok, so no additional help from SpaceX tonight at least I'm told, as they are all busy bees with the next launch (including a lot in transit to the Cape). Again, this thread is being watched, big time, and by one rather well known name at SpaceX I'm reliably informed! ;) So stay focused as much as you can with what we've got.

I'll get more eyeballs on it when I add a few paras and links to here in my F9 v1.1 Static Fire article tomorrow.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/07/2014 10:41 PM
One more note. X doesn't behave quite as you think. The bitmask is inverted. X:pos:1 actually flips the bit 8 positions away from pos. It reads the next 8 bits starting from pos with pos being the MSB of those 8 bits. If you want to flip just the target bit, do X:pos:80.

Ah, boundary conditions/endianness/alignment, how do I love thee. I start doing this manually, 01,02,04,08,10,20,40,80, increment by 8, then decided to automate it without looking at the source. :)

Empirically, it worked fairly well as a semi-targeted brute force search. The correct one should work better. :) I'm playing with a #define TRACE build, but don't understand quite what I'm doing when twiddling bits yet to narrow the search range smaller than the whole macroblock.

-Bob

I pushed up one of michaelni's changes from the other day to the repo if you want to grab it. Adds some warnings when data is possible off. Haven't found it very useful. They print out in yellow.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: John_L on 05/07/2014 10:47 PM
Ok, so no additional help from SpaceX tonight at least I'm told, as they are all busy bees with the next launch (including a lot in transit to the Cape). Again, this thread is being watched, big time, and by one rather well known name at SpaceX I'm reliably informed! ;) So stay focused as much as you can with what we've got.

I'll get more eyeballs on it when I add a few paras and links to here in my F9 v1.1 Static Fire article tomorrow.

Thanks for the heads up Chris... it's understandable with another launch coming up this weekend that everyone's busy with that ... and with that said...

Hey Elon!

(lol)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Okie_Steve on 05/07/2014 11:10 PM
Every block has 6 sub-blocks, 4 luma blocks (brightness) and 2 chroma blocks (color). The 4 luma blocks are the 4 squares in a block you often see. Both color blocks cover the entire block and are two different color planes. https://en.wikipedia.org/wiki/YUV There are 4 Y blocks and 1 U and 1 V block. Each one of those sub blocks references either the block above it or the block to the left of it (they all don't have to reference the same block). Which block they reference is determined on-the-fly (not hardcoded) so if a block is screwed up then it could cause blocks that would reference it, to not reference it or vice versa.
Hmmm, I suspect that color is not really important here, has anyone tried stripping/ignoring the chroma information and just generated "black & white"? The human visual system is *MUCH* more sensitive to intensity then color which is part of why the chroma is sampled at half (or less) of luma in most cases.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/07/2014 11:16 PM
Every block has 6 sub-blocks, 4 luma blocks (brightness) and 2 chroma blocks (color). The 4 luma blocks are the 4 squares in a block you often see. Both color blocks cover the entire block and are two different color planes. https://en.wikipedia.org/wiki/YUV There are 4 Y blocks and 1 U and 1 V block. Each one of those sub blocks references either the block above it or the block to the left of it (they all don't have to reference the same block). Which block they reference is determined on-the-fly (not hardcoded) so if a block is screwed up then it could cause blocks that would reference it, to not reference it or vice versa.
Hmmm, I suspect that color is not really important here, has anyone tried stripping/ignoring the chroma information and just generated "black & white"? The human visual system is *MUCH* more sensitive to intensity then color which is part of why the chroma is sampled at half (or less) of luma in most cases.

The data is there regardless, why not get it for free while we're at it?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: lgjy98d on 05/07/2014 11:23 PM
Is there a sense to create a grid with well-known MB locations?

I.e. 19.37.SS.mm timestamp spans MBs with following coordinates:
* colums 1 and 2 are for "19" hours,
* columns 3 and 4 are for ".35" minutes
* column 5 is for dot between minutes and seconds
* columns 6 and 7 are for SS. seconds
* column 8 and 9 are for mm miliseconds
* row 28 is for upper part of timestamp
* row 29 is for lower part

And once I'm experimenting with the image, I can see the recognizable elements, having seen them the POS in the stream would let me fixate the coordinates of that MB in the mmb string.

This would let some parts of the frame to snap in places even if there are other errors. It look like if whole row of MBs decodes properly the second row "sticks to previous one visually and lets to recognize the picture.

Is there a sense in the approach?

Besides the timestamp there are other notable features to watch for:
* dirt patch in the middle of top part of the picture.
* engine
* legs
* stage body edges (left and right)
* large dark spot above timestamp on the body of the stage
* camera lens edge (left and right bottom corners of the frame)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/07/2014 11:41 PM
iframe 11 a bit better (not much...):

-mmb 0:0:-1:1:0:1,0:12:27692,0:25:-1,1:25:82578,7:29:-1
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: michaelni on 05/07/2014 11:46 PM
How would one go about re-encoding the video with the replacement iframes, what's the command line for ffmpeg?

The following should work
assuming you have extracted  the frames with -vcodec copy into individual mpg4-img files
you could replace one with
ffmpeg -framerate 44999/3003 -i ../tests/lena.pnm -s 704x480  -aspect 22:15 -f image2 -r 44999/3003 image120.mpg4-img
the image this creates seems close enough to the original parameters to work

and the following would put it all back into one file:
ffmpeg -i image%3d.mpg4-img -vcodec copy video.avi
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: tentonine on 05/07/2014 11:51 PM
A little more detail near the top of frame 1, but not a lot better than the previous one.

40:00:-1,00:01:550,40:01:-1,00:02:4174,01:02:-1,14:02:5342,41:02:-1,00:03:8017,01:03:-1,00:04:12558,05:04:-1, 18:04:17826,37:11:-1,33:17:66620,20:21:-1,23:21:86305,33:21:-1,34:21:87382,03:28:112199,09:29:-1, 15:29:131724,20:29:-1,24:29:132485
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/08/2014 12:27 AM
iframe 7 with some tuning:

-mmb X:18494:01,18:05:-1,02:06:18794,15:12:-1,19:12:45121,19:15:-1,20:15:62858,
21:15:-1,23:15:-1:50:50:50,24:15:-1:-25:-25:-25,25:15:-1,29:15:64161,19:16:-1:-5:0:0,
21:16:68508,30:18:-1,34:18:76643,36:18:-1:20:20:20,37:18:77219,0:19:85528,14:20:-1,1:21:97689

also tried iframe 4 and 5 with no meaningful improvement
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: michaelni on 05/08/2014 01:25 AM
Another way of doing this is to say to a block: don't "listen" to your left block, instead "listen" to the left "block" of 0,0 (initial state I guess). I believe these things are called "predictions" in mepg-4.

Implemented, example:
-mmb 40:3:-2
or
-mmb 20:19:-2:32:32:32:32:50:-11
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Okie_Steve on 05/08/2014 01:37 AM
Every block has 6 sub-blocks, 4 luma blocks (brightness) and 2 chroma blocks (color). The 4 luma blocks are the 4 squares in a block you often see. Both color blocks cover the entire block and are two different color planes. https://en.wikipedia.org/wiki/YUV There are 4 Y blocks and 1 U and 1 V block. Each one of those sub blocks references either the block above it or the block to the left of it (they all don't have to reference the same block). Which block they reference is determined on-the-fly (not hardcoded) so if a block is screwed up then it could cause blocks that would reference it, to not reference it or vice versa.
Hmmm, I suspect that color is not really important here, has anyone tried stripping/ignoring the chroma information and just generated "black & white"? The human visual system is *MUCH* more sensitive to intensity then color which is part of why the chroma is sampled at half (or less) of luma in most cases.

The data is there regardless, why not get it for free while we're at it?
Because approximately half of the bit stream is chroma information and hence half of the errors needing to be dealt with. Given the way errors cascade color is probably just a distraction in this case.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: John_L on 05/08/2014 01:59 AM
More on frame 2...
this is not from the try1.ts file, the older one.

40:3:11229,16:10:18210,40:28:76955,25:8:18680,4:28:-1
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Asmegin on 05/08/2014 02:36 AM
Frame 3 is disgusting  >:(

I can't do anything with it (disclaimer: I have no idea what I'm doing)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: HALtheWise on 05/08/2014 02:44 AM
Every block has 6 sub-blocks, 4 luma blocks (brightness) and 2 chroma blocks (color). The 4 luma blocks are the 4 squares in a block you often see. Both color blocks cover the entire block and are two different color planes. https://en.wikipedia.org/wiki/YUV There are 4 Y blocks and 1 U and 1 V block. Each one of those sub blocks references either the block above it or the block to the left of it (they all don't have to reference the same block). Which block they reference is determined on-the-fly (not hardcoded) so if a block is screwed up then it could cause blocks that would reference it, to not reference it or vice versa.
Hmmm, I suspect that color is not really important here, has anyone tried stripping/ignoring the chroma information and just generated "black & white"? The human visual system is *MUCH* more sensitive to intensity then color which is part of why the chroma is sampled at half (or less) of luma in most cases.

The data is there regardless, why not get it for free while we're at it?
Because approximately half of the bit stream is chroma information and hence half of the errors needing to be dealt with. Given the way errors cascade color is probably just a distraction in this case.
Unfortunately, the information that is needed to separate the chroma and intensity values has also been corrupted in many cases. AFAIK, until the macro blocks have been correctly identified and placed, separating the streams simply isn't possible. After that, however, it may in fact make sense to focus (at least initially) on bit-level corrections for the intensity alone before worrying about chroma. Most the I-frames haven't yet reached nearly-complete Macro block identification yet, though. It can also be much easier to identify the correct placement of blocks by using the color information.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/08/2014 03:21 AM
Frame 3 is disgusting  >:(

I can't do anything with it (disclaimer: I have no idea what I'm doing)

Agreed. It seems to be totally random noise.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: dorkmo on 05/08/2014 03:45 AM
Frame 3 is disgusting  >:(

I can't do anything with it (disclaimer: I have no idea what I'm doing)

Agreed. It seems to be totally random noise.

yeah i think its just the fact that it has so few bytes, its got 8927. while the best picture so far (iframe8) has 20428. i think its interesting that iframe 6 has the at most 29622 but so far looks just as garbled as others. might be the one with most room for improvement?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: GregA on 05/08/2014 06:02 AM
Amazing work guys, I'm really looking forward to the video, and wish I had knowledge/ability to add.

With something like I Frame 3, would you be able to reconstruct based on I-Frame 2 plus all the P & B frames applied (which would theoretically be one frame before the i-frame)? I haven't seen that kind of thing mentioned here. I would assume you theoretically could - but that if i-frame 3 is gone then the preceding frames are probably also junk.

Similarly, would you replace a junk i-frame with a generic shot (plain blue plus image of rocket?) to then have the change frames act on, or would you leave the junk frame? (Of course, you need a good generic shot of the rocket for that to be possible!)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/08/2014 06:24 AM
Frame 9, cleaned up some single-bit errors up top.

20:04:-1,24:04:16023,X:16410:01,16:5:-1,17:05:18692,X:19903:1,X:27244:1,
1:15:-1,3:16:79127,41:19:-1,8:20:96220,15:04:16023

More cleanup on iframe 9. I am less convinced now that the odd shapes on the right leg are bugged bits. They seem to be consistent across multiple blocks and sub-blocks.

X:19164:10,X:32551:80,X:32743:1,20:04:-1,24:04:16023,X:16410:01,16:5:-1,
17:05:18692,X:19903:1,X:27244:1,24:08:-1,25:08:32410,1:15:-1,3:16:79127,
41:19:-1,8:20:96220,15:04:16023
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/08/2014 07:08 AM
Frame 9, cleaned up some single-bit errors up top.

20:04:-1,24:04:16023,X:16410:01,16:5:-1,17:05:18692,X:19903:1,X:27244:1,
1:15:-1,3:16:79127,41:19:-1,8:20:96220,15:04:16023

More cleanup on iframe 9. I am less convinced now that the odd shapes on the right leg are bugged bits. They seem to be consistent across multiple blocks and sub-blocks.

X:19164:10,X:32551:80,X:32743:1,20:04:-1,24:04:16023,X:16410:01,16:5:-1,
17:05:18692,X:19903:1,X:27244:1,24:08:-1,25:08:32410,1:15:-1,3:16:79127,
41:19:-1,8:20:96220,15:04:16023

They look a bit like the holding clamps which attach the legs to the body of the rocket.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/08/2014 07:52 AM
I was about to suggest that depending on the altitude at that moment, which you could probably estimate knowing the time interval between iframes 9 and 10, perhaps the apparent distortion of the center portion of the right leg could be a splash of brightly-illuminated water between the camera and the leg.

But then after looking at the emerging details of frames 7 and 8, I'm beginning to fear your suspicions may be correct that they are not bugged bits but rather a big gnarly dent in the leg.  :o Maybe a chunk of muddy ice smashed into it in just the wrong way?

Looking at the image of the freshly-installed legs it's hard to picture how something like that could have happened, but then again I've seen the heartbreaking proof of what a mere  chunk of foam can do to composite structures with my own eyes down at the USSRC in in Huntsville. I fervently hope that the meticulous grooming of the iframes you guys are doing will prove this notion wrong. Keep up the utterly amazing, amazing work! You guys deserve medals!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/08/2014 08:07 AM
I was about to suggest that depending on the altitude at that moment, which you could probably estimate knowing the time interval between iframes 9 and 10, perhaps the apparent distortion of the center portion of the right leg could be a splash of brightly-illuminated water between the camera and the leg.

But then after looking at the emerging details of frames 7 and 8, I'm beginning to fear your suspicions may be correct that they are not bugged bits but rather a big gnarly dent in the leg.  :o Maybe a chunk of muddy ice smashed into it in just the wrong way?

Looking at the image of the freshly-installed legs it's hard to picture how something like that could have happened, but then again I've seen the heartbreaking proof of what a mere  chunk of foam can do to composite structures with my own eyes down at the USSRC in in Huntsville. I fervently hope that the meticulous grooming of the iframes you guys are doing will prove this notion wrong. Keep up the utterly amazing, amazing work! You guys deserve medals!
I suspect what happened is that one of the bolts attaching the leg to the rocket did not let loose. So that part of the leg snapped off and is still attached to the body of the rocket. This event is somewhere in this video!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: luinil on 05/08/2014 08:09 AM
It would be interesting to see if we can determine if the part of the leg is still on the rocket body (that would confirm the hypothesis).
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/08/2014 08:10 AM
It would be interesting to see if we can determine if the part of the leg is still on the rocket body (that would confirm the hypothesis).
It could also have damaged the sidewall of the rocket. Maybe bending it outwards or possibly creating a hole.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: ugordan on 05/08/2014 08:40 AM
I suspect what happened is that one of the bolts attaching the leg to the rocket did not let loose. So that part of the leg snapped off and is still attached to the body of the rocket.

That sounds plausible, IMHO.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: fatdeeman on 05/08/2014 08:58 AM
I suspect what happened is that one of the bolts attaching the leg to the rocket did not let loose. So that part of the leg snapped off and is still attached to the body of the rocket. This event is somewhere in this video!

That sounds very plausible to me, those rams look similar to what you would see on an excavator so it was going to fully deploy whether the leg was partially attached or not, if the ram is as powerful as it looks then causing that damage wouldn't have made it break a sweat!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: eriblo on 05/08/2014 09:12 AM
How sure can we be that it's actual damage to the leg? The "cutout" edges matches the video block, which for at least the first and the second image is obviously corrupt in some way. That the same block would be corrupt in all three images is of course a coincidence, but there are a lot of errors and they are most visible in these high contrast blocks (going from water to leg).

Two other things: First, it's hard to see the leg cylinder in these images, shouldn't it show more clearly at least in the last image (although we might be well into sub resolution territory here ;))? Second, I have a hard time believing that the bolts would be designed to be stronger than the leg - if it fails to release the failure modes you want are either failure to release the leg or failure of the bolt, not ripping half the leg and/or chunks of the tank off.

Also remember that even if the pistons are capable of large forces (lets say about 10-15 tonnes) they have a very small leverage during the start of the extension - maybe even negative, see relevant discussion threads.

EDIT to add: Relatively large forces (mass/4 times 1.5 or so for the angle) are only necessary if the legs support the stage on gas pressure alone, which is much less likely than some sort of locking mechanism. Even so they are nowhere close to hydraulic systems that operate at a magnitude higher pressure.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/08/2014 09:20 AM
Frame 3 is disgusting  >:(

I can't do anything with it (disclaimer: I have no idea what I'm doing)

Agreed. It seems to be totally random noise.

yeah i think its just the fact that it has so few bytes, its got 8927. while the best picture so far (iframe8) has 20428. i think its interesting that iframe 6 has the at most 29622 but so far looks just as garbled as others. might be the one with most room for improvement?

Can it be that due to bit errors the size of Iframe3 is not correctly determined and part of IF3 is interpreted to be part of the next frame?

Cheers
Shanuson
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Microphobe on 05/08/2014 09:41 AM
But then after looking at the emerging details of frames 7 and 8, I'm beginning to fear your suspicions may be correct that they are not bugged bits but rather a big gnarly dent in the leg. 

Hi, I think it might be some sort of splodge of high refractive index stuff on the camera bending light around it (see attached image: red circle in same place on the two frames, aligned them using the black mark above it) Its an oval region without any internal texture (ie blurred) in all other frames AFAICS, bordering another dirty mark, and still appears to be there in frame 12 where the leg is underwater. I reckon the leg was probably OK until hitting the water, but it would be nice to get more clear images of the "splodge".

Great work guys, although I don't understand a word of what you are talking about re: video decoding !!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/08/2014 10:46 AM
Aligned iframe 1 from the previous solutions with:

-mmb 40:00:-1,14:02:5342,41:02:-1,7:05:17826,26:12:-1,0:14:50695,43:14:-1,31:16:66620,
7:21:-1,10:21:-1:-5:10:0,11:21:-1,20:21:87382,25:21:-1:0:-20:0,26:21:87827,
22:27:-1,28:27:109726,41:27:-1,03:28:112199,09:29:-1,15:29:131724,20:29:-1,24:29:132485

there are still two small sequences which I +/- randomly put: 14:02:5342,41:02:-1 and 0:14:50695,43:14:-1, and line 21 could be improved
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/08/2014 10:57 AM
Aligned iframe 1 from the previous solutions with:

-mmb 40:00:-1,14:02:5342,41:02:-1,7:05:17826,26:12:-1,0:14:50695,43:14:-1,31:16:66620,
7:21:-1,10:21:-1:0:-16:0,11:21:-1,20:21:87382,22:27:-1,28:27:109726,41:27:-1,03:28:112199,
09:29:-1,15:29:131724,20:29:-1,24:29:132485

there are still two small sequences which I +/- randomly put: 14:02:5342,41:02:-1 and 0:14:50695,43:14:-1

Be careful how you pick blocks, you can start to create an image that does not represent reality. The lower portion of the rocket is not visible.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/08/2014 11:01 AM
Frame 3 is disgusting  >:(

I can't do anything with it (disclaimer: I have no idea what I'm doing)

Agreed. It seems to be totally random noise.

yeah i think its just the fact that it has so few bytes, its got 8927. while the best picture so far (iframe8) has 20428. i think its interesting that iframe 6 has the at most 29622 but so far looks just as garbled as others. might be the one with most room for improvement?

Can it be that due to bit errors the size of Iframe3 is not correctly determined and part of IF3 is interpreted to be part of the next frame?

Cheers
Shanuson

Possible. Possibly with some better work on the transport stream that can be determined to be the case or ruled out. The try1.ts file's iframe3 is a bit larger than the others at 9005 bytes.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/08/2014 11:02 AM
Aligned iframe 1 from the previous solutions with:

-mmb 40:00:-1,14:02:5342,41:02:-1,7:05:17826,26:12:-1,0:14:50695,43:14:-1,31:16:66620,
7:21:-1,10:21:-1:0:-16:0,11:21:-1,20:21:87382,22:27:-1,28:27:109726,41:27:-1,03:28:112199,
09:29:-1,15:29:131724,20:29:-1,24:29:132485

there are still two small sequences which I +/- randomly put: 14:02:5342,41:02:-1 and 0:14:50695,43:14:-1

Be careful how you pick blocks, you can start to create an image that does not represent reality. The lower portion of the rocket is not visible.

No no, I used iframe 2 to align the image features, only the two small mentioned blocks are not where they should be. There might be condensation on the lens or the rocket was in a cloud, reducing a lot the contrast.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/08/2014 11:07 AM
In other news. Iframe8 is becoming amazing. I'm still working on the lower part, but if anyone want's to try their hand at blocks 05:21 through 27:21 or fixing the ugly color/brightness cast into the bottom right, it'd be much appreciated. That's bit 107247 to 108877 or 1630 bits spread over 23 blocks for an average of 71 bits per block.

MMB split out into multiple lines to be readable.
08:14:-1:15:-15:1:14:-2:1,
09:14:56683,
05:21:-1:0:-20:-5:2,
06:21:-1:-20:-20:6:5,
07:21:-1:0:0:3:5,
08:21:-1:0:0:0:0,
09:21:-1:10:15:0:0,
10:21:-1:15:15:0:0,
11:21:-1:5:10,
12:21:-1:5:3:0:0:-2,
13:21:-1:4:4,
14:21:-1,
28:21:108878,
X:126932:80,
04:27:-1:0:-4:13:7,
05:27:135412,
X:143273:80,
X:143386:80,
06:28:144012

As a note, in every image there should be two black triangles in the bottom right corners. I suspect this is part of the camera housing. They are good cues for reconstructing the lower part of the image.

Edit: Attached before image.

Edit2: Forgot to mention. I used try1.ts iframe8.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: ugordan on 05/08/2014 11:09 AM
Nice!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/08/2014 11:24 AM
As to the previous conversation about things breaking off the legs. In this iframe you can clearly see a white lump on the right side that would probably line up with that leg. I'm reasonably certain that that section of the image isn't corrupted, or if it is corrupted would be very minor and something of the leg would still be showing. Good chance something actually did bust off.

Edit: One more thing. If this is actually the case, getting this video showing the break-off point suddenly becomes pretty important for SpaceX if they weren't aware of it. Gives more reason for us to continue work on it after Saturday's launch.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/08/2014 11:38 AM
Hi, I think it might be some sort of splodge of high refractive index stuff on the camera bending light around it (see attached image: red circle in same place on the two frames, aligned them using the black mark above it) Its an oval region without any internal texture (ie blurred) in all other frames AFAICS, bordering another dirty mark, and still appears to be there in frame 12 where the leg is underwater. I reckon the leg was probably OK until hitting the water, but it would be nice to get more clear images of the "splodge".

I like that idea best! Phew! Good thinking! And if you peer at frame 10 in the right way you can start to make out a smudge there as well. You'd think that if it was a water droplet doing the lensing that it would move with the wind, so maybe it was some sort of damage to the window of the camera housing, or something that froze at a higher altitude and then didn't melt by the time it landed. We'll probably see more clearly when these heroic bit-wranglers persuade frames 4 and 5 to emerge and we can start to make out the deployment.

Maybe someone should try an alignment of frame 2 with the "smudge" idea in mind, coupling it with the little black quotation mark.

EDIT: Also, smudge or damage, either way it looks worse than it actually is. Take a look at the tip of the leg in the shop-floor photo: it's just a hollow shell of composite. And take a look at Grasshopper - the load-bearing pieces of the legs are the steel (or some sort of highstrengthtoweightium alloy) tubes, and even if the fairing got smashed by ice or ripped away from a stuck latch, it would have absolutely no implications for the business parts of the leg.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: MTom on 05/08/2014 11:45 AM
As to the previous conversation about things breaking off the legs. In this iframe you can clearly see a white lump on the right side that would probably line up with that leg. I'm reasonably certain that that section of the image isn't corrupted, or if it is corrupted would be very minor and something of the leg would still be showing. Good chance something actually did bust off.

Edit: One more thing. If this is actually the case, getting this video showing the break-off point suddenly becomes pretty important for SpaceX if they weren't aware of it. Gives more reason for us to continue work on it after Saturday's launch.

Not sure, the waves on the water surface also missing in this section.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: TrueBlueWitt on 05/08/2014 12:02 PM
Rampant speculation about something "damaged" when it could as easily be a video artifact or misaligned video data block is why Elon and SpaceX are leary about releasing such things in the first place. To me the anamoly it looks like it has right angled edges which align with the macro blocks.  Let's not get too crazy about this yet.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Mongo62 on 05/08/2014 12:05 PM
I have to go with the "smudge on the lens" theory as well. If (as appears to be the case) a suspiciously blank patch is present in the same location on the images from before leg deployment, as well as after the leg has been partly submerged, then it is very unlikely to be a physical chunk missing from the leg, in my non-expert opinion.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/08/2014 12:09 PM
I have to go with the "smudge on the lens" theory as well. If (as appears to be the case) a suspiciously blank patch is present in the same location on the images from before leg deployment, as well as after the leg has been partly submerged, then it is very unlikely to be a physical chunk missing from the leg, in my non-expert opinion.

That's very true. Either way, we'll find out as we get the video completed.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/08/2014 12:09 PM
Rampant speculation about something "damaged" when it could as easily be a video artifact or misaligned video data block is why Elon and SpaceX are leary about releasing such things in the first place. To me the anamoly it looks like it has right angled edges which align with the macro blocks.  Let's not get too crazy about this yet.

I wouldn't call it "rampant" - that implies "unbridled" and "widespread," and nobody here seems particularly breathless and it's only been four hours. It's a little more than "idle," yes, but not "rampant." Plus, it's only been four hours and folks may already have come up with another fixed reference point for MB alignment in addition to the quotation mark. Welcome to the wonderful and exciting world of "crowdsourcing."
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/08/2014 12:27 PM
Rampant speculation about something "damaged" when it could as easily be a video artifact or misaligned video data block is why Elon and SpaceX are leary about releasing such things in the first place. To me the anamoly it looks like it has right angled edges which align with the macro blocks.  Let's not get too crazy about this yet.

I wouldn't call it "rampant" - that implies "unbridled" and "widespread," and nobody here seems particularly breathless and it's only been four hours. It's a little more than "idle," yes, but not "rampant." Plus, it's only been four hours and folks may already have come up with another fixed reference point for MB alignment in addition to the quotation mark. Welcome to the wonderful and exciting world of "crowdsourcing."

Any shape in highly compressed MPEG will tend to "blur" because of the reduction of the pixel data to sets of something akin to 2D Fourier transforms (Discrete Cosine Transform (https://en.wikipedia.org/wiki/Discrete_cosine_transform)) resulting in "ringing" on any sharp edges in an image. Further those blurs cannot extend beyond the edge a block (or a sub-block) which can lead to sharp cutoffs.

Every 8x8 pixel sub-block is encoded like this. Example from wikipedia of encoding an image of the letter A.
(https://upload.wikimedia.org/wikipedia/commons/5/5e/Idct-animation.gif)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mikes on 05/08/2014 12:48 PM
Frame 6 got a chunk of the body

0:0:-1,0:19:121762,1:28:202509

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/08/2014 12:54 PM
In other news. Iframe8 is becoming amazing. I'm still working on the lower part, but if anyone want's to try their hand at blocks 05:21 through 27:21 or fixing the ugly color/brightness cast into the bottom right, it'd be much appreciated. That's bit 107247 to 108877 or 1630 bits spread over 23 blocks for an average of 71 bits per block.

MMB split out into multiple lines to be readable.
08:14:-1:15:-15:1:14:-2:1,
09:14:56683,
05:21:-1:0:-20:-5:2,
06:21:-1:-20:-20:6:5,
07:21:-1:0:0:3:5,
08:21:-1:0:0:0:0,
09:21:-1:10:15:0:0,
10:21:-1:15:15:0:0,
11:21:-1:5:10,
12:21:-1:5:3:0:0:-2,
13:21:-1:4:4,
14:21:-1,
28:21:108878,
X:126932:80,
04:27:-1:0:-4:13:7,
05:27:135412,
X:143273:80,
X:143386:80,
06:28:144012

As a note, in every image there should be two black triangles in the bottom right corners. I suspect this is part of the camera housing. They are good cues for reconstructing the lower part of the image.

Edit: Attached before image.

Edit2: Forgot to mention. I used try1.ts iframe8.

That's amazing!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: ugordan on 05/08/2014 12:57 PM
If this is actually the case, getting this video showing the break-off point suddenly becomes pretty important for SpaceX if they weren't aware of it.

They're bound to already have telemetry that would indicate this, if it indeed happened at all. Things like pressure vs. time profile in the cylinder compared to the other 3 indicating more resistance to deployment, etc.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/08/2014 12:58 PM
In other news. Iframe8 is becoming amazing. I'm still working on the lower part, but if anyone want's to try their hand at blocks 05:21 through 27:21 or fixing the ugly color/brightness cast into the bottom right, it'd be much appreciated. That's bit 107247 to 108877 or 1630 bits spread over 23 blocks for an average of 71 bits per block.

MMB split out into multiple lines to be readable.
08:14:-1:15:-15:1:14:-2:1,
09:14:56683,
05:21:-1:0:-20:-5:2,
06:21:-1:-20:-20:6:5,
07:21:-1:0:0:3:5,
08:21:-1:0:0:0:0,
09:21:-1:10:15:0:0,
10:21:-1:15:15:0:0,
11:21:-1:5:10,
12:21:-1:5:3:0:0:-2,
13:21:-1:4:4,
14:21:-1,
28:21:108878,
X:126932:80,
04:27:-1:0:-4:13:7,
05:27:135412,
X:143273:80,
X:143386:80,
06:28:144012

As a note, in every image there should be two black triangles in the bottom right corners. I suspect this is part of the camera housing. They are good cues for reconstructing the lower part of the image.

Edit: Attached before image.

Edit2: Forgot to mention. I used try1.ts iframe8.

That's amazing!

Key thing is that all the information is more or less still there. It's just that the nature of MPEG means that flipping a single bit from 0 to 1 (or reverse) can mean the entire image being decoded wrong. MPEG is designed to be error resilient despite this though (its how we all get our cable TV or satellite TV or over the air digital TV), just that SpaceX didn't turn on many of the error resilience features apparently that could have stopped the propagation of the errors as far as they did.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/08/2014 01:23 PM
Implemented, example:
-mmb 40:3:-2
or
-mmb 20:19:-2:32:32:32:32:50:-11

Is the source for this version available for download? Git does not work for me.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/08/2014 01:46 PM
Implemented, example:
-mmb 40:3:-2
or
-mmb 20:19:-2:32:32:32:32:50:-11

Is the source for this version available for download? Git does not work for me.

I had forgotten about it, just pushed it up to git a few hours ago. If you grab the source from the URL in the install.txt it should be the right version.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Herb Schaltegger on 05/08/2014 02:06 PM
Great work on this, mlinder. Thanks for sharing your efforts here on NSF.

ASIDE: Watching all this amateur Kremlinology on some badly-corrupted individual video frames reminds me somewhat unpleasantly of watching the internets go nuts over similarly-confusing bits of video and tidbits of information following STS-107. I can see why SpaceX didn't rush to release the video immediately. I sincerely hope there is better video (preferably from observation ships/aircraft) following Saturday's launch.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: ugordan on 05/08/2014 02:11 PM
I sincerely hope there is better video (preferably from observation ships/aircraft) following Saturday's launch.

I don't know how much "better" such video would be. On another site I saw a claim that the telemetry plane on CRS-3 was over 20 miles away. The CASSIOPE launch aerial images released also suggest a pretty large distance from the vehicle. I guess safety trumps closeup observations.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mmeijeri on 05/08/2014 02:18 PM
Maybe they can enable some of the resilience features this time round?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: ugordan on 05/08/2014 02:20 PM
If it was that simple and at no additional cost (like increase in b/w requirements) I imagine they'd already have done that. After all, even the video on the way up breaks up fairly badly at times.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Adaptation on 05/08/2014 02:23 PM
Frame 3 is disgusting  >:(

I can't do anything with it (disclaimer: I have no idea what I'm doing)
Agreed. It seems to be totally random noise.
yeah i think its just the fact that it has so few bytes, its got 8927. while the best picture so far (iframe8) has 20428. i think its interesting that iframe 6 has the at most 29622 but so far looks just as garbled as others. might be the one with most room for improvement?
Can it be that due to bit errors the size of Iframe3 is not correctly determined and part of IF3 is interpreted to be part of the next frame?

Cheers
Shanuson
Possible. Possibly with some better work on the transport stream that can be determined to be the case or ruled out. The try1.ts file's iframe3 is a bit larger than the others at 9005 bytes.

Been rather busy this week, I'm going to try to finish the todo list for my program this weekend then start in on a fresh reconstruction of the raw.ts, no guarantees it will yield much but I'm hopeful. 
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/08/2014 02:33 PM
If it was that simple and at no additional cost (like increase in b/w requirements) I imagine they'd already have done that. After all, even the video on the way up breaks up fairly badly at times.

If they'd turn off interlacing they'd get a big boost to compressibility. Compressing interlaced video at the frame level rather than field level gives you nightmare terrible compression.

I'd guess its because they got some stock "space rated" (super old technology) cameras that are old analog NTSC cameras and they transcode that to a digital video signal for downlink.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/08/2014 02:44 PM
ASIDE: Watching all this amateur Kremlinology on some badly-corrupted individual video frames reminds me somewhat unpleasantly of watching the internets go nuts over similarly-confusing bits of video and tidbits of information following STS-107. I can see why SpaceX didn't rush to release the video immediately. I sincerely hope there is better video (preferably from observation ships/aircraft) following Saturday's launch.

I think you're overreacting. The "kremlinology" you deride is in reality a genuine and heartfelt cooperative effort among enthusiastic and obviously highly-capable fans of SpaceX to suss out the precisely correct layout of each iframe, not just some pointless exercise in conspiracy-theory grandstanding. The "secretive organization" and the "limited data points" really do exist, in the form of the scrambled and uncharted bits of each iframe. If there is, in reality, a smudge or dent on the lens or a dent in the leg, then that provides important reference information for untangling the rest of the iframes. If you go grab a cup of coffee and then take another look at the astounding progress that has been made so far, I'm sure you'll feel more pleasant about it.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/08/2014 02:51 PM
iframe 13 from try1:

40:1:-1,0:10:-1:0:-10,1:10:-1:,4:10:49027,8:29:-1

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: AncientU on 05/08/2014 02:54 PM
iframe 13 from try1:

40:1:-1,0:10:-1:0:-10,1:10:-1:,4:10:49027,8:29:-1

Semi-submerged legs?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: fatdeeman on 05/08/2014 03:02 PM
iframe 13 from try1:

40:1:-1,0:10:-1:0:-10,1:10:-1:,4:10:49027,8:29:-1

Oh wow look at that! 
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: ugordan on 05/08/2014 03:02 PM
If they'd turn off interlacing they'd get a big boost to compressibility. Compressing interlaced video at the frame level rather than field level gives you nightmare terrible compression.

Yes, this is actually something I've been whining about ever since they switched to this method (CRS-2 I think) and the video is still 15 fps interlaced. I'm not getting my hopes up anymore.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Joffan on 05/08/2014 03:04 PM
iframe 13 from try1:

40:1:-1,0:10:-1:0:-10,1:10:-1:,4:10:49027,8:29:-1


Looks like the stage moved to the right, compared to the initial splash?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/08/2014 03:16 PM
iframe 13 from try1:

40:1:-1,0:10:-1:0:-10,1:10:-1:,4:10:49027,8:29:-1


Looks like the stage moved to the right, compared to the initial splash?

Hard to tell, remember that color and "brightness" of the blocks are dependent on nearby blocks, often unpredictably so, when there are errors. You should only rely on the texture patterns themselves for determining information here IMO.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Elmar Moelzer on 05/08/2014 03:19 PM
Yeah, interlacing is a terrible idea in pretty much any case, especially in the digital age. I was appalled when they decided to continue that for the HD standard(s).
Definitely bad for compression and when you are watching it on a computer.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Lars_J on 05/08/2014 03:24 PM

But then after looking at the emerging details of frames 7 and 8, I'm beginning to fear your suspicions may be correct that they are not bugged bits but rather a big gnarly dent in the leg. 

Hi, I think it might be some sort of splodge of high refractive index stuff on the camera bending light around it (see attached image: red circle in same place on the two frames, aligned them using the black mark above it) Its an oval region without any internal texture (ie blurred) in all other frames AFAICS, bordering another dirty mark, and still appears to be there in frame 12 where the leg is underwater. I reckon the leg was probably OK until hitting the water, but it would be nice to get more clear images of the "splodge".

Great work guys, although I don't understand a word of what you are talking about re: video decoding !!

Yes, I think the "dirt on lens" is the more likely explanation.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mmeijeri on 05/08/2014 03:28 PM
I'd guess its because they got some stock "space rated" (super old technology) cameras that are old analog NTSC cameras and they transcode that to a digital video signal for downlink.

If they're using a COTS encoder then they can likely turn off interlacing and maybe enable some resilience features. That either means they're not using a COTS encoder, or getting video footage wasn't a priority before.

Having worked for Siqura BV, I naturally recommend their EVE (http://www.eve.nl/) line of encoders. :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/08/2014 03:44 PM
More work on iframe8. Will post the mmb later.

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/08/2014 03:48 PM
More work on iframe8. Will post the mmb later.



Super impressive.

Now a quick task for anyone able.

Can we create a .gif, showing the original frame8 and the one above, with before and after written on them?

The .gif needs to be 350 pixels wide and whatever you require in length and then I'll use that in the article I'm going to put on shorty.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/08/2014 03:48 PM
This is a good point, and it got me thinking. If everything goes as planned, SpaceX will launch again in a few days, and I think they're planning to attempt recovery again. This could upstage the (arguably more historic) first landing video. We need to put together a video with what we've got today, tomorrow at the lastest. If necessary, I can hand-assemble something from the iframes and pieces of the pframes. It wouldn't be as pretty as what arnezami and mlindner can do, so I'll wait until I hear from them.

It's actually a lot easier than that. If no one else does it by Friday Eastern time then I'll be modifying ffmpeg to add a frame specifier to the mmb options. Then we simply pile all the mmb options into one long entry and run ffmpeg with it. viola, produced video. This would also allow us to try tweaking some p-frames if we so wish.

Producing a video now with what we have won't be very attractive yet. We need to fine tune a lot of the frames still. I'm doing that now for iframe8, which should hopefully look really really good. This means doing things like whenever you use -1 and replace the block, fine tune all the luma and chroma settings for it to have the least effect on surrounding blocks.

Are you still planning on doing the frame specifier modifications tonight? Right now we have mods for two different video streams. Are you going to be able to integrate those, or do we need to start a porting effort to a newer reconstruction?

I think it is likely that it could take weeks or months to get the video to a level of quality that is going to satisfy everyone working on it. Chris, can you ask PAO what timeframe would be most useful to them?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Zach121k on 05/08/2014 03:48 PM
I just started reading this thread today, and I want to say... this all looks beautiful.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SpaceX_MS on 05/08/2014 03:56 PM

I think it is likely that it could take weeks or months to get the video to a level of quality that is going to satisfy everyone working on it. Chris, can you ask PAO what timeframe would be most useful to them?

You guys are ahead of the game, the only real game in town actually. Provide what you can, when you can. This is an unbelievable effort on show here.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/08/2014 04:01 PM
This is a good point, and it got me thinking. If everything goes as planned, SpaceX will launch again in a few days, and I think they're planning to attempt recovery again. This could upstage the (arguably more historic) first landing video. We need to put together a video with what we've got today, tomorrow at the lastest. If necessary, I can hand-assemble something from the iframes and pieces of the pframes. It wouldn't be as pretty as what arnezami and mlindner can do, so I'll wait until I hear from them.

It's actually a lot easier than that. If no one else does it by Friday Eastern time then I'll be modifying ffmpeg to add a frame specifier to the mmb options. Then we simply pile all the mmb options into one long entry and run ffmpeg with it. viola, produced video. This would also allow us to try tweaking some p-frames if we so wish.

Producing a video now with what we have won't be very attractive yet. We need to fine tune a lot of the frames still. I'm doing that now for iframe8, which should hopefully look really really good. This means doing things like whenever you use -1 and replace the block, fine tune all the luma and chroma settings for it to have the least effect on surrounding blocks.

Are you still planning on doing the frame specifier modifications tonight? Right now we have mods for two different video streams. Are you going to be able to integrate those, or do we need to start a porting effort to a newer reconstruction?

I think it is likely that it could take weeks or months to get the video to a level of quality that is going to satisfy everyone working on it. Chris, can you ask PAO what timeframe would be most useful to them?

The mods I described would probably only support a single transport stream and changes to iframes in that transport stream. Ideally people can port their changes to try1.ts as it has more data in the frames. If not, I'm not too worried about just waiting until after the next flight before getting something out unless someone wants to work on modifying the iframes with the changes and combining them with the transport streams that way.

I won't get the changes done tonight (Eastern) will probably have to do it tomorrow.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/08/2014 04:02 PM

I think it is likely that it could take weeks or months to get the video to a level of quality that is going to satisfy everyone working on it. Chris, can you ask PAO what timeframe would be most useful to them?

You guys are ahead of the game, the only real game in town actually. Provide what you can, when you can. This is an unbelievable effort on show here.

Awesome. Thanks! Glad we could be of use. I hope this helps SpaceX to consider releasing more information to the knowledgable public in the future! We're always ready to get more candy, even if it requires some cleaning first.

The world is on SpaceX's side even if a few companies/corporations/governments aren't.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: ugordan on 05/08/2014 04:11 PM
Can we create a .gif, showing the original frame8 and the one above, with before and after written on them?

The .gif needs to be 350 pixels wide and whatever you require in length and then I'll use that in the article I'm going to put on shorty.

Something like this?

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/08/2014 04:12 PM
Can we create a .gif, showing the original frame8 and the one above, with before and after written on them?

The .gif needs to be 350 pixels wide and whatever you require in length and then I'll use that in the article I'm going to put on shorty.

Something like this?



Something like that! :)

Splendid. Thanks much!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/08/2014 04:41 PM
Ok, so this thread is now fully linked up on the news site via the Static Fire article - in which I added some paras to the end per the video repair effort.

http://www.nasaspaceflight.com/2014/05/spacex-falcon-9-static-fire-test/

Had to keep it light, given the wider audience on the news site, but linked the thread and linked Arnezami's great post on how this process is taking place.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: AncientU on 05/08/2014 04:54 PM
Nice article Chris!
The ongoing video recovery effort on NSF is wonderful and the subject even more so.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: grythumn on 05/08/2014 05:08 PM
Another day, another cygwin build. This is from today's git and adds the -2 parameter to mmb. As usual, only grab the DLLs if you don't already have them.

-Bob
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: dorkmo on 05/08/2014 05:15 PM
frame 5 so far

0:0:-1,26:12:35375,12:13:-1,16:13:38823,17:13:-1,19:13:39153,22:13:-1,27:13:40118,29:13:-1,33:13:40830,0:15:50056

2 or 3 clear blocks of leg. some of the stage, a row of it is miss aligned.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: John_L on 05/08/2014 05:32 PM
Rampant speculation about something "damaged" when it could as easily be a video artifact or misaligned video data block is why Elon and SpaceX are leary about releasing such things in the first place. To me the anamoly it looks like it has right angled edges which align with the macro blocks.  Let's not get too crazy about this yet.

What gives you the impression that SpaceX and/or Elon Musk "are leary" about releasing the video.  They specifically released the video to the public in the hopes that someone (like this band of forum members) would take up the challenge of reconstructing the video.  Alot of us have been working hard to learn mpeg decoding better, have worked directly with the curator of FFMpeg to get new functionality created within FFMpeg, creating new tools on the fly so we can edit parts of the video that have never been editable before. 

I seriously doubt that a few people speculating about the video in a forum that's read mainly by it's members, and, if I may add, SpaceX themselves, would be damaging, and, they are also following our work closely, and are happy that we are putting for the effort for nothing more than the enjoyment we're getting from positive results, and the desire for us to HELP SpaceX gain extremely useful data regarding the last launches attempt to land the stage.

We're all SpaceX fans here...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/08/2014 05:39 PM

I think it is likely that it could take weeks or months to get the video to a level of quality that is going to satisfy everyone working on it. Chris, can you ask PAO what timeframe would be most useful to them?

You guys are ahead of the game, the only real game in town actually. Provide what you can, when you can. This is an unbelievable effort on show here.

This is truly an d unbelievable effort. And is amazing to see the amount of details that have been recovered form the initial frames, and the conviction and talent of all people that have been involved....

This is what we have so far guys! And let's keep with the good job:


Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Swatch on 05/08/2014 05:44 PM
Chris, may I recommend you add links to the first post of this thread detailing how to start helping with this effort and some of the better posts in the thread describing what's going on?  Also, a link to the wikipage they setup?

I think it will help orient people without going through 24 pages of stuff.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: John_L on 05/08/2014 05:46 PM
I don't know how much "better" such video would be. On another site I saw a claim that the telemetry plane on CRS-3 was over 20 miles away. The CASSIOPE launch aerial images released also suggest a pretty large distance from the vehicle. I guess safety trumps closeup observations.

There was really bad weather, so ships and planes that were slotted to be much closer to the "action" weren't able to get there for safety reasons.  If this were a NASA launch, NASA would have likely had a mission rule stating that they wouldn't launch due to the fact that the weather (at the stage landing site) was bad, and might have scrubbed the launch.  In this case however, NASA is SpaceX's customer, and SpaceX is using the launch to test the recovery.  NASA's objective here is to get goods up to the ISS, so SpaceX had to go with things as they were, and hoped they could what data they could.  Bad weather at the 1st stage landing site wasn't going to stop the launch, but it did prevent planes and ships from getting close enough to view/better record the data.

The weather this Saturday will hopefully be better so that they can get close enough to A) record video directly from a 3rd person perspective (planes or ships or both), and B) get better data recorded from the stage itself.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: AncientU on 05/08/2014 05:52 PM

I think it is likely that it could take weeks or months to get the video to a level of quality that is going to satisfy everyone working on it. Chris, can you ask PAO what timeframe would be most useful to them?

You guys are ahead of the game, the only real game in town actually. Provide what you can, when you can. This is an unbelievable effort on show here.

This is truly an d unbelievable effort. And is amazing to see the amount of details that have been recovered form the initial frames, and the conviction and talent of all people that have been involved....

This is what we have so far guys! And let's keep with the good job:

Just curious...
The original releases (raw and 'repaired') had frames as the stage was descending through the cloud layer which fit before this sequence, and the stage falling over at the end which come after -- both best seen in raw IMO.  Are those frames somewhere in this amazing recovery effort?

Link back to OP:
http://forum.nasaspaceflight.com/index.php?topic=34597.msg1191316#msg1191316
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jakusb on 05/08/2014 05:52 PM


I think it is likely that it could take weeks or months to get the video to a level of quality that is going to satisfy everyone working on it. Chris, can you ask PAO what timeframe would be most useful to them?

You guys are ahead of the game, the only real game in town actually. Provide what you can, when you can. This is an unbelievable effort on show here.

This is truly an d unbelievable effort. And is amazing to see the amount of details that have been recovered form the initial frames, and the conviction and talent of all people that have been involved....

This is what we have so far guys! And let's keep with the good job:
Great! I was about to ask for something like this. Or make it myself. :)
One suggestion/request: a bit slower sequence? Either more realistic in terms of actual movie speed or even slower to be able to find common blocks?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/08/2014 05:59 PM
Chris, may I recommend you add links to the first post of this thread detailing how to start helping with this effort and some of the better posts in the thread describing what's going on?

I think it will help orient people without going through 24 pages of stuff.

I'd rather people read through what is only 24 pages. It takes 5-10 minutes to read and new people to the thread will gain an appreciation of the work and the processes involved. :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/08/2014 06:06 PM


I think it is likely that it could take weeks or months to get the video to a level of quality that is going to satisfy everyone working on it. Chris, can you ask PAO what timeframe would be most useful to them?

You guys are ahead of the game, the only real game in town actually. Provide what you can, when you can. This is an unbelievable effort on show here.

This is truly an d unbelievable effort. And is amazing to see the amount of details that have been recovered form the initial frames, and the conviction and talent of all people that have been involved....

This is what we have so far guys! And let's keep with the good job:
Great! I was about to ask for something like this. Or make it myself. :)
One suggestion/request: a bit slower sequence? Either more realistic in terms of actual movie speed or even slower to be able to find common blocks?

Sure. How about this:

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/08/2014 06:17 PM
I began to rebuild the iframe 6 (normal version); the MB containing the legs are visible but lots of the neighboring blocks are corrupted, so the reconstruction is not easy. I am not completely sure of the exact position of 2-3 MB sequences over the water. Maybe it works better with try1.ts?

Is it me or are the legs still not completely deployed? It seems not to be exactly the same as in the following frames.

0:0:-1,
2:1:6129,17:1:-1,
1:02:11561,6:3:-1
3:12:56455,18:12:-1,
26:12:61670,27:12:-1,16:13:-1:50,
17:13:64694,28:13:-1,
32:13:71022,4:14:-1,
27:14:78837,5:15:-1,
19:15:92898,23:15:-1,20:16:-1:0:0:0:-12,22:16:-1,
21:17:106645,27:18:-1,29:18:-1:0:0:0:-25,30:18:-1,
0:19:121762,04:21:-1:0:0:0:20,05:21:-1,
31:22:159772,0:24:-1,
30:25:184570,07:27:-1,
35:27:200466

(edit: top of frame added)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mme on 05/08/2014 06:40 PM
...
Just curious...
The original releases (raw and 'repaired') had frames as the stage was descending through the cloud layer which fit before this sequence, and the stage falling over at the end which come after -- both best seen in raw IMO.  Are those frames somewhere in this amazing recovery effort?

Link back to OP:
http://forum.nasaspaceflight.com/index.php?topic=34597.msg1191316#msg1191316
The the wiki (http://spacexlanding.wikispaces.com) has a column for a frame 0 that reads:
Quote
#IFPosition in fileSize in bytesFrame #mp4-img linkimageContributors
0~254956n/an/anot extracted yet, transport stream issue?
I'm not involved, and knew nothing about MPEG a few days ago. But for the early frames, I guess there is an issue in extracting the data for that i-frame. Of course I don't know how it's playable at all if that's the case.

As for the falling over, the repair effort is on the i-frames from which all subsequent frames are deltas. So I assume the "falling over" is after the last i-frame.  Assuming the i-frames can be reconstructed, the playback of the falling over will be better.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/08/2014 06:48 PM
frame 5 so far

0:0:-1,26:12:35375,12:13:-1,16:13:38823,17:13:-1,19:13:39153,22:13:-1,27:13:40118,29:13:-1,33:13:40830,0:15:50056

2 or 3 clear blocks of leg. some of the stage, a row of it is miss aligned.

The rocket is 2 MB rows to far up compared to IF7 or 8
otherwise nice pieces of the leg.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Swatch on 05/08/2014 07:12 PM
Chris, may I recommend you add links to the first post of this thread detailing how to start helping with this effort and some of the better posts in the thread describing what's going on?

I think it will help orient people without going through 24 pages of stuff.

I'd rather people read through what is only 24 pages. It takes 5-10 minutes to read and new people to the thread will gain an appreciation of the work and the processes involved. :)

Maybe just a link to the wiki page then?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: MarsInMyLifetime on 05/08/2014 07:25 PM
Chris, may I recommend you add links to the first post of this thread detailing how to start helping with this effort and some of the better posts in the thread describing what's going on?

I think it will help orient people without going through 24 pages of stuff.

I'd rather people read through what is only 24 pages. It takes 5-10 minutes to read and new people to the thread will gain an appreciation of the work and the processes involved. :)

Maybe just a link to the wiki page then?
I think he's saying that if you read the pages, you'll come across the link anyway, but by then you'll have much better context and appreciation for the incredible effort going on. As a Web publisher myself, I have to keep in mind (and be reminded) that I want to try to keep visitors ON my site for as long as I can. Besides, the best exhilaration is when you keep coming across ever new news here, right?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: ugordan on 05/08/2014 07:33 PM
I don't know how much "better" such video would be. On another site I saw a claim that the telemetry plane on CRS-3 was over 20 miles away. The CASSIOPE launch aerial images released also suggest a pretty large distance from the vehicle. I guess safety trumps closeup observations.

There was really bad weather, so ships and planes that were slotted to be much closer to the "action" weren't able to get there for safety reasons. 

SpaceX couldn't have possibly asked for *better* weather for the CASSIOPE launch and, yet, the photography of the 1st stage was still very distant and blurry.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Swatch on 05/08/2014 07:55 PM
...
Maybe just a link to the wiki page then?
I think he's saying that if you read the pages, you'll come across the link anyway, but by then you'll have much better context and appreciation for the incredible effort going on. As a Web publisher myself, I have to keep in mind (and be reminded) that I want to try to keep visitors ON my site for as long as I can. Besides, the best exhilaration is when you keep coming across ever new news here, right?

Duly noted... but I was trying to be lazy and have had trouble finding the link again for myself with it buried in the thread. :P 

Can't always spend a lot of time digging through the thread every time I want to see the updated pics when  there are more rockets to build!  ;)  Things are hectic around here, but it's amazing following the stuff you guys are doing with this video.  :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/08/2014 08:28 PM
Hi guys,

This is just for inspiration:

https://www.youtube.com/watch?v=G9933rIJEVw

"We've always defined ourselves by the ability to overcome the impossible"

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: CraigLieb on 05/08/2014 08:48 PM
Hi guys,

This is just for inspiration:

"We've always defined ourselves by the ability to overcome the impossible"

Regards,

arnezami

Terrific video.

And also useful to note the side view of the rocket descending into the smoke,
 much like the stage returning as it hits the surface of the water.

I suppose this is obvious to the rocket scientists that we would see just parts of the struts.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: grythumn on 05/08/2014 08:51 PM
More cleanup of iframe 9. These are from try1, but not much changed at least in the top of the frame...

Now that iframe 8 is almost done, pulled the clock position from it and set the top of this clock to that... gives us a reference point for the bottom of the frame.

X:16410:01,X:19164:10,X:19903:1,X:20037:80,X:27244:1,X:32743:1,X:32551:80,20:04:-1,
24:04:16023,16:5:-1,17:05:18692,24:08:-1,25:08:32410,1:15:-1,3:16:79127,41:19:-1,8:20:96220,
15:04:16023,4:28:112147

Bonus animation from when I was testing individual fixes against try1.ts from the old frame.

-Bob
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: John_L on 05/08/2014 08:59 PM
Just curious...
The original releases (raw and 'repaired') had frames as the stage was descending through the cloud layer which fit before this sequence, and the stage falling over at the end which come after -- both best seen in raw IMO.  Are those frames somewhere in this amazing recovery effort?

The answer is yes and no...

MPEG encoding has different types of frames... I-Frames, which contain All of the information needed to reconstruct that frame, the other types B and P frames only contain partial information, just information about what's changed since the last i-frame.  In our example here, we have a 30 second video with about 307 frames (I think), and of those 307 frames, only about a dozen or so are I-Frames.  Those are the frames we are working on, because the rest of the frames will be generated from data within the I-Frames. A given block of pixels on the screen can rely on blocks before it, after it, or from the I-Frame that was several frames back.

When you see us talking about frame 1, frame 2, frame 3, etc, we're not talking about the first 3 frames of the video, we're talking about just the iframes.  There are (by my guess) about 20-25 intermediate frames between each iframe.  We'll be getting to those once the iframes are fixed, at that point, we'll re-encode the video replacing the bad iframes with the good ones, and once the video is re-encoded, the intermediate frame data will be regenerated from the repaired iframes, and hopefully we'll have a much cleaner video.

Moral of the story... MPEG cheats... ALOT !

I tried not to get too technical with the explanation, hope that clears it up for you though.

John
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/08/2014 09:00 PM
More cleanup of iframe 9. These are from try1, but not much changed at least in the top of the frame...

Now that iframe 8 is almost done, pulled the clock position from it and set the top of this clock to that... gives us a reference point for the bottom of the frame.

X:16410:01,X:19164:10,X:19903:1,X:20037:80,X:27244:1,X:32743:1,X:32551:80,20:04:-1,
24:04:16023,16:5:-1,17:05:18692,24:08:-1,25:08:32410,1:15:-1,3:16:79127,41:19:-1,8:20:96220,
15:04:16023,4:28:112147

Bonus animation from when I was testing individual fixes against try1.ts from the old frame.

-Bob

One key thing to note about the clock. The blocks for it are abnormally large so corruption chance increases. They average around 1000 bits each for me.

Also, every timestamp in the video is of the form: 19:35:XX.XX
Importantly, the lower portion of the colons are entirely in the lower two sub-blocks of the bottom row and the upper portions are entirely in the lower two sub-blocks of the second from bottom row. Further, for the "." character its slightly wider than the lower portion of the ":" meaning you can recognize where to put it easier.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Alpha Control on 05/08/2014 09:02 PM


I think it is likely that it could take weeks or months to get the video to a level of quality that is going to satisfy everyone working on it. Chris, can you ask PAO what timeframe would be most useful to them?

You guys are ahead of the game, the only real game in town actually. Provide what you can, when you can. This is an unbelievable effort on show here.

This is truly an d unbelievable effort. And is amazing to see the amount of details that have been recovered form the initial frames, and the conviction and talent of all people that have been involved....

This is what we have so far guys! And let's keep with the good job:
Great! I was about to ask for something like this. Or make it myself. :)
One suggestion/request: a bit slower sequence? Either more realistic in terms of actual movie speed or even slower to be able to find common blocks?

Sure. How about this:

Moralec, that's terrific. Thanks for adding the frame numbers and slowing down the playback.

Really great summary of where this amazing effort is at this point! It's not my field, but it is wonderful to follow (if only at a high level) the great progress everyone is making. 
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/08/2014 09:07 PM
Hi guys,

This is just for inspiration:

"We've always defined ourselves by the ability to overcome the impossible"

Regards,

arnezami

Terrific video.

And also useful to note the side view of the rocket descending into the smoke,
 much like the stage returning as it hits the surface of the water.

I suppose this is obvious to the rocket scientists that we would see just parts of the struts.

The only problem is that in iframe 13, you can see both being the same size again....


Weird....

http://spacexlanding.wikispaces.com/
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Asmegin on 05/08/2014 09:07 PM
A bit more of the Frame 9 clock;

5:28:110675,6:29:120015,4:29:117503
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: AncientU on 05/08/2014 09:08 PM
Just curious...
The original releases (raw and 'repaired') had frames as the stage was descending through the cloud layer which fit before this sequence, and the stage falling over at the end which come after -- both best seen in raw IMO.  Are those frames somewhere in this amazing recovery effort?

The answer is yes and no...

MPEG encoding has different types of frames... I-Frames, which contain All of the information needed to reconstruct that frame, the other types B and P frames only contain partial information, just information about what's changed since the last i-frame.  In our example here, we have a 30 second video with about 307 frames (I think), and of those 307 frames, only about a dozen or so are I-Frames.  Those are the frames we are working on, because the rest of the frames will be generated from data within the I-Frames. A given block of pixels on the screen can rely on blocks before it, after it, or from the I-Frame that was several frames back.

When you see us talking about frame 1, frame 2, frame 3, etc, we're not talking about the first 3 frames of the video, we're talking about just the iframes.  There are (by my guess) about 20-25 intermediate frames between each iframe.  We'll be getting to those once the iframes are fixed, at that point, we'll re-encode the video replacing the bad iframes with the good ones, and once the video is re-encoded, the intermediate frame data will be regenerated from the repaired iframes, and hopefully we'll have a much cleaner video.

Moral of the story... MPEG cheats... ALOT !

I tried not to get too technical with the explanation, hope that clears it up for you though.

John
Thanks for the generous explanation and all effort on this reconstruction.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: John_L on 05/08/2014 09:09 PM
Interesting video about how MPEG works...

https://www.youtube.com/watch?v=P7abyWT4dss

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: AncientU on 05/08/2014 09:10 PM
Hi guys,

This is just for inspiration:

"We've always defined ourselves by the ability to overcome the impossible"

Regards,

arnezami

Terrific video.

And also useful to note the side view of the rocket descending into the smoke,
 much like the stage returning as it hits the surface of the water.

I suppose this is obvious to the rocket scientists that we would see just parts of the struts.

The only problem is that in iframe 13, you can see both being the same size again....


Weird....

http://spacexlanding.wikispaces.com/
The stage has bobbed back up to the surface (mostly) and just has the thrust structure underwater.
The thin 'legs' are actually just the pistons.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/08/2014 09:29 PM
Hi guys,

This is just for inspiration:

"We've always defined ourselves by the ability to overcome the impossible"

Regards,

arnezami

Terrific video.

And also useful to note the side view of the rocket descending into the smoke,
 much like the stage returning as it hits the surface of the water.

I suppose this is obvious to the rocket scientists that we would see just parts of the struts.

The only problem is that in iframe 13, you can see both being the same size again....


Weird....

http://spacexlanding.wikispaces.com/
The stage has bobbed back up to the surface (mostly) and just has the thrust structure underwater.
The thin 'legs' are actually just the pistons.

Yep, that's possible...

The answer lies in the b and p frames :P
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/08/2014 09:34 PM
Ocean from iframe 6 (try1). Unfortunately I have yet to think of a straightforward way to convert between the two framesets. So I can't combine this with other peoples progress. A chain file to tell which bit ranges in original frames correspond to which ranges in try1 would be of great value.
The data in this frame seems to be in fairly good condition. I think most of the frame can be perfectly recovered.

mmb 0:0:583,39:0:5496,43:0:-1,2:1:6129,21:1:8584,22:1:8796,6:3:
18105,7:3:18276,9:3:29527,30:3:32359,31:3:-1,43:3:33591,2:6:44738,4:6:46341,5:6:48794
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: MTom on 05/08/2014 09:51 PM
Hi guys,

This is just for inspiration:

"We've always defined ourselves by the ability to overcome the impossible"

Regards,

arnezami

Terrific video.

And also useful to note the side view of the rocket descending into the smoke,
 much like the stage returning as it hits the surface of the water.

I suppose this is obvious to the rocket scientists that we would see just parts of the struts.

The only problem is that in iframe 13, you can see both being the same size again....


Weird....

http://spacexlanding.wikispaces.com/

IMHO:
On frames 10-11-12 you see nearly anything about the legs because of the vapor, likely not because the legs are still partially under the water.
On frame 13 the pistons are much clearer to see - the vapor disappeared.
About from this point are the legs only partially under water (or a little bit earlier, while the rocket should cause also a big wave - it has about 4.5m diameter - this delays also a little bit that the pistons go under water).


Edit: the explanation from AncientU is also logical - the video will hopefully give an answer.

Edit2: If I see the timeline, frame13 is min. 3 seconds after engine shut down. For my theory this is too long time (and in 8 seconds it went horizontal)...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: grythumn on 05/08/2014 09:54 PM
More cleanup of iframe 9. These are from try1, but not much changed at least in the top of the frame...

Now that iframe 8 is almost done, pulled the clock position from it and set the top of this clock to that... gives us a reference point for the bottom of the frame.

X:16410:01,X:19164:10,X:19903:1,X:20037:80,X:27244:1,X:32743:1,X:32551:80,20:04:-1,
24:04:16023,16:5:-1,17:05:18692,24:08:-1,25:08:32410,1:15:-1,3:16:79127,41:19:-1,8:20:96220,
15:04:16023,4:28:112147

Bonus animation from when I was testing individual fixes against try1.ts from the old frame.

-Bob

One key thing to note about the clock. The blocks for it are abnormally large so corruption chance increases. They average around 1000 bits each for me.

Also, every timestamp in the video is of the form: 19:35:XX.XX
Importantly, the lower portion of the colons are entirely in the lower two sub-blocks of the bottom row and the upper portions are entirely in the lower two sub-blocks of the second from bottom row. Further, for the "." character its slightly wider than the lower portion of the ":" meaning you can recognize where to put it easier.

Add ,3:29:118925 for the damaged bottom portion of the clock. The 9 comes and goes, it's pretty damaged.

-Bob
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/08/2014 10:02 PM
For those that aren't aware there's a new improved online editor that has all the UI for the various options and the ability to use different sources ("coerced.ts", "try1.ts") you can get to it via the wiki here http://spacexlanding.wikispaces.com/ (Online Editor v2)

If anyone has any other .ts they are using I can upload that too, just let me know (I just need the .mp4-img files and a name for it)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/08/2014 10:35 PM
Ok....peer pressure! ;D

So if someone wants to take my Opening Post, throw in some links like Iain's Wiki page (which looks a lot better now) and maybe some highlight links to good posts throughout the thread, and put it all together.....send me the updated post via PM so I can edit it in.

And apologies to Swatch seen as it was his idea...... :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/08/2014 10:45 PM
Could someone write a detailed instruction how to update the table with the images in the wiki http://spacexlanding.wikispaces.com/home without messing everything?

Or could someone simply put the latest image versions of:
- iframe 1 (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195314#msg1195314)
- iframe 4 (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195103#msg1195103)
- iframe 5 (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195546#msg1195546)
- iframe 6 (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195584#msg1195584 and/or http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195704#msg1195704)
- iframe 8 (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195321#msg1195321 and http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195464#msg1195464)
- iframe 9 (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195709#msg1195709)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Untribium on 05/08/2014 10:48 PM
For those that aren't aware there's a new improved online editor that has all the UI for the various options and the ability to use different sources ("coerced.ts", "try1.ts") you can get to it via the wiki here http://spacexlanding.wikispaces.com/ (Online Editor v2)

If anyone has any other .ts they are using I can upload that too, just let me know (I just need the .mp4-img files and a name for it)

nice work! now I no longer have to make sure I don't accidentally modify the same block twice :D One thing though: I think it would make more sense to sort the modified blocks by y and then x as opposed to the other way around, since this is the order in which they are traversed
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/08/2014 11:00 PM
For those that aren't aware there's a new improved online editor that has all the UI for the various options and the ability to use different sources ("coerced.ts", "try1.ts") you can get to it via the wiki here http://spacexlanding.wikispaces.com/ (Online Editor v2)

If anyone has any other .ts they are using I can upload that too, just let me know (I just need the .mp4-img files and a name for it)

nice work! now I no longer have to make sure I don't accidentally modify the same block twice :D One thing though: I think it would make more sense to sort the modified blocks by y and then x as opposed to the other way around, since this is the order in which they are traversed

Done :) Thanks for the feedback
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Asmegin on 05/08/2014 11:11 PM
For those that aren't aware there's a new improved online editor that has all the UI for the various options and the ability to use different sources ("coerced.ts", "try1.ts") you can get to it via the wiki here http://spacexlanding.wikispaces.com/ (Online Editor v2)

If anyone has any other .ts they are using I can upload that too, just let me know (I just need the .mp4-img files and a name for it)

Would some sort of toggle to highlight modified macroblocks be useful here? I find myself getting a bit confused sometimes (although my knowledge level is 1/10 so it may just be me)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/08/2014 11:13 PM
Looks like the news site gave this thread a bit of a boost. Gone from 67,000 to 82,000 views in 12 hours.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/08/2014 11:15 PM
For those that aren't aware there's a new improved online editor that has all the UI for the various options and the ability to use different sources ("coerced.ts", "try1.ts") you can get to it via the wiki here http://spacexlanding.wikispaces.com/ (Online Editor v2)

If anyone has any other .ts they are using I can upload that too, just let me know (I just need the .mp4-img files and a name for it)

Would some sort of toggle to highlight modified macroblocks be useful here? I find myself getting a bit confused sometimes (although my knowledge level is 1/10 so it may just be me)

A mouseover pop-up for which block you are hovering over might be better (you can compare it to your list of changes anyway).
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: dorkmo on 05/08/2014 11:34 PM
frame 5
0:0:-1,27:10:24142,17:12:30040,25:12:-1,18:13:39153,20:13:-1,25:13:35375,29:13:-1,27:15:47754,3:16:50034

added a few things from Shanuson's
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/09/2014 12:08 AM
Could someone write a detailed instruction how to update the table with the images in the wiki http://spacexlanding.wikispaces.com/home without messing everything?

Or could someone simply put the latest image versions of:
- iframe 1 (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195314#msg1195314)
- iframe 4 (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195103#msg1195103)
- iframe 5 (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195546#msg1195546)
- iframe 6 (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195584#msg1195584 and/or http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195704#msg1195704)
- iframe 8 (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195321#msg1195321 and http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195464#msg1195464)
- iframe 9 (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195709#msg1195709)

Very simple thing to do.
1. click on edit on top.
2. Scroll down to the table, and find the frame you want to update.
3. Manually delete the old image
4. Add a new image (there is a button for that).  There is no need to upload anything, just make a reference to the image that is stored here in nsf.
5. Add the name of the author and new code.
6. Click save.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/09/2014 12:09 AM
frame 5
0:0:-1,27:10:24142,17:12:30040,25:12:-1,18:13:39153,20:13:-1,25:13:35375,29:13:-1,27:15:47754,3:16:50034

added a few things from Shanuson's

Very nice. Don't forget to add it to the wiki!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: CraigLieb on 05/09/2014 12:30 AM
Hi guys,

This is just for inspiration:

"We've always defined ourselves by the ability to overcome the impossible"

Regards,

arnezami

Terrific video.

And also useful to note the side view of the rocket descending into the smoke,
 much like the stage returning as it hits the surface of the water.

I suppose this is obvious to the rocket scientists that we would see just parts of the struts.

The only problem is that in iframe 13, you can see both being the same size again....


Weird....

http://spacexlanding.wikispaces.com/

Waves pass,  how much time between iframes?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/09/2014 12:40 AM
Hi guys,

This is just for inspiration:

"We've always defined ourselves by the ability to overcome the impossible"

Regards,

arnezami

Terrific video.

And also useful to note the side view of the rocket descending into the smoke,
 much like the stage returning as it hits the surface of the water.

I suppose this is obvious to the rocket scientists that we would see just parts of the struts.

The only problem is that in iframe 13, you can see both being the same size again....


Weird....

http://spacexlanding.wikispaces.com/

Waves pass,  how much time between iframes?

About one second
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: watermod on 05/09/2014 12:55 AM
I haven't done much with mpeg-4 but have done work with other forms of imaging (3D).   To me, one fact that sticks in the mind is that the actual upper part of the rocket should be the same (excepting albedo) from the beginning of the flight  until the end.  That should be able to provide clues or hints to reconstructing the images.  I realize that being a 2D and not a 3D image the albedo is not as easy to factor out of the image.  It would be interesting to have a 3D image of the first stage, layer the good frames early in flight  upon it's surface as an albedo cutting out the sky, ocean and such.  Then factor that back into the MPEG-4 images (both as direct missing bits and sunlight and sun angle hints).  Other then bits like the legs as they disappear into the ocean everything else should be the background sky and direct light on the first stage that is different from the light at launch.  Same for the static dirt on the lens.   It's a hint.  They are all hints to the missing bits.  From those missing bits you might get enough hints to reconstruct the ocean and remaining scenery.
(example of albedo being manipulated as a 3D skin to recover useful information: http://www.engr.uky.edu/~lgh/data/dataGraveStones2010.htm (http://www.engr.uky.edu/~lgh/data/dataGraveStones2010.htm))




Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: seanpg71 on 05/09/2014 12:59 AM
frame 5
0:0:-1,27:10:24142,17:12:30040,25:12:-1,18:13:39153,20:13:-1,25:13:35375,29:13:-1,27:15:47754,3:16:50034

added a few things from Shanuson's


That's helpful and should save me a bit of time.  I've been working from the bottom up on 5.  It's a bit of a mess.  Mostly I've just been trying to fix the bit alignment issues so that a picture shows up so far.  This is as far as I've gotten thus far.

0:0:-1,40:17:56308,43:19:69441,0:21:106970,1:21:76873,11:22:85251,12:22:85489,26:22:86772,
27:22:87056,37:22:87982,38:22:88152,10:23:90705,11:23:91098,16:23:91462,17:23:91682,
22:24:97633,23:24:97869,0:25:100325,0:27:106003,2:27:105388,22:27:110276,30:28:121538,
31:28:121730,1:29:123050,2:29:123876

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Asmegin on 05/09/2014 01:05 AM
frame 5
0:0:-1,27:10:24142,17:12:30040,25:12:-1,18:13:39153,20:13:-1,25:13:35375,29:13:-1,27:15:47754,3:16:50034

added a few things from Shanuson's


That's helpful and should save me a bit of time.  I've been working from the bottom up on 5.  It's a bit of a mess.  Mostly I've just been trying to fix the bit alignment issues so that a picture shows up so far.  This is as far as I've gotten thus far.

0:0:-1,40:17:56308,43:19:69441,0:21:106970,1:21:76873,11:22:85251,12:22:85489,26:22:86772,
27:22:87056,37:22:87982,38:22:88152,10:23:90705,11:23:91098,16:23:91462,17:23:91682,
22:24:97633,23:24:97869,0:25:100325,0:27:106003,2:27:105388,22:27:110276,30:28:121538,
31:28:121730,1:29:123050,2:29:123876

Great start. Added to Wiki
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: dorkmo on 05/09/2014 01:17 AM
frame 5
0:0:-1,27:10:24142,17:12:30040,25:12:-1,18:13:39153,20:13:-1,25:13:35375,29:13:-1,27:15:47754,3:16:50034

added a few things from Shanuson's


That's helpful and should save me a bit of time.  I've been working from the bottom up on 5.  It's a bit of a mess.  Mostly I've just been trying to fix the bit alignment issues so that a picture shows up so far.  This is as far as I've gotten thus far.

0:0:-1,40:17:56308,43:19:69441,0:21:106970,1:21:76873,11:22:85251,12:22:85489,26:22:86772,
27:22:87056,37:22:87982,38:22:88152,10:23:90705,11:23:91098,16:23:91462,17:23:91682,
22:24:97633,23:24:97869,0:25:100325,0:27:106003,2:27:105388,22:27:110276,30:28:121538,
31:28:121730,1:29:123050,2:29:123876

heres my most recent

0:0:-1,27:10:24142,16:12:30040,24:12:-1,17:13:39153, 20:13:-1,25:13:35375,29:13:-1,28:16:47754,0:17:50034, 43:19:-1,0:26:103820,13:27:-1,17:27:109631,25:27:-1, 27:27:110752,28:28:-1,33:28:121771,2:29:124970
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Asmegin on 05/09/2014 01:23 AM
Is anyone working on 3 or 4? Those are the two bad ones. 5 was bad but the last few posts show progress!

I searched Frame 3 for a while but couldn't find anything. Unrecoverable?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: dorkmo on 05/09/2014 01:46 AM
Is anyone working on 3 or 4? Those are the two bad ones. 5 was bad but the last few posts show progress!

I searched Frame 3 for a while but couldn't find anything. Unrecoverable?

judging from whats become of frame 5, i think frame 4 is going to be the most important for getting a nice shot of the leg deployment. and im afraid 3 is a lost cause. :(

i noticed that some are using a different ts file, try1? are we going to be in a world of hurt if we have mixed data or will it be okay?

ps this online editor on the wiki is SPECTACULAR! i wouldnt have been able to figure out doing it the hard way. kudos!


--edit--
added 22:24:97869 and changed some around the bottom:

0:0:-1,27:10:24142,16:12:30040,24:12:-1,17:13:39153,20:13:-1, 25:13:35375,29:13:-1,28:16:47754,0:17:50034,43:19:-1, 22:24:97869,02:27:-1,24:27:110437,13:27:-1,17:27:109631, 25:27:-1,27:27:110752,28:28:-1,33:28:121771,2:29:124970

Edit/Lar: please put spaces in every once in a while so your lines wrap. I think the parms are space agnostic (if they are after the comma but not anywhere else anyway?). thanks
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: grythumn on 05/09/2014 03:44 AM
Minor tweak to iframe13, try1: X:90932:80,40:1:-1,0:10:-1:0:-10,1:10:-1:,4:10:49027,8:29:-1
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: dorkmo on 05/09/2014 03:48 AM
Is anyone working on 3 or 4? Those are the two bad ones. 5 was bad but the last few posts show progress!

I searched Frame 3 for a while but couldn't find anything. Unrecoverable?

frame 4, so far only found a little bit of the left side and maybe a few blocks of the right
00:14:-1,05:01:-1,35:05:-1,19:05:20944,35:05:-1,43:06:27582,0:10:-1,21:13:50333,25:14:57756 ,08:15:61126,36:15:-1,00:17:71044,0:24:-1,00:27:114198

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/09/2014 04:24 AM
Wow just read through the 27 pages and am very impressed.

Trying to work on iframe 4, as a crypto guy, I was thinking of trying to develop a brute-force script to match the 19:35 timestamp. From what I have (quickly) counted, the first four blocks in the bottom row have known data. I don't have mpeg experience, but from what I've gathered from this thread, due to the down-right cascade this can only directly check the first four blocks in each row. AFAIU, corrections in those blocks could potentially help resolve the entire frame.

From the 704x480 PNG output of the custom ffmpeg command (using MMB on frames 6, 11, 12, and 13), the known pixel counts are:

0: 36/576
1: 148/576
2: 130/576
3: 154/576

Total: 468/2304 = ~.203

or about 20% which is huge in terms of a known-plaintext attack. Add in that ffmpeg errors should probably decrease with a valid flip, I think a brute force is computationally feasible. A tremendous speedup would be possible with a heuristic search algorithm, which I assume those of you that have been playing manually probably have a mental version of.

Anyone care to take a swing at describing your process of coming up with the next MMB token given the existing tokens and the next error?



Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/09/2014 05:39 AM
Wow just read through the 27 pages and am very impressed.

Trying to work on iframe 4, as a crypto guy, I was thinking of trying to develop a brute-force script to match the 19:35 timestamp. From what I have (quickly) counted, the first four blocks in the bottom row have known data. I don't have mpeg experience, but from what I've gathered from this thread, due to the down-right cascade this can only directly check the first four blocks in each row. AFAIU, corrections in those blocks could potentially help resolve the entire frame.

From the 704x480 PNG output of the custom ffmpeg command (using MMB on frames 6, 11, 12, and 13), the known pixel counts are:

0: 36/576
1: 148/576
2: 130/576
3: 154/576

Total: 468/2304 = ~.203

or about 20% which is huge in terms of a known-plaintext attack. Add in that ffmpeg errors should probably decrease with a valid flip, I think a brute force is computationally feasible. A tremendous speedup would be possible with a heuristic search algorithm, which I assume those of you that have been playing manually probably have a mental version of.

Anyone care to take a swing at describing your process of coming up with the next MMB token given the existing tokens and the next error?
Hi theshadow27!

Welcome to the forum! :)

What is mainly done is realigning the visual macroblocks to the data. Essentially trying to guess the starting bitposition of a visual block and see if it starts producing a sensible visual block. Apart from that there is also many cases where visual blocks have correct data but this data was meant for a different block: this "simply" needs to be reassigned. But its a lot of work. Due to errors in the data the visual blocks start reading at the wrong bit-position. And this happens for blocks all over the frame (not just in the lower left corner). The clock is usually easy to find and place in the right position (in many of the I-frames they have already been found). But if you have the clock in the right position it won't help you with the rest of the image.

Also since the clock is mostly static (apart from the last two digits) the P- and B-frames will have no data there (since nothing changed and these kinds of frames only contain delta information). It might be possible to use the second to last digit in order to check whether the macroblocks are alighted correctly there. But that's about it I think.

I've been looking at ways to automate the process for quite a while now. The reason this is so hard is because only humans seem to be able to decide what is a good rendered block versus a bad rendered block (apart from blocks with clear errors which could of course be used for a search algo). Without some kind of feedback (not just in the lower left part but all over the image) it doesn't seem feasible to automate the aligment of the blocks to the bitpositions. This is also the reason I decided to make it possible to let the humans be "the feedback" by creating the -mmb option in the first place.

Where I've been looking at is how to detect where macroblocks begin and end. And because mpeg-4 is so damned efficient (they have essentially no overhead bits anywhere) this turns out to be very hard. My best candidates are:

- cbpc should always be below 4. (this is sort of a 2-bit "marker")
- dquant should always be 0 (sort of a 1-bit "marker"). But I am not sure if this is true.
- other errors in general (I haven't assembled all possible errors yet, but they are probably related to the above two)

This is quite meager though. But it's something.

My best idea is so far is to use a backwards search: start at the end of the data and assume the bit before it is the start of a macroblock and decode it: if it doesn't stop reading bits exactly at the end-bit then its not a valid starting bit for a macroblock. So try the previous bit. And repeat this. If you do end correctly then use the start of that new macroblock as the new end-bit. Etc.

I've experimented with this but I ran into the trouble of getting false positives (which might cause a way too big search-space, not sure yet). And there is also no way of knowing whether I'm on the right track: you basicly need a human being for that! (well maybe statistically you can do something). Still looking into that though.

Also there are real errors in the bits so I'm not sure what to decide sometimes: is it the wrong starting position or is there an error in the data? This backwards search idea is a mechanism that doesn't work as good as humans can do it now I think. With I-frames at least. But it might be the only way to do the P- and B-frames: I think it's going to be quite hard for humans to decide between an error-block and a misplaced-block in those kinds of frames. But then again another way of determining whether the blocks are aligned right in P- and B- frames are the borders of the rocket: the blocks right outside the rocket are always changing while the blocks next to them never change. So I'm still hopeful it's still mostly doable by hand.

I have quite a lot more thoughts/ideas on this subject but it's hard to express them all :P

Regards,

arnezami

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: ctillier on 05/09/2014 05:46 AM
My meager contribution for iframe 10
0:3:8818,0:4:11564

This reveals another two rows of blocks showing sea spray, but corrupts the rest of the image in a way that I haven't figured out how to fix.  Help me SwissCheese, you're my only hope!

Fantastic effort everyone.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/09/2014 06:11 AM
Hi guys,

There is another idea I would like to share regarding P- and B-frames and how to determine whether a block is placed in the right position:

The rocket is always moving downwards (apart from the last few seconds). And the camera is looking downwards. So all visual particles that are not part of the rocket always move from the middle to the outside: their movement vector is quite predictable. Furthermore: when a particle (like a little wave in the sea) moves outwards it speeds up. Right? (pixels per second I mean).

So if mpeg-4 is using movement prediction we should be able to see the change in speed (and the direction of this change in speed) in the blocks of the P- and B-frames. Is this sound logic?

Because if so we can (roughly) see in what line (from the middle part of the image to the outside of the image) the block should be placed. Assuming we can extract this information from the data of course.

Just my 2 cents :)

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/09/2014 08:20 AM
Here's what I have so far. It's about 2-3 seconds of video almost up to splashdown:

[youtube]<object width="425" height="350"><param name="movie" value="http://www.youtube.com/v/CxheyOUAlG8"></param><param name="wmode" value="transparent"></param><embed src="http://www.youtube.com/v/CxheyOUAlG8" type="application/x-shockwave-flash" wmode="transparent" width="425" height="350"></embed></object>[/youtube]
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: MTom on 05/09/2014 08:34 AM

The rocket is always moving downwards (apart from the last few seconds)...

I would guess on the Frame 13 the rocket is not vertical yet (kipped some to right on the screen).
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jakusb on 05/09/2014 08:45 AM
Is anyone working on 3 or 4? Those are the two bad ones. 5 was bad but the last few posts show progress!

I searched Frame 3 for a while but couldn't find anything. Unrecoverable?

judging from whats become of frame 5, i think frame 4 is going to be the most important for getting a nice shot of the leg deployment. and im afraid 3 is a lost cause. :(

i noticed that some are using a different ts file, try1? are we going to be in a world of hurt if we have mixed data or will it be okay?

ps this online editor on the wiki is SPECTACULAR! i wouldnt have been able to figure out doing it the hard way. kudos!


--edit--
added 22:24:97869 and changed some around the bottom:

0:0:-1,27:10:24142,16:12:30040,24:12:-1,17:13:39153,20:13:-1, 25:13:35375,29:13:-1,28:16:47754,0:17:50034,43:19:-1, 22:24:97869,02:27:-1,24:27:110437,13:27:-1,17:27:109631, 25:27:-1,27:27:110752,28:28:-1,33:28:121771,2:29:124970

Edit/Lar: please put spaces in every once in a while so your lines wrap. I think the parms are space agnostic (if they are after the comma but not anywhere else anyway?). thanks

My first attempt to pitch in on this amazing effort... :)
I checked frame 5 with above settings and noticed the piece of mud on camera seems on position off:
27:10:24142 should be 26:10:24142
I compared with frames 8 en 9 and they are consistent in having the mud on 26:10

PS: I am having trouble figuring out the best progress per frame on spacexlanding.wikispaces.com (http://spacexlanding.wikispaces.com). It would be great if it could be made to easy copy paste the values, but also to maybe have someone judge which progress is best, or if multiple routes are chosen to separate them.
Also I guess we are getting into some trouble with people working on 2 different ts files?!?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jakusb on 05/09/2014 08:51 AM
Is anyone working on 3 or 4? Those are the two bad ones. 5 was bad but the last few posts show progress!

I searched Frame 3 for a while but couldn't find anything. Unrecoverable?

judging from whats become of frame 5, i think frame 4 is going to be the most important for getting a nice shot of the leg deployment. and im afraid 3 is a lost cause. :(

i noticed that some are using a different ts file, try1? are we going to be in a world of hurt if we have mixed data or will it be okay?

ps this online editor on the wiki is SPECTACULAR! i wouldnt have been able to figure out doing it the hard way. kudos!


--edit--
added 22:24:97869 and changed some around the bottom:

0:0:-1,27:10:24142,16:12:30040,24:12:-1,17:13:39153,20:13:-1, 25:13:35375,29:13:-1,28:16:47754,0:17:50034,43:19:-1, 22:24:97869,02:27:-1,24:27:110437,13:27:-1,17:27:109631, 25:27:-1,27:27:110752,28:28:-1,33:28:121771,2:29:124970

Edit/Lar: please put spaces in every once in a while so your lines wrap. I think the parms are space agnostic (if they are after the comma but not anywhere else anyway?). thanks

My first attempt to pitch in on this amazing effort... :)
I checked frame 5 with above settings and noticed the piece of mud on camera seems on position off:
27:10:24142 should be 26:10:24142
I compared with frames 8 en 9 and they are consistent in having the mud on 26:10

PS: I am having trouble figuring out the best progress per frame on spacexlanding.wikispaces.com (http://spacexlanding.wikispaces.com). It would be great if it could be made to easy copy paste the values, but also to maybe have someone judge which progress is best, or if multiple routes are chosen to separate them.
Also I guess we are getting into some trouble with people working on 2 different ts files?!?

When I try to align the landing legs, I do not seem to get a matching block in frames 8 and 9. Can I conclude the landing legs not yet fully deployed in frame 5?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/09/2014 08:57 AM
I posted a gif in the IRC yesterday comparing my IF5 with IF8.
I could align the lower edge of the left leg, but the right leg was a few pixel to low.
The Rocket body and the smudge above the right leg was also aligned.
My interpretation is that the right leg is almost but not fully extended yet.


Cheers
Shanuson
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jakusb on 05/09/2014 09:41 AM
I posted a gif in the IRC yesterday comparing my IF5 with IF8.
I could align the lower edge of the left leg, but the right leg was a few pixel to low.
The Rocket body and the smudge above the right leg was also aligned.
My interpretation is that the right leg is almost but not fully extended yet.


Cheers
Shanuson

Regarding Frame 5:
I tried to combine the work of both dorkmo (upper half) with seanpg71 (lower part).
Also moved the 0:0:-1 to 14:0:-1 as first blocks do appear to be ok.
I shifted a number of blocks from dorkmo to left and the richt part of body a bit down and aligns now better with frame 8

00:21:106970,00:25:100325,00:27:106003,01:21:76873,01:29:123050,02:27:105388, 02:29:123876,03:16:50034,
10:23:90705,11:22:85251,11:23:91098,12:22:85489,14:00:-1, 16:12:30040,16:23:91462,17:13:39153,17:23:91682,
20:13:-1,22:27:110276,23:24:97869,24:13:35375,25:12:-1, 26:10:24142,26:22:86772,27:16:47754,29:13:-1,
30:28:121538,37:22:87982,38:22:88152,40:17:56308,43:19:69441

PS: I simply use the Online Tool v2 in Firefox and have 2 tabs open. One of Frame 5 (working on it) and one of Frame 8 (for comparison). I simply switch often (ctrl-tab) to keep comparing alignment. Works like a charm.. You could also have a second Firefox open and use alt-tab to compare against yet another frame...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/09/2014 10:22 AM
Here is a give comparing IF5 and IF8,
legs lookl like they are not yet in their final position.

Created with via Imgflip GIF Maker (https://imgflip.com/gifgenerator)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: MP99 on 05/09/2014 10:52 AM
PS: I am having trouble figuring out the best progress per frame on spacexlanding.wikispaces.com (http://spacexlanding.wikispaces.com). It would be great if it could be made to easy copy paste the values, but also to maybe have someone judge which progress is best, or if multiple routes are chosen to separate them.
Also I guess we are getting into some trouble with people working on 2 different ts files?!?

I wonder if this would work under GIT?

Separate source for the parms for each iframe.

Each person working in their own branch(es), making trial commits, and then someone combining them into master if they look to be progressing the overall goal?

cheers, Martin
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jakusb on 05/09/2014 10:58 AM

Here is a give comparing IF5 and IF8,
legs lookl like they are not yet in their final position.

Created with via Imgflip GIF Maker (https://imgflip.com/gifgenerator)

If you use the Online Tool v2, and 2 tabs in Firefox, you can check the alignment on block level. Simply select the same block in both tabs and ctrl-tab a lot. :)
Legtips seem to be in place, but in process of getting deployed still. If so, impossible to exactly align. The tips should be both a little bit inward aligned.
I do think de upper (wider) part of legs need some work still.
The rocket body seems perfectly aligned al the way to the to corners where the camera housing cuts the view.
Time is also in correct place.
If I can find time, I will try to improve the legs a bit more. Maybe discover some hidden blocks there.
If someone else would like to try, go for it. :)
If someone could recover some sea, that would be sweet too. There seem to be lots of data available for it, although likely pretty hard to discover.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: seanpg71 on 05/09/2014 11:22 AM
I posted a gif in the IRC yesterday comparing my IF5 with IF8.
I could align the lower edge of the left leg, but the right leg was a few pixel to low.
The Rocket body and the smudge above the right leg was also aligned.
My interpretation is that the right leg is almost but not fully extended yet.


Cheers
Shanuson

Regarding Frame 5:
I tried to combine the work of both dorkmo (upper half) with seanpg71 (lower part).
Also moved the 0:0:-1 to 14:0:-1 as first blocks do appear to be ok.
I shifted a number of blocks from dorkmo to left and the richt part of body a bit down and aligns now better with frame 8

00:21:106970,00:25:100325,00:27:106003,01:21:76873,01:29:123050,02:27:105388, 02:29:123876,03:16:50034,
10:23:90705,11:22:85251,11:23:91098,12:22:85489,14:00:-1, 16:12:30040,16:23:91462,17:13:39153,17:23:91682,
20:13:-1,22:27:110276,23:24:97869,24:13:35375,25:12:-1, 26:10:24142,26:22:86772,27:16:47754,29:13:-1,
30:28:121538,37:22:87982,38:22:88152,40:17:56308,43:19:69441

PS: I simply use the Online Tool v2 in Firefox and have 2 tabs open. One of Frame 5 (working on it) and one of Frame 8 (for comparison). I simply switch often (ctrl-tab) to keep comparing alignment. Works like a charm.. You could also have a second Firefox open and use alt-tab to compare against yet another frame...

I've gotten to here with 5:

0:0:-1,26:10:24142,16:12:30040,24:12:-1,25:12:32001,28:12:-1,17:13:39153,18:13:34192,29:13:-1,
18:14:39177,23:14:-1,24:15:41044,23:16:46465,28:16:47954,40:17:56308,43:19:69441,0:21:106970,
1:21:76873,11:22:85251,12:22:85489,26:22:86772,27:22:87056,37:22:87982,38:22:88152,10:23:90705,
11:23:91098,16:23:91462,17:23:91682,22:24:97633,23:24:97869,0:25:100325,0:27:106003,2:27:105388,
22:27:110276,30:28:121538,31:28:121730,1:29:123050,2:29:123876

There's also some largish good hunks of water starting at 5895, 9085, and 22377 that I gotten around to trying to jam in there yet.


I think part of the problem I'm having getting things to line up neatly is that is seems like there's a bunch of data that's just missing - unless I'm just trying to put things in the wrong spot, it seems like there should be around 7000 bits between 40500 and 41000 for the end of row 14 and the beginning of 15.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jakusb on 05/09/2014 11:54 AM
A quick attempt at Frame 4 (try1.ts), based on earlier work from saliva_sweet.

00:03:10968,01:28:129620,05:01:4200,08:03:17911,26:10:36805,31:07:-1,34:03:23414

I see some reoccurring dirt spots on lens that help align/confirm they are consistent in frame 4,8 & 9

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/09/2014 11:55 AM
So I could align better iframe 5 (coerced.ts) with the other ones and add a few sequences, still room for improvement.

We clearly see that the legs are not completely deployed!

0:0:550,14:0:-1,
0:5:9736,26:7:-1,
4:10:22722,25:12:32001,28:12:-1,16:13:-1:0:-25:0:0,17:13:-1:-15:0:0:0,
19:13:34518,10:14:-1,
17:14:39053,24:14:-1,21:15:-1:-25:-10,22:15:-1,
34:15:42349,24:16:-1,
27:16:47754,42:19:-1,
43:19:69441,43:20:-1,
01:21:76873,11:22:-1,
12:22:85489,26:22:-1,
17:23:91682,22:24:-1,
23:24:97869,00:27:-1,
16:27:109631,24:27:-1,
25:27:110437,28:28:-1,
3:29:124970
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jakusb on 05/09/2014 12:02 PM
3 consistent dirt spots that should help to align ocean in top part
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/09/2014 12:53 PM
We clearly see that the legs are not completely deployed!

And what's more, we clearly see that the right leg is intact and that there's actually a smudge on the lens to the lower left of the black mark. Woo!  ;D Way to go!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/09/2014 12:56 PM

What is mainly done is realigning the visual macroblocks to the data. Essentially trying to guess the starting bitposition of a visual block and see if it starts producing a sensible visual block. Apart from that there is also many cases where visual blocks have correct data but this data was meant for a different block: this "simply" needs to be reassigned. But its a lot of work. Due to errors in the data the visual blocks start reading at the wrong bit-position. And this happens for blocks all over the frame (not just in the lower left corner). The clock is usually easy to find and place in the right position (in many of the I-frames they have already been found). But if you have the clock in the right position it won't help you with the rest of the image.

Also since the clock is mostly static (apart from the last two digits) the P- and B-frames will have no data there (since nothing changed and these kinds of frames only contain delta information). It might be possible to use the second to last digit in order to check whether the macroblocks are alighted correctly there. But that's about it I think.


Interesting… Do we have a model for the noise in this channel? 

At the very least, are all errors bit flips or are there dropped bits (or dropped bytes) and added bits?

If all errors are flips, and P(flip) is low and nominally time invariant, the task is several orders of magnitude easier.

I've been looking at ways to automate the process for quite a while now. The reason this is so hard is because only humans seem to be able to decide what is a good rendered block versus a bad rendered block (apart from blocks with clear errors which could of course be used for a search algo). Without some kind of feedback (not just in the lower left part but all over the image) it doesn't seem feasible to automate the aligment of the blocks to the bitpositions. This is also the reason I decided to make it possible to let the humans be "the feedback" by creating the -mmb option in the first place.


I am approaching it from a post-processing standpoint, so use ffmpeg to create a PNG, then consume the PNG with a separate application. My biggest limitation here is that the output has to go to file (due to image2 spec), so I have it writing to a ramdisk. Is there any way to have to write PNG data to stdout instead?



Just a couple of observations playing with ffmpeg more.

First, it is very sensitive to errors - most (guessing 95%+) changes will result in one or more solid (usually green) blocks showing up after the shifted block. This is trivial to detect (though time consuming, might be faster to just parse the stderr).

Second, to see which pieces "fit" due to a smooth intersection, I can get a fairly high success rate in marking bad blocks by differencing pixels that straddle N, S, E, W block boundaries using just sqrt( (r2-r1)^2 + (g2-g1)^2 + (b2-b1)^2 ) summed 22 x 4 times (pixels x each boundary), with a threshold of 0.7. This is my starting point for finding misaligned blocks, but it can't be used to find the correct alignment, since all of the ocean blocks for example will easily pass this test regardless of where they are.

Third, there when a block from a static area (faring, clock, smudge) is out of place but parsed correctly, it should have a high correlation with the same block in a "known good" frame using an appropriate fourier transform. Since there are relatively few blocks (600) a complete distance matrix could be constructed with 380k convolutions (or half that with a good mask and bad-block detection). Main problem here is determining the fastest/most accurate transform for block difference. Maybe the ffmpeg guys have some advice here?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/09/2014 01:07 PM
Hey Theshadow27! Welcome to the site's forum and this particular effort.

Incredible to see a number of new members turn up for this task - in addition to people we already had on site - members who are clearly superb at what is required here.

The best thing about the site is its community and as the fella who started this site back in 2005 - with a couple of STS-114 RTF articles and one forum section frequented by a handful Shuttle huggers, a NASA guy and a United Space Alliance guy - it's incredible how it's grown into what we have today. :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/09/2014 01:31 PM
My 2 cents on frame 3. Because of how garbled it is, its possible that the corruption is not in the data itself but in the frame header. The iframe header has lots of specifications that define how even numbers are defined (whether to use reversible or non-reversible codings). That would cause the entire frame to be decoded wrong.

Edit: Just checked, yeah the frame header is corrupted. If we fix it, large parts of the image may pop out. Other frames may also have corrupted headers but not bad enough to block header decoding. This would result in odd settings for the image still though.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/09/2014 01:33 PM
Also, for completeness, this guy commented on the wiki page but has not posted in this thread:

http://aeroquartet.com/wordpress/2014/05/08/spacex-falcon-first-stage-landing-part-ii/

Seems to know his stuff. From his page,

Quote
I have seen in some forums that people is trying to improve the video by fixing the TSHs.
This doesn’t hurt, but it will not improve the video, because while TSHs are easy to fix, a bad TSH indicates that probably several bits in the neighborhood are also bad. And those are the critical ones, and video won’t improve until you fix them all.

As mentioned previously, an error rate higher than 1/50,000 is impossible to fix. And if you have bad TSHs you probably have an order of magnitude more errors than that in the neighborhood.

Sound's pretty pessimistic to me :/
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/09/2014 01:36 PM
Also, for completeness, this guy commented on the wiki page but has not posted in this thread:

http://aeroquartet.com/wordpress/2014/05/08/spacex-falcon-first-stage-landing-part-ii/

Seems to know his stuff. From his page,

Quote
I have seen in some forums that people is trying to improve the video by fixing the TSHs.
This doesn’t hurt, but it will not improve the video, because while TSHs are easy to fix, a bad TSH indicates that probably several bits in the neighborhood are also bad. And those are the critical ones, and video won’t improve until you fix them all.

As mentioned previously, an error rate higher than 1/50,000 is impossible to fix. And if you have bad TSHs you probably have an order of magnitude more errors than that in the neighborhood.

Sound's pretty pessimistic to me :/

Quote
I’ve been repairing videos full time for the last 7 years, so I immediately knew that the bitstream was corrupt and that at best we could hope to partially fix some frames. Maybe a 50% chance of recovering one or two frames.

I think he's referring to restoring it to 100% working video. If he wasn't, we already vastly beat his predictions. Not sure whether to trust his posts or not. Either that or his experience isn't as much as he claims it is as I'm apparently doing better than him with 1 week of experience and possibly superior tools.

Repairing the TSHs is important for getting access to the frames, that was already done. We're going beyond his anticipations and repairing the iframes which in a properly repaired transport stream (or newly created transport stream) will produce a possibly working video.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/09/2014 01:56 PM
Here's what I have so far. It's about 2-3 seconds of video almost up to splashdown:

[youtube]<object width="425" height="350"><param name="movie" value="http://www.youtube.com/v/CxheyOUAlG8"></param><param name="wmode" value="transparent"></param><embed src="http://www.youtube.com/v/CxheyOUAlG8" type="application/x-shockwave-flash" wmode="transparent" width="425" height="350"></embed></object>[/youtube]

This looks good. Was this only adding the improved iframes to the video as described earlier?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/09/2014 02:26 PM
My meager contribution for iframe 10
0:3:8818,0:4:11564

This reveals another two rows of blocks showing sea spray, but corrupts the rest of the image in a way that I haven't figured out how to fix.  Help me SwissCheese, you're my only hope!

Fantastic effort everyone.

It improved a bit the image, I also added a line with the new option -2 to avoid having the flame "contaminating" the image bottom:

43:2:-1,
0:3:8818,4:5:-1,
7:5:14847,18:05:-1,
0:7:18695,23:13:-1:-30:-30:-30:-30,24:13:-1,
0:16:-2:50:50:50:50,0:17:-1,
7:17:47677

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: AncientU on 05/09/2014 02:30 PM
Here is a give comparing IF5 and IF8,
legs lookl like they are not yet in their final position.

Created with via Imgflip GIF Maker (https://imgflip.com/gifgenerator)
Agree.  You can also see the end of the (left) leg in 5 where it mates to the tip fairing -- this piece would only be visible if the legs were basically still pointing up (at say 30-60 degrees from fully stowed).  The longest extent of the legs away from the stage axis would be just before reaching 90 degrees.

Edit: typos (need more coffee)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/09/2014 02:55 PM
My 2 cents on frame 3. Because of how garbled it is, its possible that the corruption is not in the data itself but in the frame header. The iframe header has lots of specifications that define how even numbers are defined (whether to use reversible or non-reversible codings). That would cause the entire frame to be decoded wrong.

Edit: Just checked, yeah the frame header is corrupted. If we fix it, large parts of the image may pop out. Other frames may also have corrupted headers but not bad enough to block header decoding. This would result in odd settings for the image still though.

I've been doing some inspection of the transport stream and iframe3 in a hex editor. iframe3 bears no resemblance to any other iframe in its data content. I don't think its actually an iframe or even a fame. I think the decoder got confused and tried to extract the data as an iframe. There are indeed 13 iframes though, we're just not extracting the first one for whatever reason.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/09/2014 02:58 PM
I think he's referring to restoring it to 100% working video. If he wasn't, we already vastly beat his predictions. Not sure whether to trust his posts or not. Either that or his experience isn't as much as he claims it is as I'm apparently doing better than him with 1 week of experience and possibly superior tools.

MLindner, I think the key advantage which you and your one week of experience bring to the table is that you have no idea what's supposed to be impossible.  :D

Perhaps because he does video recovery as a business, he never considered the possibility of a team of people staying up until the wee hours of the morning fueled by Red Bull and Doritos twiddling bit combinations in a single macro block of a single iframe looking for the best incremental improvement in a nearly-uniform stretch of blue ocean among hundreds of variations, because no customer could ever afford that, even ULA. (I hope he's not wasting a bunch of time trying to suss out more of iframe 8 on his own.)

Since the remuneration for the folks here, and your cheering section, is not mere cash but the excitement of seeing these historic images emerge from the tangled snarl of the transport stream, it becomes an affordable effort. I wonder if they'll have this discussion thread on display next to the restored video that visitors will find looping at the kiosk in front of the first recovered Falcon 9R at the Smithsonian?  ;D
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Swatch on 05/09/2014 03:23 PM
More work on iframe8. Will post the mmb later.


Did you ever post the mmb for this?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/09/2014 03:45 PM
My 2 cents on frame 3. Because of how garbled it is, its possible that the corruption is not in the data itself but in the frame header. The iframe header has lots of specifications that define how even numbers are defined (whether to use reversible or non-reversible codings). That would cause the entire frame to be decoded wrong.

Edit: Just checked, yeah the frame header is corrupted. If we fix it, large parts of the image may pop out. Other frames may also have corrupted headers but not bad enough to block header decoding. This would result in odd settings for the image still though.

I've been doing some inspection of the transport stream and iframe3 in a hex editor. iframe3 bears no resemblance to any other iframe in its data content. I don't think its actually an iframe or even a fame. I think the decoder got confused and tried to extract the data as an iframe. There are indeed 13 iframes though, we're just not extracting the first one for whatever reason.

Also the Postion in the Video suggest that interpretation:
iFrame2 at 1146612
iFrame3 (which probably is no iFrame) at 1207712
iFrame 4 at 1446284

all other iFrames are also separeted by around 300000 bits.

Exceptions: between IF 6 and 7 it is twice as much, Are we sure there is no Iframe between these 2?
Also IF1 is at 847880. Could this mean there are 3 more iframes before that? 85/30 is ~ 3.

Cheers,
Shanuson

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/09/2014 04:07 PM
My 2 cents on frame 3. Because of how garbled it is, its possible that the corruption is not in the data itself but in the frame header. The iframe header has lots of specifications that define how even numbers are defined (whether to use reversible or non-reversible codings). That would cause the entire frame to be decoded wrong.

Edit: Just checked, yeah the frame header is corrupted. If we fix it, large parts of the image may pop out. Other frames may also have corrupted headers but not bad enough to block header decoding. This would result in odd settings for the image still though.

I've been doing some inspection of the transport stream and iframe3 in a hex editor. iframe3 bears no resemblance to any other iframe in its data content. I don't think its actually an iframe or even a fame. I think the decoder got confused and tried to extract the data as an iframe. There are indeed 13 iframes though, we're just not extracting the first one for whatever reason.

Also the Postion in the Video suggest that interpretation:
iFrame2 at 1146612
iFrame3 (which probably is no iFrame) at 1207712
iFrame 4 at 1446284

all other iFrames are also separeted by around 300000 bits.

Exceptions: between IF 6 and 7 it is twice as much, Are we sure there is no Iframe between these 2?
Also IF1 is at 847880. Could this mean there are 3 more iframes before that? 85/30 is ~ 3.

Cheers,
Shanuson

I do not know for the beginning, but for iframe 6 the blocks are "heavier"  (more bits)  than in the other iframes. The file is roughly twice larger than the other ones.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/09/2014 04:08 PM
More work on iframe8. Will post the mmb later.


Did you ever post the mmb for this?

That was a hackjob i threw together in time for Chris's article. I wasn't exactly happy with how I did some of the erasing of parts of the image. I'll release something better later. Right now I'm fussing with the transport stream to see if we can get the actual first iframe.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/09/2014 04:09 PM
My 2 cents on frame 3. Because of how garbled it is, its possible that the corruption is not in the data itself but in the frame header. The iframe header has lots of specifications that define how even numbers are defined (whether to use reversible or non-reversible codings). That would cause the entire frame to be decoded wrong.

Edit: Just checked, yeah the frame header is corrupted. If we fix it, large parts of the image may pop out. Other frames may also have corrupted headers but not bad enough to block header decoding. This would result in odd settings for the image still though.

I've been doing some inspection of the transport stream and iframe3 in a hex editor. iframe3 bears no resemblance to any other iframe in its data content. I don't think its actually an iframe or even a fame. I think the decoder got confused and tried to extract the data as an iframe. There are indeed 13 iframes though, we're just not extracting the first one for whatever reason.

Also the Postion in the Video suggest that interpretation:
iFrame2 at 1146612
iFrame3 (which probably is no iFrame) at 1207712
iFrame 4 at 1446284

all other iFrames are also separeted by around 300000 bits.

Exceptions: between IF 6 and 7 it is twice as much, Are we sure there is no Iframe between these 2?
Also IF1 is at 847880. Could this mean there are 3 more iframes before that? 85/30 is ~ 3.

Cheers,
Shanuson

I do not know for the beginning, but for iframe 6 the blocks are "heavier"  (more bits)  than in the other iframes. The file is roughly twice larger than the other ones.

I have the feeling that there is at least two additional iframes before IF1. In the original video there is a section of the rocket surrounded by clouds, with no sea visible.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mikes on 05/09/2014 04:34 PM
I am approaching it from a post-processing standpoint, so use ffmpeg to create a PNG, then consume the PNG with a separate application. My biggest limitation here is that the output has to go to file (due to image2 spec), so I have it writing to a ramdisk. Is there any way to have to write PNG data to stdout instead?
Have you tried using "-" as the output file specifier?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: david1971 on 05/09/2014 04:35 PM
The first thing that went though my head when I read about the ORBCOMM launch delay was "this means the video folks will get a few extra days to get their clean version of a landing done before SpaceX has new video."
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: rsnellenberger on 05/09/2014 04:49 PM

The first thing that went though my head when I read about the ORBCOMM launch delay was "this means the video folks will get a few extra days to get their clean version of a landing done before SpaceX has new video."

Same here...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/09/2014 04:58 PM
My 2 cents on frame 3. Because of how garbled it is, its possible that the corruption is not in the data itself but in the frame header. The iframe header has lots of specifications that define how even numbers are defined (whether to use reversible or non-reversible codings). That would cause the entire frame to be decoded wrong.

Edit: Just checked, yeah the frame header is corrupted. If we fix it, large parts of the image may pop out. Other frames may also have corrupted headers but not bad enough to block header decoding. This would result in odd settings for the image still though.

I've been doing some inspection of the transport stream and iframe3 in a hex editor. iframe3 bears no resemblance to any other iframe in its data content. I don't think its actually an iframe or even a fame. I think the decoder got confused and tried to extract the data as an iframe. There are indeed 13 iframes though, we're just not extracting the first one for whatever reason.

Also the Postion in the Video suggest that interpretation:
iFrame2 at 1146612
iFrame3 (which probably is no iFrame) at 1207712
iFrame 4 at 1446284

all other iFrames are also separeted by around 300000 bits.

Exceptions: between IF 6 and 7 it is twice as much, Are we sure there is no Iframe between these 2?
Also IF1 is at 847880. Could this mean there are 3 more iframes before that? 85/30 is ~ 3.

Cheers,
Shanuson

I do not know for the beginning, but for iframe 6 the blocks are "heavier"  (more bits)  than in the other iframes. The file is roughly twice larger than the other ones.

I have the feeling that there is at least two additional iframes before IF1. In the original video there is a section of the rocket surrounded by clouds, with no sea visible.

Further investigation.

I think there is only a single iframe not being decoded. I found all the relevant frame headers and everything at 0x03E3EC position in the file.

0x000001B0 = visual_object_sequence_start_code
0x03       = profile and level indication
0x000001B5 = visual_object_start_code

There are exactly 13 of these patterns in this sequence in the transport stream. I did some copy and paste searching of the starts of the various iframe file dumps and each one lines up to one of these code sets. Importantly, we entirely skip the first set of these codes. The first iframe is the second set of these. I'm trying to figure out why its not decoding, but not there yet.

We also need to somehow disable iframe3 from being used as that will screw up any video that could possibly be correct that would otherwise reference iframe2.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/09/2014 05:31 PM
Further investigation.

I think there is only a single iframe not being decoded. I found all the relevant frame headers and everything at 0x03E3EC position in the file.

0x000001B0 = visual_object_sequence_start_code
0x03       = profile and level indication
0x000001B5 = visual_object_start_code

There are exactly 13 of these patterns in this sequence in the transport stream. I did some copy and paste searching of the starts of the various iframe file dumps and each one lines up to one of these code sets. Importantly, we entirely skip the first set of these codes. The first iframe is the second set of these. I'm trying to figure out why its not decoding, but not there yet.

We also need to somehow disable iframe3 from being used as that will screw up any video that could possibly be correct that would otherwise reference iframe2.

Here's the current iframe frame numbers and timestamps as best as I can make them out:

 1  33 19:35:03.6
 2  53 19:35:04.?
 3  57 ?
 4  73 19:35:??.?
 5  93 19:35:07.6
 6 113 19:35:08.9
 7 150 ?
 8 169 19:35:12.9
 9 188 19:35:14.3
10 207 19:35:??.?
11 227 19:35:1?.?
12 246 19:35:18.2

Assuming a few of the sequences are contiguous, we can reconstruct some timestamps.

 1  33 19:35:03.6
 2  53 19:35:04.9
 3  57 ?
 4  73 19:35:06.3
 5  93 19:35:07.6
 6 113 19:35:08.9
 7 150 ?
 8 169 19:35:12.9
 9 188 19:35:14.3
10 207 19:35:15.6
11 227 19:35:16.9
12 246 19:35:18.2

So, here are the gaps:

 2   53 19:35:04.9
-3   ?? 19:35:05.1
 4   73 19:35:06.3
 5   93 19:35:07.6
 6  113 19:35:08.9
-7a 13? 19:35:10.2
-7b 150 19:35:11.5
 8  169 19:35:12.9

It's possible the data for iframe 3 isn't even in the stream. We are clearly missing some p and b frames too.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: seanpg71 on 05/09/2014 05:37 PM
So I could align better iframe 5 (coerced.ts) with the other ones and add a few sequences, still room for improvement.

We clearly see that the legs are not completely deployed!

0:0:550,14:0:-1,
0:5:9736,26:7:-1,
4:10:22722,25:12:32001,28:12:-1,16:13:-1:0:-25:0:0,17:13:-1:-15:0:0:0,
19:13:34518,10:14:-1,
17:14:39053,24:14:-1,21:15:-1:-25:-10,22:15:-1,
34:15:42349,24:16:-1,
27:16:47754,42:19:-1,
43:19:69441,43:20:-1,
01:21:76873,11:22:-1,
12:22:85489,26:22:-1,
17:23:91682,22:24:-1,
23:24:97869,00:27:-1,
16:27:109631,24:27:-1,
25:27:110437,28:28:-1,
3:29:124970

I added a few more sections to that.  Definitely still some more areas to get a bit of improvement from though.

0:0:550,14:0:-1,
32:4:9085,25:7:-1,
4:10:22722,25:12:32001,28:12:-1,
16:13:-1:0:-25:0:0,17:13:-1:-15:0:0:0,
18:13:34192,10:14:-1
17:14:39053,23:14:-1,
21:15:-1:-25:-10,
25:15:41044,
23:16:46465,
28:16:47954,
41:19:69012,
43:19:69441,43:20:-1,
1:21:76873,11:22:-1,
12:22:85489,26:22:-1,
27:22:87056,42:22:-1,
38:22:88152,10:23:-1,
11:23:91098,16:23:-1,
17:23:91682,22:24:-1,
23:24:97869,
0:25:100325,0:27:-1,
2:27:105388,
22:27:110276,30:28:-1,
31:28:121730,1:29:-1,
2:29:123876








Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: xcg001 on 05/09/2014 05:44 PM
Here's the current iframe frame numbers and timestamps as best as I can make them out:

 1  33 19:35:03.6
 2  53 19:35:04.?
 3  57 ?
 4  73 19:35:??.?
 5  93 19:35:07.6
 6 113 19:35:08.9
 7 150 ?
 8 169 19:35:12.9
 9 188 19:35:14.3
10 207 19:35:??.?
11 227 19:35:1?.?
12 246 19:35:18.2

Assuming a few of the sequences are contiguous, we can reconstruct some timestamps.

 1  33 19:35:03.6
 2  53 19:35:04.9
 3  57 ?
 4  73 19:35:06.3
 5  93 19:35:07.6
 6 113 19:35:08.9
 7 150 ?
 8 169 19:35:12.9
 9 188 19:35:14.3
10 207 19:35:15.6
11 227 19:35:16.9
12 246 19:35:18.2

So, here are the gaps:

 2   53 19:35:04.9
-3   ?? 19:35:05.1
 4   73 19:35:06.3
 5   93 19:35:07.6
 6  113 19:35:08.9
-7a 13? 19:35:10.2
-7b 150 19:35:11.5
 8  169 19:35:12.9

It's possible the data for iframe 3 isn't even in the stream. We are clearly missing some p and b frames too.

I am not a video specialist but looking at these timestamps there is ~1.3 seconds difference between each I frame so most likely there should be no I frame between frame 2 and 4.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/09/2014 05:50 PM
I am not a video specialist but looking at these timestamps there is ~1.3 seconds difference between each I frame so most likely there should be no I frame between frame 2 and 4.

Gah! You're right. It's too early in the morning for such advanced math.


 2   53 19:35:04.9
-3   does not exist
 4   73 19:35:06.3
 5   93 19:35:07.6
 6  113 19:35:08.9
-7a 13? 19:35:10.2
-7b 150 19:35:11.5
 8  169 19:35:12.9
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: AncientU on 05/09/2014 06:08 PM
I am not a video specialist but looking at these timestamps there is ~1.3 seconds difference between each I frame so most likely there should be no I frame between frame 2 and 4.

Gah! You're right. It's too early in the morning for such advanced math.


 2   53 19:35:04.9
-3   does not exist
 4   73 19:35:06.3
 5   93 19:35:07.6
 6  113 19:35:08.9
-7a 13? 19:35:10.2
-7b 150 19:35:11.5
 8  169 19:35:12.9

Would be consistent number-sequence wise if the first column was incremented by 20 (33, 53, 73, 93, 113, 133-7a, 153-7b, etc.) and the seconds incremented by 1.3.  This drops i frame 3 as suggested by many and also fixes7a/b.  There are also 20-ish gaps in the higher frames as well as 1.3-1.4s increments.  Somewhere in the 150-170 range, an extra DC offset  of about 6 was added.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/09/2014 06:13 PM
Ok, first try at auto-marking bad MBs based on ffmpeg log output and primitive analysis.

Blocks which were reported as errors by ffmpeg (modified, with debug on) are marked black. Internally, this tracks:
  --  the -1 and -2 override blocks from MBM,
  -- dc clipped at
  -- ac-tex at
  -- dquant at
  -- stuffing at
  -- Error at MB
although these are all painted black as the colors were confusing.

Blocks that were NOT reported as errors but probably are bad are translucent red.
   -- currently tests for the block having less than 5 individual colors.

(http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=582552)

Next step is to begin (hopefully) intelligent mutation of the MBM string to achieve a better overall score. Sample image is of frame 4 with saliva_sweet MBMs (from wiki).


Edit: Here's the Java source if anyone else wants to play. http://pastebin.com/bKgzCRqD The mammoth inner class at the bottom is for writing the animated gif :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Swatch on 05/09/2014 06:13 PM
There also appears to be, on average, 20 frames between each I-frame.  Like xcg001, I'm no video specialist, so I'm not sure if that should stay relatively consistant or not.

Edit:  What AncientU said...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/09/2014 06:17 PM
Here's what I have so far. It's about 2-3 seconds of video almost up to splashdown:

[youtube]<object width="425" height="350"><param name="movie" value="http://www.youtube.com/v/CxheyOUAlG8"></param><param name="wmode" value="transparent"></param><embed src="http://www.youtube.com/v/CxheyOUAlG8" type="application/x-shockwave-flash" wmode="transparent" width="425" height="350"></embed></object>[/youtube]

This looks good. Was this only adding the improved iframes to the video as described earlier?

I took raw.ts-nomc-ignoreerr.mkv from way back at the beginning of this thread and broke it up in to individual frames. With motion turned off, what is left is block updates and something approximating a delta between the current frame and nearby i- and p-frames. A fraction of the intermediate frames are useable with a bit of shifting around. I added in the fixed up i-frames and cut out the bad blocks. I then went sequentially through the frames adding deltas and alpha-blending the block updates.

I think the video could be vastly improved with some more tweaking, more intermediate frames, and a longer run time. However, editing the frames is obviously rather time consuming. If anyone is interested in helping out, I can post my project files to the wiki.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/09/2014 06:30 PM
There also appears to be, on average, 20 frames between each I-frame.  Like xcg001, I'm no video specialist, so I'm not sure if that should stay relatively consistant or not.

Edit:  What AncientU said...

A bit off topic, but I updated the Iframe sequence that was on the wiki, to match this speed.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Y Combinator on 05/09/2014 06:52 PM
There also appears to be, on average, 20 frames between each I-frame.  Like xcg001, I'm no video specialist, so I'm not sure if that should stay relatively consistant or not.

Edit:  What AncientU said...

A bit off topic, but I updated the Iframe sequence that was on the wiki, to match this speed.

I believe this is 15 FPS video; it looks like your animated GIF is assuming 30 FPS and as such is too fast.  Check the timestamps against a stopwatch.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/09/2014 07:03 PM
There also appears to be, on average, 20 frames between each I-frame.  Like xcg001, I'm no video specialist, so I'm not sure if that should stay relatively consistant or not.

Edit:  What AncientU said...

A bit off topic, but I updated the Iframe sequence that was on the wiki, to match this speed.

I believe this is 15 FPS video; it looks like your animated GIF is assuming 30 FPS and as such is too fast.  Check the timestamps against a stopwatch.

Yep, you are right. I updated it to a new version.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Swatch on 05/09/2014 07:32 PM
Dang... Frame 4 is tantalizing because of the leg deploy.

Also, that pace feels more accurate moralec.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: AncientU on 05/09/2014 07:34 PM
There also appears to be, on average, 20 frames between each I-frame.  Like xcg001, I'm no video specialist, so I'm not sure if that should stay relatively consistant or not.

Edit:  What AncientU said...

A bit off topic, but I updated the Iframe sequence that was on the wiki, to match this speed.

I believe this is 15 FPS video; it looks like your animated GIF is assuming 30 FPS and as such is too fast.  Check the timestamps against a stopwatch.

Yep, you are right. I updated it to a new version.
Might also want to drop i frame 3.  Leg deployment starts just after 2 and is about a quarter of the way done by frame 4, 1.3s later. Engine exhaust hitting sea surface happens between 8 and 9.  Splashdown at around 10. "Settled" vertically in sea at 13.  Not sure when engine start happens/happened.  Leg deployment was supposed to be half way through a 20ish second burn IIRC -- leg deployment to splash (about 2 -10) is 7 or 8 x 1.3s = 9.1s-10.4s which checks.

Ed: Note -- 3 could be already dropped
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/09/2014 07:48 PM
My 2 cents on frame 3. Because of how garbled it is, its possible that the corruption is not in the data itself but in the frame header. The iframe header has lots of specifications that define how even numbers are defined (whether to use reversible or non-reversible codings). That would cause the entire frame to be decoded wrong.

Edit: Just checked, yeah the frame header is corrupted. If we fix it, large parts of the image may pop out. Other frames may also have corrupted headers but not bad enough to block header decoding. This would result in odd settings for the image still though.

I've been doing some inspection of the transport stream and iframe3 in a hex editor. iframe3 bears no resemblance to any other iframe in its data content. I don't think its actually an iframe or even a fame. I think the decoder got confused and tried to extract the data as an iframe. There are indeed 13 iframes though, we're just not extracting the first one for whatever reason.

Also the Postion in the Video suggest that interpretation:
iFrame2 at 1146612
iFrame3 (which probably is no iFrame) at 1207712
iFrame 4 at 1446284

all other iFrames are also separeted by around 300000 bits.

Exceptions: between IF 6 and 7 it is twice as much, Are we sure there is no Iframe between these 2?
Also IF1 is at 847880. Could this mean there are 3 more iframes before that? 85/30 is ~ 3.

Cheers,
Shanuson

I do not know for the beginning, but for iframe 6 the blocks are "heavier"  (more bits)  than in the other iframes. The file is roughly twice larger than the other ones.

I have the feeling that there is at least two additional iframes before IF1. In the original video there is a section of the rocket surrounded by clouds, with no sea visible.

Further investigation.

I think there is only a single iframe not being decoded. I found all the relevant frame headers and everything at 0x03E3EC position in the file.

0x000001B0 = visual_object_sequence_start_code
0x03       = profile and level indication
0x000001B5 = visual_object_start_code

There are exactly 13 of these patterns in this sequence in the transport stream. I did some copy and paste searching of the starts of the various iframe file dumps and each one lines up to one of these code sets. Importantly, we entirely skip the first set of these codes. The first iframe is the second set of these. I'm trying to figure out why its not decoding, but not there yet.

We also need to somehow disable iframe3 from being used as that will screw up any video that could possibly be correct that would otherwise reference iframe2.

I think the first iframe is a lost cause. Where the user data section should start there is instead a "video_session_error_code" which according to the spec: "The video_session_error_code has been allocated for use by a media interface to indicate where uncorrectable errors have been detected."

Even replacing the code with the user data start code and the data itself with another frame's user data, the output is garbage. The decoder barfs everywhere and can't even get to decoding macroblocks.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Digitalchromakey on 05/09/2014 08:04 PM
I-Frame 6

Looks like the RHS Leg is being deployed with the LHS leg already locked in position.

Needs a lot more work to be more certain.

0:0:583,43:0:5530,
00:01:6129,02:01:6129,21:1:8584,22:1:8796,
4:3:12577,14:3:19199,15:03:-1,18:03:19433,20:3:14083,23:3:14220,42:3:21474,
0:04:-1,31:04:24629,32:4:15280,
6:5:17702,7:5:-1,13:05:27662,14:05:-1,42:0:5530,17:05:28599,18:5:13888,24:05:-1,25:05:29375,
28:7:40520,29:7:40520,32:07:-1,37:07:41235,
1:8:37105,
1:9:37105,
26:10:61670,27:10:40399,
17:11:64694,
0:13:56967'
0:16:71551,1:10:11678,
1:11:11678,
5:12:12185,18:12:58656,
07:13:-1,13:13:60221,14:13:13373,15:13:60221,
19:14:59308,
12:15:72126,
0:17:71551,2:17:71610,
0:18:71551,7:18:72226,
0:19:121762,
1:28:202509




Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/09/2014 08:11 PM
So I tried to get something with iframe 4, but still a lot to do. I have to stop now for a few days so good luck to those who wish to improve it ;)

From try1.ts

5:1:4200,00:03:-1,
02:03:10968,10:03:-1,
17:06:22427,14:11:-1,
38:16:65876,19:17:-1,
00:19:78514,28:20:-1,
0:28:129509

At 17911 and 55810 there are also valid sequences, which I do not know exactly where to put, and I did not really try to improve the bottom of the frame.

Also some sequences from http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195103#msg1195103 could help if put at the accurate place.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: cscott on 05/09/2014 08:11 PM
I think the first iframe is a lost cause.

Perhaps you could turn your attention to the missing iframe "7a"?  Maybe there's a bit flipped somewhere which is causing your search for 0x000001B0 / 0x03 / 0x000001B5 not to turn it up?  (I'm assuming the existing iframe 7 is "7b" based on the timestamp image, someone correct me if i'm wrong.)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/09/2014 08:17 PM
Have the brute-force code running on frame 4 checking 1-bit modifications, with a threshold of eliminating 4 or more erroneous blocks (starting with 00:10:-1,00:14:-1,00:17:71044,00:24:-1,00:27:114198,08:15:61126,19:05
:20944,21:13:50333,25:14:57756,35:05:-1,36:15:-1,43:06:27582). It is running pretty slowly now, about 77ms per bit (and there are a lot of bits!) but it has produced some interesting results (31 solutions and counting for a 1-bit flip).

I'm not very good at playing with the numbers manually, perhaps someone else wants to see if any of these can be improved? These are the best so far:

00:10:-1,00:14:-1,00:17:71044,00:24:-1,00:27:114198,08:
15:61126,19:05:20944,21:13:50333,25:14:57756,35:05:-1,36:15:-1,43:06:27582,X:10838:32

00:10:-1,00:14:-1,00:17:71044,00:24:-1,00:27:114198,08:
15:61126,19:05:20944,21:13:50333,25:14:57756,35:05:-1,36:15:-1,43:06:27582,X:9308:16

00:10:-1,00:14:-1,00:17:71044,00:24:-1,00:27:114198,08:
15:61126,19:05:20944,21:13:50333,25:14:57756,35:05:-1,36:15:-1,43:06:27582,X:9278:128

00:10:-1,00:14:-1,00:17:71044,00:24:-1,00:27:114198,08:
15:61126,19:05:20944,21:13:50333,25:14:57756,35:05:-1,36:15:-1,43:06:27582,X:4548:32

00:10:-1,00:14:-1,00:17:71044,00:24:-1,00:27:114198,08:
15:61126,19:05:20944,21:13:50333,25:14:57756,35:05:-1,36:15:-1,43:06:27582,X:4416:32
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Swatch on 05/09/2014 08:18 PM
I think the first iframe is a lost cause.

Perhaps you could turn your attention to the missing iframe "7a"?  Maybe there's a bit flipped somewhere which is causing your search for 0x000001B0 / 0x03 / 0x000001B5 not to turn it up?  (I'm assuming the existing iframe 7 is "7b" based on the timestamp image, someone correct me if i'm wrong.)

Not sure how long it would take to brute-force through the file in the area of concern (from pos 2054276 to 2940884), performing a bit-flip at each bit along the way and performing a search for 0x000001B0 / 0x03 / 0x000001B5 within the bits +/- the total message length.  If you don't see anything, flip the bit back and move onto the next bit.  When you see a new hit for 0x000001B0 / 0x03 / 0x000001B5, record the file location and then go back later to see if maybe a new iframe materializes.

Frustrating because I know I used to be able to do all I said above, but I've been away from coding for too long to do it in any expedient manner.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/09/2014 08:20 PM
Not sure how long it would take to brute-force through the file in the area of concern, performing a bit-flip at each bit along the way and performing a search for 0x000001B0 / 0x03 / 0x000001B5 within the bits +/- the total message length.  If you don't see anything, flip the bit back and move onto the next bit.  When you see a new hit for 0x000001B0 / 0x03 / 0x000001B5, record the file location and then go back later to see if maybe a new iframe materializes.

Frustrating because I know I used to be able to do all I said above, but I've been away from coding for too long to do it in any expedient manner.

The brute-force I just posted is for frame 4, not the mysterious missing frame. I'm just flipping bits in the MBs that ffmpeg identified.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Oersted on 05/09/2014 08:23 PM
How much intimate knowledge of video compression and codecs and whatnot would have never been discovered if it wasn't for this garbled video! :-)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/09/2014 08:34 PM
Careful with the long number strings folks. So long they break the site width. A split point half way works.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: michaelni on 05/09/2014 08:39 PM
Third, there when a block from a static area (faring, clock, smudge) is out of place but parsed correctly, it should have a high correlation with the same block in a "known good" frame using an appropriate fourier transform. Since there are relatively few blocks (600) a complete distance matrix could be constructed with 380k convolutions (or half that with a good mask and bad-block detection). Main problem here is determining the fastest/most accurate transform for block difference. Maybe the ffmpeg guys have some advice here?

Comparing blocks this way can be a bit tricky, as the DC values could be wrong as they depend on previous blocks and one row or column of AC values could be wrong when ac prediction is used. Also if the QP is wrong the block could be dequantized with a wrong scale factor. Also the blocks could be subjected to cliping when they are too bright or dark. The cliping issue could be avoided by comparing the coefficients before the IDCT.
If one did some block brute force comparisson, one could for each pair of blocks to compare rescale one to compensate for a possible QP error, adjust its DC value and if it uses ac prediction maybe use the block above or to he left if its static too to remove its influence, whats left could be compared by some means like sum of squared differences or sum of absolute differences of the coefficients.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/09/2014 08:48 PM
I-Frame 6

Looks like the RHS Leg is being deployed with the LHS leg already locked in position.

Needs a lot more work to be more certain.

0:0:583,43:0:5530,
00:01:6129,02:01:6129,21:1:8584,22:1:8796,
4:3:12577,14:3:19199,15:03:-1,18:03:19433,20:3:14083,23:3:14220,42:3:21474,
0:04:-1,31:04:24629,32:4:15280,
6:5:17702,7:5:-1,13:05:27662,14:05:-1,42:0:5530,17:05:28599,18:5:13888,24:05:-1,25:05:29375,
28:7:40520,29:7:40520,32:07:-1,37:07:41235,
1:8:37105,
1:9:37105,
26:10:61670,27:10:40399,
17:11:64694,
0:13:56967'
0:16:71551,1:10:11678,
1:11:11678,
5:12:12185,18:12:58656,
07:13:-1,13:13:60221,14:13:13373,15:13:60221,
19:14:59308,
12:15:72126,
0:17:71551,2:17:71610,
0:18:71551,7:18:72226,
0:19:121762,
1:28:202509

Please note the solution from Gnonthgol in http://spacexlanding.wikispaces.com/ (did not find his post):

0:0:583,
39:0:5496,43:0:-1,
02:01:6129,
21:1:8584,
22:1:8796,6:3:-1,
02:06:29800,2:8:-1,
12:10:46767,00:11:-1,
3:12:56455,18:12:-1,
26:12:61670,27:12:-1,
17:13:64694,28:12:-1,
32:13:71022,4:14:-1,
27:14:78837,5:15:-1,
19:15:92898,23:15:-1,
27:17:107825,27:18:-1,
0:19:121762,04:21:-1,
31:22:159772,0:24:-1,
30:25:184570,07:27:-1,
35:27:200466
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/09/2014 09:04 PM
Here's the new try1-fixed.ts Iframe3 has been converted to a b-frame.

Edit: It apparently doesn't decode at all now though. It obviously wasn't an I-frame though. We should probably remove I-frame 3 from the wiki.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/09/2014 09:25 PM
(http://content3.viralnova.com/wp-content/uploads/2014/05/25-pA0T7Mc.jpg) (http://www.viralnova.com/weekly-chalkboard-art/)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: James (Lockheed) on 05/09/2014 10:57 PM
Amazing work guys. We could have used you when we tried to get some footage down from the ET "Death Cam"!

http://www.nasaspaceflight.com/2011/07/sts-135-et-camera-ascent-no-usable-video-reentry/
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/09/2014 10:59 PM
Amazing work guys. We could have used you when we tried to get some footage down from the ET "Death Cam"!

http://www.nasaspaceflight.com/2011/07/sts-135-et-camera-ascent-no-usable-video-reentry/

Wasn't that analog? Analog = black magic.

Edit: Yeah. Analog NTSC over FM radio. If you don't have an SDR recording of it there's nothing better you'll be able to process it into AFAIK.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/10/2014 12:17 AM
Ok, I looked into try1.ts

found the postion of an I-Frame around 0x23bdd6 in try1.ts.
most bit patterns are there, but somehow set of by 4bit (half a byte, so one does not find it easily in a hex editor) and also that is probably the reason why it is not found by the decoder. The decoder wants stuff to start at the beginning of a byte not in the middle.

A typical header part, Coloured bits are similar in I-Frame headers 2,3,4,5,6,8,9,10,11,12,13,14 (numbered not as in the wiki, but by appearance in try1.ts)
This could be due to some manual editing by someone else, have not checked how the look in other .ts files.
The position mentioned is the start of sequence 00 00 01 B6 1

IF02:  0x0CEEE9;
00 00 00 01 E0 00 00 81 80 07 21 01 67 CE 5D FF FF 00 00 01 B0 03 00 00 01 B5 0D 0F 00 00 01 00 00 00 01 20 00 C4 8D C0 00 46 56 C0 1E C0 01 F2 46 9A FC
7C 2E EE 2C 08 78 28 C7 00 00 01 B2 65 6D 34 76 20 34 2E 33 2E 30 2E 30 00 C3 FF 00 00 01 B6 14

Now Frame1 (frame0 of wiki), red are the bytes that are different to every other I-Frame
IF01:  0x03E429; 
00 00 00 01 E0 00 00 81 80 07 21 01 59 79 7D FF FF 00 00 01 B0 03 00 00 01 B5 0D 0F 00 00 01 00 00 00 01 20 00 C4 8D C0 00 46 56 C0 1E C0 01 A1 62 9A FC
7C 2E EE 2C 08 78 28 C7 00 00 01 B4 7B 6D 75 76 32 34 02 27 26 5C 4E 5B 82 2A F8 00 31 30 B8 3A

Same for the frame between frame 6 and 7 of the wiki:
Also bits are shifted here by 4, compared to the ts file, therefore the .5 at the position:
IF07:  0x23BDEC.5;
40 C2 82 13 B0 01 81 8B 50 07 21 01 8D 22 8D FF FF 00 00 01 B0 03 00 00 01 B5 0D 0F 00 00 01 00 00 00 01 20 00 C4 85 C0 00 46 54 80 13 40 09 EF DE 9A FE
7C 22 EE 2C 08 78 68 C2 BC 10 49 80 6C 8D 34 76 20 34 2E 33 2E 30 2E 30 00 C3 FF 00 00 01 B6 10

You will find those parts for all 14 I-Frames in the file attached.
I dont know how one could extract that shifted frame, hexeditors do not allow to insert only 4 bits, always wants to add a byte.

Edit: I should really go to bed, but found also one at the start: to tired to colour it now compared to IF2.

IF1.5: 0x085c51;
00 00 00 01 E0 00 00 81 80 07 21 01 61 23 ED FF FF 15 38 13 20 03 00 00 41 A8 8C C7 05 06 01 14 00 00 80 80 06 C4 8D C0 40 C7 D5 C0 1E D0 01 93 1D 9A FC
7C 2E EE 2D 08 7E 28 C7 C0 00 A9 B1 7D 6D 64 76 20 34 4A 32 76 30 2E 30 00 C3 7F 03 08 01 86 28
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Swatch on 05/10/2014 01:21 AM
Attempted to color them for Shanuson... not sure if I got it right, but it does show the differences...
I realize there are some 'half-hex' pairs that are identical, but I chose to color the complete hex pair if there's any discrepancy.


Shanuson's Note: Now Frame1 (frame0 of wiki), red are the bytes that are different to every other I-Frame
IF01:  0x03E429;  (POS: 255017)
00 00 00 01 E0 00 00 81 80 07 21 01 59 79 7D FF FF 00 00 01 B0 03 00 00 01 B5 0D 0F 00 00 01 00 00 00 01 20 00 C4 8D C0 00 46 56 C0 1E C0 01 A1 62 9A FC
7C 2E EE 2C 08 78 28 C7 00 00 01 B4 7B 6D 75 76 32 34 02 27 26 5C 4E 5B 82 2A F8 00 31 30 B8 3A

IF1.5: 0x085c51;  (POS: 547921)
00 00 00 01 E0 00 00 81 80 07 21 01 61 23 ED FF FF 15 38 13 20 03 00 00 41 A8 8C C7 05 06 01 14 00 00 80 80 06 C4 8D C0 40 C7 D5 C0 1E D0 01 93 1D 9A FC
7C 2E EE 2D 08 7E 28 C7 C0 00 A9 B1 7D 6D 64 76 20 34 4A 32 76 30 2E 30 00 C3 7F 03 08 01 86 28

IF02:  0x0CEEE9;  (POS: 847593)
00 00 00 01 E0 00 00 81 80 07 21 01 67 CE 5D FF FF 00 00 01 B0 03 00 00 01 B5 0D 0F 00 00 01 00 00 00 01 20 00 C4 8D C0 00 46 56 C0 1E C0 01 F2 46 9A FC
7C 2E EE 2C 08 78 28 C7 00 00 01 B2 65 6D 34 76 20 34 2E 33 2E 30 2E 30 00 C3 FF 00 00 01 B6 14

Shanuson's Note: Also bits are shifted here by 4, compared to the ts file, therefore the .5 at the position:
IF07:  0x23BDEC.5;  (POS:  ~2342380)
40 C2 82 13 B0 01 81 8B 50 07 21 01 8D 22 8D FF FF 00 00 01 B0 03 00 00 01 B5 0D 0F 00 00 01 00 00 00 01 20 00 C4 85 C0 00 46 54 80 13 40 09 EF DE 9A FE
7C 22 EE 2C 08 78 68 C2 BC 10 49 80 6C 8D 34 76 20 34 2E 33 2E 30 2E 30 00 C3 FF 00 00 01 B6 10
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/10/2014 02:43 AM
I dont know how one could extract that shifted frame, hexeditors do not allow to insert only 4 bits, always wants to add a byte.

It should be pretty straightforward in Perl with unpack/pack. Just unpack with H*, then pack substr($str, 1) back to H. I'll take a look.

Edit: Ok, look taken.

Here's what I did:

1. Padded out my try1.ts to get it lined up with the offsets in header.txt, which may or may not match what I'm seeing in my hex editor, as it's late.
2. Cut out everything up to the 040 C2 82 13 B0 01 81 shifted region, and then searched for the next 00000001E000008180 marker. This marker turned out to be properly byte-aligned, as it happens. I then cut that out as well, leaving only the bit-shifted frame at 299297 bytes.
3. Quick and dirty Perl:
#!/usr/bin/perl
open(IN, "< bitshiftframe") or die; binmode(IN); $/=undef;

$iframe = <IN>;
$hex = substr(unpack('H*', $iframe), 1);

open(OUT, "> shanusons-bitshifted.ts") or die; binmode(OUT);
print(OUT pack('H*', $hex));


This produced a byte-aligned version of 0x23BDEC.5. I attached it here as an mp4, but it's not an MP4 file - it's a straight excerpt of the bytes of interest from the try1.ts transport stream file - the TS extension is not allowed for attachments here.

Edit: removed the bitshifted ts file because the packet headers were mangled by the operation.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/10/2014 03:25 AM
I dont know how one could extract that shifted frame, hexeditors do not allow to insert only 4 bits, always wants to add a byte.

It should be pretty straightforward in Perl with unpack/pack. Just unpack with H*, then pack substr($str, 1) back to H. I'll take a look.

PS: where did you get your try1.ts file? The one I pulled from the very bottom of the wikspaces.com page has IF01 which you list at 0x3E429 showing up in my bvi session starting at 0x3E3DB, a 30-byte offset to the left. I'll pad it out at the beginning to make it line up with your numbers, but I want to make sure we're on the same page.

There's a bigger issue is that when edited.ts and try1.ts were created they had "fixed" the transport stream headers by adding the 4 byte headers to them every 188 bytes throughout the stream. If you simply shift the bytes at the moment then all the transport stream headers will become unaligned again not to mention effectively being corruption every 188 bytes. It might be better to re-apply the processes that were done on raw.ts again and produce a new transport stream. Preferably in script form this time so that if something else is found they can quickly be re-applied.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: John_L on 05/10/2014 03:46 AM
Well, looks like we're not getting any new video anytime soon.  SpaceX has scrubbed the launch for tomorrow, and pushed it back to "later this month". :(
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/10/2014 04:01 AM
Well, looks like we're not getting any new video anytime soon.  SpaceX has scrubbed the launch for tomorrow, and pushed it back to "later this month". :(

More motivation for us to get the video done even better. :P
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Guspaz on 05/10/2014 04:10 AM
If it does come down to needing to do automated A/B testing, and humans are needed for that evaluation, I'd like to point out that there is always Amazon's Mechanical Turk. It basically allows you to farm out this sort of effort to large numbers of people fairly cheaply. I don't think you guys are at that point yet, but there's been discussions in this thread about trying to do the testing algorithmically, so I thought I'd mention it.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/10/2014 04:30 AM
There's a bigger issue is that when edited.ts and try1.ts were created they had "fixed" the transport stream headers by adding the 4 byte headers to them every 188 bytes throughout the stream. If you simply shift the bytes at the moment then all the transport stream headers will become unaligned again not to mention effectively being corruption every 188 bytes. It might be better to re-apply the processes that were done on raw.ts again and produce a new transport stream. Preferably in script form this time so that if something else is found they can quickly be re-applied.
Good point - odds are that's what happened since I shifted the entire file four bits to the right. I've been heart-attack busy lately so I've resisted the temptation to learn how MPEG encoding works. I'll take a look at the resulting bitshifted file and the video coding documentation, and if you wouldn't mind doing so as well and letting me know if you're inclined. As you can see the shifting operation is straightforward in Perl so it won't be difficult to cope with the TS headers. And also perhaps set up command line arguments to shift arbitrary ranges of nybbles.

Edit: yep, there it is, right at the end: ...FF F4 71 FF F1 6F FF...  Looking at raw.ts now.

Edit: The four-bit offset is present in raw.ts as well at 0x23BD07. Coercing the TS headers in the raw.ts may have mangled things... it's extremely difficult to discern where the missing or added four bits came from, too. Hmmm... it looks like the 00-00-01-b0-03 bytes always appear exactly 28 bytes after a ts header, let's see...

In a good packet formatted to 20 columns, we see the 47 to mark the beginning of the packet, and here "D" is the sequence number (the "3" before it means "adaptation field & payload") and 28 bytes later we see the 00-00-01-B0 pattern:

00316CE4  47 43 E8 3D 07 10 00 34 5B FF 7E 00 00 00 01 E0 00 00 81 80
00316CF8  07 21 01 A3 21 DD FF FF 00 00 01 B0 03 00 00 01 B5 0D 0F 00
00316D0C  00 01 00 00 00 01 20 00 C4 8D C0 00 46 56 C0 1E C0 01 FD 30


In the bit-shifted one, counting 28 octets back from the "0-00-00-1B-0" we land in the middle of the marked 4D, meaning that we have four extra bits between the header and the sequence:

0023BCE8  FA A9 FF 4D BE 66 AF 8C EC B9 A7 CD C3 5C 74 0C 28 21 3B 00
0023BCFC  18 18 B5 00 72 10 18 D2 28 DF FF F0 00 00 1B 00 30 00 00 1B
0023BD10  50 D0 F0 00 00 10 00 00 00 12 00 0C 48 5C 00 04 65 48 01 34


Based on what appears to be a clean packet header above this region at showing "47-1F-FF-18," at 0x23bb74 in raw.ts, the sequence number at the marked packet header above should be "A", since it's about 2x188 bytes away from the clean header.

This reveals some interesting information as well - in raw.ts, 188 bytes after this clean header is "C0-D8-61-01" which was coerced in try1.ts to "47 83 E8 11" so the sequence number there is incorrect (1 instead of 9) due to a bit flip - "0001" instead of "1001."  This apparently means that the try1.ts file is stuffed full of continuity count errors - the coerced header just after this one has a sequence number of C when it should be A, for example. And if that was ignored, then no-increment-if-no-payload was probably also ignored.

Setting that aside... let's say that the 4D is supposed to be a 47 (0111 instead of 1110, and the sequence number is supposed to be A, and look what we have here...

0023BCE8  FA A9 FF 47 BE 66 AF 8C EC B9 A7 CD C3 5C 74 0C 28 21 3B 00
0023BCFC  18 18 B5 00 72 10 18 D2 28 DF FF F0 00 00 1B 00 30 00 00 1B
0023BD10  50 D0 F0 00 00 10 00 00 00 12 00 0C 48 5C 00 04 65 48 01 34


So thanks to Shanuson we know we're looking for something along the lines of: 00 00 00 01 E0 00 00 81 80 07 21 01 59 79 7D FF FF...

0023BCE8  FA A9 FF 47 BE 66 AF 8 CE CB 9A 7C DC 35 C7 40 C2 82 13 B0 01
0023BCFC  81 8B 50 07 21 01 8D 22 8D FF FF 00 00 01 B0 03 00 00 01 B5
0023BD10  0D 0F 00 00 01 00 00 00 01 20 00 C4 85 C0 00 46 54 80 13


Taking out that one "8" nybble, for example, puts the sequence back to 28 bytes after the packet header's 0x47. So basically we know that the "AF" at the end of the packet header needs to be "1A" or "3A," and that it would be reasonable to conclude that the four extra bits fall somewhere between the 0x47 and 0xFF00. I have that dangling "8" in there, but it could be something else. Someone more familiar with the encoding may be able to recognize it. Maybe it's where the 07-21-01 match breaks.

Edit: It needs to be 3A, because the "0x02" bit is the payload-unit-start indicator which should be set at the beginning of an iframe.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/10/2014 04:55 AM
Ok, first try at auto-marking bad MBs based on ffmpeg log output and primitive analysis.

...

Edit: Here's the Java source if anyone else wants to play. http://pastebin.com/bKgzCRqD The mammoth inner class at the bottom is for writing the animated gif :)
Great stuff!!

One suggestion: when comparing a block with other block its best to only compare it with the one above and the one to the left.

This is because of how mpeg-4 works: the current block will directly affect the one below and the one to the right. So you are really comparing to yourself in a way. Also: if the current block is defect (or one until the one below is reached) it is very likely the block below is bit-shifted so you're not comparing the current block to what is supposed to be below it.

Keep that in mind.

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: grythumn on 05/10/2014 05:57 AM
Teasing away at the top of iframe 9. 23:04 starts at 15950. Best correction I got for 19:04 was X:15522:80,X:15518:80, but there were several similar solutions, and didn't try an exhaustive 2-bit search[0], much less 3-bit.

Trying to figure out 20, 21, 22... first attempt didn't end up with the correct number of bits. :/ More tomorrow, brute force search in progress.

-Bob
[0] Ran through single bit permutations to remove the largest artifact, noted the bit locations that helped, ran a second set of permutations against for each of the first list.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mikes on 05/10/2014 08:08 AM
I think the first iframe is a lost cause.

Perhaps you could turn your attention to the missing iframe "7a"?  Maybe there's a bit flipped somewhere which is causing your search for 0x000001B0 / 0x03 / 0x000001B5 not to turn it up?  (I'm assuming the existing iframe 7 is "7b" based on the timestamp image, someone correct me if i'm wrong.)

Not sure how long it would take to brute-force through the file in the area of concern (from pos 2054276 to 2940884), performing a bit-flip at each bit along the way and performing a search for 0x000001B0 / 0x03 / 0x000001B5 within the bits +/- the total message length.  If you don't see anything, flip the bit back and move onto the next bit.  When you see a new hit for 0x000001B0 / 0x03 / 0x000001B5, record the file location and then go back later to see if maybe a new iframe materializes.

Frustrating because I know I used to be able to do all I said above, but I've been away from coding for too long to do it in any expedient manner.

I think this is equivalent to flipping each of the bits of the search string, then doing a search for each of those 72 strings. I've tried that, but it didn't turn anything up.

#!/usr/bin/python
import os,sys
def binsearch(foo,ninebytes):
    want=hex(ninebytes)[2:-1].zfill(18)
    print want
    os.system("./bgrep "+want+" "+foo)
foo=sys.argv[1]
orig=0x000001b003000001b5
mask=0x800000000000000000
binsearch(foo,orig)
while mask :
    binsearch(foo,orig^mask)
    mask >>= 1


https://raw.githubusercontent.com/tmbinc/bgrep/master/bgrep.c
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/10/2014 11:25 AM
Now past the 100,000 read mark for this thread. Worth noting! :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/10/2014 12:23 PM
Ok here is a new file containing a comparison of raw.ts and try1.ts

Containing:
15 I-Frames, the start position of the '47' byte and the following 94 Bytes
Only clearly sees that all those parts are I-Frames.

Most of them are quite similar:
But IF0 and IF1 have some errors

And there is IF7 which is the by 4bit shifted one.  (In the File I manually shifted if so one can compared the bytes better)

The following is how it should look like IMHO.
There are 3 blocks of XXs which are different for each frame. The first one seems to contain a counter. The rest i dont know. But it should be mentioned in that ISO*.pdf how they should look like/what they should contain.

47 43 E8 37 07 10 00 xx xx xx 7E 00 00 00 01 E0 00 00 81 80 07 21 01 xx xx xx FF FF 00 00 01 B0 03 00 00 01 B5 0D 0F 00 00 01 00 00 00 01 20 00 C4 8D C0 00 46 56 C0 1E C0 01 xx xx 9A FC 7C 2E EE 2C 08 78 28 C7 00 00 01 B2 65 6D 34 76 20 34 2E 33 2E 30 4E 31 40 C3 FF 00 00 01 B6 1x

About the problem where the shift occurred: Frames are framed by large blocks of FFs with 47 1F FF 1X in them if they are large enough and X counting up. On Both sides of that IF7 they are already align. So the shift and reshift should have occurred between those. Maybe it is just enough to add a F at the end of the block before IF7 and remove one at the start of the block after IF7 (0x23cde9).

Cheers
Shanuson
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/10/2014 12:56 PM
My I7 MBP spent the night in the freezer (literally) running brute force bit flipping (X flags), and was still chugging like a champ when I killed the process this morning. It had run ffmpeg 2,452,480 times in 14:41:50 (8 threads). Computational feasibility demonstrated.

It came up with a few decent improvements, but no where close to what SwissCheese did by hand (on try1). The best is probably

00:10:-1,00:14:-1,00:17:71044,00:24:-1,00:27:114198,08:15:61126,
19:05:20944,21:13:50333,25:14:57756,35:05:-1,36:15:-1,43:06:27582,
X:15868:3

but I haven't looked through all 1024 yet.

Reading through the latest, it seems like I need to rework the code to run through raw.ts instead. This will probably be faster anyway, since I can manipulate bits directly in the data and skip the -mbm flags argument. Just need to catch up on what has been done.

Great work Shanuson


edit: I updated my java code to include the multithreaded ffmpeg stuff http://pastebin.com/bKgzCRqD

edit2: Just a bit of warning, this will peg your CPU at 100% for a long time, make sure it has plenty of cooling and you don't have any unsaved windows open just in case ;)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/10/2014 01:12 PM
Ok, first try at auto-marking bad MBs based on ffmpeg log output and primitive analysis.

...

Edit: Here's the Java source if anyone else wants to play. http://pastebin.com/bKgzCRqD The mammoth inner class at the bottom is for writing the animated gif :)
Great stuff!!

One suggestion: when comparing a block with other block its best to only compare it with the one above and the one to the left.

This is because of how mpeg-4 works: the current block will directly affect the one below and the one to the right. So you are really comparing to yourself in a way. Also: if the current block is defect (or one until the one below is reached) it is very likely the block below is bit-shifted so you're not comparing the current block to what is supposed to be below it.

Keep that in mind.

Regards,

arnezami

Hmm ok so it is pointless to try manipulation in blocks past the first bad block? Basically we need to completely fix one bad block at a time?

Edit: By "first bad block" I'm indexing blocks index= (x + y*width) i.e. left-to-right, top-to-bottom. Or is it in the order that they are encoded (which isn't always that)?

That's actually not so bad, really makes the problem easier. Basically the chess problem. Maybe we can ask IBM to borrow Deep Blue (or Watson :D) 
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: grythumn on 05/10/2014 01:19 PM
Ok, first try at auto-marking bad MBs based on ffmpeg log output and primitive analysis.

...

Edit: Here's the Java source if anyone else wants to play. http://pastebin.com/bKgzCRqD The mammoth inner class at the bottom is for writing the animated gif :)
Great stuff!!

One suggestion: when comparing a block with other block its best to only compare it with the one above and the one to the left.

This is because of how mpeg-4 works: the current block will directly affect the one below and the one to the right. So you are really comparing to yourself in a way. Also: if the current block is defect (or one until the one below is reached) it is very likely the block below is bit-shifted so you're not comparing the current block to what is supposed to be below it.

Keep that in mind.

Regards,

arnezami

Hmm ok so it is pointless to try manipulation in blocks past the first bad block? Basically we need to completely fix one bad block at a time?

I think you're okay fixing half-block errors that occur later in the image, but don't mess with full block or greater brightness/hue changes... they come and go freely as you fix earlier blocks. At least that's what I'm noticing on Frame 9... I'm having to revert some of my earlier edits.

-Bob
(the mad bit flipper)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/10/2014 01:57 PM
Working on those headers I found that some errors are single bit flips, but there are also stripes of bytes which are simply garbage/randomly turned bits one after an other. In an image you cant repair this I think. Substitute MBs maybe.
Headers can also be fixed, but if part of an frame MB itself is messed up like this you cant really hope to fixed it by flipping random bits.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/10/2014 02:39 PM
About the problem where the shift occurred: Frames are framed by large blocks of FFs with 47 1F FF 1X in them if they are large enough and X counting up. On Both sides of that IF7 they are already align. So the shift and reshift should have occurred between those. Maybe it is just enough to add a F at the end of the block before IF7 and remove one at the start of the block after IF7 (0x23cde9).

The X is the frame sequence number, which starts at zero and counts up through "F" and then back to 0 - this Wikipedia page about the MPEG transport stream has a good explanation of those four bytes starting with 0x47. (http://en.wikipedia.org/wiki/MPEG_transport_stream) The 1F FF 1X means:

0001 1111 1111 1111 0001

000 = no transport error, no payload unit start, no high priority
1 1111 1111 1111 = null packet identifier for fixed-bandwidth padding packets
0001 = 00 - not scrambled; 0 - no adaptation field; 1 - contains payload

This detail may help suss out what the packet headers should be in the bit-shifted packet. For instance, we know that the PID field can't take certain values like 0x0004 through 0x000F, etc.

Lining up the null packets in front of the bit-shifted one in a 47-column hex editor it's easy to see bad bytes, because we know that the 184 bytes following a 471FFF header should all be FF, such as this from raw.ts:

0023BB60  FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 47 1F FF 18 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF
0023BB8F  FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF
0023BBBE  FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF
0023BBED  FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 7F FC FF FF FF FD 7F F4 FF 7C F6 FF FF FF FF FF FF FF DE 7F 7D 76 FC F4 EB 6D 6F F6 FF 1F FD BF
0023BC1C  FF FF FF FF FF FF FF FB 03 EE 76 17 E0 1B DA 08 FF EB DF 14 C0 D8 61 01 F7 5F C8 3F FE 7B FF F8 7D 53 57 70 34 EB DC F3 3F 5C 82 FF F3 FE 3E


This packet-aligned-column view also allows you to see where the good packets pick up after this mangling - at 0x23BFDC with a 0x4703E81A, followed by 0x4703E81B. This also tells us that we have the right number of bytes in the scrambled interim, because the columns still line up.

Moving up the rows, it's easy to see where there's corrupted headers:

0023BC30 C0D86101 should be 4703E815, or something with a bit set to indicate a sequence discontinuity.
0023BCEC BE66AF8C should be 4703E816
0023BDA8 4703E817 is correct
0023BE64 77034818 should be 4703E818
0023BF20 77EB35EE should be 4703E819


So it seems like it should be practical to write a tool that can walk through the raw.ts and use the rules of the packet header formatting and sequence numbers to reconstruct a more coherent transport stream. Even just getting all the errors out of the null packets by replacing everything with FF might make it easier to discern good patterns.

Edit: For example, in raw.ts there are bytes missing or added between 0x37ACC and 0x371EC - that's the first place where the columns shift in a 47-column hex display.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/10/2014 10:09 PM
iframe 6 (try1)
Left leg is not fully deployed yet, but i'm perplexed by the tip of it. At least I think it's the tip because it doesn't look like a cloud or a foam spot on the sea. It kind of resembles the tip of the right leg on iframe 5, which shows the right leg in similar stage of deployment as left leg on this frame. The thing is it doesn't align. It's off by one block (half of macroblock). Is such an alignment error even possible?

Some macroblocks are highly suspect, they may be random noise that vaguely looks like information that I forced into the picture (18:12 and 25:12 are the main concerns).

Brace for mmb
X:82804:80,X:134444:80,0:0:583,39:0:5496,43:0:-1,2:1:6129,21:1:8584,22:1:8796,6:3:18105,
7:3:18276,9:3:29527,30:3:32359,31:3:-1,43:3:33591,2:6:44738,4:6:46341,5:6:48794,35:6:52255,
38:6:52740,40:06:52938,42:06:53105,0:7:53564,2:7:-1,4:10:55390,1:11:-1,10:11:63669,27:11:-1,
2:12:67622,17:12:70590,18:12:71241,19:12:71659,21:12:71834,23:12:-1,25:12:72549,26:12:72948,
27:12:73733,29:12:75341,33:12:-1,1:13:76584,2:13:-1,05:13:77037,30:13:83673,33:13:84191,
37:13:84690,14:14:87836,17:14:-1,19:14:88523,30:14:92989,8:15:95815,12:15:96507,15:15:96729,
22:16:109491,25:18:131296,42:18:135673,05:22:-1,27:22:173820,0:24:-1,27:24:192142,4:27:-1,
25:27:215436
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: maschnitz on 05/10/2014 10:31 PM
If you zoom in on 6, it seems to me that 17:12 is sea foam, not the leg. The color and texture isn't right. (It'd be handy to have a magnifying glass on Slapbet for stuff like this.)

Then, I suspect 15:11 is actually 15:10. The bit offsets in rows 10 and 11 jump all over the place; there's a jump from 56129 to 63669 between 09:10 and 10:10, and by 00:11 we're at 68940.  The actual end of row 10 to 11 probably lives/lived early in that range; or earlier.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jakusb on 05/10/2014 10:51 PM
An improved version of the annotated iframe8 with many more common dirt spots I see in several other iframes.
They seem to be very useful for further aligning MB's in other iframes
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/10/2014 11:00 PM
If you zoom in on 6, it seems to me that 17:12 is sea foam, not the leg. The color and texture isn't right. (It'd be handy to have a magnifying glass on Slapbet for stuff like this.)
This was my first assumption too, but there is no good place I can put it other that the top of the leg. It's very bright and causes a propagating streak if placed in other places. The range it can be in is constrained and it's in a longish piece of good data so it can't be moved much without either impinging on fixed pieces or raising the question where is the tip of the leg then. The leg section on the other hand is really dark and wants a very light macroblock on top of it.

Then, I suspect 15:11 is actually 15:10.
Can't be. 4:10 to 42:10 is a solid good piece of data anchored by the smudge on 26:10
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: maschnitz on 05/10/2014 11:22 PM
Can't be. 4:10 to 42:10 is a solid good piece of data anchored by the smudge on 26:10

Good point.

Hmm. If you switch back and forth quickly between 'common_dirt2' (frame 8 ) and 6, 6's left leg is too long, not too short. And the right leg is a little short. So maybe 15:11 is more like 16:12? Then everything else shifts down and to the left one. The lower part of the leg isn't set in stone, I don't think. Or 15:12/13? Not sure, though. Wish I was more nimble with the MPEG format.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: seanpg71 on 05/10/2014 11:23 PM
So I tried to get something with iframe 4, but still a lot to do. I have to stop now for a few days so good luck to those who wish to improve it ;)

From try1.ts

5:1:4200,00:03:-1,
02:03:10968,10:03:-1,
17:06:22427,14:11:-1,
38:16:65876,19:17:-1,
00:19:78514,28:20:-1,
0:28:129509

At 17911 and 55810 there are also valid sequences, which I do not know exactly where to put, and I did not really try to improve the bottom of the frame.

Also some sequences from http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195103#msg1195103 could help if put at the accurate place.
So I tried to get something with iframe 4, but still a lot to do. I have to stop now for a few days so good luck to those who wish to improve it ;)

From try1.ts

5:1:4200,00:03:-1,
02:03:10968,10:03:-1,
17:06:22427,14:11:-1,
38:16:65876,19:17:-1,
00:19:78514,28:20:-1,
0:28:129509

At 17911 and 55810 there are also valid sequences, which I do not know exactly where to put, and I did not really try to improve the bottom of the frame.

Also some sequences from http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195103#msg1195103 could help if put at the accurate place.

I was able to add some more dubiously placed water to this.  But not getting much of anything in the parts that are actually interesting.  Not sure threre's a whole lot more I can do with 4.

5:1:-1,
11:1:5384,30:2:-1,
2:3:10968,10:3:-1,
13:3:14762,33:3:-1,
35:3:16382,
37:3:16550,34:4:-1,
6:5:17911,
30:5:19483,32:5:-1,
36:5:20286,8:6:-1,
9:6:21857,16:10:-1,
17:10:36140,13:11:-1,
17:11:39697,38:11:-1,
1:12:41855,16:12:-1,
29:12:43895,20:13:-1,
13:14:51698,38:14:-1,
14:15:51794,36:15:-1,
37:16:65876,18:17:-1,
14:18:74612,30:20:-1,
24:27:127453,6:28:-1

Note, this is from try1.ts.  It's fairly easy to port between coerced.ts and try1.ts frames though.  Just need to add/subtract an offset to account for the extra sections of data in try1.ts that aren't in coerced.ts.

Here's the diff and offset for frame 4.


Coerced      Try      Offset
0-4225        0-4225      0
              4225-5697   
4226-46912    5698-48384   1472
              48384-51328
46913-49856   51329-54272   4416
              54272-55744   
49857-66048   55745-71936   5888
              71936-73408   
66049-104323 73409-111683   7360
              111683-114627
104324-121984   114628-132288   10304
121985 - 132287
              132289-144485


Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jakusb on 05/10/2014 11:47 PM
So I tried to get something with iframe 4, but still a lot to do. I have to stop now for a few days so good luck to those who wish to improve it ;)

From try1.ts

5:1:4200,00:03:-1,
02:03:10968,10:03:-1,
17:06:22427,14:11:-1,
38:16:65876,19:17:-1,
00:19:78514,28:20:-1,
0:28:129509

At 17911 and 55810 there are also valid sequences, which I do not know exactly where to put, and I did not really try to improve the bottom of the frame.

Also some sequences from http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195103#msg1195103 could help if put at the accurate place.
So I tried to get something with iframe 4, but still a lot to do. I have to stop now for a few days so good luck to those who wish to improve it ;)

From try1.ts

5:1:4200,00:03:-1,
02:03:10968,10:03:-1,
17:06:22427,14:11:-1,
38:16:65876,19:17:-1,
00:19:78514,28:20:-1,
0:28:129509

At 17911 and 55810 there are also valid sequences, which I do not know exactly where to put, and I did not really try to improve the bottom of the frame.

Also some sequences from http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195103#msg1195103 could help if put at the accurate place.

I was able to add some more dubiously placed water to this.  But not getting much of anything in the parts that are actually interesting.  Not sure threre's a whole lot more I can do with 4.

5:1:-1,
11:1:5384,30:2:-1,
2:3:10968,10:3:-1,
13:3:14762,33:3:-1,
35:3:16382,
37:3:16550,34:4:-1,
6:5:17911,
30:5:19483,32:5:-1,
36:5:20286,8:6:-1,
9:6:21857,16:10:-1,
17:10:36140,13:11:-1,
17:11:39697,38:11:-1,
1:12:41855,16:12:-1,
29:12:43895,20:13:-1,
13:14:51698,38:14:-1,
14:15:51794,36:15:-1,
37:16:65876,18:17:-1,
14:18:74612,30:20:-1,
24:27:127453,6:28:-1

Note, this is from try1.ts.  It's fairly easy to port between coerced.ts and try1.ts frames though.  Just need to add/subtract an offset to account for the extra sections of data in try1.ts that aren't in coerced.ts.

Here's the diff and offset for frame 4.


Coerced      Try      Offset
0-4225        0-4225      0
              4225-5697   
4226-46912    5698-48384   1472
              48384-51328
46913-49856   51329-54272   4416
              54272-55744   
49857-66048   55745-71936   5888
              71936-73408   
66049-104323 73409-111683   7360
              111683-114627
104324-121984   114628-132288   10304
121985 - 132287
              132289-144485


I changed 11:1:5384 to 22:1:5384 to properly align the dark dirt in top center... I really should give them numbers ;)
And also 30:2:-1 to 3:0:-1
Now a lot of dirt blobs start to align... Now I really gonna number them.. :)

5:1:-1,22:1:5384,0:3:-1,2:3:10968,10:3:-1,13:3:14762, 33:3:-1,35:3:16382,37:3:16550,34:4:-1,6:5:17911, 30:5:19483,32:5:-1,36:5:20286,8:6:-1,9:6:21857, 16:10:-1,17:10:36140,13:11:-1,17:11:39697,38:11:-1, 1:12:41855,16:12:-1,29:12:43895,20:13:-1,13:14:51698, 38:14:-1,14:15:51794,36:15:-1,37:16:65876,18:17:-1,14:18:74612,30:20:-1,24:27:127453,6:28:-1
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jakusb on 05/10/2014 11:51 PM
A version with the dirt spots numbered for better reference in conversation

Edit: Added dirt group 14
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jakusb on 05/11/2014 12:02 AM
iframe 4:

I see another dirt group (14) that needs to go 39 blocks left (or 1 row down and 5 to right)
However I could not get it to work yet
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/11/2014 12:12 AM
<snip>
The following is how it should look like IMHO.
There are 3 blocks of XXs which are different for each frame. The first one seems to contain a counter. The rest i dont know. But it should be mentioned in that ISO*.pdf how they should look like/what they should contain.

47 43 E8 37 07 10 00 xx xx xx 7E 00 00 00 01 E0 00 00 81 80 07 21 01 xx xx xx FF FF 00 00 01 B0 03 00 00 01 B5 0D 0F 00 00 01 00 00 00 01 20 00 C4 8D C0 00 46 56 C0 1E C0 01 xx xx 9A FC 7C 2E EE 2C 08 78 28 C7 00 00 01 B2 65 6D 34 76 20 34 2E 33 2E 30 4E 31 40 C3 FF 00 00 01 B6 1x

<snip>

The first xx xx xx is a counter. From Iframe to Iframe it increases by 60060. (the 2 red ones are calculated with this information)

Frame   Hex    Dec
IF01   2B31E7   2830823
IF02    2c1c83    2890883
IF03    2d071f    2950943
IF04    2df1bb    3011003
IF05    2edc57    3071063
IF06    2fc6f3    3131123
IF07    30b18f    3191183
IF08    319C2B   3251243
IF09    3286c7    3311303
IF10    337163    3371363
IF11    345bff    3431423
IF12    35469b    3491483
IF13    363137    3551543
IF14    371bd3    3611603
IF15    38066f    3671663


The next block of XXs is part of the Presentation Time Stamp PTS
how to get the PTS is explained quite well here: http://dvd.sourceforge.net/dvdinfo/pes-hdr.html
PTS: 21 01 xx xx xx.
We can now decode the PTS of the good frames and define the PTS of the messed up ones.

It is late again, my bed is calling.
I will calculate the PTS tomorrow, also will work out that last xx xx.
Here are the raw 5 byte PTS of the frames:

IF00: 99 31 59 14 6C
IF01: 21 01 61 23 ED
IF02: 21 01 67 CE 5D
IF03: 21 01 6F 78 CD
IF04: 21 01 77 23 3D
IF05: 21 01 7D CD AD
IF06: 21 01 85 78 1D
IF07: 21 01 8D 22 8D
IF08: 21 01 93 CC FD
IF09:  21 01 9B 77 6D
IF10: 21 01 A3 21 DD
IF11: 21 01 A9 CC 4D
IF12: 21 01 B1 76 BD
IF13: 21 01 B9 21 2D
IF14: 21 01 BF CB 9D

Cheers
Shanuson
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/11/2014 12:20 AM
Hmm. If you switch back and forth quickly between 'common_dirt2' (frame 8 ) and 6, 6's left leg is too long, not too short.

Yes while deploying the leg looks longer because of perspective and that a fully deployed leg is pointing downward.

And the right leg is a little short. So maybe 15:11 is more like 16:12? Then everything else shifts down and to the left one. The lower part of the leg isn't set in stone, I don't think. Or 15:12/13? Not sure, though. Wish I was more nimble with the MPEG format.

I can see no way that this thing can be in any other row than the one it's currently in. Leg macroblocks look in correct and unchangeable positions to me. The only place it could go is to the right. It may be possible to calculate an approximate position based on known bitpositions and average macroblock size. But try1 seems to contain large chunks of junk so that approach is questionable.

Here's what it would look like if it was shifted 1 block to the right. Pretty weird, but kind of fits. Plus this change would lighten the columns under it and darken the ones to the left. And the leg tips look pretty weird on iframe 5 as well. But again, I do not know if that such a misalignment of blocks in macroblocks is even theoretically possible.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: JBF on 05/11/2014 12:27 AM
Let me just say you guys are amazing.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/11/2014 01:00 AM
Here's what it would look like if it was shifted 1 block to the right. Pretty weird, but kind of fits. Plus this change would lighten the columns under it and darken the ones to the left. And the leg tips look pretty weird on iframe 5 as well. But again, I do not know if that such a misalignment of blocks in macroblocks is even theoretically possible.

I do like the fit of that image, as you can make out the curve of the tip of the leg. Perhaps the bright white is another camera window artifact, like the missing part of the left side of the left leg - maybe it just caught the white border in just the right way in the light, and you wound up with a magnified or brightened reflection. There's a similar sort of oddity at the tip of the leg in the current rendition of frame 5 as you observed - perhaps nailing that down more fully would help figure this one out.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: seanpg71 on 05/11/2014 03:18 AM


I changed 11:1:5384 to 22:1:5384 to properly align the dark dirt in top center... I really should give them numbers ;)
And also 30:2:-1 to 3:0:-1
Now a lot of dirt blobs start to align... Now I really gonna number them.. :)

5:1:-1,22:1:5384,0:3:-1,2:3:10968,10:3:-1,13:3:14762, 33:3:-1,35:3:16382,37:3:16550,34:4:-1,6:5:17911, 30:5:19483,32:5:-1, 36:5:20286,8:6:-1,9:6:21857, 16:10:-1,17:10:36140,13:11:-1,17:11:39697,38:11:-1, 1:12:41855,16:12:-1,29:12:43895,20:13:-1,13:14:51698, 38:14:-1,14:15:51794,36:15:-1, 37:16:65876,18:17:-1,14:18:74612,30:20:-1,24:27:127453,6:28:-1



More uninteresting parts of the image!

4:1:-1,22:1:5384,5:2:-1,6:2:7531,0:3:-1,2:3:10968,10:3:-1, 13:3:14762,33:3:-1,35:3:16382,37:3:16550,34:4:-1,6:5:17911,30:5:19483,32:5:-1, 36:5:20286,8:6:-1,9:6:21857,16:10:-1,17:10:36140,13:11:-1, 17:11:39697,38:11:-1,2:12:42062,16:12:-1,27:12:-1,29:12:43895,20:13:-1,13:14:51698, 38:14:-1,14:15:51794,36:15:-1,37:16:65876,18:17:-1,14:18:74612, 30:20:-1,41:20:91288,32:21:-1,34:21:96927, 25:22:-1,41:22:104774,0:24:-1,17:26:122506,0:27:-1,24:27:127453,6:28:-1


Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/11/2014 04:02 AM
Working on those headers I found that some errors are single bit flips, but there are also stripes of bytes which are simply garbage/randomly turned bits one after an other. In an image you cant repair this I think. Substitute MBs maybe.
Headers can also be fixed, but if part of an frame MB itself is messed up like this you cant really hope to fixed it by flipping random bits.

Yeah, been working on the raw and the corruption is much worse than I had assumed. Not only are there huge swaths of garbage, but also probable missing bits (like the 4-bit offset that was found for sure).

Now I see where the aeroquartet guy was coming from - it *should* be impossible to repair this video - but all the more reason why the work being done here is so amazing. At the current pace, we'll have silky smooth HD video before the next launch :)

I'm going to continue playing with michaelni's tsfix, but I don't see a way that this can be brute-force cleaned. Missing (and possibly added) bits expand the search space well beyond computational feasibility. We might be able to tidy up some of the intermediate frames some, but the masters will have to be done by hand. So keep up the awesome work bit flippers and block movers!

One note to the maintainer of slapbet - it would probably be very easy to integrate my ffmpeg log parsing code into the webapp to enable bad-block highlighting with a checkbox. See ~499 of http://pastebin.com/bKgzCRqD and following - just string and integer parsing in Java which is a trivial port to js. 

Edit: also, a mouseover showing the x:y would be nice :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/11/2014 04:06 AM
Working on those headers I found that some errors are single bit flips, but there are also stripes of bytes which are simply garbage/randomly turned bits one after an other. In an image you cant repair this I think. Substitute MBs maybe.
Headers can also be fixed, but if part of an frame MB itself is messed up like this you cant really hope to fixed it by flipping random bits.

Yeah, been working on the raw and the corruption is much worse than I had assumed. Not only are there huge swaths of garbage, but also probable missing bits (like the 4-bit offset that was found for sure).

Now I see where the aeroquartet guy was coming from - it *should* be impossible to repair this video - but all the more reason why the work being done here is so amazing. At the current pace, we'll have silky smooth HD video before the next launch :)

I'm going to continue playing with michaelni's tsfix, but I don't see a way that this can be brute-force cleaned. Missing (and possibly added) bits expand the search space well beyond computational feasibility. We might be able to tidy up some of the intermediate frames some, but the masters will have to be done by hand. So keep up the awesome work bit flippers and block movers!

One note to the maintainer of slapbet - it would probably be very easy to integrate my ffmpeg log parsing code into the webapp to enable bad-block highlighting with a checkbox. See ~499 of http://pastebin.com/bKgzCRqD and following - just string and integer parsing in Java which is a trivial port to js. 

Edit: also, a mouseover showing the x:y would be nice :)

This is all incremental. Anything you get us thats better than we have right now is progress!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/11/2014 07:09 AM
iframe 6 (try1)
Left leg is not fully deployed yet, but i'm perplexed by the tip of it. At least I think it's the tip because it doesn't look like a cloud or a foam spot on the sea. It kind of resembles the tip of the right leg on iframe 5, which shows the right leg in similar stage of deployment as left leg on this frame. The thing is it doesn't align. It's off by one block (half of macroblock). Is such an alignment error even possible?

If I understand your question correctly: yes.

It is possible the data from one block (not macroblock) is used by another block within the same macroblock. This would for example happen if somehow a bad bit was interpreted as a "stop-bit" of the data of a block. This way the next block (within the same macroblock) would use the data from its previous block. The other way around is also possible I believe.

This is all due to the variable length the VLC-codes can have. And they are the ones that have the "stop-bit". See here for more info on 8x8 block decoding (http://pastebin.com/pLZQReSP).
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/11/2014 10:44 AM
It's off by one block (half of macroblock). Is such an alignment error even possible?

If I understand your question correctly: yes.

It is possible the data from one block (not macroblock) is used by another block within the same macroblock. This would for example happen if somehow a bad bit was interpreted as a "stop-bit" of the data of a block. This way the next block (within the same macroblock) would use the data from its previous block. The other way around is also possible I believe.

No, I knew that, what I meant to ask was whether blocks from one macroblock can end up inside another in a way that would result in macroblock contents being offset by half of macroblock. It probably isn't. Which would mean that the odd object is not the tip of the leg.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/11/2014 12:29 PM
No, I knew that, what I meant to ask was whether blocks from one macroblock can end up inside another in a way that would result in macroblock contents being offset by half of macroblock. It probably isn't. Which would mean that the odd object is not the tip of the leg.
Ah ok. I think that can happen too: when the first macroblock thinks it has seen the 4 blocks but those stop-bits weren't all real stop-bits the next macroblock will pick-up the blocks from the previous macroblock. But then you would also expect some color-trouble because those would also be interpreted wrong I guess. I think Michael Niedermayer knows this better though.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/11/2014 12:50 PM
@Michael Niedermayer:

I've been investigating the possibility of automatically setting all L1,L2,L3,L4,C1,C2 values for all macroblocks for a frame. Basicly inputting an mmb and letting a script detect "bad lines" in the picture. Moving from the right-bottom corner to the upper-top.

I'm starting to believe (using some extra logging and some manual experimentation) that this might actually be possible. If it does this would improve the quality of the frames quite dramatically and would save a lot of manual work!

However there is one thing that I find difficult to do myself. And I think you might be able to help us. Instead of using this kind of setting:

15:14:-1:0:-40:0:0:0:20

where I replace the current macroblock with a blank one and change the DC values.

But what I really want to do is this:

15:14:57217:0:-40:0:0:0:20

In other words: I want to keep the macroblock with its (structured) data as much as I can (at the very least the blocks in it which are set to 0 above) but change only the DC "starting" values. So not blanken it, but modifying it so these DC-values make sure propogation to other blocks works the same as now with -1 (therefore fixing the color and intensity of following blocks) but I also keep the current blocks inside the current macroblock.

Is something like that possible?

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/11/2014 02:35 PM

So it seems like it should be practical to write a tool that can walk through the raw.ts and use the rules of the packet header formatting and sequence numbers to reconstruct a more coherent transport stream. Even just getting all the errors out of the null packets by replacing everything with FF might make it easier to discern good patterns.

Edit: For example, in raw.ts there are bytes missing or added between 0x37ACC and 0x371EC - that's the first place where the columns shift in a 47-column hex display.

Working on this, in the meantime, here is a plot of the bit errors per null packet in raw.ts. This was created by finding probable null packet headers (p > 0.85) then counting bits deviating from 1 within the 188 bytes.


Edit: BTW with some quick math, assuming raw.ts is ~30s, those big spikes of 5-8 packets (30-45ms) are almost certainly lightning EMI. Must have been some storm...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/11/2014 02:49 PM
I found out why iframe 0 didn't extract - the transport stream header for the first packet of the iframe had its "transport error indicator" bit flipped, and so that packet containing the very first piece of the iframe was ignored, resulting in the entire iframe being ignored.

If you take try1.ts and change offset 0x3E3D1 from 0x43 to 0x03, clearing the TEI bit, then iframe 0 should pop out. It may have some issues since the sequence numbers are rather jumbled right there - it looks like 0x3E48F should be 0x5C rather than 0x5F, and that might contain the adaptation field info for the iframe. The sequence doesn't seem to get back on track until you get to 0x3E83B.


Edit: Coffee deficit - the 0x80 bit is the TEI, not the 0x40 bit.  ::)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/11/2014 03:10 PM
I found out why iframe 0 didn't extract - the transport stream header for the first packet of the iframe had its "transport error indicator" bit flipped, and so that packet containing the very first piece of the iframe was ignored, resulting in the entire iframe being ignored.

If you take try1.ts and change offset 0x3E3D1 from 0x43 to 0x03, clearing the TEI bit, then iframe 0 should pop out. It may have some issues since the sequence numbers are rather jumbled right there - it looks like 0x3E48F should be 0x5C rather than 0x5F, and that might contain the adaptation field info for the iframe. The sequence doesn't seem to get back on track until you get to 0x3E83B.

No its more than that. The headers are wrong so it doesn't decode and by "wrong" i mean complete garbage beyond a certain point. I don't think the TEI bit is related.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/11/2014 03:26 PM
I found out why iframe 0 didn't extract - the transport stream header for the first packet of the iframe had its "transport error indicator" bit flipped, and so that packet containing the very first piece of the iframe was ignored, resulting in the entire iframe being ignored.

If you take try1.ts and change offset 0x3E3D1 from 0x43 to 0x03, clearing the TEI bit, then iframe 0 should pop out. It may have some issues since the sequence numbers are rather jumbled right there - it looks like 0x3E48F should be 0x5C rather than 0x5F, and that might contain the adaptation field info for the iframe. The sequence doesn't seem to get back on track until you get to 0x3E83B.

No its more than that. The headers are wrong so it doesn't decode and by "wrong" i mean complete garbage beyond a certain point. I don't think the TEI bit is related.

Not complete garbage, but very close - Based on the null-packet error estimation there is something like a 15% *average* error rate during iframe 0. This is substantially worse than the rest of the data, which hangs between 0.1% and 1.5%.

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/11/2014 03:32 PM
I found out why iframe 0 didn't extract - the transport stream header for the first packet of the iframe had its "transport error indicator" bit flipped, and so that packet containing the very first piece of the iframe was ignored, resulting in the entire iframe being ignored.

If you take try1.ts and change offset 0x3E3D1 from 0x43 to 0x03, clearing the TEI bit, then iframe 0 should pop out. It may have some issues since the sequence numbers are rather jumbled right there - it looks like 0x3E48F should be 0x5C rather than 0x5F, and that might contain the adaptation field info for the iframe. The sequence doesn't seem to get back on track until you get to 0x3E83B.

No its more than that. The headers are wrong so it doesn't decode and by "wrong" i mean complete garbage beyond a certain point. I don't think the TEI bit is related.

Not complete garbage, but very close - Based on the null-packet error estimation there is something like a 15% *average* error rate during iframe 0. This is substantially worse than the rest of the data, which hangs between 0.1% and 1.5%.

It was very oddly placed, at the EXACT location that the user data header was to start, it was replaced with a different "error" header with random noise in it. All the header data before that location was perfect without a single bit flip. I could not find anything that looked remotely like the standard data (thats the same for every iframe) in it. It almost looked like it was intentional and the encoder decided to not use this iframe.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/11/2014 03:33 PM
Thanks for the clarifications!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/11/2014 03:51 PM
I found out why iframe 0 didn't extract - the transport stream header for the first packet of the iframe had its "transport error indicator" bit flipped, and so that packet containing the very first piece of the iframe was ignored, resulting in the entire iframe being ignored.

If you take try1.ts and change offset 0x3E3D1 from 0x43 to 0x03, clearing the TEI bit, then iframe 0 should pop out. It may have some issues since the sequence numbers are rather jumbled right there - it looks like 0x3E48F should be 0x5C rather than 0x5F, and that might contain the adaptation field info for the iframe. The sequence doesn't seem to get back on track until you get to 0x3E83B.

No its more than that. The headers are wrong so it doesn't decode and by "wrong" i mean complete garbage beyond a certain point. I don't think the TEI bit is related.

Not complete garbage, but very close - Based on the null-packet error estimation there is something like a 15% *average* error rate during iframe 0. This is substantially worse than the rest of the data, which hangs between 0.1% and 1.5%.

It was very oddly placed, at the EXACT location that the user data header was to start, it was replaced with a different "error" header with random noise in it. All the header data before that location was perfect without a single bit flip. I could not find anything that looked remotely like the standard data (thats the same for every iframe) in it. It almost looked like it was intentional and the encoder decided to not use this iframe.

Maybe internal buffering after the encoder was brought online? I could see this with a primitive ASIC (or FPGA (or  RTOS)) encoder with a "output valid" signal that was ignored, so that with constant bitrate the resulting junk made it to the transmitter.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/11/2014 04:12 PM
Playing more with the raw.ts error data in MATLAB, I estimate 85% of the data has error rate < 1%, and ~3.0% of the data has an error rate above 12%

Edit: BTW with some quick math, assuming raw.ts is ~30s, those big spikes of 5-8 packets (30-45ms) are almost certainly lightning EMI. Must have been some storm...

Thinking out loud: If this is the case, we can reliably match "strike regions" where P(Err) is greatest in raw.ts before frame extraction, then maybe we can have ffmpeg lock C/L values within those regions. Since we can be relatively confident, i.e. P(Err) < 1%, in the packet integrity on either side of the strike, it should be easier to resync once we are "out of the weeds".

Thoughts?


Edit: to clarify my quote above, the strikes are 5-8 null-packet-counts in duration. There are 4-5 data packets between null packets. So that's locking 25-40 total packets per strike, or about 3-5k of real data.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: gospacex on 05/11/2014 04:46 PM
Parsing raw.ts using info from http://en.wikipedia.org/wiki/MPEG_transport_stream

The file consists of 188-byte ts-packets.

1st byte is always 47.

2 next bytes are 000<packetid13bits>, and apparently
they are always 03e8 for non-null ts-packets and 1fff for null ts-packets.
High-order bits mean "Transport Error Indicator" (0),
"Payload Unit Start Indicator" (0), "Transport Priority" (0).

4th byte is always 1x in both types of packets, where x cycles from 0 to f.
High-order bits 0001 mean "no scrambling" (00), "no adaptation field" (0), "payload exists" (1).

See below the listing of first few bytes of first few dozens of ts-packets, generated with
   hexdump -e '"%07.7_ax " 188/1 "%02x" "\n"' <raw.ts
command:

0000000 4703e95f81e103e2...
00000bc 4703e8103de99073...
0000178 4703e81176eeba37...
0000234 479779692f580085
00002f0 4703e413669bd0b3
00003ac 4713e8745300b09b
0000468 4703e81541a70f05
0000524 4703e8164aa9f008
00005e0 4703e817b70130e8
000069c 4703e818795c22a1
0000758 4703e8199ac05d5c
0000814 4703439a7a535507
00008d0 4703e81b06633dc1
000098c 4703e81c264b8290
0000a48 4703e81dcc29a6ab
0000b04 4703e81e107e0382
0000bc0 4703e81ff07cd801
0000c7c 4763e810a60791d7
0000d38 4703e8114386cd32
0000df4 4703e812041d1198
0000eb0 4703e813ffb86c32
0000f6c 4703e8144008f650
0001028 4703e815909cfbd4
00010e4 4703e816589e8010
00011a0 470389071ee887cb
000125c 471ac8d8ea28a720
0001318 47be1c520419650d
00013d4 474842d0f2fde345
0001490 477883798db92230
000154c 477c538a24031797
0001608 47d08588d2899b38
00016c4 477d03c3c0c0da0c
0001780 4703d41fb89610bc
000183c 4703e8301a005fff
00018f8 4740001c0000b011
00019b4 471fff12ffffffff
0001a70 471fff13ffffffff
0001b2c 4797ef1a9fdbffff
0001be8 475fff15dbb77e4e
0001ca4 474ebfbf9fffffff
0001d60 471fff17f7ffcfff
0001e1c 471ffb1827357f43
0001ed8 471fff19ffffffff
0001f94 471fd72aff5fffef
0002050 471fff1bff3ffd7f
000210c 471b1f02bfe7ffff
00021c8 478d5fe8e7476eb9
0002284 47fc19a23faa43df
0002340 470f77d412ac2e05
00023fc 47fff10fdfff3fff
00024b8 47bfe3bffffff5fb
0002574 471fff12ffffffff
0002630 471fff13ffffffff
00026ec 4f1fcf14ffffffff
00027a8 47d64ba1bfffffff

cat rawhex | sed 's/4703e81/.START=/g' | sed 's/471fff1/.NULL==/g'

and perusing resulting text file,
it's obvious that last good ts-packet header is at 0037a10:

0037a10 4703e81e0cf8280187ea...

The next +188 byte offset is:

0037acc 4703681dc10b930c9eb5b9fc....

it looks like moderately garbled ts-packet, but after that
there are no longer any recognizable ts-packet headers at +188 offsets.

The next recognizable ts-header is at 00381ec (not a +188 offset),
and from this location 188-byte packets continue from there.

The next discontinuity is at 00d6e20:

00d6bec 4b03e81c929ef0c...
00d6ca8 45abe11d923e21a...
00d6d64 4703e81e66c3272...
00d6e20 4703e81fa2080de... <== last good hdr
00d6edc 14e09aeb3fd7402...
00d6f98 a3582165fad84dd...
00d7054 11f93b7d90446f5...
00d7110 4e6f8b9b0de6ec6...

and the re-sync happens a bit later here:

00d7250 4703e815f4f261
00d730c 4703e816209781
00d73c8 4703e817784253
00d7484 4743e998e38153
00d7540 4703e8190cbc42

Oh... that's strange... I always need to seek by exactly 132 bytes to get
ts-packet headers to realign... can't be a coincidence!

Overall, there seems to be six sections of correctly-sized ts-packets:

1st_ofs - offset_of_last_recognizable_ts_packet_header
0000000 - 0037acc
00381ec - 00d6e20
00d7250 - 0215c58
0215cdc - 0356ae0
0357378 - 03dbe8c
03dd16c - 04460f0
04461aa (eof, last packet is two bytes short)

Between each section is a block of few hundred bytes.
Its size is always N*188 + 132. ???

I can "repair" the file by padding each section so that its size is a multiple of 188,
and then repair ts-packet headers: it's easy te determine whether the packet is
a null packet or payload packet, and then fix headers to be either
4703e81x or 471fff1x.

Will this be useful in any way?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: aero on 05/11/2014 05:01 PM
Quote
Oh... that's strange... I always need to seek by exactly 132 bytes to get
ts-packet headers to realign... can't be a coincidence!

hmm
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/11/2014 05:12 PM
Overall, there seems to be six sections of correctly-sized ts-packets:

1st_ofs - offset_of_last_recognizable_ts_packet_header
0000000 - 0037acc
00381ec - 00d6e20
00d7250 - 0215c58
0215cdc - 0356ae0
0357378 - 03dbe8c
03dd16c - 04460f0
04461aa (eof, last packet is two bytes short)

Between each section is a block of few hundred bytes.
Its size is always N*188 + 132. ???

I can "repair" the file by padding each section so that its size is a multiple of 188,
and then repair ts-packet headers: it's easy te determine whether the packet is
a null packet or payload packet, and then fix headers to be either
4703e81x or 471fff1x.

Will this be useful in any way?
Yes. There were 5 times 56 bytes missing (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1191647#msg1191647) in the raw.ts. Very weird indeed.

This was fixed (among other things) in the coerced.ts. I am not sure if it was fixed in the try1.ts.

But actually it's good to have a raw_aligned.ts where only this is fixed by padding 5 times 56 bytes with FF.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: gospacex on 05/11/2014 06:13 PM
The script to cut up raw.ts into sections with 188-byte ts-packets.
The last two commands prove that concatenation of sections is the same as raw.ts (both have 042433dfc39c70a51203fae581a5461f md5sum).

#!/bin/sh
mk_sect()
{
        local start=$1
        local end=$2
        dd bs=1 skip=$(($start)) count=$(($end - $start)) <raw.ts >$3.ts
        cat section?.ts >z.$$
        hexdump -s $(($start)) -v -e '"%07.7_ax " 188/1 "%02x" "\n"' <z.$$ >$3.hex
        rm z.$$
}
# 1st_ofs - offset_of_last_recognizable_ts_packet_header
# 0000000 - 0037acc
# 00381ec - 00d6e20
# 00d7250 - 0215c58
# 0215cdc - 0356ae0
# 0357378 - 03dbe8c
# 03dd16c - 04460f0
# 04461aa (eof, last packet is two bytes short)
#
# It is not obvious where exactly realignment happens.
# What can be seen is where the last packet with "old" alignment is,
# and where is the first recognizable packet with "new" alignment is.
#
rm section?.ts
mk_sect 0x0000000 0x00381ec section1
mk_sect 0x00381ec 0x00d7250 section2
mk_sect 0x00d7250 0x0215cdc section3
mk_sect 0x0215cdc 0x0357378 section4
mk_sect 0x0357378 0x03dd16c section5
mk_sect 0x03dd16c 0xfffffff section6
cat section?.ts | md5sum
cat raw.ts | md5sum
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/11/2014 06:21 PM
The script to cut up raw.ts into sections with 188-byte ts-packets.
The last two commands prove that concatenation of sections is the same as raw.ts (both have 042433dfc39c70a51203fae581a5461f md5sum).

#!/bin/sh
mk_sect()
{
        local start=$1
        local end=$2
        dd bs=1 skip=$(($start)) count=$(($end - $start)) <raw.ts >$3.ts
        cat section?.ts >z.$$
        hexdump -s $(($start)) -v -e '"%07.7_ax " 188/1 "%02x" "\n"' <z.$$ >$3.hex
        rm z.$$
}
# 1st_ofs - offset_of_last_recognizable_ts_packet_header
# 0000000 - 0037acc
# 00381ec - 00d6e20
# 00d7250 - 0215c58
# 0215cdc - 0356ae0
# 0357378 - 03dbe8c
# 03dd16c - 04460f0
# 04461aa (eof, last packet is two bytes short)
#
# It is not obvious where exactly realignment happens.
# What can be seen is where the last packet with "old" alignment is,
# and where is the first recognizable packet with "new" alignment is.
#
rm section?.ts
mk_sect 0x0000000 0x00381ec section1
mk_sect 0x00381ec 0x00d7250 section2
mk_sect 0x00d7250 0x0215cdc section3
mk_sect 0x0215cdc 0x0357378 section4
mk_sect 0x0357378 0x03dd16c section5
mk_sect 0x03dd16c 0xfffffff section6
cat section?.ts | md5sum
cat raw.ts | md5sum

Just a note. Dump any code-like text into
[code]
[/code]
blocks
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: gospacex on 05/11/2014 06:35 PM
And after this, it's instructive to check what kind of packets do we have there?

grep . section?.hex | sort -t' ' -k2

There are lots of data and null packets, of course, but also there are:

474000xx packets:

section3.hex:00e7da0 474000100000b0110000c100000000e0100001e020d36af0acfffffffff...
section3.hex:01c2c98 474000100000b0110000c100000000e0100001e020d36af0acfffffffff...
section4.hex:028cae4 474000100000b0110000c100000000e0100001e020d36af0acfffffffff...
section4.hex:0353f8c 474000100000b0110000c100000000e0100001e020d36af0acfffffffff...
section6.hex:04065a0 474000100000b0110000c100000000e0100001e020d36af0acfffffffff...

474020xx packets:

section3.hex:00eab28 474020100002b01f0001c10000e3e8f00010e3e8f0031b01f580e3e9f00081e3f3f0003f64f115ffff...
section3.hex:01c6468 474020100002b01f0001c10000e3e8f00010e3e8f0031b01f580e3e9f00081e3f3f0003f64f115ffff...
section4.hex:029d868 474020100002b01f0001c10000e3e8f00010e3e8f0031b01f580e3e9f00081e3f3f0003f64f115ffff...
section6.hex:0408768 474020100002b01f0001c10000e3e8f00010e3e8f0031b01f580e3e9f00081e3f3f0003f64f115ffff...

0x40 bit is Payload Unit Start Indicator. I have no idea what's that,
but it has quite easily recognizable bit pattern in the payload.
Meaning: it's easy to detect such packets and fix their headers even
if it's severely corrupted.

And the 474000 + 4703e8 combination also exists:

section2.hex:004817c 4743e8300710002b495d7e00400181e00000818007410019d755ffff000001b6597...
section2.hex:00ae4f0 4743e8300710002c9d8c7e00000001e000008180072101652811ffff000001b658c...
section3.hex:0117d0c 4743e8300710002df1bb7e00000001c000c081800721016f78cdffff000001b0030...
section3.hex:011bdac 4743e8300710002dfd767e00000001e0000081800721016fa7b9ffff000001b658c...

The 30 byte is "Adaptation field exist" bit 0x20 set in addition to the usual "Payload present" 0x10 bit.

Looking for less obvious "unusual" packets, "Adaptation field exist" bit seems to be present
in some payload packets too. They are visible because they have a bunch of FF bytes
in the beginning:

section3.hex:017cbf0 4703e8300b00ffffffffffffffffffff843695a94aad38fbed264a01a2503272de7134078280741e03f93613ff2abc4e935...
section1.hex:000183c 4703e8301a005ffffffffffd7fffffffffffc7ff6ffffffffffffffffffffffffffffffff5ffc3ffffffffffffffff18211...

Note 4703e83x header instead of more typical 4703e81x.
It's not a one-bit corruption because there are a number of these,
they all have ...ffffffffff... at the start of their payload.
"Usual" data packets don't.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/11/2014 06:41 PM
I can "repair" the file by padding each section so that its size is a multiple of 188, and then repair ts-packet headers: it's easy te determine whether the packet is a null packet or payload packet, and then fix headers to be either 4703e81x or 471fff1x.

Will this be useful in any way?

I think you don't want to force all headers to either of those two values - I think we would just want to clear the TEI and the scrambling bits only, and make a 0x471FFFX into a 0x471FFF1. There appear to be instances where the adaptation field and payload unit start bits are significant.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: gospacex on 05/11/2014 06:51 PM
Trivial script to concatenate sections with padding, so that every ts-packet is 188-byte aligned:
{
cat section1.ts; dd bs=56 count=1 </dev/zero
cat section2.ts; dd bs=56 count=1 </dev/zero
cat section3.ts; dd bs=56 count=1 </dev/zero
cat section4.ts; dd bs=56 count=1 </dev/zero
cat section5.ts; dd bs=56 count=1 </dev/zero
cat section6.ts
} >aligned188.ts
hexdump -v -e '"%07.7_ax " 188/1 "%02x" "\n"' <aligned188.ts >aligned188.hex
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/11/2014 08:08 PM
An improved version of the annotated iframe8 with many more common dirt spots I see in several other iframes.
They seem to be very useful for further aligning MB's in other iframes
I truly believe that in the long run this is gonna help us enormously.

Thank you Jakusb. Incredibly good!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: gospacex on 05/11/2014 09:38 PM
First cut at ts-packet fixing tool.
How to use:

(0) compile: gcc -Os -Wall fix_tspackets.c -o fix_tspackets
(1) run cut_188byte_sections.sh.txt (cuts raw.ts into sections)
(2) fix a section:

$ ./fix_tspackets -o 0x03dd16c section5.ts section5-1.ts
Loaded 2917 packets
03ea430: 4740001x packet (h:47400011 diff:0.000000)
03f0b00: 4740001x packet (h:47400012 diff:0.000000)
03fd08c: 4740001x packet (h:47800013 diff:0.000000)
03fd08c: fixing header from 47800013 to 47400013
03fd08c: fixing payload
0408d48: 4740001x packet (h:47400014 diff:0.000000)
0410094: 4740001x packet (h:47400015 diff:0.000000)
0424a50: 4740001x packet (h:47400017 diff:0.000000)
042947c: 4740001x packet (h:46000018 diff:0.083333)
042947c: fixing header from 46000018 to 47400018
042947c: fixing payload
0437188: 4740001x packet (h:47400019 diff:0.000000)
0445764: 4740001x packet (h:4740001a diff:0.000000)
044e230: 4740001x packet (h:4740001b diff:0.000000)
045e9d4: 4740001x packet (h:4740001c diff:0.000000)
045e9d4: fixing payload
Saved to 'section5-1.ts'

(-o OFFSET is cosmetic: it adjusts packet offsets so that they can match raw.ts)

(gosh... I need to add .txt to attachments for them to be posted. strip it...)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/12/2014 12:20 AM
<snip>
The next block of XXs is part of the Presentation Time Stamp PTS
how to get the PTS is explained quite well here: http://dvd.sourceforge.net/dvdinfo/pes-hdr.html
PTS: 21 01 xx xx xx.
We can now decode the PTS of the good frames and define the PTS of the messed up ones.

It is late again, my bed is calling.
I will calculate the PTS tomorrow, also will work out that last xx xx.
Here are the raw 5 byte PTS of the frames:

IF00: 99 31 59 14 6C
IF01: 21 01 61 23 ED
IF02: 21 01 67 CE 5D
IF03: 21 01 6F 78 CD
IF04: 21 01 77 23 3D
IF05: 21 01 7D CD AD
IF06: 21 01 85 78 1D
IF07: 21 01 8D 22 8D
IF08: 21 01 93 CC FD
IF09:  21 01 9B 77 6D
IF10: 21 01 A3 21 DD
IF11: 21 01 A9 CC 4D
IF12: 21 01 B1 76 BD
IF13: 21 01 B9 21 2D
IF14: 21 01 BF CB 9D

Cheers
Shanuson

Here is the next part:
PTS values are separated by 120120:
All except the first one are OK:
The first should be:
0x210159797D

Also I missed one byte that changes from IF to IF, namely the 4th in each sequence: 47 43 E8 3x ......
This X is a continuity counter. But it is different from the counter of the 47 1F FF  1Y packets.
In the better part of the file one find that X count continue for packets with 47 03 E8 1X,  47 03 E8 3X and 47 43 E8 3X. so it seems it is a counter for each packet with the same PID (13 bit long), namely 0x13E8 or 0x1FFF
So to find the right value of X for each corrupt frame one has to check the surrounding packets with the same PID, which i did:
Here are the 15 correct continuity counter X values for all IF's: (red ones contained errors in raw.ts
B, D, 7, 0, D, D, 6, 6, F, 4, D, 4, 8, 7, 5

Finally back to that 4bit shifted part:
Thanks to gospaceX work I know now what to look for:

Here the part of the file (4x188 bytes copied out of raw.ts, starting from 0x23bb74):
Black part is the start of  an I-Frame header (What you find in raw-headers.txt)
First red block: Aligned right; some errors at the end where all should be ff.
Last red block: Also aligned right and all following blocks too.
Blue Block: Errors at the start, so no 47 start byte; should look like the one before, all FFs
Green Block, Start of the I-Frame, but shifted by 4 bits to the LEFT as can be seen by comparing that one to a correct one (first 4 bits are the D at the end of the blue block)
So, the Blue block could also be shifted but it does not matter since it should contain only FFs.
Finally to fix this, I think, one only has to move that black-green block by 4 bits to the right (including that last D of the blue block and remove the last green B).


47 1F FF 18 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 7F FC FF FF FF FD 7F F4 FF 7C F6 FF FF FF FF FF FF FF DE 7F 7D 76 FC F4 EB 6D 6F F6 FF 1F FD BF FF FF FF FF FF FF FF FB 03 EE 76 17 E0 1B DA 08 FF EB DF 14

C0 D8 61 01 F7 5F C8 3F FE 7B FF F8 7D 53 57 70 34 EB DC F3 3F 5C 82 FF F3 FE 3E BB 78 7F FF FF FF FF FF BB FE 67 FF FF FF CF FF 5F FF FE FF F9 FF FF E7 FF AE FF F9 FF FF FF FF FF FF FF FF FF DF F3 3F 9F F5 FF FF F3 DF D7 67 FA 37 EF AF CF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FA 7F E7 FF FF FF FD 38 E9 69 AF E7 EC F0 8F FF 2B 30 E5 53 5F FD BF FF FF B7 78 34 FA EE FF FF FF FD FF F3 FF FF FF FF 5F FC 3F EF FD 9F 63 FC 9C 0F F7 40 FC BA 86 2D 65 44 70 BF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FC 12 BA 06 FA A9 FF 4D

BE 66 AF 8C EC B9 A7 CD C3 5C 74 0C 28 21 3B 00 18 18 B5 00 72 10 18 D2 28 DF FF F0 00 00 1B 00 30 00 00 1B 50 D0 F0 00 00 10 00 00 00 12 00 0C 48 5C 00 04 65 48 01 34 00 9E FD E9 AF E7 C2 2E E2 C0 87 86 8C 2B C1 04 98 06 C8 D3 47 62 03 42 E3 32 E3 02 E3 00 0C 3F F0 00 00 1B 61 0C F0 C0 D6 08 65 8D 9F BD C7 A4 B8 18 F1 19 57 32 42 08 44 41 E0 08 B5 27 E4 46 BA 7A ED 28 0A AA 89 BD 3F 60 BA F5 9F 96 07 05 53 CA E7 AE 0B 8A 3D 2E FA 25 D2 79 3E 7F 88 72 DE 8E B1 EC FF C1 3C 88 13 31 E7 79 26 19 C7 E9 AF 33 38 CE 46 3A 56 4F 34 34 A8 9E 73 54 76 C4 D1 76 82 EB

47 03 E8 17 3D DB B8 2A AE 02 77 05 E6 E1 48 38 8F CB 54 AB 3F CD D5 EA 29 75 70 60 A4 B3 5C A8 41 C4 26 13 A4 F3 7F 6B FB 9D AD 22 45 A1 9F B4 5D 5D 5E 39 58 FC C9 F0 77 47 99 FD CF 6E 6B 9E 4C 6C 4D 48 C9 FB 7C FC 9F EA 0F F0 15 21 27 13 F3 FA B8 79 2F D7 4F 73 D2 58 28 A7 B4 F2 90 5D 4B B8 39 00 46 E1 40 4D BA C8 9C 33 25 A0 BA F6 B9 45 E9 E5 05 1D D7 71 D5 E6 73 B9 BD B9 3D 15 73 BD FD E1 A6 75 70 FC 82 47 D2 DE ED 7F 3D E8 F4 A0 0A 94 86 F9 17 D1 17 23 EA 4D BF D5 84 AF A7 EB 6B 1E 1B DB D2 BE 14 D0 60 90 B7 94 B8 AA 86 B0 20 47 18 0E 0A 23 E5 C9 8A 0B


Tomorrow, I will work out that last XX XX of the headers. I will then change the raw.ts file to include the right headers and that shift and upload it.

Cheers
Shanuson
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: gospacex on 05/12/2014 12:21 AM
How to use:

(-1) download these files into an empty dir, drop raw.ts there too
(0) run cut_188byte_sections.sh (cuts raw.ts into sections)
(1) compile: gcc -Os -Wall fix_tspackets.c -o fix_tspackets
(2) run fix_tspackets_in_sections.sh
    - fixes each section
    - copies fixed sections into fixed_sections/ subdir
    - glues them into "new" fixed_sections/raw.ts
    - runs cut_188byte_sections.sh on it
    - produces raw.hex.diff for you to peruse

With this source, resulting "new" raw.ts plays no worse than original
(maybe even better, it's hard to tell).

There is a catch, though. The most interesting part of this exercise
is to decode more data from packets with damaged headers.
The part of code which fixes packets is here:

        fix_4740001x_packets(pkt);
        fix_4740201x_packets(pkt);
        fix_4743e83x_packets(pkt);
        fix_null_packets(pkt);

and the part which tries to decide which packets are data, follows:

        // Does a bit too much damage by "fixing":
        //fix_data_packets(pkt, 0.3);
        // Fares better:
        //fix_data_packets(pkt, 0.25);

As you see, it is commented out. There are some heuristics in fix_data_packets() which avoid a few cases where damage is done, but apparently not enough.

I encourage interested people to uncomment last line, run the program,
look at raw.hex.diff, and suggest improvements to fix_data_packets() logic.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: michaelni on 05/12/2014 12:31 AM
Ah ok. I think that can happen too: when the first macroblock thinks it has seen the 4 blocks but those stop-bits weren't all real stop-bits the next macroblock will pick-up the blocks from the previous macroblock. But then you would also expect some color-trouble because those would also be interpreted wrong I guess. I think Michael Niedermayer knows this better though.

yes, with enough (bad) luck a block could end up in the wrong position, but the next MBs MCBPC & cbpy would then not be where they are expected very likely
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: michaelni on 05/12/2014 12:41 AM
Parsing raw.ts using info from http://en.wikipedia.org/wiki/MPEG_transport_stream

The file consists of 188-byte ts-packets.

1st byte is always 47.

2 next bytes are 000<packetid13bits>, and apparently
they are always 03e8 for non-null ts-packets and 1fff for null ts-packets.

there are more than 2 PID values in the ts packets/ file
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: michaelni on 05/12/2014 12:43 AM

Yes. There were 5 times 56 bytes missing (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1191647#msg1191647) in the raw.ts. Very weird indeed.

This was fixed (among other things) in the coerced.ts. I am not sure if it was fixed in the try1.ts.

IIRC edited, coerced and try1 all had the 188byte packets aligned (except the 4bit thing)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: gospacex on 05/12/2014 12:53 AM

Yes. There were 5 times 56 bytes missing (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1191647#msg1191647) in the raw.ts. Very weird indeed.

This was fixed (among other things) in the coerced.ts. I am not sure if it was fixed in the try1.ts.

IIRC edited, coerced and try1 all had the 188byte packets aligned (except the 4bit thing)

Aligning is easy, and possibly not that useful.
For one, it changes file offsets of ts-packets, making it harder for us to refer to the packets in speech.
E.g. "packet at offset XYZ needs its header fixed this way" - offset XYZ in *which .ts file*?

The hard part is repairing damaged headers, and subsequently, data.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/12/2014 02:38 AM
Operating on the ts packet would be easier if you set up the struct to map it all out:

typedef struct {
    unsigned char fourseven;
    unsigned tei:1;
    unsigned pus:1;
    unsigned pri:1;
    unsigned pid:13;
    unsigned scramble:2
    unsigned hasadapt:1;
    unsigned haspayload:1;
    unsigned count:4;
    unsigned char payload[184];
} tspkt;

See http://www.cs.cf.ac.uk/Dave/C/node13.html for more info.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/12/2014 04:12 AM

Yes. There were 5 times 56 bytes missing (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1191647#msg1191647) in the raw.ts. Very weird indeed.

This was fixed (among other things) in the coerced.ts. I am not sure if it was fixed in the try1.ts.

IIRC edited, coerced and try1 all had the 188byte packets aligned (except the 4bit thing)

Aligning is easy, and possibly not that useful.
For one, it changes file offsets of ts-packets, making it harder for us to refer to the packets in speech.
E.g. "packet at offset XYZ needs its header fixed this way" - offset XYZ in *which .ts file*?

The hard part is repairing damaged headers, and subsequently, data.

Aligning is easy, but also critical. There is no way to decode misaligned data. Luckily for us, michaelni has already written code to do this automatically (tsalign.c).

Referring to packet header locations is not that useful, as they are easily automatically repaired by the other program michaelni has written (tsfix.c).

It's as simple as

cat raw.ts | tsalign | tsfix > fixed.ts


The only hard part is repairing the data.

P.S. I saw you read all about mpeg on wikipedia, but were I you I'd listen to what he says:

Wooow! I didn't realize this at first. But we have Michael Niedermayer (http://www.ffmpeg.org/consulting.html) in the house!  8)  One of the creators of ffmpeg itself.

Just saying  ;)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Axiom on 05/12/2014 06:10 AM
Hi! After following this forum for several years as a silent consumer, I felt compelled today to try and add something to this superb effort (first post, so go easy :)

About the Iframes, i was thinking that manual moving the macroblocks around will be difficult if you don't have a lot of reference (i.e. if there aren't many features on the block). So what about getting some help from statistics. So, would it be helpful to statistically compare all valid macroblocks for the closest matching edges, then automatically reshuffle the macroblocks. Some results may be false, yet if you create several iterations of the image for visual confirmation, those could rapidly be checked. You could even pin correctly placed macroblocks in place, excluding them from the automatic reshuffle. Would that be of help? 

Furthermore, and this may not be feasable but wanted to post the idea anyway, would it be possible to use the last frame before the next iframe, as a reference point. So if you have a very good Iframe upstream, say iframe8, let the video run until just one frame before iframe9, and use that as a reference for reconstructing Iframe9. Just my 50c.

Cheers!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Adaptation on 05/12/2014 06:11 AM
Alpha 3 of my transport stream search and repair tool is finished.

Seems like michaelni has been doing a good job at making me redundant.   ;)
Really some top notch work here, I'm impressed. 

Hopefully my program can still help with manual searches and bit manipulation needs... for people with windows. 
 

Made the search capable of finding results that are not byte aligned.  (This feature includes a wildcard filter and can return results with a specified tolerated number of bit errors.) 
Enabled bit level insertion and deletion. (will bit shift everything after that point in the file)
Bit deletion at arbitrary point in the byte, bit insertion is after the byte selected, (first bit of next byte) to insert in the middle of a byte you can bit shift the whole file so your desired insertion point is between two bytes or select the byte ahead, insert and twiddle the bits back to how you want them. 

Switched most of the operations out of managed managed memory and most of the math is now 64 bit.

Fixed quite a few bugs but several remain, it will still crash if you operate most of the controls before opening a ts file so make sure to open one first.

Still plan on adding a bit level find/replace feature, adding an autosave/undo and colorizing the background between packet boundaries and search results in the hex editor. 

<edit> removed transport stream program </edit>
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/12/2014 06:43 AM
Hi guys,

I have a bit of a mystery going on and it's holding me back from doing great stuff. ;) Hopefully somebody else has an idea what is going on.

This is essentially about how luminance/chrominance (errors) propagate. From what I understood so far each (macro)block propagates to its right (macro)block and to its bottom (macro)block. Luminance does it on a block (8x8pixel) basis while chrominance does it on a macroblock basis. So if you set macroblock 0,0 to "more red" the entire picture becomes that much "more red". Right?

Now here is my mystery example.

Take iframe 12 (try1) and enter 0:0:-1,13:24:72730 as mmb. It shows the bottom part of the picture and it looks quite nice.

Now enter 0:0:-1,12:24:72687 and suddenly you see a luminosity error appearing far from the block we just changed. How is this possible??

I suspect this has to do with the fact that the lower-right block of 24:26 is getting different input from above or left, but I don't understand why it could be so different from before. Maybe some kind of dc-clipping? (but there is no error log of that I think) Is this block reading the bits differently because it has different input? Because I always thought that the reading/meaning of the bits was not determined by the input.

I'm confused by this.

Regards,

arnezami

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/12/2014 07:16 AM
Hi guys,

I have a bit of a mystery going on and it's holding me back from doing great stuff. ;) Hopefully somebody else has an idea what is going on.

This is essentially about how luminance/chrominance (errors) propagate. From what I understood so far each (macro)block propagates to its right (macro)block and to its bottom (macro)block. Luminance does it on a block (8x8pixel) basis while chrominance does it on a macroblock basis. So if you set macroblock 0,0 to "more red" the entire picture because that much "more red". Right?

Now here is my mystery example.

Take iframe 12 (try1) and enter 0:0:-1,13:24:72730 as mmb. It shows the bottom part of the picture and it looks quite nice.

Now enter 0:0:-1,12:24:72687 and suddenly you see a luminosity error appearing far from the block we just changed. How is this possible??

I suspect this has to do with the fact that the lower-right block of 24:26 is getting different input from above or left, but I don't understand why I could be so different from before. Maybe some kind of dc-clipping? (but there is no error log of that I think) Is this block reading the bits differently because it has different input? Because I always thought that the reading/meaning of the bits was not determined by the input.

I'm confused by this.

Regards,

arnezami



I've seen this too. Either the offending block is messed up in some way that propagates the error, or MPEG behaves badly when values are clipped. The problem seems to be worse on frames where the top row is missing. I attached an image that shows the path from the bad block to where it gets noticeably bad.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: gospacex on 05/12/2014 07:33 AM

Yes. There were 5 times 56 bytes missing (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1191647#msg1191647) in the raw.ts. Very weird indeed.

This was fixed (among other things) in the coerced.ts. I am not sure if it was fixed in the try1.ts.

IIRC edited, coerced and try1 all had the 188byte packets aligned (except the 4bit thing)

Aligning is easy, and possibly not that useful.
For one, it changes file offsets of ts-packets, making it harder for us to refer to the packets in speech.
E.g. "packet at offset XYZ needs its header fixed this way" - offset XYZ in *which .ts file*?

The hard part is repairing damaged headers, and subsequently, data.

Aligning is easy, but also critical. There is no way to decode misaligned data. Luckily for us, michaelni has already written code to do this automatically (tsalign.c).

Referring to packet header locations is not that useful, as they are easily automatically repaired by the other program michaelni has written (tsfix.c).

It's as simple as

cat raw.ts | tsalign | tsfix > fixed.ts


The only hard part is repairing the data.

Unfortunately, it is not that easy to automatically
fix headers without damaging data even more.

I think it's better to have the corrections explained (so that they can be cross-checked and tuned).
For example, take edited.ts and coerced.ts. edited.ts has this sequence of packets:

00011a0 4703e8171ee887cbdfca43de18b9c2ea1a00c602180987dea67319134
000125c 4703e818ea28a720cc02d4aab0f256ba257cbc142a249ece63371f217
0001318 47be1c520419650d5d53bbcdb599034d0a9671ca97fbe0f475e8a40e0
00013d4 474842d0f2fde345f6dfbfd879228b38560c9b12c310942450f11ef62
0001490 477883798db922308145ac41ec97875bbc361389625238f2a0c8ec2ae
000154c 477c538a240317970fb67007230fca81c10d7055a74983e73efec042c
0001608 47d08588d2899b38c980f24655dd453b0f1fef99c87541c3458f9a9e4
00016c4 477d03c3c0c0da0c5ea8033e10419583170d016759e418bc1003fc9a0
0001780 4703e81fb89610bc12a1b164f81e27fe900e06540e126806979704207
000183c 4703e8301a005ffffffffffd7fffffffffffc7ff6ffffffffffffffff
00018f8 4740001c0000b0110000c100000000e0100001e020d36af0acfffffff
00019b4 471fff12ffffffffffffffffffffffffffffffffffffffff8480e4ffe
0001a70 471fff13fffffffffffffffffffffffffffffffffffffffffffffffff
0001b2c 4703e8149fdbfffffffffffffffffffffffffffffdfff3ffffaffe1fe
0001be8 4703e815dbb77e4e2fffefff9fffcfff5fffe17fb8cfe799def8fe67f

and in coerced.ts, the corresponding packets are

0001318 4703e8171ee887cbdfca43de18b9c2ea1a00c602180987dea67319134
00013d4 4703e818ea28a720cc02d4aab0f256ba257cbc142a249ece63371f217
0001490 471fff12fffffffffffffffffffffffffffffffffffffffffffffffff
000154c 471fff13fffffffffffffffffffffffffffffffffffffffffffffffff
0001608 471fff14fffffffffffffffffffffffffffffffffffffffffffffffff
00016c4 471fff15fffffffffffffffffffffffffffffffffffffffffffffffff
0001780 471fff16fffffffffffffffffffffffffffffffffffffffffffffffff
000183c 471fff17fffffffffffffffffffffffffffffffffffffffffffffffff
00018f8 4703e819b89610bc12a1b164f81e27fe900e06540e126806979704207
00019b4 4703e83a1a005ffffffffffd7fffffffffffc7ff6ffffffffffffffff
0001a70 474000110000b0110000c100000000e0100001e020d36af0acfffffff
0001b2c 471fff18fffffffffffffffffffffffffffffffffffffffffffffffff
0001be8 471fff19fffffffffffffffffffffffffffffffffffffffffffffffff
0001ca4 4703e81b9fdbfffffffffffffffffffffffffffffdfff3ffffaffe1fe
0001d60 4703e81cdbb77e4e2fffefff9fffcfff5fffe17fb8cfe799def8fe67f

As you see, packets 2..8 were marked as "null" and filled with FF's.
Granted, their headers look severely corrupted, but it's wrong
to turn them to "nulls". Check 8'th nibble. First two packets
have 7, 8 there. Continue incrementing it through the corrupted packets.
9,a,b,c,d,e... When you reach F, you get a match again, with the
4703e81fb89610bc12a1b... packet (see? it's awful when packets
have no fixed offsets - I need to refer to them by *contents*...)
Next is 0 - matches next packet.
Next is 1 - corrupted.
Next is 2 - match. 3 - match. 4 - match. 5 - match!

Those six packets with headers has been auto-fixed to "nulls", subsequent packets have their seq fields renumbered. This destroyed some data which was there.

We need to be more careful than that.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jakusb on 05/12/2014 07:49 AM

Hi guys,

I have a bit of a mystery going on and it's holding me back from doing great stuff. ;) Hopefully somebody else has an idea what is going on.

This is essentially about how luminance/chrominance (errors) propagate. From what I understood so far each (macro)block propagates to its right (macro)block and to its bottom (macro)block. Luminance does it on a block (8x8pixel) basis while chrominance does it on a macroblock basis. So if you set macroblock 0,0 to "more red" the entire picture because that much "more red". Right?

Now here is my mystery example.

Take iframe 12 (try1) and enter 0:0:-1,13:24:72730 as mmb. It shows the bottom part of the picture and it looks quite nice.

Now enter 0:0:-1,12:24:72687 and suddenly you see a luminosity error appearing far from the block we just changed. How is this possible??

I suspect this has to do with the fact that the lower-right block of 24:26 is getting different input from above or left, but I don't understand why I could be so different from before. Maybe some kind of dc-clipping? (but there is no error log of that I think) Is this block reading the bits differently because it has different input? Because I always thought that the reading/meaning of the bits was not determined by the input.

I'm confused by this.

Regards,

arnezami

I had a similar experience a few days back. I cannot remember which frame. I think 4 or 5.
I had to -1 the very last mb to have it mess up the clock. It is very strange to have a block down stream mess up the image up stream. Or isn't it?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/12/2014 08:52 AM
I've seen this too. Either the offending block is messed up in some way that propagates the error, or MPEG behaves badly when values are clipped. The problem seems to be worse on frames where the top row is missing. I attached an image that shows the path from the bad block to where it gets noticeably bad.
Yeah. I'm starting to realize that it is even harder than I thought it would be.

I wanted to create a "clean enviroment" for checking bad block propagation by starting at the end and working backward therefore avoiding the issue of having propagation from above and left.

But this won't work if block propagation below and to the right is different when adding more blocks to the left and above.

Hmmm. A rethink is required here.

Thanks for that diff!

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: hutchel on 05/12/2014 11:38 AM
I've seen this too. Either the offending block is messed up in some way that propagates the error, or MPEG behaves badly when values are clipped. The problem seems to be worse on frames where the top row is missing. I attached an image that shows the path from the bad block to where it gets noticeably bad.
Yeah. I'm starting to realize that it is even harder than I thought it would be.

I wanted to create a "clean enviroment" for checking bad block propagation by starting at the end and working backward therefore avoiding the issue of having propagation from above and left.

But this won't work if block propagation below and to the right is different when adding more blocks to the left and above.

Hmmm. A rethink is required here.

Thanks for that diff!

Regards,

arnezami


Quick thought from a quiet computer Geek in the audience.... Can Space X provide you with a good 30 sec chunk of video from the same setup so you can see what good video is supposed to look like?  That would get you ground truth as to formatting etc.  It would also give you the ability to selectively damage the bit stream to better understand propagation related errors.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/12/2014 02:33 PM

Unfortunately, it is not that easy to automatically
fix headers without damaging data even more.

I think it's better to have the corrections explained (so that they can be cross-checked and tuned).
For example, take edited.ts and coerced.ts. edited.ts has this sequence of packets:



Those six packets with headers has been auto-fixed to "nulls", subsequent packets have their seq fields renumbered. This destroyed some data which was there.

We need to be more careful than that.

If you look at tsfix.c, it defaults to assume a packet is video rather than null. And it does not fill null packets with 0xFF, it just fixes the header so they are skipped by ffmpeg. Also, I have been working on statistical analysis of packet contents (rather than just the header) and context (previous and next packets) to increase accuracy.

The problem with your approach is that it assumes the error rate in the header is low enough for a human to be able to recognize the different parts. It isn't, at least in a lot of the data, the error rate is 16-20% everywhere, including the header, not just the data. Humans can't help finding patterns in things, even if they are not there. Especially when you start taking packets out of order/context, it is hard to get the whole picture.

For example, as I posted earlier, there are spikes in noise level where error rate spikes above 30% for a few thousand bytes. If this occurred in a null packet it might look to you like a mildly corrupt data packet, but in reality it is junk.

The goal of cleaning the raw stream is to give the folks here the very best mpg4-img files to tweak visually. There are 23838 packets in raw.ts - IMO It is better to tune the algorithm rather than numbering the data and tuning by hand. I don't understand your concern, for any given code run on any given input, the output will always be the same. It's not like my output will be different than yours, cross referencing is always possible by offset.

Edit: gospacex - look at http://pastebin.com/HmDAzu1q to see my version of null packet identification. Looking again, it seems to be similar to yours. My next iteration will count runs of diffs instead of just the number of diffs.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/12/2014 03:06 PM
Ok, I am done with my work on the headers

attached you find a raw_edit1.ts (edit: file contained a bug with frame169(old Iframe8), you find the corrected version  here (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1197348#msg1197348)) where only the 15 I-Frame headers are changed (plus a few '47' bytes maybe), Also the one 188 byte block is reshifted by 4 bits to align it.

The last XX XX was part of vbv_occupancy (1 marker bit and the frist 7 of the latter half of vbv_occupancy)
According to michaelni, the actually value is not needed to extrace I-frames.

Cheers
Shanuson
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/12/2014 04:34 PM
I encourage interested people to uncomment last line, run the program,
look at raw.hex.diff, and suggest improvements to fix_data_packets() logic.

After wading through this code, I think we have roughly the same idea. Just took me a while to get used to your coding style before I could see what you were doing.

I've started to add your extra header patterns to the paste bin code, and tweaked what michaelni had done for fixing packets. Overall, I think of the three the newest style is the most concise and fastest performing.

The things that still need to be done:
(1) check the rest of your packet types, and adjust the masks as needed.

(2) integrate min(score) over packet number, and use this to allow the probabilities to relax in high-error regions. When p(error) exceeds a certain value, it might be best to "drop" packets all together rather than feed them to the decoder.

(3) rather than use a simple bit-count as the diff, we should really be checking the number of continuous matching bits. In other words, we should compute Poisson rather than binomial probability.

(4) in addition to expected and mask, there should be a third row of bit probability. when the expected is not known (thus, nothing to diff). This would solve the issue of null body matching, since the p(1) should be close to .5 for real data and close to 1 for null data. Note this only matters when (3) has been implemented, otherwise expected = ..0x55 0x55 0x55.. has the same effect for the binomial probability.

Edit (5): evaluate all possibilities, rather than the first best match, then choose the highest.

Edit (6): fix sequence numbers where possible 
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/12/2014 04:57 PM
I think we can also assume that the transport error and the scrambling control bits should never be set in any header, right? That may help constrain the problem a bit. Turning off the transport priority bit in everything may also be helpful, since we're dealing with a file rather than an over-the-air stream and wouldn't expect an overworked decoder to need to choose which packet to deal with first.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: cscott on 05/12/2014 05:19 PM
About the Iframes, i was thinking that manual moving the macroblocks around will be difficult if you don't have a lot of reference (i.e. if there aren't many features on the block).

As it turns out, it appears that the dirty splash on liftoff has been a great help, as there are lots of small dirty smudges in each frame which provide reference features.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jakusb on 05/12/2014 06:21 PM
About the Iframes, i was thinking that manual moving the macroblocks around will be difficult if you don't have a lot of reference (i.e. if there aren't many features on the block).

As it turns out, it appears that the dirty splash on liftoff has been a great help, as there are lots of small dirty smudges in each frame which provide reference features.

Yes, see my earlier post with an image annotated with 14 dirt spots that clear are on the lens.


A version with the dirt spots numbered for better reference in conversation

Edit: Added dirt group 14

(http://img.tapatalk.com/d/14/05/13/7eje5ete.jpg)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/12/2014 06:35 PM
About the Iframes, i was thinking that manual moving the macroblocks around will be difficult if you don't have a lot of reference (i.e. if there aren't many features on the block).

As it turns out, it appears that the dirty splash on liftoff has been a great help, as there are lots of small dirty smudges in each frame which provide reference features.

Yes, see my earlier post with an image annotated with 14 dirt spots that clear are on the lens.


A version with the dirt spots numbered for better reference in conversation

Edit: Added dirt group 14

(http://img.tapatalk.com/d/14/05/13/7eje5ete.jpg)

Dont forget the one that is overlapping with part of the right leg. (wiki IF 5 shows a complete right leg)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/12/2014 06:40 PM
Hi guys,

I have a bit of a mystery going on and it's holding me back from doing great stuff. ;) Hopefully somebody else has an idea what is going on.

This is essentially about how luminance/chrominance (errors) propagate. From what I understood so far each (macro)block propagates to its right (macro)block and to its bottom (macro)block. Luminance does it on a block (8x8pixel) basis while chrominance does it on a macroblock basis. So if you set macroblock 0,0 to "more red" the entire picture becomes that much "more red". Right?

Now here is my mystery example.

Take iframe 12 (try1) and enter 0:0:-1,13:24:72730 as mmb. It shows the bottom part of the picture and it looks quite nice.

Now enter 0:0:-1,12:24:72687 and suddenly you see a luminosity error appearing far from the block we just changed. How is this possible??

I suspect this has to do with the fact that the lower-right block of 24:26 is getting different input from above or left, but I don't understand why it could be so different from before. Maybe some kind of dc-clipping? (but there is no error log of that I think) Is this block reading the bits differently because it has different input? Because I always thought that the reading/meaning of the bits was not determined by the input.

I'm confused by this.

Regards,

arnezami

So this confused me for a while too. First off a block does not propagate to the left AND right. It is better to say that a given block propagates from EITHER the left OR the top block. Further, this is not defined in the file specifically. It is defined by comparing the block above, the block to the left, AND the block to the upper left. Based on the DC (brightness/color) of the blocks in these directions it PICKs which block to propagate from. Because of this when you adjust a value you can cause a given block to swap which block it inherits from.

This is even worse because this prediction direction even determines the scan order through the DCT table. So causing a block to swap which block it inherits from can cause the DCT scan order to become screwed up resulting in the pixels in the block to be transposed (mirrored and flipped vertically).
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Lars_J on 05/12/2014 06:42 PM
A version with the dirt spots numbered for better reference in conversation

Edit: Added dirt group 14

Also, don't forget the large smudge under dirt #3 (just below to the left), which obscures/blurs part of the leg in every frame. (that has a leg)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/12/2014 06:42 PM
Hi guys,

I just noticed that there are no B-frames in the file. Only I-frames and P-frames.

Quote
./ffprobe try1.ts -show_frames > frames.txt

Not sure yet what the implications are yet. Hopefully it will get easier that way.

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/12/2014 06:45 PM
So this confused me for a while too. First off a block does not propagate to the left AND right. It is better to say that a given block propagates from EITHER the left OR the top block. Further, this is not defined in the file specifically. It is defined by comparing the block above, the block to the left, AND the block to the upper left. Based on the DC (brightness/color) of the blocks in these directions it PICKs which block to propagate from. Because of this when you adjust a value you can cause a given block to swap which block it inherits from.
Aha! Interesting!

Thank you for that. That way it does indeed make more sense.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/12/2014 06:56 PM
I've seen this too. Either the offending block is messed up in some way that propagates the error, or MPEG behaves badly when values are clipped. The problem seems to be worse on frames where the top row is missing. I attached an image that shows the path from the bad block to where it gets noticeably bad.
Yeah. I'm starting to realize that it is even harder than I thought it would be.

I wanted to create a "clean enviroment" for checking bad block propagation by starting at the end and working backward therefore avoiding the issue of having propagation from above and left.

But this won't work if block propagation below and to the right is different when adding more blocks to the left and above.

Hmmm. A rethink is required here.

Thanks for that diff!

Regards,

arnezami


Quick thought from a quiet computer Geek in the audience.... Can Space X provide you with a good 30 sec chunk of video from the same setup so you can see what good video is supposed to look like?  That would get you ground truth as to formatting etc.  It would also give you the ability to selectively damage the bit stream to better understand propagation related errors.

Yes we asked about that. But nothing has come yet. Supposedly something is in the works.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/12/2014 07:06 PM
Parsing raw.ts using info from http://en.wikipedia.org/wiki/MPEG_transport_stream

The file consists of 188-byte ts-packets.

1st byte is always 47.

2 next bytes are 000<packetid13bits>, and apparently
they are always 03e8 for non-null ts-packets and 1fff for null ts-packets.

there are more than 2 PID values in the ts packets/ file


To find out how many different TS-PIDs exists, I counted all different "0x47xxxx" appearances in raw_edit1.ts:
This are all which occured 20 times or more:
20 471be8
20 4733e8
21 470fe8
25 4703e9
63 474703
74 474020
81 474000
269 4743e8
4968 471fff
15016 4703e8


I think the first 4 are permutations of the last ones by bitflips.
4743e8 and 4703e8 have the same PID,
When one checks where 474703 is found, one sees that the next byte is E8. So the first 47 is the last byte of a previous packet, and therefore 474703 is not part of a TS-header.
So we end up with only 4 PIDs (13 bit long):
0x0000, 0x0020, 0x03E8,0x1FFF
Each PID has its own continuity counter, which should help deciding which packet is which.
 
With some different flagbits we end with the following TS-headers:
0x4740001W
0x4740201X
0x4703E81Y, 0x4703E83Y and 0x4743E83Y
0x471FFF1Z
with the 4 different counters W,X,Y,Z
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/12/2014 07:46 PM
Offtopic: An updated version of the sequence from video I-Frames that I posed some pages ago, now with the legs clearly moving!


Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jakusb on 05/12/2014 08:35 PM
Offtopic: An updated version of the sequence from video I-Frames that I posed some pages ago, now with the legs clearly moving!
Nice! Clearly great progress already.
If only we could somehow get the legs in Iframe 4 resolved...
I assume they should be just opened and slightly away from the stage. Or not

Ps: very much on topic if you ask me. ;)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/12/2014 09:11 PM
Ok, I am done with my work on the headers

attached you find a raw_edit1.ts where only the 15 I-Frame headers are changed (plus a few '47' bytes maybe), Also the one 188 byte block is reshifted by 4 bits to align it.

The last XX XX was part of vbv_occupancy (1 marker bit and the frist 7 of the latter half of vbv_occupancy)
According to michaelni, the actually value is not needed to extrace I-frames.

Cheers
Shanuson

I tried it, some iframes appear to have extra data and are clearer, others are worse. iframe8 is missing, and I'm not sure if there are new iframes or not. Several of the frames have apparently significantly less errors though.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IRobot on 05/12/2014 09:19 PM
As soon as the work finishes on recovering the information, I guess some post processing can be done, namely noise filtering/smothering and some morphing to create intermediate frames!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jakusb on 05/12/2014 09:20 PM

Ok, I am done with my work on the headers

attached you find a raw_edit1.ts where only the 15 I-Frame headers are changed (plus a few '47' bytes maybe), Also the one 188 byte block is reshifted by 4 bits to align it.

The last XX XX was part of vbv_occupancy (1 marker bit and the frist 7 of the latter half of vbv_occupancy)
According to michaelni, the actually value is not needed to extrace I-frames.

Cheers
Shanuson

I tried it, some iframes appear to have extra data and are clearer, others are worse. iframe8 is missing, and I'm not sure if there are new iframes or not. Several of the frames have apparently significantly less errors though.

Any idea yet how to proceed?
Is it useful to add them to the online editor v2?
I am getting pretty curious to see this potential new or improved data in actual images...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: AncientU on 05/12/2014 09:33 PM
About the Iframes, i was thinking that manual moving the macroblocks around will be difficult if you don't have a lot of reference (i.e. if there aren't many features on the block).

As it turns out, it appears that the dirty splash on liftoff has been a great help, as there are lots of small dirty smudges in each frame which provide reference features.

Yes, see my earlier post with an image annotated with 14 dirt spots that clear are on the lens.


A version with the dirt spots numbered for better reference in conversation

Edit: Added dirt group 14

(http://img.tapatalk.com/d/14/05/13/7eje5ete.jpg)

Dont forget the one that is overlapping with part of the right leg. (wiki IF 5 shows a complete right leg)
Also, don't forget the rocket body and it's dirty water 'paint job' which should remain constant except for lighting effects.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/12/2014 09:37 PM

Ok, I am done with my work on the headers

attached you find a raw_edit1.ts where only the 15 I-Frame headers are changed (plus a few '47' bytes maybe), Also the one 188 byte block is reshifted by 4 bits to align it.

The last XX XX was part of vbv_occupancy (1 marker bit and the frist 7 of the latter half of vbv_occupancy)
According to michaelni, the actually value is not needed to extrace I-frames.

Cheers
Shanuson

I tried it, some iframes appear to have extra data and are clearer, others are worse. iframe8 is missing, and I'm not sure if there are new iframes or not. Several of the frames have apparently significantly less errors though.

Any idea yet how to proceed?
Is it useful to add them to the online editor v2?
I am getting pretty curious to see this potential new or improved data in actual images...

First there is a small python script which helps to get the iframes.
After ffmpeg -vstats -i file.ts-vcodec copy -an -f image2 frame%d.mpg4-img produced all frames
use the following python script

import os
import sys

ld=os.listdir('./')

for ff in ld:
    if "frame" in ff:
        with open(ff, "rb") as f:
            b1 = f.read(4).encode("hex")
            i=0       
            #print b1,ff
            if b1=='000001b0':
                print ff
                os.system('cp '+ff+' i'+ff)


It will make copys of those frames that are I-frames.

attached you find a zipfile containing all 15. But some still need some work, 1 is only 78 byte large, will another is of 30Kb.
But this is progress. Maybe to early to add them to the wiki, i dont know.

Edit: maybe should give a maping from the new ones to the old ones:
New -> Old:
iframe1.mpg4-img -> IF0
iframe14.mpg4-img ->
iframe32.mpg4-img ->  IF1
iframe52.mpg4-img -> IF2
iframe72.mpg4-img -> IF4
iframe92.mpg4-img -> IF5
iframe112.mpg4-img  ->  IF6
iframe131.mpg4-img  ->
iframe150.mpg4-img  ->  IF7
iframe169.mpg4-img  ->  IF8 
iframe190.mpg4-img  -> IF9
iframe210.mpg4-img  -> IF10
iframe230.mpg4-img  -> IF11
iframe249.mpg4-img  -> IF12
iframe269.mpg4-img  -> IF13
 

Edit2:Fixed the 78byte IF, it is now 20472 Bytes large
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/12/2014 09:56 PM
As soon as the work finishes on recovering the information, I guess some post processing can be done, namely noise filtering/smothering and some morphing to create intermediate frames!

Already working on it. Want to help?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/12/2014 10:00 PM
As soon as the work finishes on recovering the information, I guess some post processing can be done, namely noise filtering/smothering and some morphing to create intermediate frames!

Already working on it. Want to help?

A lot of work still needs to be done on iframes before that can be done.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/12/2014 10:16 PM
To find out how many different TS-PIDs exists, I counted all different "0x47xxxx" appearances in

Back when I was messing with the raw.ts I looked at the PMTs (because I thought corrupted PMTs may mess up decoding I made a consensus PAT and PMT and attached those to the start of the file and removed all others, don't think it made much difference). I remember there were 3 PIDs defined. 3e8 - video, 3e9 and 3f0. I have no idea what the last two are, but I think they're unneeded. Those packets may still be present though, but I don't remember actually seeing one. Even if the header said 3e9 the counter suggested it was actually still 3e8.

This is even worse because this prediction direction even determines the scan order through the DCT table. So causing a block to swap which block it inherits from can cause the DCT scan order to become screwed up resulting in the pixels in the block to be transposed (mirrored and flipped vertically).

That's just disgusting.

If only we could somehow get the legs in Iframe 4 resolved...
I assume they should be just opened and slightly away from the stage. Or not

I've spent the last two evenings on this and it's been awful. I'm inching closer, but the data is in dreadful condition. I've seen no leg at all. They appear out of thin air in iframe 5 for all I can tell. I'm calling it a night for today. My best effort so far is pretty abysmal, but still shows a bit more than the previous efforts (I hope it's more or less correct)

mmb: 5:1:4200,0:3:10968,8:3:14762,28:3:16550,24:4:20286,40:4:-1,9:6:21857,13:11:39488,37:11:41549,
10:12:42693,29:12:43895,21:13:50986,10:14:-1,13:14:-1,21:15:-1:60:60::::,22:15:55249,2:16:-1,
8:16:58952,13:16:59649,27:16:63538,18:17:69075,27:17:-1,13:18:74459
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/12/2014 10:17 PM
As soon as the work finishes on recovering the information, I guess some post processing can be done, namely noise filtering/smothering and some morphing to create intermediate frames!

Already working on it. Want to help?

A lot of work still needs to be done on iframes before that can be done.

Everything that I am working on involves transformations on the i- and p- frames. I can fold decoding improvements back into the project as they happen.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/12/2014 10:27 PM
Here are my I-Frames.
Yes for various reasons am i sure that these are the I-Frames, and that they are the correct ones.
But it can be that some data is missing from them or that there is data in them from the next frame.

As for the different Iframe mlindner and i got, i think that is because he is using raw_edit1.ts directly while i put it through tsalign and tsfix first. those i got 269 frames.
here are the pngs from those:
Also as can be seen by the numbering: most of them are 20 frames apart, some are less, indicating where something is still not ok.

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/12/2014 10:33 PM
Those look very very nice!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/12/2014 10:56 PM
Here is the fixed.ts where those and the img-files in that zipfolder a few posts back came from:

Edit: And the raw_edit1.zip file with the header of frame169 fixed. I will delete the wrong one.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: bilbo on 05/12/2014 10:59 PM
Has any work been done on the section where the stage tips over?
so far, I have only seen work on the landing, and initial splashdown.
Keep up the good work!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: gospacex on 05/12/2014 11:06 PM
For example, as I posted earlier, there are spikes in noise level where error rate spikes above 30% for a few thousand bytes. If this occurred in a null packet it might look to you like a mildly corrupt data packet, but in reality it is junk.

I was thinking about it and here sequence numbers are a godsend. They are often corrupted too, yes, but it would take a lot of corruption to make long run of them completely unrecognizable!

And as long as you can reasonably guess and restore them, most of the time it conclusively answers the question "is this packet data or null?" - they have separately counting sequences.

Quote
The goal of cleaning the raw stream is to give the folks here the very best mpg4-img files to tweak visually.

Exactly. My worry is that by dropping just one data ts-packet, even corrupted one, we can easily make it orders of magnitude harder to restore the frame.

ts-packet fixing operates on 188-byte blocks, macroblock fixing works on BITS! Spending a minute more on fixing a packet can save hours!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/12/2014 11:08 PM
Here are my I-Frames.

Wow. That iframe 6 is almost perfect.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/12/2014 11:11 PM
Going to port my iframe8 changes to this new iframe8.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/12/2014 11:28 PM
Going to port my iframe8 changes to this new iframe8.
How do you do that?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/12/2014 11:48 PM
Alpha 3 of my transport stream search and repair tool is finished.

Were you able to find any other sub-byte shifts aside from the one we already know about?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/13/2014 12:01 AM
I have tsfix.c matching the patterns from gospacex correctly now, and fixing sequence numbers: http://pastebin.com/dirZVxfS

It seems like there are a ton of sequence number errors, mostly for PID=1000 (which is forced more often than not as all PIDs not 0, 8191, 32, or 64 are forced to 1000).

This does make the sequence number errors in ffmpeg go away, but brings on many new errors, such as:

DTS 4301262034 < 8597009660 out of order
timestamp discontinuity -47730595911, new offset= -47770843290
etc.

The output video looks different, but I would not say better (just playing in VLC).
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/13/2014 12:10 AM
The time stamp discontinuities should be able to be fixed without too much heartache since we know this is a monotonic stream. There should be a valid range of values and I suspect the flipped bits would become apparent upon inspection of the prior and subsequent stamps, much like the last four bits of the header.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/13/2014 12:26 AM
i fixed manually (by hand) the TS-Headers including the continuity counter throughout that 2nd IFrame (frame13 in raw_edit1.ts or frame14 in fixed.ts) in both those files.
before it had ~9KB in fixed.ts.
After fixing the sequence in fixed.ts It got a bit larger ~10Kb. But there was one 188 KB block of mostly FFs that somehow got included into fixed.ts by tsfix (or less likely tsalign) that could not be accounted for with a continuous continuity counter.
So i also handfixed raw_edit1.ts the same way, which results in the attached png and img-file :D. It is now 16kb large.

Also i am not sure some script can fix all those TS-headers correctly. Sometimes the bytes are completely wrong.

Edit: this is not frame14 at 0x85bf8, but the next one, old IF1, sorry for messing up the frames
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/13/2014 12:58 AM

To find out how many different TS-PIDs exists, I counted all different "0x47xxxx" appearances in raw_edit1.ts:
This are all which occured 20 times or more:
20 471be8
20 4733e8
21 470fe8
25 4703e9
63 474703
74 474020
81 474000
269 4743e8
4968 471fff
15016 4703e8


I think the first 4 are permutations of the last ones by bitflips.
4743e8 and 4703e8 have the same PID,
When one checks where 474703 is found, one sees that the next byte is E8. So the first 47 is the last byte of a previous packet, and therefore 474703 is not part of a TS-header.
So we end up with only 4 PIDs (13 bit long):
0x0000, 0x0020, 0x03E8,0x1FFF
Each PID has its own continuity counter, which should help deciding which packet is which.
 
With some different flagbits we end with the following TS-headers:
0x4740001W
0x4740201X
0x4703E81Y, 0x4703E83Y and 0x4743E83Y
0x471FFF1Z
with the 4 different counters W,X,Y,Z

For completeness, I counted PIDs by absolute bit position in the raw.ts, here are the top 50:

PID       HEX      COUNT 
1000    x03E8      15592   -- Video
8191    x1FFF      5070     -- Null packet
   0    x0000      84
  32    x0020      75
5096    x13E8      49       -- 1 bit off from 1000
1001    x03E9      48        -- 1 bit off from 1000
 904    x0388      40                -- 2 bit off from 1000
7144    x1BE8      30       -- 2 bit off from 1000
 984    x03D8      29                -- 2 bit off from 1000
1003    x03EB      29       -- 1 bit off from 1000
 616    x0268      28
3048    x0BE8      28
1512    x05E8      27
2000    x07D0      27
 996    x03E4      26
1008    x03F0      26
1002    x03EA      25
4095    x0FFF      24
 500    x01F4      23
1006    x03EE      23
4072    x0FE8      23
8190    x1FFE      23
 808    x0328      19
1004    x03EC      19
 992    x03E0      17
 232    x00E8      16
 488    x01E8      16
 968    x03C8      16
1016    x03F8      16
2024    x07E8      16
6120    x17E8      15
 872    x0368      14
 936    x03A8      14
4000    x0FA0      14
 680    x02A8      13
 744    x02E8      13
1005    x03ED      13
2047    x07FF      13
2536    x09E8      12
8167    x1FE7      11
8189    x1FFD      11
7807    x1E7F      10
 840    x0348      9
5119    x13FF      9
7423    x1CFF      9
8095    x1F9F      9
8127    x1FBF      9
8185    x1FF9      9
 125    x007D      8
 994    x03E2      8


All 1650 are in the attached csv.

I think your 4-PID theory is correct.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/13/2014 01:24 AM
Going to port my iframe8 changes to this new iframe8.
How do you do that?

Apparently the new and the old iframe8 are byte identical. Was hoping some of the issues would be fixed, but apparently not.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/13/2014 01:52 AM
I think your 4-PID theory is correct.

And it makes sense, too - it's not like it's a very complicated stream with a bunch of different elementary streams in it. One camera and you're done. The 0x0000 is the program association table which "contains a directory listing of all Program Map Tables," (Wikipedia) then 0x0020 is the first available number in the range of assignable data tables, and finally 1000 is a nice round number in decimal.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Adaptation on 05/13/2014 02:01 AM
Hi guys,

I just noticed that there are no B-frames in the file. Only I-frames and P-frames.

Quote
./ffprobe try1.ts -show_frames > frames.txt

Not sure yet what the implications are yet. Hopefully it will get easier that way.

Regards,

arnezami

That is very fortunate happenstance.  P frames will only have compound errors from any damage from the frames before them in the sequence.  B frames would have compound errors from both before and after them in the sequence. 
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Swatch on 05/13/2014 03:36 AM

To find out how many different TS-PIDs exists, I counted all different "0x47xxxx" appearances in raw_edit1.ts:
...

For completeness, I counted PIDs by absolute bit position in the raw.ts, here are the top 50:
...

I think your 4-PID theory is correct.


Out of curiosity, did anyone do the counting (based on absolute bit position) on the streams that were re-aligned to 188 bits (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1196838#msg1196838)?  I was under the impression that raw.ts was not, so this might throw off your count if you're doing absolute bit position.

How many differenct PID permutations might that produce?

Then it might help for Adaptation's tool (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1197014#msg1197014) to be able to display/color based on the PID permutations.  That might also help identify what type of PID a corrupted packet is supposed to be.

Just a suggestion after digesting all of your work the last few pages.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/13/2014 04:46 AM

To find out how many different TS-PIDs exists, I counted all different "0x47xxxx" appearances in raw_edit1.ts:
...

For completeness, I counted PIDs by absolute bit position in the raw.ts, here are the top 50:
...

I think your 4-PID theory is correct.


Out of curiosity, did anyone do the counting (based on absolute bit position) on the streams that were re-aligned to 188 bits (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1196838#msg1196838)?  I was under the impression that raw.ts was not, so this might throw off your count if you're doing absolute bit position.

How many differenct PID permutations might that produce?

Then it might help for Adaptation's tool (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1197014#msg1197014) to be able to display/color based on the PID permutations.  That might also help identify what type of PID a corrupted packet is supposed to be.

Just a suggestion after digesting all of your work the last few pages.

Sorry, to be clear I always run raw.ts | tsalign before doing anything on it. So the packets are lined up correctly in the stats above.

Edit: the reason I like to start with raw.ts is that it would be nice to have the final solution be generated completely automatically. Maybe then Michael will add a -spacex flag to ffmpeg that will allow raw.ts to play flawlessly ;)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Digitalchromakey on 05/13/2014 05:01 AM
Here are my I-Frames.

Wow. That iframe 6 is almost perfect.
I think iframe112-mpg4.img.png is decoded from the ES video data set that was previously used to decode iframe_6 (coerced).

The time stamp in the new  iframe131.mpg4-img.png  frame is 19:35:10:23, whereas the time stamp you can see in iframe_6 (coerced) is 19:35:08:23.

As iframe_8 has a time stamp of 19:35:12:97 - a difference of nearly 5 seconds, it looks like maybe a new I frame has been teased out of the transport stream between the old iframe_6 and iframe_8.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/13/2014 06:46 AM
Alpha 3 of my transport stream search and repair tool is finished.

Were you able to find any other sub-byte shifts aside from the one we already know about?

I recall there are at least two ts packets in raw.ts in iframe 8 that are missing one bit and the next packet has an extra bit causing the following ones to be aligned again. I expect there are more in other frames.

Going to port my iframe8 changes to this new iframe8.
How do you do that?

Apparently the new and the old iframe8 are byte identical. Was hoping some of the issues would be fixed, but apparently not.

That made for an easy port.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: JamesH on 05/13/2014 08:45 AM
Hi guys,

I just noticed that there are no B-frames in the file. Only I-frames and P-frames.

Quote
./ffprobe try1.ts -show_frames > frames.txt

Not sure yet what the implications are yet. Hopefully it will get easier that way.

Regards,

arnezami

That is very fortunate happenstance.  P frames will only have compound errors from any damage from the frames before them in the sequence.  B frames would have compound errors from both before and after them in the sequence.

Not really that 'fortunate' - there are very few if any real time encoders of the sort that would be used here that use B-frames. They need to encode in near real time, which means basing frames on frames that haven't yet appeared doesn't really work.  B-frames tends to work best when doing offline encoding. It does give a better compression ratio, but is more computationally expensive.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: seanpg71 on 05/13/2014 08:52 AM
...

Going to port my iframe8 changes to this new iframe8.
How do you do that?

Apparently the new and the old iframe8 are byte identical. Was hoping some of the issues would be fixed, but apparently not.

Comparing try1 to the frames from the edit1 zip above:
iframes 1, 4, 5, 7, 8, 9 and 13 match the frames from try1 exactly.
iframe 11 and 12 match up except for a couple bitflips
iframes 2, 6, and 10 have extra bits in try1 that aren't in the edit1 (dunno if they're important bits).

So we're probably not going to get any better versions of the current frames using the new iframes in Shanuson's edit1-fixed-iframe.zip.

The new frames are exciting though.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Kaputnik on 05/13/2014 09:38 AM
A long thread but worth it. I feel like I have just been on a crash course in digital video processing!
Keep up the amazing work- the world is watching :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/13/2014 11:07 AM
So I quickly tried the new frame 131 (between 6 and 7)! I threw away some valid blocks so it should be possible to improve it further.

0:0:-1,
1:3:7967,0:7:-1,
12:8:23866,26:14:-1,
34:16:65834,25:20:-1,
14:21:92080,24:25:-1,
14:27:120887,10:29:-1
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: gospacex on 05/13/2014 12:07 PM

To find out how many different TS-PIDs exists, I counted all different "0x47xxxx" appearances in raw_edit1.ts:
This are all which occured 20 times or more:
20 471be8
20 4733e8
21 470fe8
25 4703e9
63 474703
74 474020
81 474000
269 4743e8
4968 471fff
15016 4703e8


I think the first 4 are permutations of the last ones by bitflips.
4743e8 and 4703e8 have the same PID,
When one checks where 474703 is found, one sees that the next byte is E8. So the first 47 is the last byte of a previous packet, and therefore 474703 is not part of a TS-header.
So we end up with only 4 PIDs (13 bit long):
0x0000, 0x0020, 0x03E8,0x1FFF
Each PID has its own continuity counter, which should help deciding which packet is which.
 
With some different flagbits we end with the following TS-headers:
0x4740001W
0x4740201X
0x4703E81Y, 0x4703E83Y and 0x4743E83Y
0x471FFF1Z
with the 4 different counters W,X,Y,Z

For completeness, I counted PIDs by absolute bit position in the raw.ts, here are the top 50:

PID       HEX      COUNT 
1000    x03E8      15592   -- Video
8191    x1FFF      5070     -- Null packet
   0    x0000      84
  32    x0020      75
5096    x13E8      49       -- 1 bit off from 1000
1001    x03E9      48        -- 1 bit off from 1000
 904    x0388      40                -- 2 bit off from 1000
7144    x1BE8      30       -- 2 bit off from 1000
 984    x03D8      29                -- 2 bit off from 1000
1003    x03EB      29       -- 1 bit off from 1000
 616    x0268      28
3048    x0BE8      28
1512    x05E8      27
2000    x07D0      27
 996    x03E4      26
1008    x03F0      26
1002    x03EA      25
4095    x0FFF      24
 500    x01F4      23
1006    x03EE      23
4072    x0FE8      23
8190    x1FFE      23
 808    x0328      19
1004    x03EC      19
 992    x03E0      17
 232    x00E8      16
 488    x01E8      16
 968    x03C8      16
1016    x03F8      16
2024    x07E8      16
6120    x17E8      15
 872    x0368      14
 936    x03A8      14
4000    x0FA0      14
 680    x02A8      13
 744    x02E8      13
1005    x03ED      13
2047    x07FF      13
2536    x09E8      12
8167    x1FE7      11
8189    x1FFD      11
7807    x1E7F      10
 840    x0348      9
5119    x13FF      9
7423    x1CFF      9
8095    x1F9F      9
8127    x1FBF      9
8185    x1FF9      9
 125    x007D      8
 994    x03E2      8


All 1650 are in the attached csv.

I think your 4-PID theory is correct.

I think "ultimate fix" for ts-packet stream is to identify the nature and sequence numbers of nearly every packet. It is in fact possible despite instances of heavy local corruption.

Example how it can be done (looking at raw.hex):

00d9ce8 471fff1bffffffffffffffffffffffffffffffffffffffff...
00d9da4 071e7f1cfffffffffffce7ffaffffffffffbffe7ffffffff...
00d9e60 5e9eff1bfffffc3ff77fffffffffffff53ffffffffffffff...
00d9f1c 465eff389fcebbfd67f0ffffffbffe79ffebffaffe1fdf7f
00d9fd8 26bebae7fbffdfe73fafff646f669f7ecd5c63a4e271d4a5

above packets are null packets. last two are heavily corrupted.
Now data packets follow:

00da094 4743e8380710002d2a507e00000001e00000818007210169
00da150 4703e819cdc0466941eb5c9df5f0320710bc3a330e091ba7
00da20c c703e81a059c70c40701a982190958cd31e2f06e6605a0b3
00da2c8 470fe41bb1d1204529aa6b20c9e3161765247f8f27400625
00da384 4703e81c2b371e0c9e2220df2056a413bc0c4f06c727fd05
00da440 4703e81ddfeeb416075502c8320cc4680dab3f6dafe923dc
00da4fc 4703e81e8d3801135e80608ecd4ee8a6b831473a5c0704c6
00da5b8 4703e81f19c3e015e44317a641ab463daf6c9474f5404178
00da674 5303e810845492731ce7088df908d5c69c9d8650f83de0c9
00da730 c0aa09d1aff4edb39423532036d6414be3bff3774fec9ff7
00da7ec 3870d4b906f3dc1101ac21f8154e663f444e7a684e908e5f
00da8a8 98167426424d9a350043e7e988b7648e76139028fef4b339
00da964 5060f71f0180f1b65c02ad0414880a6a0594d1211547cc01
00daa20 8e07902a9908c0d17d52b1e4cd2a16ed7895f5209a251ca7

last four packets above are heavily corrupted - header is completely
unrecognizable, but they *must be* data packets, otherwise seq numbers
don't match: there is 8,9,a,b,c,d,e,f,0,1 sequence before them,
and below the same sequence continues: 6,7,8,9,...
So these four packets above *have to be* 2,3,4,5,
and since seq numbers are counting independently for different PIDs,
they must be data packets.

00daadc 4703e816f0cd370cc3300b7a55347680436640e0c8456c32
00dab98 4703e81752d05c5b6234200ae027f951e53499e182195038
00dac54 4703e81893671ee0c8058f24c320c9ce0b250c9cf878496e
00dad10 4703e81973877ea096f1181004e2301e774163c04f72c038
00dadcc 4703e81a40b2a83cdc8e085b9d3413de15028243200af9e7
00dae88 4703e81bb9ec065fc3da74f299fd281e0b551ffcd3aab013
00daf44 4703e81c5938420482ded1e067066230400bdd77cf833832
00db000 4f03d81d380ff2904f9ae02286676e2bd40704b73e1a17e9
00db0bc 4703e8127c77ab0958cc54ac742ce54c00c3a0b2fd029871

Corrupted seq no above, should be 'e'.

00db178 471fff10ffffffffffffffffffffffffffffffffffffffff
00db234 4703e81f82ba45e0090670958f067b80dac02da97f1904f8

As you see, every packet can be identified in this example, despite
some headers being shot to hell.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jakusb on 05/13/2014 12:21 PM

Offtopic: An updated version of the sequence from video I-Frames that I posed some pages ago, now with the legs clearly moving!

Could someone maybe make a small addition where in top right corner per frame an estimation from how stage would look from the side?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/13/2014 01:12 PM

last four packets above are heavily corrupted - header is completely
unrecognizable, but they *must be* data packets, otherwise seq numbers
don't match: there is 8,9,a,b,c,d,e,f,0,1 sequence before them,
and below the same sequence continues: 6,7,8,9,...
So these four packets above *have to be* 2,3,4,5,
and since seq numbers are counting independently for different PIDs,
they must be data packets.


Careful with this assumption - the continuity counter/sequence number is just 4 bits. It is not special nor immune to noise than any other four bits in the header or data. Yes, including it in the match will increase the chances of getting the right PID, but it's not a magic bullet in itself.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/13/2014 01:18 PM
Careful with this assumption - the continuity counter/sequence number is just 4 bits. It is not special nor immune to noise than any other four bits in the header or data. Yes, including it in the match will increase the chances of getting the right PID, but it's not a magic bullet in itself.

The info is not in the 4 bits of an individual counter, but the sequence of thousands of them. The question is whether including completely garbled packets does more good or harm.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/13/2014 01:34 PM
Careful with this assumption - the continuity counter/sequence number is just 4 bits. It is not special nor immune to noise than any other four bits in the header or data. Yes, including it in the match will increase the chances of getting the right PID, but it's not a magic bullet in itself.
I don't think he's trying to say that, but rather merely pointing out that there's a second dimension from which the correct header can be derived, kind of like a macroblock referring to the one above or to the left. There's only four PIDs in the stream, and only two which occur frequently (0x03e8 and 0x1fff), so even with a completely trashed header like 0x5060F71, pulling these known quantities together you can figure out with a very high level of confidence that it's supposed to be 0x4703E814.

Of course, it might be 0x34, not 0x14, if it's an adaptation packet, but there's also context available to figure that out too - such as if the next byte after the header is greater than 184 (0xB8) since that would mean the adaptation field is longer than the rest of the packet and so either that byte is wrong or the 0x34 should be 0x14.

Now whether the likely completely trashed payload that goes with that completely trashed header will be of any use is another question, but at least we have the payload.

Of course this doesn't help much if you go too far beyond 16 packets in either direction (3008 bytes), but I don't think we've seen that much contiguous unrecognizable data anywhere in the transport stream.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/13/2014 01:39 PM
I updated frame 131 (and can finally make .png as output):

0:0:-1,
7:2:5895,4:7:-1,
11:7:20490,26:14:-1,
18:15:56376,13:16:-1,
14:16:62505,20:16:-1,
21:16:64068,27:16:-2,28:16:-1,
32:16:65707,25:20:-1,
14:21:92080,24:25:-1,
14:26:117098,41:26:-1,
14:27:120887,10:29:-1

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/13/2014 02:35 PM
Here are the best results on the first 2 I-frames IF0 and IF 0.5 and the ts-file where they came from:

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Okie_Steve on 05/13/2014 02:37 PM
As you see, every packet can be identified in this example, despite
some headers being shot to hell.
Interesting approach. Do you see more 0->1 or 1->0 bit flips are they about equal? If non-random then the people doing genetic bit flips might want to bias their attempts.

Also, might be interesting to see an "XOR" file the same size as the input that could be applied to generate the "repaired" headers if someone wants to look for patterns and apply them to the data too.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/13/2014 04:50 PM
@Michael Niedermayer:

I've been investigating the possibility of automatically setting all L1,L2,L3,L4,C1,C2 values for all macroblocks for a frame. Basicly inputting an mmb and letting a script detect "bad lines" in the picture. Moving from the right-bottom corner to the upper-top.

I'm starting to believe (using some extra logging and some manual experimentation) that this might actually be possible. If it does this would improve the quality of the frames quite dramatically and would save a lot of manual work!

However there is one thing that I find difficult to do myself. And I think you might be able to help us. Instead of using this kind of setting:

15:14:-1:0:-40:0:0:0:20

where I replace the current macroblock with a blank one and change the DC values.

But what I really want to do is this:

15:14:57217:0:-40:0:0:0:20

In other words: I want to keep the macroblock with its (structured) data as much as I can (at the very least the blocks in it which are set to 0 above) but change only the DC "starting" values. So not blanken it, but modifying it so these DC-values make sure propogation to other blocks works the same as now with -1 (therefore fixing the color and intensity of following blocks) but I also keep the current blocks inside the current macroblock.

Is something like that possible?

Regards,

arnezami
Hi guys,

I've created a new feature that implements the above.

Simply put you can now do something like this:

15:14:57217:0:-40:0:0:0:20

This will keep the macroblock (so not nullify it) but it will change the DC values. With this you can really fix the lum/chrom issues in the current I-frames. Should have a big effect. I hope.

[edit] Forgot: it now also logs the 6 DC values for each macroblock. Which can come in handy in combination with this new feature.

Also you can do this now:

15:14:57217:0:0:0:0:0:0:0
15:14:57217:0:0:0:0:0:0:63

What this does is change direction from where a DC prediction are coming/inherited from: left or top. For each 1-bit in this number it tells it to get the DC-prediction from the top for a specific DC (4 x lum, 2 x chrom). A 0-bit means left. The number consists of 6 bit (hence max is 63). So in effect a 0 is all left, a 63 is all top. I don't know yet whether this feature is going to be very useful but it can't hurt.

I've attached the 3 source files that were changed. If somebody can check if it doesn't conflict with the existing branch (and maybe test it) and then push it to github, that would be great. Its these files btw:

libavcodec/mpegvideo.h
libavcodec/mpeg4video.h
libavcodec/h263dec.c

[edit] Fixed a bug: when using -2 it would get stuck in that mode. zip re-uploaded.

I need some sleep now ;)

Regards,

arnezami

PS. I was also trying to make a reverse-macroblock finder of sorts. So instead of telling its starting bitposition you would tell it its ending bitposition (which would usually be a starting bitposition of a mb you already have). But I ran into a lot of issues. Even when you try all possible DC-directions it still won't find the right starting position. Even when I fake the right DC values left to the block. I'm not sure why yet. Probably something I don't understand yet. This part of the code is in there but its commented out. It was also extremely error-log-heavy...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/13/2014 05:28 PM
Hi guys,

I just noticed that there are no B-frames in the file. Only I-frames and P-frames.

Quote
./ffprobe try1.ts -show_frames > frames.txt

Not sure yet what the implications are yet. Hopefully it will get easier that way.

Regards,

arnezami

That is very fortunate happenstance.  P frames will only have compound errors from any damage from the frames before them in the sequence.  B frames would have compound errors from both before and after them in the sequence.

Not really that 'fortunate' - there are very few if any real time encoders of the sort that would be used here that use B-frames. They need to encode in near real time, which means basing frames on frames that haven't yet appeared doesn't really work.  B-frames tends to work best when doing offline encoding. It does give a better compression ratio, but is more computationally expensive.

I am trying to recreate the full movie using the I-frames, a few "nice-looking" P-frames and fill the rest with empty P-frames. How can I convert a png into a P-frame? Or should I create the empty P-frames differently?

For the I-frames, I use for example:
ffmpeg -i frame_131.png -s 704x480 -aspect 22:15 -f image2 -r 44999/3003 frame131.mpg4-img
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/13/2014 06:47 PM
Careful with this assumption - the continuity counter/sequence number is just 4 bits. It is not special nor immune to noise than any other four bits in the header or data. Yes, including it in the match will increase the chances of getting the right PID, but it's not a magic bullet in itself.
I don't think he's trying to say that, but rather merely pointing out that there's a second dimension from which the correct header can be derived, kind of like a macroblock referring to the one above or to the left. There's only four PIDs in the stream, and only two which occur frequently (0x03e8 and 0x1fff), so even with a completely trashed header like 0x5060F71, pulling these known quantities together you can figure out with a very high level of confidence that it's supposed to be 0x4703E814.

Of course, it might be 0x34, not 0x14, if it's an adaptation packet, but there's also context available to figure that out too - such as if the next byte after the header is greater than 184 (0xB8) since that would mean the adaptation field is longer than the rest of the packet and so either that byte is wrong or the 0x34 should be 0x14.

Now whether the likely completely trashed payload that goes with that completely trashed header will be of any use is another question, but at least we have the payload.

Of course this doesn't help much if you go too far beyond 16 packets in either direction (3008 bytes), but I don't think we've seen that much contiguous unrecognizable data anywhere in the transport stream.

Yes, and if null packets occurred one at a time (or even N at a time, N in known set) every ~M data packets, this would be useful, and I agree that looking at any particular segment you might believe you have recovered a header based on the counter, but it adds much less value than you might hope. The problem is that null packets do not occur in a predictable way. I ran some stats to illustrate..

The first graph shows the frequency of such runs vs their length. There is obviously some outliers resulting from problematic data-packet matching, but the variance between 1 and 32 looks real enough to me. Strings of 1 occur often, but percentage-wise (third graph) they only constitute 2.58% of null packets.

The second graph shows the distance between null packets. Excluding the case where d=0 (i.e. a continuous string per above, 5103 counts) and presuming that d=1..3 (120 counts) might be d=0 with over-opomistic data-packet identification, there is still a huge variance in the potential number of data packets before the next null packet.

The third graph is the most important, since it answers the question "suppose I know for sure I have N null packets, what is the chance that the next packet is also a null packet". The chance is pretty even to about N=38, meaning that the 16 values offered by the continuity counter could alias twice with equal probability.

That said, I did implement the counter into the packet-matcher data (see attached). As I expected, it does not seem to have made much of a difference.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/13/2014 07:32 PM
I am trying to recreate the full movie using the I-frames, a few "nice-looking" P-frames and fill the rest with empty P-frames. How can I convert a png into a P-frame? Or should I create the empty P-frames differently?

For the I-frames, I use for example:
ffmpeg -i frame_131.png -s 704x480 -aspect 22:15 -f image2 -r 44999/3003 frame131.mpg4-img

When you convert the p-frames to PNGs, some information about what to do with the blocks is lost. I don't think it is possible to reassemble the movie that way.

I think probably mlindner's idea to add a frame specifier to the mmb parameter is going to work best for contiguous sequences of i- and p-frames. In cases where we are missing p-frames, there is going to need to be some manual interpolation and reassembly.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/13/2014 07:39 PM
Regarding the update of the ffmpeg:

I just fixed a bug: "when using -2 it would get stuck in that mode. zip re-uploaded".

Please re-download the new one above.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/13/2014 08:08 PM
Could pframes be used to help fine tune iframes? If you start with a high quality iframe like 6, and then look for spots where the pframes indicate no change, could you simply carry the block from the good iframe forward (or backward) or does the compression in the iframe make that impractical?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/13/2014 08:20 PM
Hi guys,

I just noticed that there are no B-frames in the file. Only I-frames and P-frames.

Quote
./ffprobe try1.ts -show_frames > frames.txt

Not sure yet what the implications are yet. Hopefully it will get easier that way.

Regards,

arnezami

That is very fortunate happenstance.  P frames will only have compound errors from any damage from the frames before them in the sequence.  B frames would have compound errors from both before and after them in the sequence.

Not really that 'fortunate' - there are very few if any real time encoders of the sort that would be used here that use B-frames. They need to encode in near real time, which means basing frames on frames that haven't yet appeared doesn't really work.  B-frames tends to work best when doing offline encoding. It does give a better compression ratio, but is more computationally expensive.

I am trying to recreate the full movie using the I-frames, a few "nice-looking" P-frames and fill the rest with empty P-frames. How can I convert a png into a P-frame? Or should I create the empty P-frames differently?

For the I-frames, I use for example:
ffmpeg -i frame_131.png -s 704x480 -aspect 22:15 -f image2 -r 44999/3003 frame131.mpg4-img

How does it look with just the iframes? care to join us in the IRC channel?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/13/2014 08:49 PM
Hi guys,

I just noticed that there are no B-frames in the file. Only I-frames and P-frames.

Quote
./ffprobe try1.ts -show_frames > frames.txt

Not sure yet what the implications are yet. Hopefully it will get easier that way.

Regards,

arnezami

That is very fortunate happenstance.  P frames will only have compound errors from any damage from the frames before them in the sequence.  B frames would have compound errors from both before and after them in the sequence.

Not really that 'fortunate' - there are very few if any real time encoders of the sort that would be used here that use B-frames. They need to encode in near real time, which means basing frames on frames that haven't yet appeared doesn't really work.  B-frames tends to work best when doing offline encoding. It does give a better compression ratio, but is more computationally expensive.

I am trying to recreate the full movie using the I-frames, a few "nice-looking" P-frames and fill the rest with empty P-frames. How can I convert a png into a P-frame? Or should I create the empty P-frames differently?

For the I-frames, I use for example:
ffmpeg -i frame_131.png -s 704x480 -aspect 22:15 -f image2 -r 44999/3003 frame131.mpg4-img

How does it look with just the iframes? care to join us in the IRC channel?

So I created empty P-frames with an hex editor (just taking a valid P-frame and replacing everything except the 4 first bytes with FF, somehow it seems it worked ;))

I used then the I-frames, the empty P-frames and the following nearly intact P-frames (fixed.ts): 135, 172, 188, 199, 225, 233, 251

And finally created the movie with: ffmpeg -framerate 44999/3003 -i frame%d.mpg4-img -vcodec copy fixed_ts.avi

Not really better than the gif, but it's a beginning ;)

Does someone work on a way to edit the P-frames?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/13/2014 09:06 PM
Could pframes be used to help fine tune iframes? If you start with a high quality iframe like 6, and then look for spots where the pframes indicate no change, could you simply carry the block from the good iframe forward (or backward) or does the compression in the iframe make that impractical?

You could in theory carry blocks forward from one i-frame to the next i-frame. However you would need a good decode for that block in every one of the ~19 p-frames in between. I think we are unlikely to see that in practice.

The p-frames also contain block updates, so you could carry those forward to update the following i-frame. That doesn't help much either, because the blocks that are most likely to get updates are the most dynamic. There are lots of updates of the water which is either spinning or full of smoke, spray, or waves. Not so many updates of the rocket body which is most in need of "filling in". Up to iframe 11, at least. This method might be useful for reconstructing some of the water after splashdown.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/13/2014 09:32 PM
Been talking to SpaceX today about things. Got another awesome from them on this effort, so that's for you guys.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/13/2014 09:42 PM
My thought was to deal with what I presume is low-hanging fruit: blocks with no or virtually no change from one iframe to the next. If there's no updates, does that reduce the pframe coding length enough that it'd be more likely to have dodged corruption?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/13/2014 09:55 PM
My thought was to deal with what I presume is low-hanging fruit: blocks with no or virtually no change from one iframe to the next.

Do P-frame blocks within a given frame propagate like I-frame blocks? Or is it strictly a diff from the same MB?

If we can create fake P-frames with no change, then fill in the data changes one at a time, we might make faster progress. But if they are interdependent you'd need to guess C/L data for each which could get annoying fast (and give people creative license to make the video into whatever)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/13/2014 09:58 PM
My thought was to deal with what I presume is low-hanging fruit: blocks with no or virtually no change from one iframe to the next. If there's no updates, does that reduce the pframe coding length enough that it'd be more likely to have dodged corruption?

Heh. It's a rocket landing. With a hover-slam. There are virtually no blocks with no change from one frame to the next.

The pframes seem to be an all-or-nothing deal as far as corruption goes, as you might expect. Some of them are quite good, but a lot of them are really bad.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/13/2014 10:01 PM
Do P-frame blocks within a given frame propagate like I-frame blocks? Or is it strictly a diff from the same MB?

If we can create fake P-frames with no change, then fill in the data changes one at a time, we might make faster progress. But if they are interdependent you'd need to guess C/L data for each which could get annoying fast (and give people creative license to make the video into whatever)

No, they do not seem to suffer from the same error propagation problems as the i-frames.

Edit: Actually, some of them have groups of block updates which can propagate errors. It will be much easier to fix, though, because there are fewer opportunities for errors to get compounded.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/14/2014 10:19 AM
Actually we can edit the P-frames the same as the I-frames, the generated PNG image then looks good, but I cannot manage to write them back as P-frames, also when using something like this:

ffmpeg -s:0 704:480 -i frameXXX.mpg4-img -f image2 frameXXX_modified.mpg4-img

I hoped it would just copy the file, but it automatically converts the data into an I-frame it seems.

I also tried to use -f mp4 or to add -vf '[out]select=eq(pict_type\,P)', with no more success.

Does someone know how to force ffmpeg to write a P-frame?

Edit:

I finally managed to do it with:
ffmpeg -s:0 704:480 -i frameXXX.mpg4-img -vcodec copy -an -f image2 frameXXX_mod.mpg4-img

Edit(2):

It copies the P-frame but does not handle the -mmb ...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/14/2014 01:32 PM
Hi All,

Just a little update on the Wiki.

The current table with iFrames is a bit of a mess, as it has frames extracted from three different sources (coerced.ts from arnezami, try1.ts from mlindner and -until minutes ago- edit5.ts from Shanuson).

The most recent version (edit5.ts) is going to have the complete set of  15 workable IFrames. In order to organize things a bit, I created a separate page on the wiki to focus on these, hoping  these will be the definitive iframes. You can access from the main page, or directly here http://spacexlanding.wikispaces.com/Progress+on+edit5.ts . The previous table still has all the edits on frames from coerced.ts  and try1.

As of today, there is only one iframe in the new page (the one of frame #14 in the video) as its the only one that is completely fixed and ready to be edited. If anyone has some time available, please work on that one.  As other iframes become available (Shanuson is working on some), they will start appearing there. The online editor 2 will also have an updated version of those iframes for those that are using that tool.

And important note regarding backward compatibility. The new set of iframes includes previously unseen frames, but also some that are  equivalent to those we have already fixed. However, due to changes in iframe sizes, positions of MBs will change. This means that for existing iframes, previous -mmb codes will be only be partially applicable.  Keep this in mind as some .mmb codes will have to get reworked.

Thanks everyone for the good work!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/14/2014 04:18 PM
The online editor 2 will also have an updated version of those iframes for those that are using that tool.

Just to add on this: Big thanks to Iain for implementing some pretty cool new features in http://spacex.slapbet.org

- MB Errors (from the log) are now shown next to the MB X:Y display above the image when an MB is clicked. This includes DC clipping, dquant, ibpc, ac-tex, and stuffing (though I'm not even sure what they all mean ;) ).

- Total number of MB with errors are show on the right. It would be expected that a good -mbm would reduce the total number of MB with errors by more than one. This also allows a quick check to see which of two similar MBM strings is "better"

- Checkbox to enable highlighting MB with errors. This provides invaluable visual assistance in identifying where problems start. It would have been almost impossible to get this clear of a picture just looking at the log. It is interesting to watch how blocks with problems don't always look like it, and how visually defective blocks don't have any errors. This might drastically simplify debugging mbms.

So if you are using Online Editor V2 and haven't refreshed the page in a while, give it a F5. Great work Iain!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/14/2014 07:28 PM
So I was building the latest FFMpeg for windows from here https://github.com/michaelni/FFmpeg/tree/spacexdebug1

I managed to actually get it working (yay for me) but when I was testing it I found that the output for the same image with no -mmb applied was different to the previous version of ffmpeg I was using.

OLD: http://i.imgur.com/CeHOIrB.png
NEW: http://i.imgur.com/sMInZkY.png

The output seems a bit cleaner but I don't know why it's different, if anyone knows why this is, or more importantly if it's actually expected and OK, then I'll deploy it to the Online Editor to include the latest functionality.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/14/2014 08:09 PM
So I was building the latest FFMpeg for windows from here https://github.com/michaelni/FFmpeg/tree/spacexdebug1

I managed to actually get it working (yay for me) but when I was testing it I found that the output for the same image with no -mmb applied was different to the previous version of ffmpeg I was using.

OLD: http://i.imgur.com/CeHOIrB.png
NEW: http://i.imgur.com/sMInZkY.png

The output seems a bit cleaner but I don't know why it's different, if anyone knows why this is, or more importantly if it's actually expected and OK, then I'll deploy it to the Online Editor to include the latest functionality.
I'm not sure what version you were on but I also noticed some positive changes a few days ago when I updated. I think somebody did some bugfixes/improvements. Very similar to yours.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/14/2014 09:22 PM
Cool, I put the latest ffmpeg on the image handler the editor uses. If someone can summarise what the changes to the editor itself should be (I think it's an additional parameter fr the macroblock operation but I don't know what it's for) then I can add it :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/14/2014 09:36 PM
The output seems a bit cleaner but I don't know why it's different, if anyone knows why this is, or more importantly if it's actually expected and OK, then I'll deploy it to the Online Editor to include the latest functionality.

Iain, the new version of that frame you're seeing was extracted from a cleaned-up version of the transport stream, where errors in the transport stream packet identifiers, flag bits, and sequence numbers were corrected in order to allow more 184-byte pieces of the MPEG data to be incorporated. If you review from about page 38, that's where Shanuson posts the first pass at the extracted images from the error-corrected transport stream, and prior pages are where we were discussing how to get from point A to pont B (or point 0x03E8) with the protocol.

So yes, it's expected and OK, and you can go ahead and deploy.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/14/2014 09:36 PM
Only I ran the new ffmpeg on the old images not the new ones :/

Anyway it's deployed, if it's wrong (I can't tell) I can revert easily enough.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Joffan on 05/14/2014 10:15 PM
Only I ran the new ffmpeg on the old images not the new ones :/

Anyway it's deployed, if it's wrong (I can't tell) I can revert easily enough.

Hi Iain
Interface request for http://spacexlanding.wikispaces.com/Online+Editor : can you set it so that when the user clicks on a block in the image, the relevant part of the log is shown? and maybe fill in the existing pos value in the edit window (if it's a previously unspecified block).
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/14/2014 10:20 PM
Which bits of the log do you need? There's already quite a bit there above the image.

As for auto adding pos, currently, because of the way it's built, that would end up adding a bunch of effectively useless parts to the mmb (i.e. if you just click around and don't actually do anything) There might be a way round this, I'll have a think tomorrow when I get back from work =)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/14/2014 10:20 PM
@Michael Niedermayer:

I've been investigating the possibility of automatically setting all L1,L2,L3,L4,C1,C2 values for all macroblocks for a frame. Basicly inputting an mmb and letting a script detect "bad lines" in the picture. Moving from the right-bottom corner to the upper-top.

I'm starting to believe (using some extra logging and some manual experimentation) that this might actually be possible. If it does this would improve the quality of the frames quite dramatically and would save a lot of manual work!

However there is one thing that I find difficult to do myself. And I think you might be able to help us. Instead of using this kind of setting:

15:14:-1:0:-40:0:0:0:20

where I replace the current macroblock with a blank one and change the DC values.

But what I really want to do is this:

15:14:57217:0:-40:0:0:0:20

In other words: I want to keep the macroblock with its (structured) data as much as I can (at the very least the blocks in it which are set to 0 above) but change only the DC "starting" values. So not blanken it, but modifying it so these DC-values make sure propogation to other blocks works the same as now with -1 (therefore fixing the color and intensity of following blocks) but I also keep the current blocks inside the current macroblock.

Is something like that possible?

Regards,

arnezami
Hi guys,

I've created a new feature that implements the above.

Simply put you can now do something like this:

15:14:57217:0:-40:0:0:0:20

This will keep the macroblock (so not nullify it) bit it will change the DC values. With this you can really fix the lum/chrom issues in the current I-frames. Should have a big effect. I hope.

[edit] Forgot: it now also logs the 6 DC values for each macroblock. Which can come in handy in combination with this new feature.

Also you can do this now:

15:14:57217:0:0:0:0:0:0:0
15:14:57217:0:0:0:0:0:0:63

What this does is change direction from where a DC prediction are coming/inherited from: left or top. For each 1-bit in this number it tells it to get the DC-prediction from the top for a specific DC (4 x lum, 2 x chrom). A 0-bit means left. The number consists of 6 bit (hence max is 63). So in effect a 0 is all left, a 63 is all top. I don't know yet whether this feature is going to be very useful but it can't hurt.

I've attached the 3 source files that were changed. If somebody can check if it doesn't conflict with the existing branch (and maybe test it) and then push it to github, that would be great. Its these files btw:

libavcodec/mpegvideo.h
libavcodec/mpeg4video.h
libavcodec/h263dec.c

[edit] Fixed a bug: when using -2 it would get stuck in that mode. zip re-uploaded.

I need some sleep now ;)

Regards,

arnezami

PS. I was also trying to make a reverse-macroblock finder of sorts. So instead of telling its starting bitposition you would tell it its ending bitposition (which would usually be a starting bitposition of a mb you already have). But I ran into a lot of issues. Even when you try all possible DC-directions it still won't find the right starting position. Even when I fake the right DC values left to the block. I'm not sure why yet. Probably something I don't understand yet. This part of the code is in there but its commented out. It was also extremely error-log-heavy...

Iain, is it possible to add this in the online editor interface? It is quite useful :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/14/2014 10:25 PM
Yup, I'll look at that tomorrow too.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Joffan on 05/14/2014 10:28 PM
Which bits of the log do you need? There's already quite a bit there above the image.

As for auto adding pos, currently, because of the way it's built, that would end up adding a bunch of effectively useless parts to the mmb (i.e. if you just click around and don't actually do anything) There might be a way round this, I'll have a think tomorrow when I get back from work =)

I meant the scrolling image interpretation log below the image - can you position that to show the relevant block as maybe the third line in the window? And actually now you point out the line above the image (which I had been looking for and never found for some reason), that has the current position of the block anyway so it's fairly easy to cut and paste across.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/15/2014 03:36 AM
Which bits of the log do you need? There's already quite a bit there above the image.

As for auto adding pos, currently, because of the way it's built, that would end up adding a bunch of effectively useless parts to the mmb (i.e. if you just click around and don't actually do anything) There might be a way round this, I'll have a think tomorrow when I get back from work =)

I meant the scrolling image interpretation log below the image - can you position that to show the relevant block as maybe the third line in the window? And actually now you point out the line above the image (which I had been looking for and never found for some reason), that has the current position of the block anyway so it's fairly easy to cut and paste across.

I suspect you hadn't seen it because it was added yesterday :) I think this change (the extra info at the top) is better than scrolling the log, right? Or do you still see utility in that? This might actually be relevant with the new dc value logging (see edit2).


IainCole: Since the MB already auto-populates the mmb field, I think he's referring to the byte offset of the MB into the "Invert Bits" box. In your code, that's var pos = parseInt(match[4]); near line 714. It could just fill in a new invert bits with a mask of 0.

Edit: I don't know about utility, but it would also be fun to have 8 checkboxes rather than typing in the hex. It would make it easier for non-software people to visualize. Maybe a shift left/right/invert button. I can code it if you're busy

Edit2: I didn't think that it was actually possible to scroll a textarea in js but apparently it is http://features.sheep.art.pl/AutoscrolledTextarea , though due to interop issues it's non-trivial. The trick would be to append one line at a time, recording the length of the string when you hit a mb_pos_info and storing it with the rest of the MB data. 
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/15/2014 04:10 AM
The online editor 2 will also have an updated version of those iframes for those that are using that tool.

Just to add on this: Big thanks to Iain for implementing some pretty cool new features in http://spacex.slapbet.org

- MB Errors (from the log) are now shown next to the MB X:Y display above the image when an MB is clicked. This includes DC clipping, dquant, ibpc, ac-tex, and stuffing (though I'm not even sure what they all mean ;) ).

...

Just to follow up with this, michaelni gave a nice description of each error on the IRC:

Quote from: michaelni
dquant means that a MB that changes the QP has occured, thats no error but possibly is not used by the encoder which in that case could be used to detect errors

ac-tex damaged means that something went wrong during decoding of the ac coefficients, for example the sum of runs being larger than the space in a 8x8 block or a invalid vlc

I cbpy damaged means that where the cbpy is stored there was a bit sequence that maps to no valid intra cbpy

marker bit missing in X. esc means that a bit that is required to be 1 in the ac coefficient escape coding was not 1

stuffing is a method used to increase the bitrate, aka to stuff (useless) bits in the stream, its allowed in mpeg4 but i suspect the encoder doesnt use it so it could be used to identify errors. That is, one MB type is for stuffing and when that occurs the decoder throws it away and reads the mb type again until it finds a non stuffing one, that way the bitstream can be filled up to reach some bitrate. So like the null TS packets, but at a MB level, probably meaning that the MB type is corrupt
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: michaelni on 05/15/2014 04:50 AM
Hi guys,

I've created a new feature that implements the above.

Simply put you can now do something like this:

15:14:57217:0:-40:0:0:0:20

This will keep the macroblock (so not nullify it) bit it will change the DC values. With this you can really fix the lum/chrom issues in the current I-frames. Should have a big effect. I hope.

[edit] Forgot: it now also logs the 6 DC values for each macroblock. Which can come in handy in combination with this new feature.

Also you can do this now:

15:14:57217:0:0:0:0:0:0:0
15:14:57217:0:0:0:0:0:0:63

What this does is change direction from where a DC prediction are coming/inherited from: left or top. For each 1-bit in this number it tells it to get the DC-prediction from the top for a specific DC (4 x lum, 2 x chrom). A 0-bit means left. The number consists of 6 bit (hence max is 63). So in effect a 0 is all left, a 63 is all top. I don't know yet whether this feature is going to be very useful but it can't hurt.

I've attached the 3 source files that were changed. If somebody can check if it doesn't conflict with the existing branch (and maybe test it) and then push it to github, that would be great. Its these files btw:

libavcodec/mpegvideo.h
libavcodec/mpeg4video.h
libavcodec/h263dec.c

pushed changes  into github  (except what was commented out)

thanks
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: michaelni on 05/15/2014 05:10 AM
My thought was to deal with what I presume is low-hanging fruit: blocks with no or virtually no change from one iframe to the next.

Do P-frame blocks within a given frame propagate like I-frame blocks? Or is it strictly a diff from the same MB?

If we can create fake P-frames with no change, then fill in the data changes one at a time, we might make faster progress. But if they are interdependent you'd need to guess C/L data for each which could get annoying fast (and give people creative license to make the video into whatever)

P- frames can contain both intra MBs (like I frames) and inter MBs, each inter MB has 1(common) or 4(not so common) motion vector(s) that points to some location in the previous frame from where data is copied, such MB then stores the difference to that previous area. MVs are predicted somewhat similar to DC values of intra MBs, allthough MVs use the left, top and top right MBs for prediction. You can visualize the motion vectors with -vismv 7
MVs are in half pixel units
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/15/2014 08:53 AM
PS. I was also trying to make a reverse-macroblock finder of sorts. So instead of telling its starting bitposition you would tell it its ending bitposition (which would usually be a starting bitposition of a mb you already have). But I ran into a lot of issues. Even when you try all possible DC-directions it still won't find the right starting position. Even when I fake the right DC values left to the block. I'm not sure why yet. Probably something I don't understand yet. This part of the code is in there but its commented out. It was also extremely error-log-heavy...
Good news: I finally got the reverse searching of macroblocks to work!  8)

It's still quite hacky right now but hopefully I can release some working code within a day or so.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/15/2014 10:14 AM
Hi Guys,

Here is my by-hand-aligned raw.ts, and all iframes after piping it through tsfix (tsalign still complains about one region but that's an misreport. Compare that part to raw.ts and you will see that all 47's are there and at the right position in that part. so don't use tsalign my that ts.)
I will update  http://spacexlanding.wikispaces.com/Progress+on+edit5.ts  (http://spacexlanding.wikispaces.com/Progress+on+edit5.ts) after this post.
But i have to go back to do my original work, So this will probably be my last contribution. 
I used the continuity bit and my best judgement to edit raw.ts to get that outcome.
There was on 56byte missing part where i am not 100% sure I add the 56 FF at the right position, I was a gap of 5 188 blocks with one missing 56 bytes. I added the 56 byte to the last one and marked it by 12 34 56 78 for the first 4 of those 56 Bytes.

Have fun with it :D

Cheers Shanuson
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/15/2014 10:18 AM
And here a Zip with all img-files and the raw_edit8.ts
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/15/2014 10:37 AM
An here is the rest of the frames.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/15/2014 12:41 PM
Amazing job! The wiki page with these net iframes is up and working :)
Looking forward to see your edits
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mhenderson on 05/15/2014 02:39 PM
Progress on old iFrame 10 / frame 209 in edit5.ts.

X:14630:01,X:14708:01,X:15737:01,X:15817:1,X:15843:1,X:17180:1,43:2:8818,2:3:
8916,5:5:-2:16:25:25:16,6:5:14708,18:5:-1:-1:1:1:
-1:0:0,21:5:-1:-4:1:-5:-5,23:5:-1:-2:3:-4:2:0:0,25:5:16467,31:5:-1:-5:0:0:-5:
0:0,32:5:16740,37:5:-2:0:0:-5:-5:0:0,0:6:-1,0:7:-1,21:7:21000,0:15:-2,7:17:55313

This is basically back to where it was before the switch to edit5.ts. Some improvements in row 5.

I use the online v2 editor. Thanks so much Iain.  :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/15/2014 02:59 PM
Remember to break up your code lines or the width of the thread goes haywire.

And a welcome to mhenderson! :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/15/2014 03:21 PM
And here a Zip with all img-files and the raw_edit8.ts

Thank you for your work!

Two questions:

How can I get the "fixed_edit8.ts" version? I read about the "tsfix.c" and "tsalign.c" and the command line "cat raw.ts | tsfix | tsalign > fixed.ts", do I have to compile the files and how? For now the command line does not work. I found the files in the "FFmpeg-spacexdebug1/tools" folder.

When directly extracting the frames from raw_edit8.ts, I get only 241 frames, is it normal?

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/15/2014 03:32 PM
And here a Zip with all img-files and the raw_edit8.ts

Thank you for your work!

Two questions:

How can I get the "fixed_edit8.ts" version? I read about the "tsfix.c" and "tsalign.c" and the command line "cat raw.ts | tsfix | tsalign > fixed.ts", do I have to compile the files and how? For now the command line does not work. I found the files in the "FFmpeg-spacexdebug1/tools" folder.

When directly extracting the frames from raw_edit8.ts, I get only 241 frames, is it normal?

yes raw_edit8.ts only gives 241 frames or so. that why you need to use tsfix too.

simply do gcc -o tsfix tsfix.c and likewise for tsalign. After that only do cat raw_edit8.ts | tsfix > fixed_edit8.ts
and dont use tsalign on raw_edit8.ts

Cheers
Shanuson
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/15/2014 04:00 PM
And here a Zip with all img-files and the raw_edit8.ts

Thank you for your work!

Two questions:

How can I get the "fixed_edit8.ts" version? I read about the "tsfix.c" and "tsalign.c" and the command line "cat raw.ts | tsfix | tsalign > fixed.ts", do I have to compile the files and how? For now the command line does not work. I found the files in the "FFmpeg-spacexdebug1/tools" folder.

When directly extracting the frames from raw_edit8.ts, I get only 241 frames, is it normal?

yes raw_edit8.ts only gives 241 frames or so. that why you need to use tsfix too.

simply do gcc -o tsfix tsfix.c and likewise for tsalign. After that only do cat raw_edit8.ts | tsfix > fixed_edit8.ts
and dont use tsalign on raw_edit8.ts

Cheers
Shanuson

Now I can run it, but it only creates a 32 bytes file... I made a print screen of the command prompt output. Is it possible to post your fixed_edit8.ts file?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/15/2014 04:15 PM
Now I can run it, but it only creates a 32 bytes file... I made a print screen of the command prompt output. Is it possible to post your fixed_edit8.ts file?

Are you on windows? The 'cat' command is *nix, IIRC the equivalent on windows is 'type'. So type raw_edit8.ts | tsfix > fixed_edit8.ts

Also, tsfix < raw_edit8.ts > fixed_edit8.ts might work better.

 (disclaimer: I don't have Windows)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/15/2014 04:23 PM

yes raw_edit8.ts only gives 241 frames or so. that why you need to use tsfix too.

simply do gcc -o tsfix tsfix.c and likewise for tsalign. After that only do cat raw_edit8.ts | tsfix > fixed_edit8.ts
and dont use tsalign on raw_edit8.ts

Cheers
Shanuson

Now I can run it, but it only creates a 32 bytes file... I made a print screen of the command prompt output. Is it possible to post your fixed_edit8.ts file?

I can do when i am back at home.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/15/2014 04:28 PM
Yes I am on windows (using MinGW).

So I tried again your propositions and got the same as before, I compiled tsfix.c with Visual Studio and also got the same, and finally logged to a Unix machine and managed to do it there.

So I finally got all the frames, thank you for the help! :)

edit: I put the file in attachment
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Untribium on 05/15/2014 04:42 PM
Some quick work on frame 52/old iframe 2. The bit flips are pretty much random...
X:25254:1,X:25299:1,X:53423:1,40:3:-1,6:4:9364,4:5:-1,6:5:11443,7:5:11487,0:7:-1,28:11:25250
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/15/2014 04:43 PM
So I adapted iframe32 using the new options with:

X:57182:4,
40:0:-1,
17:1:5444,13:2:-1,
5:4:14914,43:4:-1,
7:5:19298,14:06:24261:0:0:0:0:0:0:0,23:11:45354:0:0:0:0:0:0:0,26:12:-1,
7:13:52167,7:14:-1,
17:15:62855,28:15:-1,
27:16:68962,21:17:72509:0:0:0:0:0:0:0,21:18:76689:0:0:0:0:0:0:0,24:18:76921:0:0:0:0:0:0:0,7:21:-1,
10:21:-1:-5:10:0,
11:21:-1,
20:21:90326:10:0:0:0,22:27:-1,
27:27:112406,0:28:-1,
3:28:115143,9:29:-1,
33:29:135545
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/15/2014 04:52 PM
So back to the P-frames...

So this is an example of a P-frame (frame192). I ploted the motion vectors to see if they are also correctly modified. I used the following command lines for both images:

ffmpeg -debug mb_pos_size -err_detect ignore_err -vismv 7 -s:0 704:480 -i frame192.mpg4-img -f image2 frame_192_mv.png
ffmpeg -mmb 2:0:-1,19:0:1092 -debug mb_pos_size -err_detect ignore_err -vismv 7 -s:0 704:480 -i frame192.mpg4-img -f image2 frame_192_mv_mod.png

Now instead of creating a png, is it possible to directly store the modified one as a P-frame? If yes, how? Or does something have to be implemented?

When using:
ffmpeg -mmb 2:0:-1,19:0:1092 -debug mb_pos_size -err_detect ignore_err -s:0 704:480 -i frame192.mpg4-img -f image2 frame192_mod.mpg4-img
it stores an I-frame (we can check the frame type with option -vf 'showinfo')

When using:
ffmpeg -mmb 2:0:-1,19:0:1092 -debug mb_pos_size -err_detect ignore_err -s:0 704:480 -i frame192.mpg4-img -vodec copy -an -f image2 frame192_mod.mpg4-img
it just copies the P-frame without modifying it.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/15/2014 06:17 PM
So I adapted iframe32 using the new options with:

X:57182:4,
40:0:-1,
17:1:5444,13:2:-1,
5:4:14914,43:4:-1,
7:5:19298,14:06:24261:0:0:0:0:0:0:0,23:11:45354:0:0:0:0:0:0:0,26:12:-1,
7:13:52167,7:14:-1,
17:15:62855,28:15:-1,
27:16:68962,21:17:72509:0:0:0:0:0:0:0,21:18:76689:0:0:0:0:0:0:0,24:18:76921:0:0:0:0:0:0:0,7:21:-1,
10:21:-1:-5:10:0,
11:21:-1,
20:21:90326:10:0:0:0,22:27:-1,
27:27:112406,0:28:-1,
3:28:115143,9:29:-1,
33:29:135545
Just to be sure: the extra six 0's (0:0:0:0:0:0) won't do anything here I believe. The 6 numbers are only needed if you fill in at least one non-zero number. They change the lum/chrom by the number. They don't set it to 0.

Just in case I wasn't very clear about that.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/15/2014 06:23 PM
So I adapted iframe32 using the new options with:

X:57182:4,
40:0:-1,
17:1:5444,13:2:-1,
5:4:14914,43:4:-1,
7:5:19298,14:06:24261:0:0:0:0:0:0:0,23:11:45354:0:0:0:0:0:0:0,26:12:-1,
7:13:52167,7:14:-1,
17:15:62855,28:15:-1,
27:16:68962,21:17:72509:0:0:0:0:0:0:0,21:18:76689:0:0:0:0:0:0:0,24:18:76921:0:0:0:0:0:0:0,7:21:-1,
10:21:-1:-5:10:0,
11:21:-1,
20:21:90326:10:0:0:0,22:27:-1,
27:27:112406,0:28:-1,
3:28:115143,9:29:-1,
33:29:135545
Just to be sure: the extra six 0's (0:0:0:0:0:0) won't do anything here I believe. The 6 numbers are only needed if you fill in at least one non-zero number. They change the lum/chrom by the number. They don't set it to 0.

Just in case I wasn't very clear about that.

I know, it's the 7th "0" which is useful there
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/15/2014 06:36 PM
At the risk of introducing additional confusion and coding difficulty, may I suggest IPV6-style notation for zeros,

:: --> 0:0:0:0:0:0

23:11:45354:0:0:0:0:0:0:0
23:11:45354::0

The '::' can only occur once and only fills zeros.

if the 7th 0 is different than having only 6 zeros, maybe change it to requiring a '-0' rather than '0' to distinguish the cases.

Edit: I have this implemented in a fork, waiting on michaelni to merge
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/15/2014 06:36 PM
So I adapted iframe32 using the new options with:

X:57182:4,
40:0:-1,
17:1:5444,13:2:-1,
5:4:14914,43:4:-1,
7:5:19298,14:06:24261:0:0:0:0:0:0:0,23:11:45354:0:0:0:0:0:0:0,26:12:-1,
7:13:52167,7:14:-1,
17:15:62855,28:15:-1,
27:16:68962,21:17:72509:0:0:0:0:0:0:0,21:18:76689:0:0:0:0:0:0:0,24:18:76921:0:0:0:0:0:0:0,7:21:-1,
10:21:-1:-5:10:0,
11:21:-1,
20:21:90326:10:0:0:0,22:27:-1,
27:27:112406,0:28:-1,
3:28:115143,9:29:-1,
33:29:135545

I have not been able to replicate this using the online editor. Can you confirm that this is edit5?  ;)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/15/2014 06:38 PM
Some quick work on frame 52/old iframe 2. The bit flips are pretty much random...
X:25254:1,X:25299:1,X:53423:1,40:3:-1,6:4:9364,4:5:-1,6:5:11443,7:5:11487,0:7:-1,28:11:25250

Same here! I´m afraind you guys were using an old version of edit5  :P
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/15/2014 06:40 PM
That's assuming I haven't done something wrong ;)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/15/2014 06:53 PM
hehehe indeed. Can someone using the FFmpeg utility directly confirm to us if the output of the editor is consistent?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/15/2014 06:54 PM
I know, it's the 7th "0" which is useful there
Oooh. Sorry. I didn't count ;)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Untribium on 05/15/2014 07:15 PM
Some quick work on frame 52/old iframe 2. The bit flips are pretty much random...
X:25254:1,X:25299:1,X:53423:1,40:3:-1,6:4:9364,4:5:-1,6:5:11443,7:5:11487,0:7:-1,28:11:25250

Same here! I´m afraind you guys were using an old version of edit5  :P

Yeah I just realized that...oh well, here we go again :D

EDIT: The new version is a lot cleaner! Great work Shanuson!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/15/2014 07:40 PM
Hi Guys,

Here is my by-hand-aligned raw.ts, and all iframes after piping it through tsfix (tsalign still complains about one region but that's an misreport. Compare that part to raw.ts and you will see that all 47's are there and at the right position in that part. so don't use tsalign my that ts.)
I will update  http://spacexlanding.wikispaces.com/Progress+on+edit5.ts  (http://spacexlanding.wikispaces.com/Progress+on+edit5.ts) after this post.
But i have to go back to do my original work, So this will probably be my last contribution. 
I used the continuity bit and my best judgement to edit raw.ts to get that outcome.
There was on 56byte missing part where i am not 100% sure I add the 56 FF at the right position, I was a gap of 5 188 blocks with one missing 56 bytes. I added the 56 byte to the last one and marked it by 12 34 56 78 for the first 4 of those 56 Bytes.

Have fun with it :D

Cheers Shanuson

I just want to give you a very big thank you for this.  :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: michaelni on 05/15/2014 07:48 PM
So back to the P-frames...

So this is an example of a P-frame (frame192). I ploted the motion vectors to see if they are also correctly modified. I used the following command lines for both images:

ffmpeg -debug mb_pos_size -err_detect ignore_err -vismv 7 -s:0 704:480 -i frame192.mpg4-img -f image2 frame_192_mv.png
ffmpeg -mmb 2:0:-1,19:0:1092 -debug mb_pos_size -err_detect ignore_err -vismv 7 -s:0 704:480 -i frame192.mpg4-img -f image2 frame_192_mv_mod.png

Now instead of creating a png, is it possible to directly store the modified one as a P-frame? If yes, how? Or does something have to be implemented?

You can flip bits or move bits around in the frame and or the ts file using a hex editor or some other tool, keeping the data otherwise exactly intact. Or all the changes which are possible through mmb could be applied while decoding, this would require to make mmb work with multiple frames.
Some changes now possible with mmb like changing dc prediction direction can not be represented within the bitstream so they could not be added into frames.  Unless the frames are simply reencoded to high quality I-Frames (which in some sense is cheating but still may be a usefull tool)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Untribium on 05/15/2014 07:59 PM
Frame 52 (now using the most recent ts :)) )
40:3:-1,6:4:9364,4:5:-1,6:5:11443,7:5:11487,5:6:13496:0:0:0:0:0:-2,0:7:-1,38:11:26080:0:-4:-1:2:0:0,
31:12:28067:3:1:0:0,34:12:28226:-1,40:14:34617:-3:0:2:0,40:16:42426:-1:0:-4:-5,33:17:44961:0:-7:0:0:-2:0
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: FutureSpaceTourist on 05/15/2014 08:11 PM
Amazing thread; it's fantastic to see what you are all achieving.

Am I the only one thinking some of the frames currently look very Turneresque?!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/15/2014 08:21 PM
Frame 52 (now using the most recent ts :)) )
40:3:-1,6:4:9364,4:5:-1,6:5:11443,7:5:11487,5:6:13496:0:0:0:0:0:-2,0:7:-1,38:11:26080:0:-4:-1:2:0:0,
31:12:28067:3:1:0:0,34:12:28226:-1,40:14:34617:-3:0:2:0,40:16:42426:-1:0:-4:-5,33:17:44961:0:-7:0:0:-2:0

Perfect! Great work!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/15/2014 09:02 PM
Haven't had time to work on this for a while, but here's my latest attempt at the cursed frame 72 (ex iframe4), incorporating work by SwissCheese and seanpg71. I've been trying to get to the badly corrupted business end of the rocket but I fear this is little more than bad art and figments of my imagination. My current techiques don't allow much better, may need to develop new ones for this.

mmb attached as .txt file:
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/15/2014 09:18 PM
Ugg dumb question guys.

I assumed (as does the logic in the online editor) for X:pos:mask, mask was a hexadecimal value ("00" to "FF").

When I was adding the "::" to h263dec.c, I noticed the sscanf is using %d, which means it's decimal.

Which way is right/intended? Edit: so that I can update the wiki with the correct info…


Edit2: Ok, I was looking at the wrong line. The X:pos:mask IS a hex mask

but

The new DC inherit direction is a decimal mask. This should probs be changed to hex, if not too many people are using it.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: lewing on 05/15/2014 09:25 PM
Here is my best attempt at frame 150 from edit8.ts

X:18494:1,20:5:-1,33:6:23509,14:12:-1,18:12:46628,15:13:-1:24,16:13:50797:20,19:14:57637,
18:15:-1:0:0:4:2::14,25:15:66746,24:16:72453,32:18:-1:1:10:-3:6,35:18:87166,39:19:94065,
43:19:-1,0:20:94827,12:20:-1,43:20:101568,31:21:106253,33:22:112514,22:23:-1,
23:23:119825,26:23:-2,0:26:128548,5:27:-1,9:27:133159,0:29:142356,2:29:147166
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/15/2014 09:34 PM
Haven't had time to work on this for a while, but here's my latest attempt at the cursed frame 72 (ex iframe4), incorporating work by SwissCheese and seanpg71. I've been trying to get to the badly corrupted business end of the rocket but I fear this is little more than bad art and figments of my imagination. My current techiques don't allow much better, may need to develop new ones for this.

Perhaps someone from SpaceX can tell us whether or not we should expect to see some indication of leg deployment 1.3 seconds before what's depicted in the next iframe, shown here:

(http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=582517;image)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/15/2014 10:07 PM
Haven't had time to work on this for a while, but here's my latest attempt at the cursed frame 72 (ex iframe4), incorporating work by SwissCheese and seanpg71. I've been trying to get to the badly corrupted business end of the rocket but I fear this is little more than bad art and figments of my imagination. My current techiques don't allow much better, may need to develop new ones for this.

mmb: 5:1:4200,0:3:10968,8:3:14762,28:3:16550,24:4:20286,40:4:-1,9:6:21857,13:11:39488,37:11:41549,
10:12:-1,11:12:42693,30:12:-1,32:12:43895,24:13:-1,18:14:50986,8:15:-1,19:15:54479,20:15:54791,
2:16:-1,8:16:58952,13:16:-1,15:16:59896,20:16:61472,24:16:-1,27:16:63538,18:17:69075,27:17:-1,
13:18:74459,30:20:-1,41:20:91288,32:21:-1,34:21:96927,25:22:-1,41:22:104774,0:24:-1,
17:26:122506,0:27:-1,24:27:127453,6:28:-1

Looks amazing. But what .ts are you using? I´m getting something slightly differnt on edit8
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/15/2014 10:19 PM

Looks amazing. But what .ts are you using? I´m getting something slightly differnt on edit8

Edit5. The last one in the online editor. It's a useful tool. It was the same in edit5 and try1 so I assumed it would be the same in edit8 too. I'll check edit8.

Edit: I got the same thing from edit8. The one posted here: http://forum.nasaspaceflight.com/index.php?topic=34597.msg1198706#msg1198706

Edit2: I have somehow managed to post a bad mmb, I apologize.

Still bad, something's wrong with my pasting skills.
5:1:4200,0:3:10968,8:3:14762,28:3:16550,24:4:20286,40:4:-1,9:6:21857,13:11:39488,37:11:41549,
10:12:-1,11:12:42693,30:12:-1,32:12:43895,24:13:-1,18:14:50986,8:15:-1,19:15:54479,
20:15:54791,2:16:-1,8:16:58952,13:16:-1,15:16:59896,20:16:61472,24:16:-1,27:16:63538,
18:17:69075,27:17:-1,13:18:74459,30:20:-1,41:20:91288,32:21:-1,34:21:96927,25:22:-1,
41:22:104774,0:24:-1,17:26:122506,0:27:-1,24:27:127453,6:28:-1
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/15/2014 10:26 PM
Here is also the fixed_edit8.ts for those of you using windows :D
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/15/2014 10:36 PM

Looks amazing. But what .ts are you using? I´m getting something slightly differnt on edit8

Edit5. The last one in the online editor. It's a useful tool. It was the same in edit5 and try1 so I assumed it would be the same in edit8 too. I'll check edit8.

Edit: I got the same thing from edit8. The one posted here: http://forum.nasaspaceflight.com/index.php?topic=34597.msg1198706#msg1198706

I'm getting a bulish version on the online editor... no idea why  :o

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Oersted on 05/15/2014 10:44 PM
Am I the only one thinking some of the frames currently look very Turneresque?!

Turneresque indeed, with a bit of Mondrian thrown in, unfortunately!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/15/2014 10:45 PM
I'm getting a bulish version on the online editor... no idea why  :o

I'm attaching the mmb as text. For some reason it's not pasting correctly.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/15/2014 10:51 PM
Sweet! That was it!  ;)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/15/2014 11:08 PM
A few updates to the UI of the editor today, so refresh if you haven't recently :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/16/2014 06:00 AM
Hi guys,

Here is the source for reverse searching of macroblocks.  8)

Only this file was changed:

libavcodec/h263dec.c

Please test first and then push to github.

How it works:

Lets say you know that a correct macroblock starts here:

0:0-1,19:06:22832

and you want to find 18:06 but you don't know its bitposition. So you do this:

0,0:0:-1,18:06:--22832:0,19:06:22832

With this you give the instruction: when reaching mb 18:06 start with the position 22832 (which is the starting bitposition of 19:06) and try to find a possible starting bitposition before 22832. What it will do is first set bitposition of 18:06 to 22831, then decode from there and if the decoding of the macroblock does not end at 22832 it will start from 22830 etc. If it does at some point end at 22832 it will keep that as the solution. You can see the final position in the log.

If you don't think it's the real solution you can tell it to skip the one or more candidates:

0,0:0:-1,18:06:--22832:1,19:06:22832

This will skip the first hit and will take the second hit as the solution.

Now after experimenting with this I saw there can be quite a few false positives: for one example I found 11 possible solutions. So you will still have to determine yourselves which one is the most likely starting position (probably the one with the lowest bitposition number). In practice the real starting position is the one where the macroblock before that (in this case 17:06) ends again. So I think you can see where this is going ;)

There is a max of 2048 tries at the moment btw. If it doesn't find a solution it will set the starting position at the postion you told it to start searching from (22832 in this case).

Sometimes it crashes after finding no solution. I'm not sure why yet. So handle with care.

Important note: do not post a reverse searching command in your solution online! If you find a bitposition using this technique then replace the search-command with a normal command (eg 18:06:22712). Otherwise you will create a LOT of unnecessary log to others.

Regards,

arnezami

PS. This technique will cause quite a bit of error-log because you are trying to start from a lot of wrong bitpositions. Never do two searches at the same time!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/16/2014 06:38 PM
Still trying to get something out of the p-frames...

We can visualize quite well the macroblock types with -debug vis_mb_type and the motion vectors with -vismv 7.  It's nice but does not help so much as it is only a visual information. Would it be possible to write the macroblock type and motion vector values besides the macroblock dc values? I think it would help quite a lot.

edit: found that we can write out the MB types with -debug mb_type

edit2: wrote some code lines to do it :), will post tomorrow
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/17/2014 11:34 AM
The updated version displaying the MB type and the motion vectors is in attachment, it was based on the latest version of Arnezami. I hope it does not introduce any bug ;)

I also tried to rebuild frames +/- "as the encoder does" with some basic matlab code, the problem is that many P-frames are highly corrupted... but it worked not too badly for frames 169 (I-frame), 171 and 172.

I don't know if it's a good idea to go this way, but I was stuck trying to repair P-frames: as soon as I touched them (tried to correct them with matlab), they would not be used during the video encoding (even when simply reading and rewriting a frame, without changing anything).

Btw some P-frames like 171 or 176 have issues in their first data bytes or at the end of the header (bytes 4/5/6), I don't really know, and are not converted to PNG, a copy/paste of these bytes from a valid frame solves the issue.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/17/2014 11:41 AM
Hi guys,

I noticed a problem in the online editor. It is removing zero's in the DC sections. This goes wrong if you want to do something like this: 16:05:65003:0:0:-5:0:0:0. Now it makes 16:05:65003:::-5,0 out of that. Which is wrong.

Only if all 6 DC values (L1:L2:L3:L4:C1:C2) are zero should it remove the 6 zero's (L1:L2:L3:L4:C1:C2). Otherwise there should always be 6 numbers there. The direction-number after that is optional.

Also I think it is not updated to the latest commit (https://github.com/michaelni/FFmpeg/commit/77d24d6a036e18064bcd25281f8d458e92de9b7e) where the short version 16:05:65003::15 of changing the directions was introduced.

Regards,

arenzami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/17/2014 12:01 PM
The updated version displaying the MB type and the motion vectors is in attachment, it was based on the latest version of Arnezami. I hope it does not introduce any bug ;)

I also tried to rebuild frames +/- "as the encoder does" with some basic matlab code, the problem is that many P-frames are highly corrupted... but it worked not too badly for frames 169 (I-frame), 171 and 172.

I don't know if it's a good idea to go this way, but I was stuck trying to repair P-frames: as soon as I touched them (tried to correct them with matlab), they would not be used during the video encoding (even when simply reading and rewriting a frame, without changing anything).

Btw some P-frames like 171 or 176 have issues in their first data bytes or at the end of the header (bytes 4/5/6), I don't really know, and are not converted to PNG, a copy/paste of these bytes from a valid frame solves the issue.
Cool! And interesting. The engine is just igniting! :)

Regarding the P-frames: I think it would be a good idea to extend the -mmb syntax with a framenumber.

So you would have something like this:

-mmb FRAME168:16:04:37259,23:02:73229=FRAME169:01:01:2342,34:02:-1  etc.

Something like that.

That way you can give it either the complete .ts or a .ts that starts with an I-frame until the P-frame you are actually working on. It would run through all the frames changing their macroblocks. And the P-frame you're interested in will inherit all the fixes (and errors of course) that have preceded it.

The only big problem is that the mmb will get quite big. We would need some kind of script that assembles all the mmb's of the frames and calls ffmpeg with that long mmb.

Good idea?

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Avron on 05/17/2014 12:13 PM
...I was stuck trying to repair P-frames:...



Amazing, quite amazing to see P- frames. I would hope Elon send you all an invite to tour the factory for the work effort he triggered, I am so sure he will do something great to say thanks.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/17/2014 12:42 PM
The only big problem is that the mmb will get quite big. We would need some kind of script that assembles all the mmb's of the frames and calls ffmpeg with that long mmb.

In condor_submit, you just specify the "-append" argument multiple times to add more parameters. Seems like the same idea would work here, and allow a bit more flexibility in the context of this idea.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/17/2014 01:40 PM
The updated version displaying the MB type and the motion vectors is in attachment, it was based on the latest version of Arnezami. I hope it does not introduce any bug ;)

I also tried to rebuild frames +/- "as the encoder does" with some basic matlab code, the problem is that many P-frames are highly corrupted... but it worked not too badly for frames 169 (I-frame), 171 and 172.

I don't know if it's a good idea to go this way, but I was stuck trying to repair P-frames: as soon as I touched them (tried to correct them with matlab), they would not be used during the video encoding (even when simply reading and rewriting a frame, without changing anything).

Btw some P-frames like 171 or 176 have issues in their first data bytes or at the end of the header (bytes 4/5/6), I don't really know, and are not converted to PNG, a copy/paste of these bytes from a valid frame solves the issue.
Cool! And interesting. The engine is just igniting! :)

Regarding the P-frames: I think it would be a good idea to extend the -mmb syntax with a framenumber.

So you would have something like this:

-mmb FRAME168:16:04:37259,23:02:73229=FRAME169:01:01:2342,34:02:-1  etc.

Something like that.

That way you can give it either the complete .ts or a .ts that starts with an I-frame until the P-frame you are actually working on. It would run through all the frames changing their macroblocks. And the P-frame you're interested in will inherit all the fixes (and errors of course) that have preceded it.

The only big problem is that the mmb will get quite big. We would need some kind of script that assembles all the mmb's of the frames and calls ffmpeg with that long mmb.

Good idea?

Regards,

arnezami
Hi guys,

I've extended the mmb-syntax so it can now handle multiple frames at the time. See the above quote.

The syntax is:

Quote
-mmb FRAME0:16:04:37259,23:02:73229=FRAME1:01:01:2342,34:02:-1

Note the '=' sign!

The old syntax still works on files with one frame in it.

I've also incorporated SwissCheese' logging (and changed back the && into & ;))

I haven't actually tested it with a file with multiple frames in it. But it *should* work. Let me know!

[edit] Important: the frame numbers should be the frame numbers in your file. If you use a file with say 3 frames in it their frame numbers should be 0, 1 and 2. So not 168, 169, 170 in that case. Should I add the frame number in the log too? Not sure.

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/17/2014 01:51 PM
Coming up on the 150,000 view mark for this thread/effort. Noting it as there's a small active team of people really working hard on this, but also a HUGE amount of people watching that work. Always nice to have a large appreciative audience, both inside and outside the space industry.

You'd be astonished how many views are from IP addresses in a place called Hawthorne, California! ;D
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/17/2014 02:02 PM
The updated version displaying the MB type and the motion vectors is in attachment, it was based on the latest version of Arnezami. I hope it does not introduce any bug ;)

I also tried to rebuild frames +/- "as the encoder does" with some basic matlab code, the problem is that many P-frames are highly corrupted... but it worked not too badly for frames 169 (I-frame), 171 and 172.

I don't know if it's a good idea to go this way, but I was stuck trying to repair P-frames: as soon as I touched them (tried to correct them with matlab), they would not be used during the video encoding (even when simply reading and rewriting a frame, without changing anything).

Btw some P-frames like 171 or 176 have issues in their first data bytes or at the end of the header (bytes 4/5/6), I don't really know, and are not converted to PNG, a copy/paste of these bytes from a valid frame solves the issue.
Cool! And interesting. The engine is just igniting! :)

Inspiring work SwissCheese! this is Amazing! Yet another piece of the puzzle (engine ignition) has been uncovered.

The idea of using matlab sounds good to me. I think we should accept that some of the frames of the video are beyond recovery.  Considering this, a good approach could be to deconstruct the video frame by frame recovering as many usable frames as possible, and then reconstruct it back again keeping only the good frames, to then interpolate (like the guys on the curiosity landing did) the missing ones in order to obtain a decent dynamic video.

My guess is the p-frames immediately after a fixed iframe will be the best looking, as only some macro blocks should have changed..... but as the sequence continues moving forward, the subsequent frames will start to accumulate errors and they will look worse and worse. That is, naturally, until the video gets to the next iframe that has been manually fixed.

For how many frames did you apply the process?  Could you the same technique for the frames after of after 72 or 209 (which could show leg deployment and touchdown)? do you thing my hyphotesis of constant deterioration will hold?

Edit: On other news, I tried to apply your very long mmb code for frame 92 (the frame formally known as iframe # 5) to the newest version (the one obtained from edit5.ts) and unfortunately is not working anymore. AS you where the main author, maybe you will be able to figure out what has changed. I'm guessing a small adjustment should be sufficient to get that frame visible again! It could also be a problem on how the code was pasted in the forum: I believe this happened to Saliva_sweet
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: billh on 05/17/2014 02:16 PM
The updated version displaying the MB type and the motion vectors is in attachment, it was based on the latest version of Arnezami. I hope it does not introduce any bug ;)

I also tried to rebuild frames +/- "as the encoder does" with some basic matlab code, the problem is that many P-frames are highly corrupted... but it worked not too badly for frames 169 (I-frame), 171 and 172.

I don't know if it's a good idea to go this way, but I was stuck trying to repair P-frames: as soon as I touched them (tried to correct them with matlab), they would not be used during the video encoding (even when simply reading and rewriting a frame, without changing anything).

Btw some P-frames like 171 or 176 have issues in their first data bytes or at the end of the header (bytes 4/5/6), I don't really know, and are not converted to PNG, a copy/paste of these bytes from a valid frame solves the issue.
Cool! And interesting. The engine is just igniting! :)

I don't think that's the engine igniting. It looks more like the plume is just beginning to interact with the ocean surface.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: edfishel on 05/17/2014 02:25 PM
For the small group of people who are working really, really hard on this...thank you!  It's beyond my comprehension.  But I thoroughly appreciate the work you are doing.  Chris Bergin is so right.  You are doing wonderful work on behalf of all of us.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/17/2014 02:29 PM
The updated version displaying the MB type and the motion vectors is in attachment, it was based on the latest version of Arnezami. I hope it does not introduce any bug ;)

I also tried to rebuild frames +/- "as the encoder does" with some basic matlab code, the problem is that many P-frames are highly corrupted... but it worked not too badly for frames 169 (I-frame), 171 and 172.

I don't know if it's a good idea to go this way, but I was stuck trying to repair P-frames: as soon as I touched them (tried to correct them with matlab), they would not be used during the video encoding (even when simply reading and rewriting a frame, without changing anything).

Btw some P-frames like 171 or 176 have issues in their first data bytes or at the end of the header (bytes 4/5/6), I don't really know, and are not converted to PNG, a copy/paste of these bytes from a valid frame solves the issue.
Cool! And interesting. The engine is just igniting! :)

Regarding the P-frames: I think it would be a good idea to extend the -mmb syntax with a framenumber.

So you would have something like this:

-mmb FRAME168:16:04:37259,23:02:73229=FRAME169:01:01:2342,34:02:-1  etc.

Something like that.

That way you can give it either the complete .ts or a .ts that starts with an I-frame until the P-frame you are actually working on. It would run through all the frames changing their macroblocks. And the P-frame you're interested in will inherit all the fixes (and errors of course) that have preceded it.

The only big problem is that the mmb will get quite big. We would need some kind of script that assembles all the mmb's of the frames and calls ffmpeg with that long mmb.

Good idea?

Regards,

arnezami
Hi guys,

I've extended the mmb-syntax so it can now handle multiple frames at the time. See the above quote.

The syntax is:

Quote
-mmb FRAME0:16:04:37259,23:02:73229=FRAME1:01:01:2342,34:02:-1

Note the '=' sign!

The old syntax still works on files with one frame in it.

I've also incorporated SwissCheese' logging (and changed back the && into & ;))

I haven't actually tested it with a file with multiple frames in it. But it *should* work. Let me know!

[edit] Important: the frame numbers should be the frame numbers in your file. If you use a file with say 3 frames in it their frame numbers should be 0, 1 and 2. So not 168, 169, 170 in that case. Should I add the frame number in the log too? Not sure.

Regards,

arnezami

I don't exactly understand how to use it, could you make a small .zip package with an example? (text file with command lines and used .ts or .mp4-img files)

(sorry for the &  ::)  :P)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/17/2014 02:32 PM
The updated version displaying the MB type and the motion vectors is in attachment, it was based on the latest version of Arnezami. I hope it does not introduce any bug ;)

I also tried to rebuild frames +/- "as the encoder does" with some basic matlab code, the problem is that many P-frames are highly corrupted... but it worked not too badly for frames 169 (I-frame), 171 and 172.

I don't know if it's a good idea to go this way, but I was stuck trying to repair P-frames: as soon as I touched them (tried to correct them with matlab), they would not be used during the video encoding (even when simply reading and rewriting a frame, without changing anything).

Btw some P-frames like 171 or 176 have issues in their first data bytes or at the end of the header (bytes 4/5/6), I don't really know, and are not converted to PNG, a copy/paste of these bytes from a valid frame solves the issue.
Cool! And interesting. The engine is just igniting! :)

Inspiring work SwissCheese! this is Amazing! Yet another piece of the puzzle (engine ignition) has been uncovered.

The idea of using matlab sounds good to me. I think we should accept that some of the frames of the video are beyond recovery.  Considering this, a good approach could be to be to deconstruct the video frame by frame recovering as many usable frames as possible, and then reconstruct it back again keeping only the good frames, to then interpolate (like the guys on the curiosity landing did) in order to obtain a decent dynamic video.

My guess is the p-frames immediately after a fixed iframe will be the best looking, as only some macro blocks should have changed..... but as the sequence continues moving forward, the subsequent frames will start to accumulate errors and they will look worse and worse. That is, naturally, until the video gets to the next iframe that has been manually fixed.

For how many frames did you apply the process?  Could you the same technique for the frames after of after 72 or 209 (which could show leg deployment and touchdown)? do you thing my hyphotesis of constant deterioration will hold?

Edit: On other news, I tried to apply your very long mmb code for frame 92 (the frame formally known as iframe # 5) to the newest version (the one obtained from edit5.ts) and unfortunately is not working anymore. AS you where the main author, maybe you will be able to figure out what has changed. I'm guessing a small adjustment should be sufficient to get that frame visible again! It could also be a problem on how the code was pasted in the forum: I believe this happened to Saliva_sweet

Yes it is exactly the problem, it works just after the I-frame and as long as we have good quality P-frames, but if a serie of P-frames is missing/corrupted it does not give great results anymore... Also seen in frame 176: some MB have somehow a wrong type (coded as stand-alone MB, when they are clearly difference MB...) and other have really low values (very dark) for no reason that I understand.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/17/2014 04:15 PM
Interpolated descent between frames 150 and 168.

Many thanks to SwissCheese for demonstrating how to apply p-frames.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/17/2014 04:20 PM
Video time  8)

I've used the new multiframe command on fixed_edit8.ts using all known mmb's for the I-frames that work on edit8 (from the wiki). I then reencoded it back into a video.

I used this command:

Quote
./ffmpeg -r 44999/3003 -mmb `cat mmb.txt` -debug mb_pos_size -err_detect ignore_err  -i fixed_edit8.ts -c:v mpeg4 -q:v 3 -tag:v xvid spacex_langing_nsf_20140517.avi

The mmb.txt file is attached. As you can see it's simply all the mmb's concatenated with '=' between them and 'FRAMExxx:' in front of each.

[edit] Almost forgot: for some reason the framenumbers are offset by 8. I think there are some frames before the first I-frame that cause this.

If you just want all the pictures you can do this:

Quote
./ffmpeg -r 44999/3003 -mmb `cat mmb.txt` -debug mb_pos_size -err_detect ignore_err  -i fixed_edit8.ts -f image2 img-%03d.png

The video (also attached) already looks very cool :) At some point you can actually see it flying!

https://www.youtube.com/watch?v=PgY2NZwRxBY

Of course the P-frames disturb the video heavily. So there is still a lot of work to do.

Enjoy!

arnezami

@SwissCheese: I've tried to add the P-frame mmb's you provided. But somehow this doesn't work: the changed P-frames and the P-frames after that stopped working. We'll have to figure out why. I thought that should have worked.

[edit] Added youtube video.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/17/2014 04:25 PM
Video time  8)

I've used the new multiframe command on fixed_edit8.ts using all known mmb's for the I-frames that work on edit8 (from the wiki). I then reencoded it back into a video.

I used this command:

Quote
./spx/ffmpeg-spacex-spacexdebug1/ffmpeg -r 44999/3003 -mmb `cat mmb.txt` -debug mb_pos_size -err_detect ignore_err  -i fixed_edit8.ts -c:v mpeg4 -q:v 3 -tag:v xvid spacex_langing_nsf_20140517.avi

The mmb.txt file is attached. As you can see it's simply all the mmb's concatenated with =FRAMExxx: between them and just FRAMExxx: at the beginning.

[edit] Almost forgot: for some reason the framenumbers are offset by 8. I think there are some frames before the first I-frame that cause this.

If you just want all the pictures you can do this:

Quote
./spx/ffmpeg-spacex-spacexdebug1/ffmpeg -r 44999/3003 -mmb `cat mmb.txt` -debug mb_pos_size -err_detect ignore_err  -i fixed_edit8.ts -f image2 img-%03d.png

The video (also attached) already looks very cool :) At some point you can actually see it flying!

Of course the P-frames disturb the video heavily. So there is still a lot of work to do.

Enjoy!

arnezami

@SwissCheese: I've tried to add the P-frame mmb's you provided. But somehow this doesn't work: the changed P-frames and the P-frames after that stopped working. We'll have to figure out why. I thought that should have worked.

That things looks amazing!!!!!!!!!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/17/2014 04:29 PM
I don't know if it's a good idea to go this way, but I was stuck trying to repair P-frames: as soon as I touched them (tried to correct them with matlab), they would not be used during the video encoding (even when simply reading and rewriting a frame, without changing anything).

Are there some headers and metadata around the pframes that might be able to be repaired in a similar way as the transport stream and iframe headers were? I can't seem to find a good explanation so far as to how pframes are encoded, though. If there's some reference points like timestamp values and what have you, that might help resuscitate some of them.

Edit: NICE jobs on that video and animation, you two! I think that explains why the orange color in iframe 209 wants to get pulled down farther than you'd expect - it's reflecting off the rocket body.

Edit: By the way, at the very beginning of the video are we perhaps seeing some evidence that the first couple of iframes were almost completely fogged in? Until we get up to iframe 32 which SwissCheese combed out, there's a huge amount of grey MBs in there, and then just a hint of orange as the engine glow passes through the fog. What if we hardcoded the first iframe or two to just a solid grey foggy expanse and see what happens as the pframes come through?

Would the triangles at the lower corners be visible in dense fog, also? Seems like they would be, but that would depend on how the housing is designed.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/17/2014 04:38 PM
Hi guys,

I've added a youtube video link to my previous post (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1199895#msg1199895).

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Sohl on 05/17/2014 04:46 PM
Video time  8)

That is fantastic!

But I was wondering, is there a relatively simple way to coerce the hue/color values in the p-frames to something in the blue-to-gray/white range before re-encoding the video?  Well, and maybe some yellow/orange near the frame center when the plume is visible.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Asmegin on 05/17/2014 04:52 PM
Amazing video! Nice to see all the hard work starting to come together.

Will the work on the P-Frames be more automatic or manual similar to the I-Frames? I'm sure we have enough people here with a bit of knowledge to help work on them if lots of manual labour is required. Otherwise it's all up to the few smart guys here :D

The next daytime landing won't be for many months, so there is lots of time to produce a good product. The upcoming Orbcomm launch will likely be in the early morning darkness, and the following two Asiasat launches won't have legs. Which means we have to wait until CRS-4, which I believe is scheduled for early August.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: adrianwyard on 05/17/2014 05:23 PM
Apologies if that has come up before (this is a big thread!) but has any consideration been given to post-processing the final 'best' rendering of the actual data?

It's a good bet that others have tried to recover highly corrupted MPEG streams before so there may be filters and techniques already developed out there beyond NSF. But some ideas:

+ Restrict the color palette to those we know are real. For those crazy colored larger macroblocks, substitute a color by averaging that of the neighbors. A really simple version of this would be to just forgo color and go black-and-white.

+ Detect hard 90 degree edges (of macroblocks) and blur those away; we know those edges didn't exist in the camera source image.

+ At some point you need to determine if there's any useful engineering data in this project. If so, then you'd be cautious about adding blur and generating colors, if if not, then post-processing choices would be geared to producing the most pleasing perceived video. You may find that adding generated 'snow' produces subjectively 'better' video because we're more used to seeing signals degraded this way than bright-green macroblocks.

Just my $0.02. Thanks everyone for your hard work; it may be just a footnote, but this landing is going in the history books.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: meadows.st on 05/17/2014 05:24 PM
Amazing video! Nice to see all the hard work starting to come together.

Will the work on the P-Frames be more automatic or manual similar to the I-Frames? I'm sure we have enough people here with a bit of knowledge to help work on them if lots of manual labour is required. Otherwise it's all up to the few smart guys here :D

The next daytime landing won't be for many months, so there is lots of time to produce a good product. The upcoming Orbcomm launch will likely be in the early morning darkness, and the following two Asiasat launches won't have legs. Which means we have to wait until CRS-4, which I believe is scheduled for early August.

First, let me say that I echo the comment on the Amazing video and simulation and offer any brute-force labor that may be required for P-Frames.

re: "The next daytime landing", I first started to do a calculation for what the light might be like at between ~1000m and 2000m ASL where the landing burn should start on 27 May, 2014 and then I found SunCalc.net that seems to indicate that the landing area should be between "civil twilight and sunrise" between 03:00 and 03:30 EDT on 27 May, 2014.  If the launch actually happens, this light should make for some reasonable (but perhaps not great) footage.  I have attached a screenshot of the resulting map presented by the SunCalc simulation.

EDIT: Actually, I suspect the launch images will be spectacular in the early light if the weather is clear and the further East the landing is located, the more light there will be for the landing (so no pressure) ;).

@Comga gives more detailed (and contradictory) info here:
http://forum.nasaspaceflight.com/index.php?topic=33089.msg1199888#msg1199888

EDIT2: Sadly I think that SunCalc is giving the time in my local timezone (PDT) so unfortunately I (now) agree with @Comga that the flight will still be in darkness. :(  In my own defence, I am used to higher latitudes (49N) and it starts to get light at 4AM around here as we approach the summer solstice. (added updated image with time in EDT)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Llian Rhydderch on 05/17/2014 07:57 PM
With the "NSF community-edited video" getting further along, it is probably not too early to ask this question:

What will be the license attached to any video that comes of this community editing effort by NSFers?

It would be great if the finished product ends up with one of the acceptable Creative Commons licenses (https://en.wikipedia.org/wiki/Wikipedia:Copyrights) on it that would make the video okay for Wikimedia and all the Wikipedia Foundation projects.  Two of the acceptable licenses for use there include the Creative Commons Attribution-Sharealike 3.0 Unported License (CC-BY-SA) and the GNU Free Documentation License (GFDL). 

Given that this started out as a SpaceX raw file, with what license I don't know, what will the edited/cleaned final version license be?  ... or should it be?


Edited:  fixed a single-letter typo
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/17/2014 09:24 PM
Edit: On other news, I tried to apply your very long mmb code for frame 92 (the frame formally known as iframe # 5) to the newest version (the one obtained from edit5.ts) and unfortunately is not working anymore. AS you where the main author, maybe you will be able to figure out what has changed. I'm guessing a small adjustment should be sufficient to get that frame visible again! It could also be a problem on how the code was pasted in the forum: I believe this happened to Saliva_sweet
This is my best attempt so far:

0:0:550,14:0:-1,
25:10:17410,
17:11:33650,
17:12:37648,
25:12:39009,29:12:-1,32:12:-1,10:14:-1,
17:14:52765,18:14:-1,24:14:-1,21:15:-1,22:15:-1,
34:15:56061,24:16:-1,
27:16:61466,42:19:-1,
43:19:83153,43:20:-1,
1:21:90585,11:22:-1,
12:22:99201,26:22:-1,
17:23:105394,22:24:-1,
23:24:111581,0:27:-1,
16:27:124815,24:27:-1,
25:27:125621,28:28:-1,
3:29:140154

Hope that helps others to make it better again. I've tried to use as much as possible from the last (much better) attempt (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195949#msg1195949). This though is using edit8.

But somehow I feel (while more data is certainly added to this frame) something got lost too. Not sure. Particularly around the legs. But I'm not so good at this. Hope I'm wrong.

Please help.

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Asmegin on 05/17/2014 10:28 PM

This is my best attempt so far:

0:0:550,14:0:-1,
25:10:17410,
17:11:33650,
17:12:37648,
25:12:39009,29:12:-1,32:12:-1,10:14:-1,
17:14:52765,18:14:-1,24:14:-1,21:15:-1,22:15:-1,
34:15:56061,24:16:-1,
27:16:61466,42:19:-1,
43:19:83153,43:20:-1,
1:21:90585,11:22:-1,
12:22:99201,26:22:-1,
17:23:105394,22:24:-1,
23:24:111581,0:27:-1,
16:27:124815,24:27:-1,
25:27:125621,28:28:-1,
3:29:140154


Bit more of legs: 19:13:42998, 18:14:47626
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/17/2014 11:03 PM
Interpolated descent between frames 150 and 168.

Many thanks to SwissCheese for demonstrating how to apply p-frames.

It looks good! How did you do the interpolation? Did you use information from the P-frames? I was thinking about interpolating the moving vector fields for the non-usable P-frames, or somehow better using them, but I'm stuck...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/17/2014 11:12 PM

This is my best attempt so far:
Bit more of legs: 19:13:42998, 18:14:47626

This looks very close to the first attempt! If someone can tease out more of the ocean, and perhaps find dirt spot #3 again, we'd be in good shape.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Asmegin on 05/17/2014 11:15 PM

This is my best attempt so far:
Bit more of legs: 19:13:42998, 18:14:47626

This looks very close to the first attempt! If someone can tease out more of the ocean, and perhaps find dirt spot #3 again, we'd be in good shape.

Dirt spot: 26:10:31150. Kinda messes up the rest though.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/17/2014 11:19 PM

This is my best attempt so far:
Bit more of legs: 19:13:42998, 18:14:47626

This looks very close to the first attempt! If someone can tease out more of the ocean, and perhaps find dirt spot #3 again, we'd be in good shape.

Dirt spot: 26:10:31150

Love that dirt spot! They should put them in the camera for the next flight too.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/17/2014 11:25 PM
Better if they turn off interlacing and enable CRC instead. :D
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: somepitch on 05/17/2014 11:58 PM
Coming up on the 150,000 view mark for this thread/effort. Noting it as there's a small active team of people really working hard on this, but also a HUGE amount of people watching that work. Always nice to have a large appreciative audience, both inside and outside the space industry.

You'd be astonished how many views are from IP addresses in a place called Hawthorne, California! ;D

How many from Centennial, CO or Evry-Courcouronnes, France...?  ;D
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Asmegin on 05/18/2014 12:24 AM

This is my best attempt so far:

0:0:550,14:0:-1,
25:10:17410,
17:11:33650,
17:12:37648,
25:12:39009,29:12:-1,32:12:-1,10:14:-1,
17:14:52765,18:14:-1,24:14:-1,21:15:-1,22:15:-1,
34:15:56061,24:16:-1,
27:16:61466,42:19:-1,
43:19:83153,43:20:-1,
1:21:90585,11:22:-1,
12:22:99201,26:22:-1,
17:23:105394,22:24:-1,
23:24:111581,0:27:-1,
16:27:124815,24:27:-1,
25:27:125621,28:28:-1,
3:29:140154


Bit more of legs: 19:13:42998, 18:14:47626
Dirt: 26:10:31150

I found what appears to be more of the left leg @ 17:13:42121 and 19:13:42516, but they only kindaaa fit and really mess up everything else. Maybe someone else can find the correct ones??
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jakusb on 05/18/2014 12:30 AM
A repost for convenience. :)
For those new:
The image has annotated most of the common dirt groups that reoccur in all frames. It helps align the macroblocks. :)(http://img.tapatalk.com/d/14/05/18/emugerav.jpg)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/18/2014 02:19 AM
Here's the collected mmb's for frame 92 from the above posts:

0:0:550,14:0:-1,25:10:17410,26:10:31150,
17:11:33650,17:12:37648,25:12:39009,
29:12:-1,32:12:-1,17:13:42121,18:13:42526,
10:14:-1,17:14:52765,18:14:-1,24:14:-1,
21:15:-1,22:15:-1,34:15:56061,24:16:-1,
27:16:61466,42:19:-1,43:19:83153,43:20:-1,
1:21:90585,11:22:-1,12:22:99201,26:22:-1,
17:23:105394,22:24:-1,23:24:111581,0:27:-1,
16:27:124815,24:27:-1,25:27:125621,28:28:-1,
3:29:140154
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Untribium on 05/18/2014 03:00 AM
Some progress on the top half of the frame and the left leg. Also removed some duplicate -1s
0:0:550,14:0:-2:0:0:0:0:12:-6,7:5:14749,12:5:15312,39:7:-1,18:9:27558,
25:10:17410,26:10:31150,17:11:33650,17:12:37648,25:12:39009,29:12:-1,
32:12:-1,17:13:42121,18:13:42526,10:14:-1,18:14:47626,26:14:-1,34:15:56061,
24:16:-1,27:16:61466,42:19:-1,43:19:83153,43:20:-1,1:21:90585,11:22:-1,
12:22:99201,26:22:-1,17:23:105394,22:24:-1,23:24:111581,0:27:-1,
16:27:124815,24:27:-1,25:27:125621,28:28:-1,3:29:140154
I think 23:14 needs some bits flipped...the part of the right leg is one block (half a macro block) off, which is pretty bad...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Asmegin on 05/18/2014 03:27 AM
Some progress on the top half of the frame and the left leg. Also removed some duplicate -1s
0:0:550,14:0:-2:0:0:0:0:12:-6,7:5:14749,12:5:15312,39:7:-1,18:9:27558,
25:10:17410,26:10:31150,17:11:33650,17:12:37648,25:12:39009,29:12:-1,
32:12:-1,17:13:42121,18:13:42526,10:14:-1,18:14:47626,26:14:-1,34:15:56061,
24:16:-1,27:16:61466,42:19:-1,43:19:83153,43:20:-1,1:21:90585,11:22:-1,
12:22:99201,26:22:-1,17:23:105394,22:24:-1,23:24:111581,0:27:-1,
16:27:124815,24:27:-1,25:27:125621,28:28:-1,3:29:140154
I think 23:14 needs some bits flipped...the part of the right leg is one block (half a macro block) off, which is pretty bad...

Nice job on the ocean! Before I had to step out I got a bit at the top but I didn't save it! Ahh, will keep trying. Also gonna try and work on the base of the legs. Dunno about that block on the right leg, I'm not knowledgeable enough to figure out what's happening there  ;D
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Untribium on 05/18/2014 03:42 AM
Some progress on the top half of the frame and the left leg. Also removed some duplicate -1s
-snip-
I think 23:14 needs some bits flipped...the part of the right leg is one block (half a macro block) off, which is pretty bad...

Nice job on the ocean! Before I had to step out I got a bit at the top but I didn't save it! Ahh, will keep trying. Also gonna try and work on the base of the legs. Dunno about that block on the right leg, I'm not knowledgeable enough to figure out what's happening there  ;D

Happened to me with frame 52, closed the tab by accident :'( :) The ocean seems to be pretty much intact, it's just really time consuming to find the address of the next block after a bad one...brute force ftw! :D
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Asmegin on 05/18/2014 04:04 AM
Frame 92:

23:14:49203 gives you a piece of the circle between the legs. Only the left blocks are correct though... Doesn't help the right side of the right leg either :p

Also there is something at 54419. I thought it was the edge of the rocket base but it doesn't fit anywhere. Might be a wave...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/18/2014 04:35 AM
Interpolated descent between frames 150 and 168.

Many thanks to SwissCheese for demonstrating how to apply p-frames.

It looks good! How did you do the interpolation? Did you use information from the P-frames? I was thinking about interpolating the moving vector fields for the non-usable P-frames, or somehow better using them, but I'm stuck...

I used the ocean blobs from the i-frames at each end and p-frames 153, 159, and 166 as control points, then I interpolated each transformation variable with a spline. I scripted Hugin to do the transformations. P-frame 159 was applied directly, except for a bit of shifting and masking to get the timestamp in the right place. P-frames 151 and 154 were also applied but only half of the ocean for each was useable. Basically I took the previous frame, pretended it was an i-frame, created a movie with just that and one p-frame, and then broke it up into images again.

Bits and pieces of other p-frames were pasted in, but I did not attempt to apply the mpg4-img in those cases.

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Asmegin on 05/18/2014 04:52 AM
Frame 229

0:0:-1,4:13:63400,20:15:74510,2:16:77453
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/18/2014 04:53 AM
Hi!

You guys are amazing! I really thought the legs were a goner on that frame. But oooh no! :)

Here is my version of 229 with edit8. It was still missing from the video.

0:0:-1,
9:12:59273:25:1:1:1:1:1,
0:21:99181,0:25:-1,
1:25:113405,7:29:-1

It needs some love though. ;)

I am going to make a new video again with these new frames.

Regards,

arnezami

PS. The 1:1:1:1:1 is in these because the online editor removes the 0:0:0:0:0. Maybe somebody can take a look at that (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1199765#msg1199765)...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Asmegin on 05/18/2014 04:58 AM
Looks like we had the same idea. Yours looks a bit cleaner though :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/18/2014 06:01 AM
Hmmm. I've got some issues. I don't understand whats causing them. Looks like ffmpeg is not cooperating...

For some reason when I use this command frames 189, 229 and 268 are missing. (these are frames 197, 237 and 276 according to ffmpeg btw).

Quote
./ffmpeg -r 44999/3003 -mmb `cat mmb.txt` -debug mb_pos_size -err_detect ignore_err  -i fixed_edit8.ts -f image2 img-%03d.png 2> debug.log

See attached mmb.txt (you will have to remove the newlines before using it).

If you look at the pictures it generated (careful the frame number in the filename is 1 higher then the frame number according to ffmpeg) you see all the i-frames except the above mentioned.

When I run the same command with the three i-frame commands removed then 189 appears again (in its original form) while 229 and 268 are still missing.

Why is ffmpeg not adding these frames to the output?? I can't see anything in the log either. Can we force it somehow? But I'm no expert at using ffmpeg. Hopefully somebody else can help here.

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/18/2014 09:46 AM
Hmmm. I've got some issues. I don't understand whats causing them. Looks like ffmpeg is not cooperating...

For some reason when I use this command frames 189, 229 and 268 are missing. (these are frames 197, 237 and 276 according to ffmpeg btw).

Quote
./ffmpeg -r 44999/3003 -mmb `cat mmb.txt` -debug mb_pos_size -err_detect ignore_err  -i fixed_edit8.ts -f image2 img-%03d.png 2> debug.log

See attached mmb.txt (you will have to remove the newlines before using it).

If you look at the pictures it generated (careful the frame number in the filename is 1 higher then the frame number according to ffmpeg) you see all the i-frames except the above mentioned.

When I run the same command with the three i-frame commands removed then 189 appears again (in its or