NASASpaceFlight.com Forum

SpaceX Vehicles and Missions => SpaceX Reusability => Topic started by: Chris Bergin on 04/30/2014 07:43 pm

Title: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 04/30/2014 07:43 pm
Certainly needs cleaning up! Video experts around the world requested to help SpaceX with this.

http://www.youtube.com/watch?v=7m8H8OlJ3o8

Best repair so far:
https://www.youtube.com/watch?v=er66BActC4E

Background:

CRS-3 Falcon 9 first stage to sport legs and attempt soft splashdown
https://www.nasaspaceflight.com/2014/02/crs-3-falcon-9-first-stage-sport-legs-attempt-soft-splashdown/

SpaceX outlines CRS-3 landing legs plan toward first stage recovery ambitions
https://www.nasaspaceflight.com/2014/02/spacex-crs-3-landing-legs-plan-first-stage-recovery-ambitions/

SpaceX Falcon 9 successfully launches CRS-3 Dragon
https://www.nasaspaceflight.com/2014/04/spacex-crs-3-dragon-new-milestones/

Rockets that return home – SpaceX pushing the boundaries
https://www.nasaspaceflight.com/2014/04/rockets-return-home-spacex-pushing-boundaries/

--

UPDATE:

The video updates, in order, are available on this channel:
https://www.youtube.com/channel/UCyZDgyJBYz3OXD3JbDJzNww

Here's the post-competition technical feature article - by the team's Lourens Veen!

https://www.nasaspaceflight.com/2014/06/recovering-falcon-9-ocean-landing-video-done/


UPDATE:

Five year anniversary:
https://twitter.com/NASASpaceflight/status/1123300237647519744
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 04/30/2014 07:53 pm
Guy on twitter (not sure if he's on here, but in response to me on twitter....)

Ryan Ziolko ‏@MediumFidelity  6m
@nasaspaceflight It's going to be tricky, the file contains large sections like this: pic.twitter.com/E5Csw1sjXy
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Lars_J on 04/30/2014 07:53 pm
The original data files are available for download from SpaceX here:
http://www.spacex.com/news/2014/04/29/first-stage-landing-video

 
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: StephenB on 04/30/2014 07:59 pm
If the data is just not there (like those ffff's), then no recovery is possible, right?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jarnis on 04/30/2014 08:04 pm
If the data is just not there (like those ffff's), then no recovery is possible, right?

Well, some data could be transferred from known good images from the same camera as far as the fixed part of the image (rocket body) goes. Beyond that, it would require some imagination and frame-by-frame touching up. It would be partially fiction at that point...

If there only were some additional fully intact frames, a bunch of damaged ones between them could be interpolated but that just isn't there I think.

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: swlee on 04/30/2014 08:07 pm
On the SpaceX web page, they preface the videos with:
"Below is a look at the original video data from the first stage landing, recovered from the Falcon 9 onboard camera and shot right before splashdown".

Does this imply that the camera itself was recovered (and, by extension, at least some part of the first stage)?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Lars_J on 04/30/2014 08:10 pm
No, it is the *data* that has been recovered. (although they did find some pieces)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: billh on 04/30/2014 08:17 pm
No, it is the *data* that has been recovered. (although they did find some pieces)

Looks like they only found "some pieces" of the data, too.  :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: adrianwyard on 04/30/2014 08:18 pm
It's interesting to me that this demonstrates the value of using analog rather than digital signal transmission in some circumstances. Even an extremely noisy analog signal would have been easier to reconstruct than this one, with its macroblocking/i-frame image-segment tracking weirdness. It can be done, but would be easiest if you had access to the exact compression code used on the camera. If SpaceX are smart they'll release that info.

Before now I've been disappointed to see the Wireless Video System used for EVAs had quite a lot of static and dropouts (given that we live in an HD digital world) but in retrospect perhaps that is the best signal to send.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Lee Jay on 04/30/2014 08:21 pm
My recommendation for cleaning this up is to repeat the test.  :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: MTom on 04/30/2014 08:21 pm
If the data is just not there (like those ffff's), then no recovery is possible, right?

Well, some data could be transferred from known good images from the same camera as far as the fixed part of the image (rocket body) goes. Beyond that, it would require some imagination and frame-by-frame touching up. It would be partially fiction at that point...

If there only were some additional fully intact frames, a bunch of damaged ones between them could be interpolated but that just isn't there I think.

The top of many frames are more or less intact. Maybe they could be a good starting point.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: meekGee on 04/30/2014 08:32 pm
Which means, IMO, they should rotate the image 90 degrees between successive key frames, as far as the compression is concerned.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Lars_J on 04/30/2014 08:34 pm
Which means, IMO, they should rotate the image 90 degrees between successive key frames, as far as the compression is concerned.


So... Basically use an completely new codec?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 04/30/2014 08:44 pm
Hi guys,

Today I have done a low-level analysis of the raw.ts file. Most notably the "Transport Stream Headers" (see http://en.wikipedia.org/wiki/MPEG_transport_stream )

While it is bad, its not as bad as is being suggested above. The raw file consists of 23,838 ts-packets (containing 188 bytes each). Each one of these has a 4-byte header. These should always begin with the hexadecimal value "47". Of these 23,838 headers the following are correct:

4703e8: 15,014 times
471fff: 4,966 times
4743e8: 263 times
474000: 81 times
474020: 74 times

The actual video-packets have the code "4703e8" and as you can see there are quite a lot of them! And they contain a LOT of data.

The code "471fff" identify the so called null-packets and these contain a lot of FF's. But this is totally normal. Since they are "used for fixed bandwidth padding".

The other correct packets are basicly overhead. I will not go into them. But they are essential for the video working. There are a lot more than shown above btw.

The problem lies in the remaining bad headers (still several thousand): they get ignored by the video-player which causes it to act strangely. Also: a lot of bits (not just in the header but also in the content) are flipped. But my hope is that this is not the biggest issue: mpeg-4 is quite resilient.

I'm trying to figure out if I can "repair" most of the headers so the video player can at least decide what to do with the data in it (now it will just ignore it because they have the wrong PID, invalidating valid packets in the meantime).

This however is no simple task. :P

Fingers crossed.

Regards,

arnezami

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 04/30/2014 09:13 pm
Good luck Arnezami! If you have any success, also PM me as I'll hook you up with the SpaceX people (the e-mail address they provide is just an "inbox pile" address, I can get you direct).
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: meadows.st on 04/30/2014 09:30 pm
Hi guys,

Today I have done a low-level analysis of the raw.ts file. Most notably the "Transport Stream Headers" (see http://en.wikipedia.org/wiki/MPEG_transport_stream )

While it is bad, its not as bad as is being suggested above. The raw file consists of 23,838 ts-packets (containing 188 bytes each). Each one of these has a 4-byte header. These should always begin with the hexadecimal value "47". Of these 23,838 headers the following are correct:

4703e8: 15,014 times
471fff: 4,966 times
4743e8: 263 times
474000: 81 times
474020: 74 times

The actual video-packets have the code "4703e8" and as you can see there are quite a lot of them! And they contain a LOT of data.

The code "471fff" identify the so called null-packets and these contain a lot of FF's. But this is totally normal. Since they are "used for fixed bandwidth padding".

The other correct packets are basicly overhead. I will not go into them. But they are essential for the video working. There are a lot more than shown above btw.

The problem lies in the remaining bad headers (still several thousand): they get ignored by the video-player which causes it to act strangely. Also: a lot of bits (not just in the header but also in the content) are flipped. But my hope is that this is not the biggest issue: mpeg-4 is quite resilient.

I'm trying to figure out if I can "repair" most of the headers so the video player can at least decide what to do with the data in it (now it will just ignore it because they have the wrong PID, invalidating valid packets in the meantime).

This however is no simple task. :P

Fingers crossed.

Regards,

arnezami

Great work @arnezami. If there is a way to "crowdsource" the crowdsource (e.g. by breaking the effort into small brute force chunks that the rest of us can do and that you could reassemble), I volunteer.  I don't have the expertise to do the manual parsing but am happy to do some scripting or even manual editing if instructed on what needs to be done.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: MP99 on 04/30/2014 09:40 pm
While it is bad, its not as bad as is being suggested above. The raw file consists of 23,838 ts-packets (containing 188 bytes each). Each one of these has a 4-byte header. These should always begin with the hexadecimal value "47". Of these 23,838 headers the following are correct:

4703e8: 15,014 times
471fff: 4,966 times
4743e8: 263 times
474000: 81 times
474020: 74 times

The actual video-packets have the code "4703e8" and as you can see there are quite a lot of them! And they contain a LOT of data.

The code "471fff" identify the so called null-packets and these contain a lot of FF's. But this is totally normal. Since they are "used for fixed bandwidth padding".

It seems a real shame that they didn't use the ~25% of unwanted bandwidth for parity / CRC / redundancy. Ah well, roll on flight #10.

Anyway - best of luck with this.

Cheers, Martin
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: QuantumG on 04/30/2014 10:10 pm
So.... I did a little digging on that video stream.

Everything I've seen online so far, including what SpaceX put out, just looks like they used ffmpeg to pull out the frames.. which is what I did too. However, I also did ffmpeg -i raw.ts raw.mpg and then compared the bytes in the mpg to the ts.

It skips the first 847k of the ts, but after that it hums along. Mostly the data comes across in 184 byte chunks, with an occasional truncated or dropped chunk. There's some really big gaps though.

If you know how to extract I frames manually from a stream, you could probably easily pull at least one out of those big gaps.

The top 5 are:

8864 bytes at 0x0011d470
11120 bytes at 0x00170664
15952 bytes at 0x000d6260
23716 bytes at 0x000df508
23904 bytes at 0x001502dc

I wrote a little code to attempt it but I only saw one extra I frame and couldn't get it to convert to an image. Now I'm out of time.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IRobot on 04/30/2014 10:37 pm
The first thing I would do to improve video quality on future flights would be to encode a monochrome image. Color is useless for this type of image.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SpaceX_MS on 04/30/2014 10:45 pm
Great thread. If one of you guys can help, it will be worth your time and effort. This is an Elon call to arms.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Lar on 04/30/2014 11:04 pm
Great thread. If one of you guys can help, it will be worth your time and effort. This is an Elon call to arms.

Can you get the specs on the camera used including the software load (and it sounds like, especially the codec) ??
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: docmordrid on 05/01/2014 01:38 am
Yes! We need confing and codec specs like GOP length and other mpeg-(4?) parameters.

And I agree: using monochromatic analog video would solve a LOT of problems where signal strength and  interference is an issue. Newer isn't always better.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: ugordan on 05/01/2014 01:52 am
The first thing I would do to improve video quality on future flights would be to encode a monochrome image. Color is useless for this type of image.

MPEG already operates on luminance/chrominance components and most of the bits allocated are for luminance (i.e. monochrome) component. I'm not sure what you're suggesting, it wouldn't do much for improving video quality and color actually helps a lot when you have objects with similar brightness in the scene.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: neilh on 05/01/2014 02:34 am
A "SpaceX team member" (same account they used for an AMA last year) posted the following comment

http://www.reddit.com/r/spacex/comments/2bsn2/first_stage_landing_video/ch6f8io
Quote
[–]spacexdevtty [+1] 8 points 2 hours ago (12|4)
Hey all, SpaceX team here. Just wanted to answer some frequently asked questions we've been getting:
Q: Why is the video so bad? A: This was recorded over a very lossy RF link.
Q: Why release the video? A: We've had some success here (and with a little outside help as well, see http://aeroquartet.com/[1] ) manually stitching the video back together, but it's time consuming work. We don't expect the video to be 100% recoverable but we're hoping folks out there with more MPEG expertise than we have can provide assistance recovering more of the video.
Q: What codec? Settings? A: MPEG 4 Part 2, P/I 15, 15fps, NTSC, fixed bitrate
Q: How did you create repair1.ts? A: This was a joint effort between us and an outside consultant. First we took a pass on the data to align every MPEG packet on a 188 byte boundary and set the packet start byte to 0x47. Then we identified blocks within keyframes that contain bit errors, and then manually flipping bits in those corrupt blocks to see if it recovers more of the image.
Q: What is a TS file? A: http://lmgtfy.com/?q=MPEG+TS[2]
Feel free to post more questions here, we'll try and respond. We really appreciate all the help and great ideas! Thank you!!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IRobot on 05/01/2014 07:53 am
The first thing I would do to improve video quality on future flights would be to encode a monochrome image. Color is useless for this type of image.

MPEG already operates on luminance/chrominance components and most of the bits allocated are for luminance (i.e. monochrome) component. I'm not sure what you're suggesting, it wouldn't do much for improving video quality and color actually helps a lot when you have objects with similar brightness in the scene.
I think we would all be happy with objects with similar brightness, if we could see them. I wouldn't mind if the whole rocket looked of the same brightness as long as I could see the flame, legs and ocean.

So color is useless here.

Also slight color noise introduces a lot of information between frames.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Adaptation on 05/01/2014 08:11 am
Quote
First we took a pass on the data to align every MPEG packet on a 188 byte boundary and set the packet start byte to 0x47.

I'm not seeing that.  For instance location 26EC is divisible by 188 (0XBC) but its value is 4F not 47.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: docmordrid on 05/01/2014 08:13 am
Color video, via chroma subsampling, also reduces apparent resolution. An example is NTSC where colors are spread across 2 horizontal lumance pixels. You end up with ˝ the horizontal resolution and full vertical resolution.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/01/2014 09:27 am
Quote
First we took a pass on the data to align every MPEG packet on a 188 byte boundary and set the packet start byte to 0x47.

I'm not seeing that.  For instance location 26EC is divisible by 188 (0XBC) but its value is 4F not 47.
I can concur. In the repair1.ts file the sync bytes have not been "fixed" to 47 (hex). Maybe they uploaded the wrong file? Not that this does much: the rest of the header is usually broken as well. Also, in the raw.ts there are 5 places where I had to add exactly 56 bytes in order for the headers to align on the 188 bytes grid.

Anyway. Back to trying to get a little more life out of this video.  ;)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: ugordan on 05/01/2014 11:24 am
I think we would all be happy with objects with similar brightness, if we could see them. I wouldn't mind if the whole rocket looked of the same brightness as long as I could see the flame, legs and ocean.

So color is useless here.

You're not getting me. When everything is the same brightness, you have no idea what you're looking at, where one object ends and the next one begins. This is where color is important, to distinguish between different materials.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: ugordan on 05/01/2014 11:26 am
Color video, via chroma subsampling, also reduces apparent resolution. An example is NTSC where colors are spread across 2 horizontal lumance pixels. You end up with ˝ the horizontal resolution and full vertical resolution.

Chroma subsampling is precisely why I said luminance takes the most bits for encoding, because the 2 chroma channels are reduced in resolution by a factor of 4 or more. Luminance is not subsampled, unless you tell the encoder to do that for some reason.

The upshot of all this is that having color as additional information in the stream really doesn't have as big an impact as some would like to believe. It's not like it's 3 full red/green/blue frames encoded and 3x the bandwidth of a single monochrome image.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: input~2 on 05/01/2014 12:24 pm
This frame at 14 sec. seems interesting
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: AJA on 05/01/2014 12:51 pm
Ok..let me throw this out there: Would having the telemetry bitstream as opposed to only the video bitstream help? I'm thinking on the lines of whether additional error correction coding etc. would introduce errors in the TSH, or in the error correction code of the videostream. Somewhat like a frame-shift mutation, or an insertion/deletion mutation.


Color video, via chroma subsampling, also reduces apparent resolution. An example is NTSC where colors are spread across 2 horizontal lumance pixels. You end up with ˝ the horizontal resolution and full vertical resolution.

Chroma subsampling is precisely why I said luminance takes the most bits for encoding, because the 2 chroma channels are reduced in resolution by a factor of 4 or more. Luminance is not subsampled, unless you tell the encoder to do that for some reason.

The upshot of all this is that having color as additional information in the stream really doesn't have as big an impact as some would like to believe. It's not like it's 3 full red/green/blue frames encoded and 3x the bandwidth of a single monochrome image.


How about a monochrome image through a red-filter? Reduced data (although not a 3x reduction as you point out), without reduction in discerning boundaries. After all, differential luminance in one channel is what contributes to discernibility in the composite. The ocean would be dark, while the surf kicked up (being white) would still give you a sense of "sea level". You'd probably be able to write some compression code that wouldn't bother transmitting pixels below a certain black value anyway. Plus, we know the colour of the ocean, and the colour of Falcon's legs to a good degree... so as long as you took the channel with the highest variance, you'd be able to reconstruct a fairly decent "false colour" image..?


I still think they should put a hundred memory cards inside small, sealed, brightly painted empty plastic boxes (only a data cable coming out), and stick it with strong, but water soluble adhesive to the inner wall of the first stage-second stage interface. Then offer a monetary reward to the local Florida fishing community for whoever returns most boxes :P


Or simply have a drone fly much closer to the stage than would be allowed for manned airborne assets.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Sohl on 05/01/2014 12:58 pm
There's a lot of interesting ideas presented here to get better video data, but the next scheduled flight is probably too soon to make any changes on the vehicle side.  Perhaps they can do better with aircraft or ship assets to be in a better position for signal acquisition (especially if weather cooperates).  Later on, it should not matter as much as they get close to and hopefully fully succeeding at a land landing.

But let's hope for calm seas on this next flight!   ;)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mmeijeri on 05/01/2014 12:59 pm
I still think they should put a hundred memory cards inside small, sealed, brightly painted empty plastic boxes (only a data cable coming out), and stick it with strong, but water soluble adhesive to the inner wall of the first stage-second stage interface. Then offer a monetary reward to the local Florida fishing community for whoever returns most boxes :P

They had what they called "Talon pods" on earlier flights where they tried to recover v1.0 first stages. I haven't heard anything about that this time.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: ugordan on 05/01/2014 01:42 pm
How about a monochrome image through a red-filter? Reduced data (although not a 3x reduction as you point out), without reduction in discerning boundaries.

But why bother? Assuming 4x4 subsampled chrominance channels, color video only has 12.5% more bits to encode than the same video where only luminance is encoded. Seeing as how bad the transmission losses were, that wouldn't have made any difference whatsoever.

Again, why go around making up solutions that would only be relevant for a couple of seconds before splashdown? The end goal is recovering the stage, not getting better video of a landing stage that ends up being lost anyway. A better approach is to just record the stream onboard for later playback and even that would not be needed for land boostback where the stage doesn't go over the horizon to primary range tracking.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IRobot on 05/01/2014 02:18 pm
How about a monochrome image through a red-filter? Reduced data (although not a 3x reduction as you point out), without reduction in discerning boundaries.

But why bother? Assuming 4x4 subsampled chrominance channels, color video only has 12.5% more bits to encode than the same video where only luminance is encoded. Seeing as how bad the transmission losses were, that wouldn't have made any difference whatsoever.
@AJA why red filter? Maybe a luminance filter with IR cut, but can't understand the reason to use the red filter.

12.5% is actually a lot. When receiving such error-prone signal, it makes a lot of difference. It also reduces transmission power requirements (for the same frame rate), therefore more power available for transmission, therefore better S/N ratio.

Also true monochrome cameras are up to 3x more sensitive, meaning less (camera) noise to start with.

Again, why go around making up solutions that would only be relevant for a couple of seconds before splashdown?
Because the way up is quite well documented. The way down is not.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: ugordan on 05/01/2014 02:29 pm
12.5% is actually a lot. When receiving such error-prone signal, it makes a lot of difference. It also reduces transmission power requirements (for the same frame rate), therefore more power available for transmission, therefore better S/N ratio.

Maybe. *If* that's the only telemetry sent. Who's to say the video wasn't multiplexed along all the other, high rate vehicle telemetry so the 12.5% for video is more of a noise in the total bandwidth budget?

Also true monochrome cameras are up to 3x more sensitive, meaning less (camera) noise to start with.

Seriously? At the codec quality settings they're using, the camera dirt that's deposited on the way down you're worried about camera noise?

Again, why go around making up solutions that would only be relevant for a couple of seconds before splashdown?
Because the way up is quite well documented. The way down is not.

Oh, I'm sure they have it quite well-documented. Just not in a format your typical rocket pr0n enthusiast likes. It's in the form of vehicle telemetry. That's gold, anything else is just gravy.

It still doesn't change my argument that any such solutions are just too much trouble for the amount of use they'll have eventually.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: rst on 05/01/2014 02:36 pm
Just a note:  for those who may not have noticed, there's a parallel discussion of the video and what may or may not be visible on particular frames on the CRS-3/SpX-3 discussion thread in the missions section:

http://forum.nasaspaceflight.com/index.php?topic=31513.1650
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: hrissan on 05/01/2014 02:41 pm
12.5% is actually a lot. When receiving such error-prone signal, it makes a lot of difference. It also reduces transmission power requirements (for the same frame rate), therefore more power available for transmission, therefore better S/N ratio.

Maybe. *If* that's the only telemetry sent. Who's to say the video wasn't multiplexed along all the other, high rate vehicle telemetry so the 12.5% for video is more of a noise in the total bandwidth budget?

Also true monochrome cameras are up to 3x more sensitive, meaning less (camera) noise to start with.

Seriously? At the codec quality settings they're using, the camera dirt that's deposited on the way down you're worried about camera noise?

Again, why go around making up solutions that would only be relevant for a couple of seconds before splashdown?
Because the way up is quite well documented. The way down is not.

Oh, I'm sure they have it quite well-documented. Just not in a format your typical rocket pr0n enthusiast likes. It's in the form of vehicle telemetry. That's gold, anything else is just gravy.

It still doesn't change my argument that any such solutions are just too much trouble for the amount of use they'll have eventually.
Worked on sort of "telemetry", we divided data into what's important (10%, in this case sensors data) and what's less important (90%, in this case video feed). Both were CRC-ed, but the first got retransmitted if corrupted or lost, the second was not. Sort of TCP and UDP.

On worsening channel the retransmissions of sensor data occupied more and more bits until no bits were available to video.

I guess SpaceX does the same, so if we see SOME video, it means ALL sensor data was received without gaps.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IRobot on 05/01/2014 02:47 pm
Oh, I'm sure they have it quite well-documented. Just not in a format your typical rocket pr0n enthusiast likes. It's in the form of vehicle telemetry. That's gold, anything else is just gravy.
If you want to go that way, we don't need a video. And yet we have one.

It still doesn't change my argument that any such solutions are just too much trouble for the amount of use they'll have eventually.
What trouble? Switch the camera for a mono version? Or reconfigure the camera to transmit in gray scale? You make it sound like mono cameras are troublesome, but in fact they are EXACTLY the same! Usually manufacturers supply the same camera in mono and color version.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: ugordan on 05/01/2014 03:00 pm
Oh, I'm sure they have it quite well-documented. Just not in a format your typical rocket pr0n enthusiast likes. It's in the form of vehicle telemetry. That's gold, anything else is just gravy.
If you want to go that way, we don't need a video. And yet we have one.

Exactly. We don't *need* a video. Having video and needing video are separate things.

It still doesn't change my argument that any such solutions are just too much trouble for the amount of use they'll have eventually.
What trouble? Switch the camera for a mono version? Or reconfigure the camera to transmit in gray scale? You make it sound like mono cameras are troublesome, but in fact they are EXACTLY the same! Usually manufacturers supply the same camera in mono and color version.

Did you read what I wrote about monochrome video bandwidth impacts and your point about camera noise? I'm not sure why you're thinking that switching to grayscale video would simply sort out all issues with transmission, which is ultimately the only issue we had here. It's not color. It's not camera noise. It's packet loss.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Lars_J on 05/01/2014 03:27 pm
Ugordan is right. Switching to monochrome would not save much bandwidth. (if any) Don't argue if you don't understand how chroma (color) is sub-sampled compared to luminosity (brightness).
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IRobot on 05/01/2014 04:05 pm
Ugordan is right. Switching to monochrome would not save much bandwidth. (if any) Don't argue if you don't understand how chroma (color) is sub-sampled compared to luminosity (brightness).
I do understand better than you think. I've designed scientific grade CCD cameras :)

And I work with Colorimeters and Spectrometers. Part of my day's work is dealing with YCbCr, XYZ, xyY CIELab, etc.

My point is that any useless information should be discarded to save bandwidth. 12.5% might be enough to save a frame.

EDIT: the used codec can have color subsampling of 4:2:2 or 4:4:4, meaning at best a 2:1 relation between luminance and chroma. So it is more likely that the color information is 33% of the video bandwidth, not considering compression effects of similarities between frames. Where did you get the 12.5%?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: R7 on 05/01/2014 04:52 pm
EDIT: the used codec can have color subsampling of 4:2:2 or 4:4:4, meaning at best a 2:1 relation between luminance and chroma. So it is more likely that the color information is 33% of the video bandwidth, not considering compression effects of similarities between frames. Where did you get the 12.5%?

Wouldn't 4:2:2 mean 1:1 ratio between luma and chroma bandwidths? (=8 luma samples and 4 Cb + 4 Cr samples)

The 12.5% is assuming (non-mpeg4 standard?) 4x4 subsampling, kind of "4:1:0:0:0".

Btw anyone know what kind of error correction methods the RF-link had? Some sort of Reed-Solomon encapsulation layer?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: ugordan on 05/01/2014 05:09 pm
I just checked the *.ts SpaceX posted, the format is 4:2:0 so that would bring the total number of bits to encode to 50% more than monochromatic video so I stand corrected. I must have been thinking of older codecs like Indeo that did allow as high as 4x4 subsampling of chroma.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: rickyramjet on 05/01/2014 06:02 pm
The problem is the noisy signal, not the codec.  The reason for the noisy signal is signal strength, pure and simple, worsened by the distance from the rocket and the bad weather.  The easiest solution is to install a more powerful transmitter.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Adaptation on 05/01/2014 06:04 pm
Quote
First we took a pass on the data to align every MPEG packet on a 188 byte boundary and set the packet start byte to 0x47.

I'm not seeing that.  For instance location 26EC is divisible by 188 (0XBC) but its value is 4F not 47.
I can concur. In the repair1.ts file the sync bytes have not been "fixed" to 47 (hex). Maybe they uploaded the wrong file? Not that this does much: the rest of the header is usually broken as well. Also, in the raw.ts there are 5 places where I had to add exactly 56 bytes in order for the headers to align on the 188 bytes grid.

Anyway. Back to trying to get a little more life out of this video.  ;)

Well some bits in the header should be able to be restored as well. 

Here is a prototype for the two header bytes.
1000 0111  (this is the G or 47)
0Y0Y YYYY
YYYY YYYY
00X1 XXXX

Where 1's and 0's are values that should be set regardless of contents of packets.  X's contain data that may be determined by analysing headers before and after this packet.  Y's contain data that could possibly be determined by analyzing data within the packet, knowing the identifiers associated with the codec and several invalid values could be excluded.

Without doing much analysis we can make some bitwise filters for the second and fourth bytes to fix five bits. 

0101 1111  to be and with the second byte (this sets the packet to not be ignored and not to give it special priority)
0011 1111  to be and with the fourth byte  (this declares the stream to be unencrypted)
0001 0000  to be ored with the fourth byte  (this sets packet contains payload to true, which is possibly too big of an assumption)

http://en.wikipedia.org/wiki/MPEG_transport_stream#Packet
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IRobot on 05/01/2014 06:11 pm
The problem is the noisy signal, not the codec.  The reason for the noisy signal is signal strength, pure and simple, worsened by the distance from the rocket and the bad weather.  The easiest solution is to install a more powerful transmitter.
More or less. If you use a codec that reduces the information to 50% (for example by not using color), and keeping a frame rate of 15fps, you can actually send the same frame twice (or send the same transmission packet twice). But as that would require a deep software change, the other option would be to double the number of frames, 30fps. Then differences between frames would be reduced to half, increasing compression ratio.

I'm no codec expert, so there are probably better solutions on how to use bandwidth to reduce transmission noise (data corruption). Still, a change in the codec is probably a lot easier than replacing the transmitter. A more powerful transmitter also uses more energy.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Lars_J on 05/01/2014 06:49 pm
Someone on Youtube did a decent effort of cleaning it up:

I'm not sure how accurate it is, and I don't think the legs were extended in the beginning of the clip? Nonetheless it looks neat.

https://www.youtube.com/watch?v=CnyrleS782g
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IRobot on 05/01/2014 06:56 pm
That guy got the rocket image from the best frame and overimposed it on all frames... it offers a visual cue, but that's all.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Adaptation on 05/01/2014 07:01 pm
the other option would be to double the number of frames, 30fps. Then differences between frames would be reduced to half, increasing compression ratio.

Doubling the framerate would do little to solve the problem on a modern codec.  You would need a full frame codec like MJPEG for that to really work.  You could reduce the threshold for sending a keyframe or have keyframes sent twice.  As they are using a fixed bit rate stream there may be room for some of these tricks. 

The best thing they could do is know better where the rocket will come down and have adequate downlink capability there. 

Higher transmit power is nice but it only gets you so far, double the power and you get 1/4 more range, you can only double the power so many times before the strategy gets out of hand.  But using for instance a 28 dbi directional antenna on the receiver gives you the same result as multiplying your transmit power 500 times.  The only problem is you have to be able to point it very accurately, if you're off by just 5° you are only going to get 250x the receive power but it steeply drops from there a few more degrees and its the same as sticking blinders up on your receiver. 

AFIK this launch did not aggressively attempt landing at a precise spot because more velocity was given to the dragon to assure the highest possible margins for mission success. 
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: AJA on 05/01/2014 07:02 pm
@AJA why red filter? Maybe a luminance filter with IR cut, but can't understand the reason to use the red filter.

Falcon's white. The legs too.. have a white border (scroll to find input~2's YT screenshot ITT). The ocean's blue. Now that means that the RGB luminances of Falcon are (say) r1, g1, b1, whereas the ocean's is ~0, ~g2(bodies of water do look greenish sometimes don't they? Plus, plankton?), b2. The difference |b2-b1|, is I would assume the smallest of the three pairs and doesn't really help in terms of establishing where the legs end and water begins (in the image data). |g2-g1| would probably be more than the blue channel differences, but the largest, by far would be |r2-0| (I'm assuming that the ocean's black in the red channel..or very close to it atleast). So this allows you to differentiate.

While it may seem useless, and like a really poor version of a grasshopper video if you're not able to tell if Falcon is moving up and down in response to the waves... I'm counting on the fact that as waves break, the surf is going to be white...and will be visible in the image as well.

They may still want to cut out IR.. because the water might be radiating, and once the engine lights, it'll probably saturate the sensor.

I don't think this'd require much modification at all. Unless they're using some special space qualified camera, with a custom chip, a custom form factor etc.... can't you get a black and white camera and stick a red filter in front of it? If they keep the same data payload, they can trade two channels for more dynamic resolution...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: MP99 on 05/01/2014 07:21 pm
12.5% is actually a lot. When receiving such error-prone signal, it makes a lot of difference. It also reduces transmission power requirements (for the same frame rate), therefore more power available for transmission, therefore better S/N ratio.

Maybe. *If* that's the only telemetry sent. Who's to say the video wasn't multiplexed along all the other, high rate vehicle telemetry so the 12.5% for video is more of a noise in the total bandwidth budget?

Also true monochrome cameras are up to 3x more sensitive, meaning less (camera) noise to start with.

Seriously? At the codec quality settings they're using, the camera dirt that's deposited on the way down you're worried about camera noise?

Again, why go around making up solutions that would only be relevant for a couple of seconds before splashdown?
Because the way up is quite well documented. The way down is not.

Oh, I'm sure they have it quite well-documented. Just not in a format your typical rocket pr0n enthusiast likes. It's in the form of vehicle telemetry. That's gold, anything else is just gravy.

It still doesn't change my argument that any such solutions are just too much trouble for the amount of use they'll have eventually.
Worked on sort of "telemetry", we divided data into what's important (10%, in this case sensors data) and what's less important (90%, in this case video feed). Both were CRC-ed, but the first got retransmitted if corrupted or lost, the second was not. Sort of TCP and UDP.

On worsening channel the retransmissions of sensor data occupied more and more bits until no bits were available to video.

I guess SpaceX does the same, so if we see SOME video, it means ALL sensor data was received without gaps.

You guys are worried about how many bits the chroma component takes when some of analysis says they included substantial fill-in packets to bump up the data rate to make it a fixed rate transmission??

If they had infinite time to work on the transmission system it would have been nice to optimise it with lots of redundancy data instead of fill-in "0xffffffff" packets. (Though maybe those "ffff"s make it easier to re-synchronise the stream once major errors start to bite??)

But, I suspect this is more of an off-the-shelf system that was stymied by weather conditions on the day.

Next launch / splashdown should have an easier time of it.

cheers, Martin
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: eeergo on 05/01/2014 07:56 pm
Someone on Youtube did a decent effort of cleaning it up:

I'm not sure how accurate it is, and I don't think the legs were extended in the beginning of the clip? Nonetheless it looks neat.

https://www.youtube.com/watch?v=CnyrleS782g

The overlay this person made is actually quite misleading: there are some misplaced pixels at 0:14-0:15 from the engine exhaust that appear as yellow artifacts to the left of the image - in the original video they were not so apparent since there was a lot of noise, but here they take the context away and get quite distracting. Also, it makes the splashdown and subsequent tipping over of the stage very confusing to watch, since the legs should be submerged.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Lars_J on 05/01/2014 08:07 pm
Yes, the overlay works better for the first part of the video.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: michaelni on 05/02/2014 03:08 am
The video seems simple profile level 3 mpeg4 video in mpeg TS.
It seems none of the error resilience features of mpeg4 have been used when encoding it. Which is a pitty, had slices been used then the decoder could resume decoding of a frame at the next slice start, had data partitioning been used then the more important low resolution information and motion vectors would have been coded first in each slice making errors less likely to damage them. And had rvlcs been used then slices could have been decoded from both the start and end again, limiting the impact of bit errors.

I know nothing about how the video was generated or how it was transmitted, but if there was some FEC in there then then it should be possible in principle to re-run FEC decoding after manual fixing up all mpeg-TS and mpeg4-ES headers. And as such manual fixing would decrease the errors, FEC would then have fewer errors to deal with and might in a few rare cases be able to fix a few more errors.
Also if some kind of CRCs have been used, CRCs can also be used to correct bit errors as long as the number of errors is sufficiently small, which each CRC would need to correct, the exact number that could be corrected that way depends on the packet size and  crc polynomial being used.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/02/2014 04:52 am
Yeah. It's a real challenge. Pretty stuck here.

But I still got a few ideas I want to try...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SVBarnard on 05/02/2014 06:12 am
can someone please explain to me why spacex still hasn't released the footage they got from their airplane? Why are they being so secretive about it? I mean seeing is believing so why not just release the footage and prove to everyone in the world they really accomplished such an unprecedented feat?

I mean they did actually record it from their airplane right?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: luinil on 05/02/2014 06:15 am
the airplane might not have been close enough to take a video.

Remember the weather was pretty heavy, NASA renounced to send their plane.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Lars_J on 05/02/2014 06:16 am
can someone please explain to me why spacex still hasn't released the footage they got from their airplane? Why are they being so secretive about it? I mean seeing is believing so why not just release the footage and prove to everyone in the world they really accomplished such an unprecedented feat?

I mean they did actually record it from their airplane right?

I suspect we will see more footage when SpaceX releases their usual mission highlights video.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/02/2014 09:03 am
Over at the reddit thread (http://www.reddit.com/r/spacex/comments/24bsn2/first_stage_landing_video/) somebody posted his very useful analysis. I will quote it here:

Quote
Argh. I have some MPEG-2/MPEG-4 experience, and had a go at fixing the file too. Sadly, it's no better than the version SpaceX posted, in terms of useable frames and recognizable objects in the frame. I think there's just too much lost data (and the I-frames made up too much of the missing data) to get much back. But I'd love to be wrong!

The order and detail of my efforts was:

 * Identify discontinuities in the file -- where 0x47 sync bytes jumped offset. Unlike the SpaceX file (which apparently removed data), I added 0xFFs to pad the data to align the next identifiable packet to a 188 byte multiple.

 * Identify the different PIDs in the file. There were the usual suspects: 0x0000 (PAT: Program Allocation Table), 0x0020 (PMT: Program Map Table referenced by the PAT), 0x03e8 (PES video stream, referenced by the PMT), and 0x1fff (padding packets to align the transport stream bit clock with the Program Clock Reference).

 * Identify video PES streams by start code and ID (0x000001e0). I created a template packet that looked like this (hex): 47 43 e8 3_ 07 10 00 __ __ __ __ __ 00 00 01 E0 00 00 81 80 07 I tested each packet against this template, computing the Hamming distance and printing out packets below 23 bits difference. This showed me all the likely PES headers, which I could examine and repair. The five "__" bytes contain the Program Clock Reference value for that point in the transport stream. The PCR advances at 90kHz, with a 300-count fraction/extension that advances at 27MHz. In this file, the PCR advances at multiples of 6006 (decimal) or 66.733 milliseconds. I managed to recover 276 PCR values, and the time stamps span 318 frames (at 6006 PCR counts per frame) or 21.221 seconds. For PCR values that were not multiples of 6006, but were clearly close or off by one or two flipped bits, I flipped them back (hex file editor) to get the interval right.

 * I also normalized the Presentation Time Stamp (PTS) values, which appear to lag the PCR of the packet by exactly 10000 counts (111.111 milliseconds). I used the PCR and PTS intervals to cross-check values (looking for multiples of 6006 in the PCR and a difference of +10000 between the PCR and PTS) and fixed anything that was out of sorts.

 * I identified all transport stream packets as being either PAT, PMT, video, or padding, and coerced bit errors in the header to have the correct sync byte (0x47) and flags (turning off the transport stream error indicator, transport priority, and scrambling bits. I used the continuity counter value to tell whether successive packets might actually be the same PID (usually 0x3e8, video). Based on all this information, I coerced PIDs to the most likely value, and turned any obviously junk packets into padding (null) packets. I fixed up the continuity counter on the packets of each PID so they were incrementing without gaps.

 * I then dug into the MPEG-4 visual object sequence and visual object structures. I saw damage on some of the bits that described the dimensions of the image frame, which were causing VLC and ffmpeg some consternation. So I normalized those.

At this point, I ran out of time. I have a lot of other projects I should be working on. So I share all this in hopes others can build on this, or point out a fatal flaw in my approach and improve on it.

Here's the file I've hand-edited to correct all the PCR and PTS timing issues I could find:

edited.ts (http://www.sharebrained.com/spacex/edited.ts)

And here's that file run through a script to coerce the transport stream headers into more sensible values (which might or might not actually be an improvement):

coerced.ts (http://www.sharebrained.com/spacex/coerced.ts)

Good luck! And I hope SpaceX gets better weather and position for their RF link on the next launch. Being a software radio nerd, I can appreciate the challenges they face catching several megabits of data from an object they don't want to be too close to when it comes down.

    - Jared
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: michaelni on 05/02/2014 03:35 pm
here are two videos which where transcoded with a modified FFmpeg.
The modifications force the decoder to continue decoding frames even once errors where detected, this should show more parts of the frames at the expense of also significnatly more random damaged and ugly looking blocks.
Also motion compensation and error concealment where disabled, the way neither correct nor incorrect parts of previous frames would leak in the current frame.
Overall the video certainly is less pleasing to watch that way but it should contain more information for analysis and be easier to make sense of if someone wants to analize it frame by frame

The first is based on raw.ts, the second is based on the coerced.ts file from Jarred which was posted here
https://ffmpeg.org/~michael/space-x-video/raw.ts-nomc-ignoreerr.mkv
and
https://ffmpeg.org/~michael/space-x-video/Jared-coerced.ts-nomc-ignoreerr.mkv
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: michaelni on 05/02/2014 03:59 pm
and here are teh 3 videos, raw.ts and the 2 from jared decoded with ffmpeg with error concealment tuned a bit (that is favoring predicting from the previous frame with MV 0,0), which might look slightly more pleasing than the original, but shouldnt contain more information

https://ffmpeg.org/~michael/space-x-video/raw.ts-tunedec.mkv
https://ffmpeg.org/~michael/space-x-video/Jared-edited.ts-tunedec.mkv
https://ffmpeg.org/~michael/space-x-video/Jared-coerced.ts-tunedec.mkv
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/02/2014 06:36 pm
here are two videos which where transcoded with a modified FFmpeg.
The modifications force the decoder to continue decoding frames even once errors where detected, this should show more parts of the frames at the expense of also significnatly more random damaged and ugly looking blocks.
Also motion compensation and error concealment where disabled, the way neither correct nor incorrect parts of previous frames would leak in the current frame.
Overall the video certainly is less pleasing to watch that way but it should contain more information for analysis and be easier to make sense of if someone wants to analize it frame by frame

The first is based on raw.ts, the second is based on the coerced.ts file from Jarred which was posted here
https://ffmpeg.org/~michael/space-x-video/raw.ts-nomc-ignoreerr.mkv
and
https://ffmpeg.org/~michael/space-x-video/Jared-coerced.ts-nomc-ignoreerr.mkv

Wooow! I didn't realize this at first. But we have Michael Niedermayer (http://www.ffmpeg.org/consulting.html) in the house!  8)

One of the creators of ffmpeg itself. Gulp.

Hope we can work together to figure this out. I like your approach though. Currently I'm focusing on the I-frames and altering the decoder to give me more information about what is wrong with certain (macro)blocks. But I've seen the decoder source code for the first time today, while you may know it by heart.

Ok guys. I think we're in good hands now ;)

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/02/2014 06:44 pm
Interesting work guys! Keep up the good work!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Helodriver on 05/02/2014 06:52 pm
From that cleanup it appears the video definitely ends before the engine quits and definitely before the vehicle begins to topple. Still no definitive answer as to what the "8 seconds" entailed. Nice work on the video.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Marams on 05/02/2014 08:02 pm
here are two videos which where transcoded with a modified FFmpeg.
The modifications force the decoder to continue decoding frames even once errors where detected, this should show more parts of the frames at the expense of also significnatly more random damaged and ugly looking blocks.
Also motion compensation and error concealment where disabled, the way neither correct nor incorrect parts of previous frames would leak in the current frame.
Overall the video certainly is less pleasing to watch that way but it should contain more information for analysis and be easier to make sense of if someone wants to analize it frame by frame

The first is based on raw.ts, the second is based on the coerced.ts file from Jarred which was posted here
https://ffmpeg.org/~michael/space-x-video/raw.ts-nomc-ignoreerr.mkv
and
https://ffmpeg.org/~michael/space-x-video/Jared-coerced.ts-nomc-ignoreerr.mkv

Nice work. The following frames from the raw.ts where quite interesting because they clearly show the time stamp in the lower left corner. There are a lot frames with clearly visible last numbers of time stamps.

MECO was around 19:28:19
landing video started at 19:35:03 - 1s or so
landing video ended at 19:35:19

In every frame there is 19:35 in the lower left corner with the exact same pixels. For scam-analysis a known plain text would help for sure :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: michaelni on 05/02/2014 08:08 pm
Hope we can work together to figure this out. I like your approach though. Currently I'm focusing on the I-frames and altering the decoder to give me more information about what is wrong with certain (macro)blocks. But I've seen the decoder source code for the first time today, while you may know it by heart.

If we assume that a frame has been damaged by a flipped bit, then in general the decoder will not detect it when reading that bit but rather will either show artifacts and continue without a detected error or more likely get desynced and detect some kind or inconsistency like a out of range value or incorrect marker bit at a later point. For example if theres an error detected in MB at location XxY the actual bit error that caused it could be a few macroblocks before it,
MBs are 16x16 pixel in mpeg4 and are stored in raster scan order left to right and top to bottom.
mpeg4 (A)SP uses variable length coding (huffman coding with a few unused codes), so bit errors can throw things out of sync (vlc not matching up between encoder and decoder or symbols not matching up, that is for example a MB type could end up being decoded as a motion vector after a bit error). If detected inconsistencies are ignored then the decoder can end up falling into sync again by chance and continue to decode the bitstream somewhat reasonably. After such resync various values like motion vectors, quantization parameters or values used for AC/DC prediction could/would be wrong though causing some artifacts but could/should still contain some human recognizeable features of the original.

I suspect if the number of flipped bits per frame is small enough and nothing is missing of a frame, someone with alot of time could find the bits by trial and error and by starting from the location in raster scan order where artifacts are first vissible. error concealment would have to be disabled in the used decoder though so that one can see where the artifacts start.
The error concealment code in ffmpeg by default conceals several MBs prior to a detected error.
Error concealment can be disabled with -ec 0

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/02/2014 08:16 pm
Nice work. The following frames from the raw.ts where quite interesting because they clearly show the time stamp in the lower left corner. There are a lot frames with clearly visible last numbers of time stamps.

MECO was around 19:28:19
landing video started at 19:35:03 - 1s or so
landing video ended at 19:35:19

In every frame there is 19:35 in the lower left corner with the exact same pixels. For scam-analysis a known plain text would help for sure :)

Great find!!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Michael Bloxham on 05/02/2014 08:23 pm
Looks like the stage is still going reasonably fast when it hits the water, and plunges deep into the water?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mmeijeri on 05/02/2014 08:25 pm
https://ffmpeg.org/~michael/space-x-video/raw.ts-tunedec.mkv
https://ffmpeg.org/~michael/space-x-video/Jared-edited.ts-tunedec.mkv
https://ffmpeg.org/~michael/space-x-video/Jared-coerced.ts-tunedec.mkv

Won't play for me with either Windows Media Player or DivX. What software can I use to play it?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: YetAnotherLurker on 05/02/2014 08:30 pm
Won't play for me with either Windows Media Player or DivX. What software can I use to play it?

VLC Media Player
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/02/2014 08:31 pm
https://ffmpeg.org/~michael/space-x-video/raw.ts-tunedec.mkv
https://ffmpeg.org/~michael/space-x-video/Jared-edited.ts-tunedec.mkv
https://ffmpeg.org/~michael/space-x-video/Jared-coerced.ts-tunedec.mkv

Won't play for me with either Windows Media Player or DivX. What software can I use to play it?

Something based on ffmpeg or libav. http://mpv.io/installation/ will work
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mheney on 05/02/2014 09:34 pm
Xine on a linux box works as well.  I believe you can download a version of xine for Windows ...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: MTom on 05/02/2014 10:47 pm
Nice work. The following frames from the raw.ts where quite interesting because they clearly show the time stamp in the lower left corner. There are a lot frames with clearly visible last numbers of time stamps.

MECO was around 19:28:19
landing video started at 19:35:03 - 1s or so
landing video ended at 19:35:19

In every frame there is 19:35 in the lower left corner with the exact same pixels. For scam-analysis a known plain text would help for sure :)

Great find!!

And this is placed on the bottom of the frames.
This sounds me (without any knowledge of video) as a good point: could it be a good reference point for the correction? (The top of the frames have a good quality on the raw video, after identifiing the bottom lines could it be like a "picture frame" for the work?)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jcc on 05/03/2014 01:17 am
Xine on a linux box works as well.  I believe you can download a version of xine for Windows ...

I just downloaded the latest RealPlayer update "RealPlayer Cloud" on win7, and it plays mkv files.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: AS-503 on 05/03/2014 01:24 am
All Windows based OS can benefit greatly from installing a complete CODEC package like K-Lite (mega). This will not only allow playback of virtually everything, but also enable thumbnails and super-charge Windows Media Player to playback almost all video and audio file extensions.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: MarsInMyLifetime on 05/03/2014 02:29 am
FWIW, I was able to watch these mkv files using J River Media Center which seems to have (or get) the right codec.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IslandPlaya on 05/03/2014 02:52 am
Just download VLC player. It saves downloading all them dodgy codec packs.
If VLC can't play it then forget it...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: MP99 on 05/03/2014 08:13 am
Just download VLC player. It saves downloading all them dodgy codec packs.
If VLC can't play it then forget it...

Concur.

Cheers, Martin
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: grythumn on 05/03/2014 11:38 am
For windows, Media Player Classic Homecinema or VLC are the standard applications. I usually have better luck with MPC for hardware acceleration (not important for this particular case.)

-R C
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Digitalchromakey on 05/03/2014 02:54 pm
here are two videos which where transcoded with a modified FFmpeg.
The modifications force the decoder to continue decoding frames even once errors where detected, this should show more parts of the frames at the expense of also significnatly more random damaged and ugly looking blocks.
Also motion compensation and error concealment where disabled, the way neither correct nor incorrect parts of previous frames would leak in the current frame.
Overall the video certainly is less pleasing to watch that way but it should contain more information for analysis and be easier to make sense of if someone wants to analize it frame by frame

The first is based on raw.ts, the second is based on the coerced.ts file from Jarred which was posted here
https://ffmpeg.org/~michael/space-x-video/raw.ts-nomc-ignoreerr.mkv
and
https://ffmpeg.org/~michael/space-x-video/Jared-coerced.ts-nomc-ignoreerr.mkv
The second transport stream above which is based on Jared's coerced ts with motion vectors and error concealment turned off, does make very interesting viewing on a frame by frame basis. There are several fixed features in each frame that we know are spatially invariant on a frame by frame basis - the legs, the first stage edges. the real time read out display and to an extent the rocket plume.

Looking on a frame by frame basis you can clearly see the edges for the legs, often as (a) cohesive block(s), but located in clearly the wrong frame position.

It should be possible, but incredible tedious for someone wit the right tools to manually reverse engineer a transport stream, using these measurable, known fixed special errors as frame by frame variable special offsets and moving all the associated data (including motion vectors, etc) within these clearly defined mislocated blocks over to their known correct spacial position in a reverse engineered transport stream.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Carreidas 160 on 05/03/2014 02:58 pm
I'm neither a video encoding/decoding expert nor qualified rocket engineer but...

Is there a way to clean up a noisy video signal using genetic algorithms?

I.e. randomly keep switching bits (turn 0s into 1s) in parts of the signal that are known to be corrupt, and see whether these variations clean up the picture or not? And subsequently lock the bits that improve quality?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/03/2014 09:17 pm
Hi guys,

Litte update.

I have been making some progress. I can now tell the codec to skip bad macroblocks (16x16 pixel blocks). And I can tell it when to start the next good macroblock and where its data source begins (on a bit-level).

As a proof of concept I have taken the coerced.ts and from that basicly recreated the I-frame SpaceX released earlier. See the attached image. (btw I haven't spend time to repair the lower part of the image)

The difference is that with this I can "fix" the output of an I-frame without actually fixing the data itself. I "just" have to say what blocks are "bad" and where the new "good" block begins (bit wise). The latter is also very time consuming right now. But I have some ideas that maybe everybody can help. Maybe.

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: dorkmo on 05/03/2014 09:37 pm
awesome stuff.

to comment on the content of what can be seen so far,
it looks to me like the legs both came out a little bit. then only the left leg progressed, then stopped. then the right leg extended fully. then finally the left leg extended fully.

i wonder if thats typical of whats been seen in testing
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Adaptation on 05/04/2014 12:01 am
Hi guys,

Litte update.

I have been making some progress. I can now tell the codec to skip bad macroblocks (16x16 pixel blocks). And I can tell it when to start the next good macroblock and where its data source begins (on a bit-level).

As a proof of concept I have taken the coerced.ts and from that basicly recreated the I-frame SpaceX released earlier. See the attached image. (btw I haven't spend time to repair the lower part of the image)

The difference is that with this I can "fix" the output of an I-frame without actually fixing the data itself. I "just" have to say what blocks are "bad" and where the new "good" block begins (bit wise). The latter is also very time consuming right now. But I have some ideas that maybe everybody can help. Maybe.

Regards,

arnezami

That is really freakin good work.  Would you mind sharing your latest .ts and a little more on how you are determining which blocks are good and bad.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/04/2014 04:02 am
Hi guys,

Litte update.

I have been making some progress. I can now tell the codec to skip bad macroblocks (16x16 pixel blocks). And I can tell it when to start the next good macroblock and where its data source begins (on a bit-level).

As a proof of concept I have taken the coerced.ts and from that basicly recreated the I-frame SpaceX released earlier. See the attached image. (btw I haven't spend time to repair the lower part of the image)

The difference is that with this I can "fix" the output of an I-frame without actually fixing the data itself. I "just" have to say what blocks are "bad" and where the new "good" block begins (bit wise). The latter is also very time consuming right now. But I have some ideas that maybe everybody can help. Maybe.

Regards,

arnezami

That is really freakin good work.  Would you mind sharing your latest .ts and a little more on how you are determining which blocks are good and bad.

Hi Adaptation,

I have not changed any .ts lately. The coerced.ts file from Jared is pretty good for now.

What I'm doing is changing the behavior of the decoder itself: libavcodecs/h263dec.c (http://git.videolan.org/?p=ffmpeg.git;a=blob;f=libavcodec/h263dec.c;h=523fca9e7f9406211c519f0380a722fd6b0268fa;hb=HEAD) to be more exact. Thats the code that loops through the macroblocks. The code that actually decodes the bits themselves is also very important: libavcodec/ituh263dec.c (http://git.videolan.org/?p=ffmpeg.git;a=blob;f=libavcodec/ituh263dec.c;h=d9170df8266f0274adee5c19d758574f39e0a540;hb=HEAD). Although I don't change it.

There are essentially two ways I'm determining whether a macroblock is bad: either ffmpeg gives an error like "Error in MB: " or I see it visually. That's basicly it.

What is a good block and more importantly where its starts in the datastream is much harder to derermine. This is because these blocks have a variable length and nowhere does it say how long they are (it usually varies from 30 to ~150 bits, but sometimes its a couple hundred bits). This is for I-frames btw: the frames that are basically independent pictures in the stream and require no other frames to be decoded. There are 13 I-frames in the file btw. We have seen only two partials, so there is a lot to be discovered still.

Anyway, this is the real challenge: figuring out the starting positions of the good macroblocks after a few bad ones. And maybe we can all collaborate here. Maybe. But for now what I have tried to do is simply let the codec iterate through lot of starting positions for a block and decode from there, and if there is no error it assumes it is ok. Semi-automated trial and error essentially.

This however doesn't always work: sometimes there is no error but clearly the block (visually) isn't correct. But when trying all kinds of random starting positions another block (later in the stream) will become visible (albeit in the wrong position as already discussed in this thread). Since I log all the bit-positions of all the macroblocks I can now determine what the starting position (bit-wise) is of that block. Then I "tell" (aka hack) the codec to read that block earlier when it is supposed to be shown.

But for now I hack this in the code and I want to make this a little bit more user friendly.

Hope that answers your questions.

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/04/2014 08:53 am
Hi guys,

Here is an overview of the I-frames in the file:

#Position in fileSize in bytesFrame numbermp4-img linkimage linkCurrent state
0~254956n/an/anot extracted yet, transport stream issue?
18478801667333iframe_1.zip (http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=581910)0% (gray)
211466121103953iframe_2.zip (http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=581911)iframe_2.png (http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=581924)10%
31207712892757iframe_3.zip (http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=581912)0% (gray)
414462841653673iframe_4.zip (http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=581913)0% (gray)
517448281709493iframe_5.zip (http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=581914)0% (gray)
6205427629622113iframe_6.zip (http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=581915)0% (gray)
7264196415064150iframe_7.zip (http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=581946)iframe_7.png (http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=581934)15%
8294088420428169iframe_8.zip (http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=581947)iframe_8.png (http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=581936)45% - 60%
9323942815730188iframe_9.zip (http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=581918)iframe_9.png (http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=581942)15%
10353797210009207iframe_10.zip (http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=581919)iframe_10.png (http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=581944)10%
11384065214113227iframe_11.zip (http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=581920)0% (gray)
12413581212169246iframe_12.zip (http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=581921)0% (gray)

Some explanation about this table:

The column "Position in file" means the position (in bytes) in the coerced.ts file. So where it's data (according to ffmpeg) begins. "Size in bytes" is the size according to ffmpeg. "Frame number" is the frame number according to ffmpeg. All based on the current coerced.ts file. Note that iframe_0 hasn't been extracted yet and several iframes don't give any output at the moment (just blank grey).

For each I-Frame I've also attached an .mp4-img file and a .png picture produced by the standard ffmpeg.

The mp4-img files were produced like this: (tip from michaelni)

Quote
./ffmpeg -i coerced.ts -vcodec copy -an -f image2 frame%d.mpg4-img

In his words:

Quote
This would split the mpeg-ts into 1 file per frame, each containing just the mpeg4-es data, no mpeg-ts anymore

This is going to be useful so we can target 1 I-frame at the time. I'm currently working on a slightly altered ffmpeg version (and some scripts) with which you can manipulate multiple macroblock positions, and semi-automatically iterate over many bit-positions in order to find a "good" block. That way anyone can help recover these images. But I still have a lot more work to do before that happens ;).

Regards,

arnezami

[edit] You need to add the option "-s:0 704x480" to ffmpeg if you want it to accept an .mp4-img file as input.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/04/2014 04:51 pm
Hi guys,

I've created a way so that everybody (with some basic linux skills) can help restoring these I-frames.

[edit]Attached is one file: install.txt. It contains instructions on how to install this adapted version of ffmpeg.

Here is the last part of the install.txt:

Quote
In your current directory there is now a new ffmpeg file
You can now use this new ffmpeg with additional options, namely:

1) "-debug mb_pos_size" : this enables logging of the x, y, bitposition and bitsize of the decoded macroblocks
2) "-mmb x1:y1:bitpos1,x2:y2:bitpos2" : with this you can overrule the bitposition of multiple macroblocks (so they start at the correct data)
3) "-err_detect ignore_err" : this makes sure decoding doesn't stop if there are errors (mainly due to the bad alignment of data) (this change was done by Michael Niedermayer)

Example usage:
./ffmpeg -mmb 07:14:17233,19:14:57914 -debug mb_pos_size -err_detect ignore_err -s:0 704:480 -i iframe_8.mpg4-img -f image2 img-%03d.png

In the above example we do serveral things:

- Since the macroblock at x:7 y:14 is defect we overrule its datapoint with a block at 17233. The data of this block is chosen because it contains almost no data (only 30 bits) which ensures that when put here it won't affect neighbouring blocks too much. It's a "silent" block if you will, which doesn't create too much noise.
- We have found out (after experimenting and examining the log) that a specific part of the left leg of the stage begins at bitposition 57914. Since we know where to place it (x:19 y:14) we assign this block to that datapoiint.
- We use "iframe_8.mpg4-img" as the source (see the forum, there is a list of all I-frames, http://forum.nasaspaceflight.com/index.php?topic=34597.msg1193316#msg1193316) but therefore we need to add the option "-s:0 704:480", otherwise ffmpeg cant determine the size of the video
- the option "-f image2 img-%03d.png" makes sure that the I-frame is being exported as a lossless png (the %03d is not needed when there is only one frame in the source)
- In the resulting .png you can see that the I-frame looks pretty good already!

Good luck and have fun!

Regards,
arnezami

PS. If you have any questions (or it doens't work somehow) please let me know.
PPS. Because the forum doesn't let me upload a tar.gz file I had to double compress it with zip. Sorry.

[edit] New install.txt uploaded because I used an odd version of ffmpeg which could screw up your own.
[edit] Even newer install.txt since we now have it on git! Thanks mlindner!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/04/2014 08:02 pm
Hi Guys,

iframe2 is starting to become alive!  ;D

No legs deployed!

btw I did a trick to make this work, because of that everything is sort of shifted up.

Have fun!

arnezami

@mlindner: I have responded to you in a pm.

[edit] almost forgot, I did this: "-mmb 40:03:11229,42:03:11229,42:05:19620"
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/04/2014 09:35 pm
Hi Guys,

iframe2 is starting to become alive!  ;D

No legs deployed!

btw I did a trick to make this work, because of that everything is sort of shifted up.

Have fun!

arnezami

@mlindner: I have responded to you in a pm.

[edit] almost forgot, I did this: "-mmb 40:03:11229,42:03:11229,42:05:19620"

Awesome!

What does your code do when it skips a block?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/04/2014 09:54 pm
Awesome!

What does your code do when it skips a block?
Blocks are not really skipped as such, they are on-the-fly given a different point where they retrieve their data. So if a blocks' data would normally be corrupt I just tell it to look for different data (from a block earlier or later). And with a later block I point it its proper data point again. Therefore avoiding the corrupt data of one block affecting other blocks.

My newest strategy is to turn use detect_err = ignore, and look for blocks that seem to contain visual data (rocket structure and such). And then (by counting x and y on my screen) determining the x and y of that block, look at the log and see the bit position. Then set block x:0 y:0 to that bitposition! That works great for finding real data parts. :)

Btw. Attached is a tantalizing very sketchy version of iframe11...

Can anyone confirm this is AFTER landing? Because SpaceX might be interested in this. Be careful though looks can be deceiving with this much noise. But do I see an intact fuselage? And a leg? Or is it artefact?

Ooh and there is the timestamp :P

Hihi this is fun! :)


[edit] Forgot it again. I did: "-mmb 0:0:10020" here.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/04/2014 11:09 pm
Awesome!

What does your code do when it skips a block?
Blocks are not really skipped as such, they are on-the-fly given a different point where they retrieve their data. So if a blocks' data would normally be corrupt I just tell it to look for different data (from a block earlier or later). And with a later block I point it its proper data point again. Therefore avoiding the corrupt data of one block affecting other blocks.

My newest strategy is to turn use detect_err = ignore, and look for blocks that seem to contain visual data (rocket structure and such). And then (by counting x and y on my screen) determining the x and y of that block, look at the log and see the bit position. Then set block x:0 y:0 to that bitposition! That works great for finding real data parts. :)

Btw. Attached is a tantalizing very sketchy version of iframe11...

Can anyone confirm this is AFTER landing? Because SpaceX might be interested in this. Be careful though looks can be deceiving with this much noise. But do I see an intact fuselage? And a leg? Or is it artefact?

Ooh and there is the timestamp :P

Hihi this is fun! :)


[edit] Forgot it again. I did: "-mmb 0:0:10020" here.

Arnezami, now that we have (and are getting more) partially good I-frames, how do we apply these same changes to the video so that the following frames can reference them properly?

Also I don't understand your description of what you do with blocks. So when it uses "different data" is it using data from other blocks or possibly just finds the correct location of its own data?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: michaelni on 05/04/2014 11:39 pm
The mp4-img files were produced like this: (tip from michaelni)

Quote
./ffmpeg -i coerced.ts -vcodec copy -an -f image2 frame%d.mpg4-img

Its possible to extract a few more frames at the start with
./ffmpeg -i coerced.ts -vcodec copy -an -copyinkf  -f image2 frame%d.mpg4-img
They are not keyframes though
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: michaelni on 05/05/2014 01:45 am
Ive put arnezamis changes on github:
(with some simplifications to the implementation, and very basic docs from mlindner)

should be easier to download that way

https://github.com/michaelni/FFmpeg/tree/spacexdebug1
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/05/2014 02:04 am
Ive put arnezamis changes on github:
(with some simplifications to the implementation, and very basic docs from mlindner)

should be easier to download that way

https://github.com/michaelni/FFmpeg/tree/spacexdebug1

Just a correction, the docs are from arnezami. I only put them up in an initial github tree which michaelni merged changes from.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/05/2014 02:56 am
@mlindner: I can confirm the steps your new install.txt work. I've added it to my original post and removed the .zip file.

Thanks mlindner!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: grythumn on 05/05/2014 03:31 am
Couple of notes:

1) This compiles (slowly) and runs fine under cygwin, so windows people can get in on the fun too.
2) When you replace a bad MMB with a small one, it keeps going from that point in the bitstream. You need to restart the stream of the next good MMB at its original point in the bitstream or the picture looks much worse and you get ghosts.

Original  error:
[mpeg4 @ 0x8001af20] MB pos/size: 0 19:04:15480 73
[mpeg4 @ 0x8001af20] MB pos/size: 0 20:04:15553 209
[mpeg4 @ 0x8001af20] ac-tex damaged at 21 4
[mpeg4 @ 0x8001af20] MB pos/size: -1 21:04:15762 73
[mpeg4 @ 0x8001af20] Error at MB: 201
[mpeg4 @ 0x8001af20] illegal dc vlc
[mpeg4 @ 0x8001af20] MB pos/size: -1 22:04:15835 10
[mpeg4 @ 0x8001af20] Error at MB: 202
[mpeg4 @ 0x8001af20] I cbpc damaged at 23 4
[mpeg4 @ 0x8001af20] MB pos/size: -1 23:04:15845 6
[mpeg4 @ 0x8001af20] Error at MB: 203
[mpeg4 @ 0x8001af20] MB pos/size: 0 24:04:15851 78
[mpeg4 @ 0x8001af20] MB pos/size: 0 25:04:15929 96

 ./ffmpeg -mmb 21:04:3656 -debug mb_pos_size -err_detect ignore_err -s:0 704:480 -i iframe_9.mpg4-img -f image2 img-%03d.png
[mpeg4 @ 0x80022240] MB pos/size: 0 19:04:15480 73
[mpeg4 @ 0x80022240] MB pos/size: 0 20:04:15553 209
[mpeg4 @ 0x80022240] MB pos/size: 0 21:04:10340 22
[mpeg4 @ 0x80022240] MB pos/size: 0 22:04:10362 22
[mpeg4 @ 0x80022240] MB pos/size: 0 23:04:10384 23
[mpeg4 @ 0x80022240] MB pos/size: 0 24:04:10407 102
[mpeg4 @ 0x80022240] MB pos/size: 0 25:04:10509 226


 ./ffmpeg -mmb 21:04:3656,24:04:15851 -debug mb_pos_size -err_detect ignore_err -s:0 704:480 -i iframe_9.mpg4-img -f image2 img-%03d.png
[mpeg4 @ 0x80022460] MB pos/size: 0 19:04:15480 73
[mpeg4 @ 0x80022460] MB pos/size: 0 20:04:15553 209
[mpeg4 @ 0x80022460] MB pos/size: 0 21:04:3656 22
[mpeg4 @ 0x80022460] MB pos/size: 0 22:04:3678 22
[mpeg4 @ 0x80022460] MB pos/size: 0 23:04:3700 22
[mpeg4 @ 0x80022460] MB pos/size: 0 24:04:15851 78
[mpeg4 @ 0x80022460] MB pos/size: 0 25:04:15929 96
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/05/2014 03:42 am
Couple of notes:

1) This compiles (slowly) and runs fine under cygwin, so windows people can get in on the fun too.
2) When you replace a bad MMB with a small one, it keeps going from that point in the bitstream. You need to restart the stream of the next good MMB at its original point in the bitstream or the picture looks much worse and you get ghosts.

Original  error:
[mpeg4 @ 0x8001af20] MB pos/size: 0 19:04:15480 73
[mpeg4 @ 0x8001af20] MB pos/size: 0 20:04:15553 209
[mpeg4 @ 0x8001af20] ac-tex damaged at 21 4
[mpeg4 @ 0x8001af20] MB pos/size: -1 21:04:15762 73
[mpeg4 @ 0x8001af20] Error at MB: 201
[mpeg4 @ 0x8001af20] illegal dc vlc
[mpeg4 @ 0x8001af20] MB pos/size: -1 22:04:15835 10
[mpeg4 @ 0x8001af20] Error at MB: 202
[mpeg4 @ 0x8001af20] I cbpc damaged at 23 4
[mpeg4 @ 0x8001af20] MB pos/size: -1 23:04:15845 6
[mpeg4 @ 0x8001af20] Error at MB: 203
[mpeg4 @ 0x8001af20] MB pos/size: 0 24:04:15851 78
[mpeg4 @ 0x8001af20] MB pos/size: 0 25:04:15929 96

 ./ffmpeg -mmb 21:04:3656 -debug mb_pos_size -err_detect ignore_err -s:0 704:480 -i iframe_9.mpg4-img -f image2 img-%03d.png
[mpeg4 @ 0x80022240] MB pos/size: 0 19:04:15480 73
[mpeg4 @ 0x80022240] MB pos/size: 0 20:04:15553 209
[mpeg4 @ 0x80022240] MB pos/size: 0 21:04:10340 22
[mpeg4 @ 0x80022240] MB pos/size: 0 22:04:10362 22
[mpeg4 @ 0x80022240] MB pos/size: 0 23:04:10384 23
[mpeg4 @ 0x80022240] MB pos/size: 0 24:04:10407 102
[mpeg4 @ 0x80022240] MB pos/size: 0 25:04:10509 226


 ./ffmpeg -mmb 21:04:3656,24:04:15851 -debug mb_pos_size -err_detect ignore_err -s:0 704:480 -i iframe_9.mpg4-img -f image2 img-%03d.png
[mpeg4 @ 0x80022460] MB pos/size: 0 19:04:15480 73
[mpeg4 @ 0x80022460] MB pos/size: 0 20:04:15553 209
[mpeg4 @ 0x80022460] MB pos/size: 0 21:04:3656 22
[mpeg4 @ 0x80022460] MB pos/size: 0 22:04:3678 22
[mpeg4 @ 0x80022460] MB pos/size: 0 23:04:3700 22
[mpeg4 @ 0x80022460] MB pos/size: 0 24:04:15851 78
[mpeg4 @ 0x80022460] MB pos/size: 0 25:04:15929 96

Yeah took me a bit to realize that too. It's somewhat intentional as the errors may come in a string and also because a given block is dependent on the previous block for its start location. So if you replace a block with a different block then the start location of the next block is obviously wrong so its useful to let it continue along and wrap back to the same data to possibly "hit" the data at a better bit position.

Edit: attached my best try at iframe 8 thus far here. mmb setting 08:14:17233,11:14:56840 Also we need to get these iframes as perfect as possible as every other following frame references them for their own data. Afterwards we might get a pretty good looking video.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/05/2014 04:43 am
It seems like the errors aren't all that common, but they're pretty uniformly spread around and it appears that SpaceX didn't encode the mpeg4 with lots of the settings that could detect and correct for errors that are built into the mpeg4 format.

Mpeg4 is a self-referential format meaning nearly every piece of data depends on some other piece of data and errors will propagate through the system as they accumulate until you have garbage referencing garbage. Those are all fixed when you hit an I-frame (what we're working on fixing), but there's very few I-frames, only 13, in the whole video that everything else references. When we fix these errors, they will still accumulate but they'll accumulate slower so things may become more recognizable.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/05/2014 04:53 am
Hi guys,

Nice to see its starting to work for you too. :)

I'm going to explain a bit more how this all  works and how you can use it. Because I feel there is a need for it.

Attached are 4 illustrations representing 4 scenarios. I think these are going to be useful in understanding how it all works and what you can do with this tool.

1. All is perfect.

In the first illustration you see that all is perfect. The top part represents the bits from the stream. You can see that there are "end markers" in the stream. These basicly tell the decoder that the data for a block has ended. (btw I'm not entirely sure how this actually works, but this I think is close to it). At each end-marker the decoder happily switches to the next block of 16x16 pixels. Note that the length of the data for each 16x16 block is not the same.

The lower part represents the 16x16 pixel blocks. They have identifying numbers. And as you can see they have the colors (representing its content) of the correct bits-stream-parts. So far so good.

2. Defect in bitstream

In the second illustration you see there is a defect in the bitstream (yellow). A bitsteam error can simply mean the content of a 16x16 pixel block doesn't seem quite right (without affecting other blocks too much). However lets assume that this time its worse: the defective bits are now interpreted by the decoder as an end-marker! Ouch. This is big trouble because firstly block 2 is broken off too early and will contain some nasty data (yellow in lower part) which can severely affect all blocks to its right and bottom). But secondly block 3 starts with completely wrong data! It is now on the complete wrong path since all bits are interpreted wrongly and the chance that some of these bits are (wrongly) interpreted as end-markers increases too. This is actually happening in this illustration/scenario. And at some point it even reads a sequence of bits that don't make sense at all. The decoder stops in such a situation. This leaves blocks 5-9 grey. This is the reason only the top parts of the I-frames were visible and the lower parts were grey.

In order to fix this you could try to restore the bitstream itself. But this is very hard to do and extremely time consuming (fixing compressed data is no joke!). The worse is: you don't know when the bad bits stop. We need to know where the good bits start again and try to avoid being dependent on having to do tedious and time consuming work in order to make any progress.

3. Reassigning the bitpointer twice

The problem now becomes: how do you "skip" a block? This seems impossible to do. There is no nice index of the bitstream (as fas as I know).

The first thing to do is make sure is not to halt the decoder when it encounters an error. This way the decoder sometimes rediscovers actual end-marks. And when it does it usually will keep finding more valid end-marks. Of course these will be read by the wrong blocks (aka you see them appearing in the wrong position in the picture). But still useful.

So we now know some valid end-marks (which also mark the start of block-data of course). In order to make use of them we should on-the-fly reassign the bitstream pointer of a block to such an end/start-marker. But first we need to make sure that block 2 doesn't read the bad bits.

So we do two reassignments:

1) Just before the decoder starts with decoding block 2 we reassign its bitpointer to (for example) a block in the past. In this case we set its bitpointer to the bits of block 1 (see third illustration). This is mainly to prevent that really bad visual effects come into our pixels (maybe there is a way to simply "do nothing" with a block, which may be better. I don't know, maybe others do).

2) Just before block 3 starts we assign its bitstream to the end-mark of block 2 (aka start-mark of block 3. Btw  we assume we found/guessed this somehow. This prevents us from reading the bad bits. The bad data is "skipped".

In the lower part of illustration 3 you can see how the content will look like if we do the above. Only one block is "broken" (contains data from a different block that is) and all the other blocks get their own intended data.

4. Reassigning the bitpointer once

A useful technique I discovered was that if you know the bitposition of certain good block-data (using a combination of your eyes which make out the x,y coordinate and the log which contains the corresponding bitpointer at that time) you can simply set the bitpointer of block 0 to that bitposition. I do this is in illustration 4. Then you can easily see how much good data there is and where something is wrong again. If you do this randomly (assigning block 0 to random bitpositions) you can easely detect good data parts in the stream. Which is what we need. So with this you can create a "data-map" of good and bad data. And that will eventually allow you to carefully "skip" the bad data.

Hopefully that helps. My fingers hurt :P

Regards and have a good day,

arnezami

[edit] grammar
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: dorkmo on 05/05/2014 05:18 am
awesome stuff.

to comment on the content of what can be seen so far,
it looks to me like the legs both came out a little bit. then only the left leg progressed, then stopped. then the right leg extended fully. then finally the left leg extended fully.

i wonder if thats typical of whats been seen in testing

Are you serious? I don't see how you could have possibly seen such detail from any version of the video so far. SpaceX has the actual flight data of all these systems and they say the legs deployed correctly as hoped. I'm gonna go with that.

watching the coerced video, it looks like the entire deployment occured durring the clock time of 00:06 to 00:10. theres only a sliver of blocks in a handful of frames and the image jumps around from left to right a lot.

also it looks like around 15 seconds you stop seeing any signs of flame

and to me it looks like maybe at 17 you can see a splash up of water from impact or a leaning/settling of the rocket to the side, which is up in the video
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/05/2014 05:22 am
<snip>

arnezami I've been talking to michaelni a lot today in learning how to do this process and I think your concept of "end markers" is off. Every block has a pre-defined size which is determined by its format. When a block is mis-interpreted then the size automatically becomes wrong and the block "eats" extra bits from the next block or misses bits from its block shoving them into the next block. Occasionally these errors cancel each other out (because there is a small finite number of block types and sizes) and the streaming is "resynced" to the proper block ordering so you get a string of correct blocks. You often don't see this resync because after its resynced the blocks are still referencing all the previous corrupted blocks. This is why after you fix one block suddenly lots more blocks become readable IMO.

If michaelni could comment though that would probably clear more things up.

Edit: The key difference that matters from your explanation is that changing the start position of a block can _increase_ or _decrease_ how far away the stop position is for that block because of how the block is interpreted.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: grythumn on 05/05/2014 05:29 am
Best image so far of Frame 9.

-Bob

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IRobot on 05/05/2014 05:35 am
I wonder if we reach a point from where some data alignment becomes impossible to correctly to determine. In that case, would it be possible to generate a couple of hundred of thousands of combinations, generate a result image for each, put them on a web server and allow the community to evaluate them visually?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/05/2014 05:39 am
I wonder if we reach a point from where some data alignment becomes impossible to correctly to determine. In that case, would it be possible to generate a couple of hundred of thousands of combinations, generate a result image for each, put them on a web server and allow the community to evaluate them visually?

I've actually been doing exactly that myself. You can throw out some settings because there are errors, but lots are "valid" data wise but are still garbage image-wise. I just have a text editor open with the setting in question, a terminal open so i can hit up arrow key (last command) and enter to run it, and have it open an image viewer with the new image. So i have a feedback time of about 5 seconds per value change. I've been thinking of turning it into a script that others could run.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/05/2014 05:41 am
Here's all my best iframe8's this far.

Edit: And just a note, I've only been working on the middle section thus far which is why the bottom section stays garbage. Also, the settings used are in the filenames, drop the iframe8- section and change the -'s to :'s.

Edit2: Working on these videos has wanted to make me go find the guy who wrote their rocket video encoder and shake them. Why the hell do they interlace their video and then re-encode the interlacing? It increases the bandwidth for their videos a crap ton by doing that which allows for higher error rate. Further, they didn't add all the stuff built into the codec that anyone streaming video uses to ensure error correction. It's almost like the cameras were slapped on as an afterthought purely for PR. It was #1 on my list if I ever got an internship at SpaceX would be to look into their codebase, look up who did the commits on this code and go bug them about it.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/05/2014 06:18 am
<snip>

arnezami I've been talking to michaelni a lot today in learning how to do this process and I think your concept of "end markers" is off. Every block has a pre-defined size which is determined by its format. When a block is mis-interpreted then the size automatically becomes wrong and the block "eats" extra bits from the next block or misses bits from its block shoving them into the next block. Occasionally these errors cancel each other out (because there is a small finite number of block types and sizes) and the streaming is "resynced" to the proper block ordering so you get a string of correct blocks. You often don't see this resync because after its resynced the blocks are still referencing all the previous corrupted blocks. This is why after you fix one block suddenly lots more blocks become readable IMO.

If michaelni could comment though that would probably clear more things up.

Edit: The key difference that matters from your explanation is that changing the start position of a block can _increase_ or _decrease_ how far away the stop position is for that block because of how the block is interpreted.
Very interesting! If that's true than that would probably help greatly in (semi-)automating this process! Because I had almost given up on that. So I'm very interested in an explanation of how this actually works.

Regards,

arnezami

[edit] When I look at my log though I almost do not see any of the same length of bitsizes per block. So I don't think macroblocks have any sort of fixed sizes.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/05/2014 06:36 am
<snip>

arnezami I've been talking to michaelni a lot today in learning how to do this process and I think your concept of "end markers" is off. Every block has a pre-defined size which is determined by its format. When a block is mis-interpreted then the size automatically becomes wrong and the block "eats" extra bits from the next block or misses bits from its block shoving them into the next block. Occasionally these errors cancel each other out (because there is a small finite number of block types and sizes) and the streaming is "resynced" to the proper block ordering so you get a string of correct blocks. You often don't see this resync because after its resynced the blocks are still referencing all the previous corrupted blocks. This is why after you fix one block suddenly lots more blocks become readable IMO.

If michaelni could comment though that would probably clear more things up.

Edit: The key difference that matters from your explanation is that changing the start position of a block can _increase_ or _decrease_ how far away the stop position is for that block because of how the block is interpreted.
Very interesting! If that's true than that would probably help greatly in (semi-)automating this process! Because I had almost given up on that. So I'm very interested in an explanation of how this actually works.

Regards,

arnezami

[edit] When I look at my log though I almost do not see any of the same length of bitsizes per block. So I don't think macroblocks have any sort of fixed sizes.

They're content-based sizes I think. Maybe I'm wrong. When michaelni gets up he can answer.

Just as a note, in my (or michaelni's repo) version of your code, re-compile while uncommenting the "//#define TRACE" variable in libavcodec/get_bits.h. It prints a bunch of info about the contents of every block as it decodes it (run with your debug option on).
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/05/2014 07:07 am
I need some "truth" data. Do we have any camera views from any other video that shows what the lower portion of the image (the rocket itself) looks like at all? I'm trying to figure out something remarkable about it so I can tell one block from another.

I need to order a bunch of puzzle pieces. Each has numbers on the bottom that need to be in order, they are perfectly square, varying flat white and there are either too few pieces or too many pieces and I need to either make some new ones or throw some away.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Digitalchromakey on 05/05/2014 07:09 am
<snip>

arnezami I've been talking to michaelni a lot today in learning how to do this process and I think your concept of "end markers" is off. Every block has a pre-defined size which is determined by its format. When a block is mis-interpreted then the size automatically becomes wrong and the block "eats" extra bits from the next block or misses bits from its block shoving them into the next block. Occasionally these errors cancel each other out (because there is a small finite number of block types and sizes) and the streaming is "resynced" to the proper block ordering so you get a string of correct blocks. You often don't see this resync because after its resynced the blocks are still referencing all the previous corrupted blocks. This is why after you fix one block suddenly lots more blocks become readable IMO.

If michaelni could comment though that would probably clear more things up.

Edit: The key difference that matters from your explanation is that changing the start position of a block can _increase_ or _decrease_ how far away the stop position is for that block because of how the block is interpreted.
Very interesting! If that's true than that would probably help greatly in (semi-)automating this process! Because I had almost given up on that. So I'm very interested in an explanation of how this actually works.

Regards,

arnezami

[edit] When I look at my log though I almost do not see any of the same length of bitsizes per block. So I don't think macroblocks have any sort of fixed sizes.
The MPEG 4 spec has the algorithm that allows you to parse the elementary visual bit stream, which can be of varying bit length, however the sequence_end_code x000001B7 is always byte aligned with defined stuffing bit codes preceding the end code depending on the number of bits required.

The sequence start codes are also similarly byte aligned.

Quote
Bits to be stuffed Stuffing Codeword
1 0
2 01
3 011
4 0111
5 01111
6 011111
7 0111111
8 01111111
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/05/2014 07:17 am
<snip>

arnezami I've been talking to michaelni a lot today in learning how to do this process and I think your concept of "end markers" is off. Every block has a pre-defined size which is determined by its format. When a block is mis-interpreted then the size automatically becomes wrong and the block "eats" extra bits from the next block or misses bits from its block shoving them into the next block. Occasionally these errors cancel each other out (because there is a small finite number of block types and sizes) and the streaming is "resynced" to the proper block ordering so you get a string of correct blocks. You often don't see this resync because after its resynced the blocks are still referencing all the previous corrupted blocks. This is why after you fix one block suddenly lots more blocks become readable IMO.

If michaelni could comment though that would probably clear more things up.

Edit: The key difference that matters from your explanation is that changing the start position of a block can _increase_ or _decrease_ how far away the stop position is for that block because of how the block is interpreted.
Very interesting! If that's true than that would probably help greatly in (semi-)automating this process! Because I had almost given up on that. So I'm very interested in an explanation of how this actually works.

Regards,

arnezami

[edit] When I look at my log though I almost do not see any of the same length of bitsizes per block. So I don't think macroblocks have any sort of fixed sizes.
The MPEG 4 spec has the algorithm that allows you to parse the elementary visual bit stream, which can be of varying bit length, however the sequence_end_code x000001B7 is always byte aligned with defined stuffing bit codes preceding the end code depending on the number of bits required.

The sequence start codes are also similarly byte aligned.

Quote
Bits to be stuffed Stuffing Codeword
1 0
2 01
3 011
4 0111
5 01111
6 011111
7 0111111
8 01111111

A link to the spec section in question?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Syrinx on 05/05/2014 07:26 am
I'm anxious to assist any way that I can.  I've got the custom ffmpeg built and installed (from the github link).  I'm a good engineer and programmer but I know next to nothing about MPEG or graphics in general.

I guess I'm confused about what I'm supposed to do now.  What do I give ffmpeg and what should I expect to get out?  Do I need to download the video or iframes from somewhere?  If so, I don't want to repair what you guys have already repaired.  How are you guys staying in sync?  Is there any coordination going on?

Would it be more productive for me to wait until I can assist in a more brute force fashion?  By that I mean using my eyeballs rather than my engineering skills.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/05/2014 07:31 am
I'm anxious to assist any way that I can.  I've got the custom ffmpeg built and installed (from the github link).  I'm a good engineer and programmer but I know next to nothing about MPEG or graphics in general.

I guess I'm confused about what I'm supposed to do now.  What do I give ffmpeg and what should I expect to get out?  Do I need to download the video or iframes from somewhere?  If so, I don't want to repair what you guys have already repaired.  How are you guys staying in sync?  Is there any coordination going on?

Would it be more productive for me to wait until I can assist in a more brute force fashion?  By that I mean using my eyeballs rather than my engineering skills.

Download the iframes linked previously. Then use the command that was listed to generate .png from that. Then modify the -mmb (the hard part) in various ways to make the image better. Format is listed in previous post as well: X, Y, block_start. Start by using an mmb that one of us mentioned and the given iframe and make sure you get the same image, then start manipulating those values to start teaching yourself what its doing.

./ffmpeg -mmb mmb_stuff_here -debug mb_pos_size -err_detect ignore_err -s:0 704:480 -i path/to/iframe/file/here -f image2 img.png
To see the original iframe just remove the -mmb option.

No coordination other than a couple of PMs between me and arnezami and this forum. I was also talking directly to michaelni on IRC in the #ffmpeg channel on freenode, but that was mostly to ask questions to learn about the format. Just assume that if no one's mentioned their work on a frame then no one has started on it yet. I'm personally stuck on iframe8 right now as I can't figure out the mess at the bottom. It's 3:40 AM here so I'll be heading to bed shortly, but I'll be resuming when I get up. No work or school tomorrow/today (Monday).

If enough people want to coordinate to work on it by tomorrow then I'll probably set up an IRC channel and we can coordinate/teach each other there live.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Syrinx on 05/05/2014 07:58 am
OK, thanks for the info.

The iframes can be downloaded from reply #89 in this thread.

I downloaded iframe 11, ran this:

ffmpeg -mmb 0:0:10020 -debug mb_pos_size -err_detect ignore_err -i iframes/iframe_11.mpg4-img -f image2 img-11.png

and got the same image that had been posted earlier.

Since there are only 12 iframes, does that mean there are only 12 images in the entire video?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jarnis on 05/05/2014 08:04 am
I need some "truth" data. Do we have any camera views from any other video that shows what the lower portion of the image (the rocket itself) looks like at all? I'm trying to figure out something remarkable about it so I can tell one block from another.

I need to order a bunch of puzzle pieces. Each has numbers on the bottom that need to be in order, they are perfectly square, varying flat white and there are either too few pieces or too many pieces and I need to either make some new ones or throw some away.

There are images from the rocketcam in this thread

http://forum.nasaspaceflight.com/index.php?topic=34502.105

However, I think these are actually taken from 2nd stage (higher up in the stack, slightly different angle) and a camera on the 1st stage was used only for this landing footage, so the image is not entirely same. You'd still get some idea as to what the rocket body looks like.

I do agree that it would probably help if SpaceX would release at least a couple of frames from the 1st stage camera from higher up. Reportedly they have good video of the braking burns but chose not to release it, perhaps to keep their secret sauce secret.

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jakusb on 05/05/2014 08:07 am
I need some "truth" data. Do we have any camera views from any other video that shows what the lower portion of the image (the rocket itself) looks like at all? I'm trying to figure out something remarkable about it so I can tell one block from another.

I need to order a bunch of puzzle pieces. Each has numbers on the bottom that need to be in order, they are perfectly square, varying flat white and there are either too few pieces or too many pieces and I need to either make some new ones or throw some away.

There are images from the rocketcam in this thread

http://forum.nasaspaceflight.com/index.php?topic=34502.105

However, I think these are actually taken from 2nd stage (higher up in the stack, slightly different angle) and a camera on the 1st stage was used only for this landing footage, so the image is not entirely same. You'd still get some idea as to what the rocket body looks like.

I do agree that it would probably help if SpaceX would release at least a couple of frames from the 1st stage camera from higher up. Reportedly they have good video of the braking burns but chose not to release it, perhaps to keep their secret sauce secret.

Maybe something for Chris to pitch in? :)

Edit/Lar: I PMed Chris. If you want to make sure Chris sees it, PMing him is a better approach than just mentioning his name in a post. He's godlike but not omniescent :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/05/2014 08:18 am
I think I got a bit more of iframe 8, hard to tell what the correct alignment is.

ffmpeg -mmb 07:14:17233,14:14:57111,08:15:17233,09:15:63546,19:16:17233,21:16:73568,08:21:17233,00:22:111664 -debug mb_pos_size -err_detect ignore_err -s:0 704:480 -i iframe_8.mpg4-img -f image2 img-%03d.png
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Digitalchromakey on 05/05/2014 08:27 am
<snip>

arnezami I've been talking to michaelni a lot today in learning how to do this process and I think your concept of "end markers" is off. Every block has a pre-defined size which is determined by its format. When a block is mis-interpreted then the size automatically becomes wrong and the block "eats" extra bits from the next block or misses bits from its block shoving them into the next block. Occasionally these errors cancel each other out (because there is a small finite number of block types and sizes) and the streaming is "resynced" to the proper block ordering so you get a string of correct blocks. You often don't see this resync because after its resynced the blocks are still referencing all the previous corrupted blocks. This is why after you fix one block suddenly lots more blocks become readable IMO.

If michaelni could comment though that would probably clear more things up.

Edit: The key difference that matters from your explanation is that changing the start position of a block can _increase_ or _decrease_ how far away the stop position is for that block because of how the block is interpreted.
Very interesting! If that's true than that would probably help greatly in (semi-)automating this process! Because I had almost given up on that. So I'm very interested in an explanation of how this actually works.

Regards,

arnezami

[edit] When I look at my log though I almost do not see any of the same length of bitsizes per block. So I don't think macroblocks have any sort of fixed sizes.
The MPEG 4 spec has the algorithm that allows you to parse the elementary visual bit stream, which can be of varying bit length, however the sequence_end_code x000001B7 is always byte aligned with defined stuffing bit codes preceding the end code depending on the number of bits required.

The sequence start codes are also similarly byte aligned.

Quote
Bits to be stuffed Stuffing Codeword
1 0
2 01
3 011
4 0111
5 01111
6 011111
7 0111111
8 01111111

A link to the spec section in question?
ISO/IEC 144696-2 Section 6 Video Bitstream Syntax and Semantics.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: grythumn on 05/05/2014 08:30 am
Some more notes:

The macro blocks are 16x16. Sometimes you want to replace a block that isn't throwing an error because it messes up the image, and that helps work out what number to mess with.

Work sequentially, if possible... changing stuff early in the picture can cause green blocks to reappear in what was junky, but clear, picture.

Some green blocks just won't go away.

All I've gotten from frame 4 is the top of the timestamp, at position 119316:

-Bob
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Syrinx on 05/05/2014 09:07 am
I think I understand what's going on now.

There are 44 * 30 macroblocks in the picture.  ffmpeg is dumping the x-y locations of the macroblocks, and the byte offset and byte size of each macroblock.

E.g.,

[mpeg4 @ 0x60007a520] MB pos/size: 0 30:00:12347 104
[mpeg4 @ 0x60007a520] MB pos/size: 0 31:00:12451 53
[mpeg4 @ 0x60007a520] MB pos/size: 0 32:00:12504 91
[mpeg4 @ 0x60007a520] MB pos/size: 0 33:00:12595 116
[mpeg4 @ 0x60007a520] MB pos/size: 0 34:00:12711 77
[mpeg4 @ 0x60007a520] MB pos/size: 0 35:00:12788 133
[mpeg4 @ 0x60007a520] MB pos/size: 0 36:00:12921 73
[mpeg4 @ 0x60007a520] MB pos/size: 0 37:00:12994 95
[mpeg4 @ 0x60007a520] MB pos/size: 0 38:00:13089 110
[mpeg4 @ 0x60007a520] ac-tex damaged at 39 0
[mpeg4 @ 0x60007a520] MB pos/size: -1 39:00:13199 7


The fields are:

[mpeg4 @ 0x60007a520] MB pos/size: return-code x:y:offset size]

You guys are manually looking at the picture, picking out a familiar macroblock and placing it at the correct x-y location using the -mmb override.  When a macroblock is placed at the correct x-y location, it might also make other macroblocks snap in to place.  Then you repeat for another macroblock.

A return code of -1 means ffmpeg couldn't decode the macroblock.  0 means success, but it doesn't necessarily mean the macroblock is in the correct x-y location.

Is that all correct?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/05/2014 09:52 am
I need some "truth" data. Do we have any camera views from any other video that shows what the lower portion of the image (the rocket itself) looks like at all? I'm trying to figure out something remarkable about it so I can tell one block from another.

I need to order a bunch of puzzle pieces. Each has numbers on the bottom that need to be in order, they are perfectly square, varying flat white and there are either too few pieces or too many pieces and I need to either make some new ones or throw some away.

There are images from the rocketcam in this thread

http://forum.nasaspaceflight.com/index.php?topic=34502.105

However, I think these are actually taken from 2nd stage (higher up in the stack, slightly different angle) and a camera on the 1st stage was used only for this landing footage, so the image is not entirely same. You'd still get some idea as to what the rocket body looks like.

I do agree that it would probably help if SpaceX would release at least a couple of frames from the 1st stage camera from higher up. Reportedly they have good video of the braking burns but chose not to release it, perhaps to keep their secret sauce secret.

Na, I need an image from the lower camera. Preferably during retroburn or something, but any time works.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/05/2014 10:00 am
I think I understand what's going on now.

There are 44 * 30 macroblocks in the picture.  ffmpeg is dumping the x-y locations of the macroblocks, and the byte offset and byte size of each macroblock.

E.g.,

[mpeg4 @ 0x60007a520] MB pos/size: 0 30:00:12347 104
[mpeg4 @ 0x60007a520] MB pos/size: 0 31:00:12451 53
[mpeg4 @ 0x60007a520] MB pos/size: 0 32:00:12504 91
[mpeg4 @ 0x60007a520] MB pos/size: 0 33:00:12595 116
[mpeg4 @ 0x60007a520] MB pos/size: 0 34:00:12711 77
[mpeg4 @ 0x60007a520] MB pos/size: 0 35:00:12788 133
[mpeg4 @ 0x60007a520] MB pos/size: 0 36:00:12921 73
[mpeg4 @ 0x60007a520] MB pos/size: 0 37:00:12994 95
[mpeg4 @ 0x60007a520] MB pos/size: 0 38:00:13089 110
[mpeg4 @ 0x60007a520] ac-tex damaged at 39 0
[mpeg4 @ 0x60007a520] MB pos/size: -1 39:00:13199 7


The fields are:

[mpeg4 @ 0x60007a520] MB pos/size: return-code x:y:offset size]

You guys are manually looking at the picture, picking out a familiar macroblock and placing it at the correct x-y location using the -mmb override.  When a macroblock is placed at the correct x-y location, it might also make other macroblocks snap in to place.  Then you repeat for another macroblock.

A return code of -1 means ffmpeg couldn't decode the macroblock.  0 means success, but it doesn't necessarily mean the macroblock is in the correct x-y location.

Is that all correct?

We're not placing macroblocks at the correct locations as is replacing corrupted macroblocks with blocks from _elsewhere_ in the image. Every macroblock in the image is a reference+changes to either the block directly above it or the block directly to the left (this choice is made when the video stream is created). If you replace a macroblock with another block that has "roughly" the same "+changes" as the actual block then the created block will look roughly like the original and blocks that reference it will look roughly how they are supposed to.

In some cases where things are too messed up we look for a macroblock that has changes that we know should be at a specific location (the legs) and forcibly put it there. That doesn't fix any previous blocks, but as you mention it often makes other blocks that follow it snap into place. Ideally after you put that block in place you should work backwards (right to left, bottom to top) toward the other bad blocks trying to replace those as well while maintaining the same "block address" of the block you fixed.

Everything else is correct.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/05/2014 10:03 am
Since there are only 12 iframes, does that mean there are only 12 images in the entire video?

There are 12 iframes which are complete images that reference no previous images. Every other frame is a reference to one of those frames. To be specific, any frames in between two iframes reference the previous iframe, other frames that also reference that iframe or pieces of the frame reference other pieces of the same frame. Welcome to the world of high-compression video codecs. :P Errors propagate like hell.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/05/2014 10:26 am
Na, I need an image from the lower camera. Preferably during retroburn or something, but any time works.

This picture of iframe2 contains data at the (nearly) bottom:

 http://forum.nasaspaceflight.com/index.php?topic=34597.msg1193527#msg1193527 (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1193527#msg1193527)

Note that the clock is the lowest block. So this is what you want I think. It's just that everything in this picture is moved up because I "fast-forwarded" at some point (essentially skipped like 4 rows). So especially the left is what you want. And the stage is probable symetrical. So could be useful.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/05/2014 10:37 am
Na, I need an image from the lower camera. Preferably during retroburn or something, but any time works.

Maybe you can check the Cassiope launch highlights movie (at 2:40) where they show the reentry burn: http://forum.nasaspaceflight.com/index.php?topic=32946.msg1109241#msg1109241
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: fatdeeman on 05/05/2014 12:53 pm
Absolutely in awe of what's being achieved here. I had heard before that videos with errors still contained a lot of usable information but I had no idea it could be retrieved to this extent. I had visions of having to reconstruct entire frames by moving blocks around like a puzzle and guessing if they were in the right place.

Absolutely fantastic work being done here!

One more thing, Elon mentioned fan sourcing helped restore video footage before. Does any one know of any examples?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Asmegin on 05/05/2014 01:33 pm
Absolutely in awe of what's being achieved here. I had heard before that videos with errors still contained a lot of usable information but I had no idea it could be retrieved to this extent. I had visions of having to reconstruct entire frames by moving blocks around like a puzzle and guessing if they were in the right place.

Absolutely fantastic work being done here!

One more thing, Elon mentioned fan sourcing helped restore video footage before. Does any one know of any examples?

+1 to being amazed. I wish I could help, but I literally have no idea what you guys are talking about :D

As for the video, it's possible he was referencing the Mars Curiosity Landing video which was amazing after someone took 4 weeks to fix it: https://www.youtube.com/watch?v=Esj5juUzhpU
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: fatdeeman on 05/05/2014 01:46 pm
Absolutely in awe of what's being achieved here. I had heard before that videos with errors still contained a lot of usable information but I had no idea it could be retrieved to this extent. I had visions of having to reconstruct entire frames by moving blocks around like a puzzle and guessing if they were in the right place.

Absolutely fantastic work being done here!

One more thing, Elon mentioned fan sourcing helped restore video footage before. Does any one know of any examples?

+1 to being amazed. I wish I could help, but I literally have no idea what you guys are talking about :D

As for the video, it's possible he was referencing the Mars Curiosity Landing video which was amazing after someone took 4 weeks to fix it: https://www.youtube.com/watch?v=Esj5juUzhpU

Ah yes I was watching this the other day funnily enough but it didn't click! The amount of detail in this one compared to the version NASA released is incredible!

I couldn't do what these guys are doing either! A lot of the things being said make sense to me but as far as doing it myself I wouldn't have a clue! I'd be happy if there was anything slightly less perplexing like cheap labour!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: llanitedave on 05/05/2014 04:51 pm
Wow, all that and sound too!  Beautiful!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jeff Lerner on 05/05/2014 05:21 pm
I also want to say how amazed I am at the effort and progress being made with this video.

Having said that, I thought I'd toss in my $0.02 as a retired IT project manager of 38 years. I often  found that the  developers working for me would get so caught up developing their code that they has no time to see the bigger picture...that's where I came in as PM..

So,having said that, I'm wondering if anyone is aware of any other efforts on any other space related sites, photo or video sites or for that matter, even at SpaceX, to answer Elon's call to arms to fix up this video...I'm just thinking that it would be sad to have all this great effort wasted because some where else, someone else has already done this work.....

...and thats maybe where Chris can help out as he has some contacts at SpaceX and can inquire as to whether they are aware of any other progress being made...if little else, Chris could at least make SpaceX know that people on this site are working on a solution and might require some info from SpaceX.....
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Syrinx on 05/05/2014 05:46 pm
We're not placing macroblocks at the correct locations as is replacing corrupted macroblocks with blocks from _elsewhere_ in the image. Every macroblock in the image is a reference+changes to either the block directly above it or the block directly to the left (this choice is made when the video stream is created). If you replace a macroblock with another block that has "roughly" the same "+changes" as the actual block then the created block will look roughly like the original and blocks that reference it will look roughly how they are supposed to.

In some cases where things are too messed up we look for a macroblock that has changes that we know should be at a specific location (the legs) and forcibly put it there. That doesn't fix any previous blocks, but as you mention it often makes other blocks that follow it snap into place. Ideally after you put that block in place you should work backwards (right to left, bottom to top) toward the other bad blocks trying to replace those as well while maintaining the same "block address" of the block you fixed.

Everything else is correct.

Is it really just a matter of finding a suitable stand-in for corrupt macroblocks?  Maybe the process is not so bad then!

But I was looking at iframe 11.  Macroblock 0:0 is valid (not corrupt) as reported by the debug info, but somebody replaced it anyway (with 5:0 I think) and it greatly improved the image.

(Maybe I am mistaken.  Maybe iframe 11 0:0 is corrupt.  I would check but I'm not on my home computer right now.)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: swervin on 05/05/2014 05:48 pm
Perhaps the folks who assisted in the cleaned up Curiosity video could be contacted for 'lessons learned' on that video? ... Or even assist on this one if they were interested.

I thought it would be cool if the 'code' was broken on how to clean this up, if that was made available, a la SETI @ home, to use the computing power of a bunch of computers to clean it up faster. Then again, maybe not much computing power is req'd.

This coming from someone who has no idea how to do this ... Haha

Cheers,
Splinter
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: grythumn on 05/05/2014 05:52 pm
I also want to say how amazed I am at the effort and progress being made with this video.

Having said that, I thought I'd toss in my $0.02 as a retired IT project manager of 38 years. I often  found that the  developers working for me would get so caught up developing their code that they has no time to see the bigger picture...that's where I came in as PM..

So,having said that, I'm wondering if anyone is aware of any other efforts on any other space related sites, photo or video sites or for that matter, even at SpaceX, to answer Elon's call to arms to fix up this video...I'm just thinking that it would be sad to have all this great effort wasted because some where else, someone else has already done this work.....

Aside from the youtube video, only attempt I've seen is:

temp bana/dtZOQ

That's a photoshop, not actual data.

-Bob
EDIT: Apparently I can't link to it. Try an imaging site that begins with an i and ends with an r. Popular on facebook and such.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Lars_J on 05/05/2014 05:52 pm
Perhaps the folks who assisted in the cleaned up Curiosity video could be contacted for 'lessons learned' on that video? ... Or even assist on this one if they were interested.

The Curiosity video is a completely different scenario. No cleanup of re-interpretation of missing data was needed - and if it was, not anywhere near this level.

The real achievement of the Curiosity video was the smooth frame interpolation from still frames to create the illusion of a full frame rate video.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/05/2014 05:56 pm
I'm truly amazed on the outstanding work that is being done in recovering the I-frames. Kudos to arnezami for developing the approach, and for everyone else that has been involved. Even if only those 12 images are recovered, I'm sure they are going to be really valuable for Space X.

Just a question: Is anybody collecting/monitoring the progress on this task? As several of you are working on parallel, I believe a monitoring tool would be useful in order to lower the risk of having several individuals doing the same thing. 

To start, may I suggest on keeping a table like the one originally describing the I-Frames (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1193316#msg1193316) in order for everyone to know who is working on each one?


I'll be glad to take that responsibility if you, the guys actually working on the images,  find it useful...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: neoforce on 05/05/2014 05:57 pm
Perhaps the folks who assisted in the cleaned up Curiosity video could be contacted for 'lessons learned' on that video? ... Or even assist on this one if they were interested.

The Curiosity video is a completely different scenario. No cleanup of re-interpretation of missing data was needed - and if it was, not anywhere near this level.

The real achievement of the Curiosity video was the smooth frame interpolation from still frames to create the illusion of a full frame rate video.

Some additional details on the differences... 

In the Falcon 9 video, we have compression that lost blocks due to transmission fall out.  In the Curiosity video, it was by design 4 frames per second.  All of the data for each of those 4 frames per second were recovered. So what he did was "create" an additional 26 frames for each second of footage to get it to 30 frames per second.

It is a very different problem.  His was enhancing, adding full frames between existing frames of complete information.  In this thread, here they are trying to repair data within each frame of the video.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/05/2014 06:00 pm
<snip>

arnezami I've been talking to michaelni a lot today in learning how to do this process and I think your concept of "end markers" is off. Every block has a pre-defined size which is determined by its format. When a block is mis-interpreted then the size automatically becomes wrong and the block "eats" extra bits from the next block or misses bits from its block shoving them into the next block. Occasionally these errors cancel each other out (because there is a small finite number of block types and sizes) and the streaming is "resynced" to the proper block ordering so you get a string of correct blocks. You often don't see this resync because after its resynced the blocks are still referencing all the previous corrupted blocks. This is why after you fix one block suddenly lots more blocks become readable IMO.

If michaelni could comment though that would probably clear more things up.

Edit: The key difference that matters from your explanation is that changing the start position of a block can _increase_ or _decrease_ how far away the stop position is for that block because of how the block is interpreted.
Hi,

I think I found most the answer to this question.

In the MPEG 4-2 specification ( I googled for "Video Bitstream Syntax and Semantics" and found a doc file called "MPEG-2_Video_IS.doc") its says in section 6.2.6 that blocks end with "End of block"-bits (10b I believe). Later on in section 8.2 it says that macroblocks will end with the "End of block"-symbol of its last block. We have 4 blocks in a macroblock I believe. See the attachments.

So I think my intuition of the macroblocks ending with stop-bits ("end markers") is pretty spot on, albeit with a little nuance that the stop-bits are in the blocks not in the macroblock. The document also gives a lot of info on bits that may be recognizable somehow. Not sure yet how we can use this for semi-automation.

Regards,

arnezami

[edit] I'm sorry this is of course MPEG-2. Duh. This probably does not apply to MPEG-4.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: vanderhilst on 05/05/2014 06:12 pm
Hi all, I'm really enjoying to learn from the inspiring progress that is made by all you guys,
as well as by the efforts of SpaceX to create this video in the first place!!

As an experienced programmer in Mathematica, I wasn't able to chip in up to this point, but it seems to me that you have come to a point where automated calls to ffmpeg may be beneficial to the process of recovering the I-frames. I'm willing to jump in, and have two questions:

1) Did anybody listen to Jeff Lerner's comments, reply #132?

2) Is it sensible to use this way to have ffmpeg run under windows? http://www.wikihow.com/Install-FFmpeg-on-Windows

Thanks and enjoy the revealing!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/05/2014 06:19 pm
I was toying with the idea of creating a web app so that people could do this visually and would make it easier to distribute the workload. I think it would be a good idea but I also think that by the time it's done people will have managed to get most of the way there. I don't know if something like that would have any re-use value save for something like this happening again.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/05/2014 06:25 pm
Aha!

I think I found a way to detect the beginning of macroblocks! :)

They seem to have a fixed 11-bit macroblock_escape code (0000000100b) followed by an 1-11 bit macroblock_address_increment! (very useful for detecting where the macroblock should be). If this is actually true then finding the start of macroblocks might actually become quite easy. And its even followed by a macroblock_type of 1-9 bits. That's a lot of bits to be non-random.

See attachments.

Regards,

arnezami

[edit] I'm sorry this is of course MPEG-2. Duh. This probably does not apply to MPEG-4.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: laszlo on 05/05/2014 06:25 pm
can someone please explain to me why spacex still hasn't released the footage they got from their airplane? Why are they being so secretive about it? I mean seeing is believing so why not just release the footage and prove to everyone in the world they really accomplished such an unprecedented feat?

I mean they did actually record it from their airplane right?

The left landing gear didn't deploy and they didn't want the embarrassment of showing it skidding off the runway. Oh, wait. Wrong landing. My bad.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Syrinx on 05/05/2014 06:26 pm
2) Is it sensible to use this way to have ffmpeg run under windows? http://www.wikihow.com/Install-FFmpeg-on-Windows

First, you need to install the custom ffmpeg version that's been made available in this thread on github.  You probably already knew that, but I wanted to make it explicit.

I didn't try the install mechanism presented at that link.  I installed mine on top of Cygwin.  It look about an hour and I had no problems, but of course you also need to install Cygwin if you don't have it.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: grythumn on 05/05/2014 06:56 pm
2) Is it sensible to use this way to have ffmpeg run under windows? http://www.wikihow.com/Install-FFmpeg-on-Windows

This might work without a full cygwin install (at least cygcheck thinks I got all the DLLs). But I recommend it anyway, or at least some sort of improved shell.

EDIT: This is a command line utility. Please only use if you know how to use such.

-Bob
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: vanderhilst on 05/05/2014 07:10 pm
Yay! Thank you both, Syrinx and Bob!

The ffmpeg as downloadable in the post by Bob above, combined with the method in the wikihow link I posted a few posts earlier, actually allow me to get a correct response to a command line command "ffmpeg -version" under 64-bit Windows 7.

I will proceed to try and run the other commands explained before to recreate the results posted by others.
If successful, I will try to see whether automation will do anything useful. I will start with I-frame 11, if any.

edit: for clarity: I did NOT install Cygwin.  :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: GraniteHound92 on 05/05/2014 07:23 pm
I asked Doug Ellison, the JPL guy who did the MSL descent video, if he had any tips.

@danieljphilo:@doug_ellison They're trying to improve the SpaceX first stage landing video at http://nasaspaceflight.com  Any tips? http://forum.nasaspaceflight.com/index.php?topic=34597.135 …

@doug_ellison:@danieljphilo Nope. Beyond my skills.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jarnis on 05/05/2014 07:29 pm
I asked Doug Ellison, the JPL guy who did the MSL descent video, if he had any tips.

@danieljphilo:@doug_ellison They're trying to improve the SpaceX first stage landing video at http://nasaspaceflight.com  Any tips? http://forum.nasaspaceflight.com/index.php?topic=34597.135 …

@doug_ellison:@danieljphilo Nope. Beyond my skills.

Not surprised. MSL video was about doing high quality interpolation from fairly good keyframes.

Here the keyframes are in tatters and you really need to do painstaking manual reconstruction...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/05/2014 07:31 pm
I asked Doug Ellison, the JPL guy who did the MSL descent video, if he had any tips.

@danieljphilo:@doug_ellison They're trying to improve the SpaceX first stage landing video at http://nasaspaceflight.com  Any tips? http://forum.nasaspaceflight.com/index.php?topic=34597.135 …

@doug_ellison:@danieljphilo Nope. Beyond my skills.

Top man. Met him in real life and really happy he ended up at JPL.

Keep up the good work here. It's all over my head, but clearly a hardcore process. Very impressive!

Remember, any big breakthrough, send me a message and I'll fast track you to someone at SpaceX. There's probably a free ticket to ride on Dragon in it for a win! ;)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Syrinx on 05/05/2014 07:41 pm
We're not placing macroblocks at the correct locations as is replacing corrupted macroblocks with blocks from _elsewhere_ in the image. Every macroblock in the image is a reference+changes to either the block directly above it or the block directly to the left (this choice is made when the video stream is created). If you replace a macroblock with another block that has "roughly" the same "+changes" as the actual block then the created block will look roughly like the original and blocks that reference it will look roughly how they are supposed to.

In some cases where things are too messed up we look for a macroblock that has changes that we know should be at a specific location (the legs) and forcibly put it there. That doesn't fix any previous blocks, but as you mention it often makes other blocks that follow it snap into place. Ideally after you put that block in place you should work backwards (right to left, bottom to top) toward the other bad blocks trying to replace those as well while maintaining the same "block address" of the block you fixed.

Everything else is correct.

Is it really just a matter of finding a suitable stand-in for corrupt macroblocks?  Maybe the process is not so bad then!

But I was looking at iframe 11.  Macroblock 0:0 is valid (not corrupt) as reported by the debug info, but somebody replaced it anyway (with 5:0 I think) and it greatly improved the image.

(Maybe I am mistaken.  Maybe iframe 11 0:0 is corrupt.  I would check but I'm not on my home computer right now.)

OK, I double-checked and macroblock 0:0 is indeed valid:

[mpeg4 @ 0x80062980] MB pos/size: 0 00:00:10020 128
[mpeg4 @ 0x80062980] MB pos/size: -1 01:00:10148 102
[mpeg4 @ 0x80062980] Error at MB: 1
[mpeg4 @ 0x80062980] MB pos/size: 0 02:00:10250 109
[mpeg4 @ 0x80062980] I cbpy damaged at 3 0
[mpeg4 @ 0x80062980] MB pos/size: -1 03:00:10359 4
[mpeg4 @ 0x80062980] Error at MB: 3
[mpeg4 @ 0x80062980] I cbpc damaged at 4 0
[mpeg4 @ 0x80062980] MB pos/size: -1 04:00:10363 6
[mpeg4 @ 0x80062980] Error at MB: 4
[mpeg4 @ 0x80062980] MB pos/size: 0 05:00:10369 102
[mpeg4 @ 0x80062980] MB pos/size: 0 06:00:10471 64
[mpeg4 @ 0x80062980] MB pos/size: 0 07:00:10535 92
[mpeg4 @ 0x80062980] MB pos/size: 0 08:00:10627 52
[mpeg4 @ 0x80062980] MB pos/size: 0 09:00:10679 64


So should I "simply" find suitable stand-ins for corrupt macroblocks?  Should I replace (some) valid macroblocks?  Both?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/05/2014 08:01 pm
OK, I double-checked and macroblock 0:0 is indeed valid:

[mpeg4 @ 0x80062980] MB pos/size: 0 00:00:10020 128
[mpeg4 @ 0x80062980] MB pos/size: -1 01:00:10148 102
[mpeg4 @ 0x80062980] Error at MB: 1
[mpeg4 @ 0x80062980] MB pos/size: 0 02:00:10250 109
[mpeg4 @ 0x80062980] I cbpy damaged at 3 0
[mpeg4 @ 0x80062980] MB pos/size: -1 03:00:10359 4
[mpeg4 @ 0x80062980] Error at MB: 3
[mpeg4 @ 0x80062980] I cbpc damaged at 4 0
[mpeg4 @ 0x80062980] MB pos/size: -1 04:00:10363 6
[mpeg4 @ 0x80062980] Error at MB: 4
[mpeg4 @ 0x80062980] MB pos/size: 0 05:00:10369 102
[mpeg4 @ 0x80062980] MB pos/size: 0 06:00:10471 64
[mpeg4 @ 0x80062980] MB pos/size: 0 07:00:10535 92
[mpeg4 @ 0x80062980] MB pos/size: 0 08:00:10627 52
[mpeg4 @ 0x80062980] MB pos/size: 0 09:00:10679 64


So should I "simply" find suitable stand-ins for corrupt macroblocks?  Should I replace (some) valid macroblocks?  Both?

First. I'm not sure you have already but if not please read/look at this fairly thourough explanation of how it all works:

http://forum.nasaspaceflight.com/index.php?topic=34597.msg1193685#msg1193685 (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1193685#msg1193685)

Second. I did intentionally replace 0,0 with a different block. But this is just for experimentation.

The reason why 0,0 has advantages is that there are no blocks above or left of it. How mpeg-4 works is that a block "inherits" the traits of the blocks left and above it. So if you're at 0,0 there is no corruption from other blocks before that.

You have to understand that in mpeg-4 everything is chained somehow: vertical, horizontal and time-wise. But its all chained in one direction: left to right, top to bottom, previous to next. This really saves in the number of bytes it takes to store/transmit a video but it also makes it very hard to reconstruct it when something is only slightly off, because a lot of other things depend on it being right: errors propagate rapidly. This is the difficulty we are trying to overcome.

But when you start a block at 0,0 you eliminate most of all this domino-effect troubles. You lose the information what you would have gotten from a top and left block when you started at your normal x and y, but its just to see which blocks are good or bad. Which is much easier to determine from 0,0. So I just used 0,0 to prove this point. Don't worry about it. The first block from 0,0 might be correct and completely ok. But I didn't care for that in this particular experiment: I wanted a starting block with no left and upper blocks! And there is only one block in the game that has that: 0,0.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/05/2014 08:04 pm
We're not placing macroblocks at the correct locations as is replacing corrupted macroblocks with blocks from _elsewhere_ in the image. Every macroblock in the image is a reference+changes to either the block directly above it or the block directly to the left (this choice is made when the video stream is created). If you replace a macroblock with another block that has "roughly" the same "+changes" as the actual block then the created block will look roughly like the original and blocks that reference it will look roughly how they are supposed to.

In some cases where things are too messed up we look for a macroblock that has changes that we know should be at a specific location (the legs) and forcibly put it there. That doesn't fix any previous blocks, but as you mention it often makes other blocks that follow it snap into place. Ideally after you put that block in place you should work backwards (right to left, bottom to top) toward the other bad blocks trying to replace those as well while maintaining the same "block address" of the block you fixed.

Everything else is correct.

Is it really just a matter of finding a suitable stand-in for corrupt macroblocks?  Maybe the process is not so bad then!

But I was looking at iframe 11.  Macroblock 0:0 is valid (not corrupt) as reported by the debug info, but somebody replaced it anyway (with 5:0 I think) and it greatly improved the image.

(Maybe I am mistaken.  Maybe iframe 11 0:0 is corrupt.  I would check but I'm not on my home computer right now.)

"Suitable" is still rather difficult though. I personally don't have a good grasp of "suitable" other than trial and error.

I haven't looked at the image in question, but often a block is corrupted such that its still has valid data. Corruption is not binary. That means a 1 bit change could just slightly tweak the color for example and would still be completely valid. I've been looking into making some more modifications to allow for overriding from the command line the contents of a block so we can try to fix the block rather than replace it.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: didacticus on 05/05/2014 08:55 pm
Very interesting reading all the discussion here! Just wanted to say I appreciate the effort you are all putting in to retrieve this video. Good luck!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: LucR on 05/05/2014 09:34 pm
I think I found a way to detect the beginning of macroblocks! :)
They seem to have a fixed 11-bit macroblock_escape code (0000000100b) followed by an 1-11 bit macroblock_address_increment!

Sorry, the macroblock_escape won't help for decoding I-frames, because:
Quote from: MPEG-2_Video_IS.doc section 6.3.17 Macroblock
macroblock_escape -- The macroblock_escape is a fixed bit-string ‘0000 0001 000’ which is used when the difference between macroblock_address and previous_macroblock_address is greater than 33. It causes the value of macroblock_address_increment to be 33 greater than the value that will be decoded by subsequent macroblock_escape and the macroblock_address_increment codewords.
In an I-frame all macroblocks are typically coded (i.e. not skipped, "There shall be no skipped macroblocks in I-pictures except [...]" at the end of section 6.3.7) so there will be no occurrences of macroblock_escape.

However, in I-frames the immediately following macroblock_address_increment code should encode a value of 1 which are coded as the bitstream bits 0000 0101 01 (from Table B-1 --- Variable length codes for macroblock_address_increment).
This is still 10 bits of constant data, hope that helps some.

If there is no spatial scalability, Table B-2 applies for the immediately following macroblock_type code, which means that it is either 1 (Intra) or 01 (Intra, Quant); another 1 or 2 constant bits.

I haven't looked at the actual bitstreams (yet); are slices start codes not used? This would make it easier to restart part-way through a frame.

By the way, for background: I have some amount of MPEG experience but it's not at the macroblock level. However, I do know how to read the bitstream syntax specs.

Edit: Added note about macroblock_type.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/05/2014 09:39 pm
In figuring out good methods, here's how I'm doing it.

I use the CLI line: ./ffmpeg -mmb `cat mmb.txt` -debug mb_pos_size -err_detect ignore_err -s:0 704:480 -i iframes/iframe_8.mpg4-img -f image2 img-001.png && open img-001.png

I use ` ` to pipe the text document into the CLI so I can rapidly change it and the open command (Mac OS X specific) to open the image.

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/05/2014 09:43 pm
I think I found a way to detect the beginning of macroblocks! :)
They seem to have a fixed 11-bit macroblock_escape code (0000000100b) followed by an 1-11 bit macroblock_address_increment!

Sorry, the macroblock_escape won't help for decoding I-frames, because:
Quote from: MPEG-2_Video_IS.doc section 6.3.17 Macroblock
macroblock_escape -- The macroblock_escape is a fixed bit-string ‘0000 0001 000’ which is used when the difference between macroblock_address and previous_macroblock_address is greater than 33. It causes the value of macroblock_address_increment to be 33 greater than the value that will be decoded by subsequent macroblock_escape and the macroblock_address_increment codewords.
In an I-frame all macroblocks are typically coded (i.e. not skipped, "There shall be no skipped macroblocks in I-pictures except [...]" at the end of section 6.3.7) so there will be no occurrences of macroblock_escape.

However, in I-frames the immediately following macroblock_address_increment code should encode a value of 1 which are coded as the bitstream bits 0000 0101 01 (from Table B-1 --- Variable length codes for macroblock_address_increment).
This is still 10 bits of constant data, hope that helps.

I haven't looked at the actual bitstreams (yet); are slices start codes not used? This would make it easier to restart part-way through a frame.

By the way, for background: I have some amount of MPEG experience but it's not at the macroblock level. However, I do know how to read the bitstream syntax specs.

Thanks for the tips. I don't know MPEG as well as you so if you could investigate and find that out and see if that works that'd be awesome!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/05/2014 09:47 pm
I am also really impressed by the work done here.

Attached you find a version of the ISO/IEC 14496 which defines the MPEG 4-2 standard.
(Edit: all these sections and tables and stuff mentioned in LucR's posting are found in there)
If I read it correctly there are no fixed end-codes of a block or marcoblock, but I only jump through the document.
Llke a good specification document it is really hard to read, at least for me and my knowledge of not too many such documents ;)
Maybe it helps to understand more of the problem.

Good Luck with finding the right bits :D

Shanuson
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/05/2014 10:03 pm
mlindner was asking for a non-corrupted image, so this one comes from the reentry burn of Cassiope launch

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: LucR on 05/05/2014 10:08 pm
If I read it correctly there are no fixed end-codes of a block or marcoblock, but I only jump through the document.
AFAIK this is correct, the "End of block" code varies, it can be either 10 (Table B-14) or 0110 (Table B-15), as described in 7.2.2 Other coefficients. In either case the bitstring is probably too short to provide for meaningful recovery.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Adaptation on 05/05/2014 10:37 pm
I updated my Transport stream tool.  It's probably not going to help much at this point but it does have a hex editor and powerful search feature with a bit level wildcard filter, ability to return results with a specified tolerated number of single bit errors and the results are easy to visualize and locate. 

To do:
Disable controls until a ts file is opened.
Make the search capable of finding results that are not byte aligned. 
Add a replace feature with a wildcard filter.
Enable bit level insertion and deletion ability. (would bit shift everything after that point in the file)


I think its a good companion tool for this transport stream editor. 
http://www.videohelp.com/tools/TS-Packet-Editor

-Update, version 3 is out-
http://forum.nasaspaceflight.com/index.php?topic=34597.msg1197014#msg1197014
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: TrueBlueWitt on 05/05/2014 10:48 pm
I feel like we're missing an important piece of this puzzle.

Was the video data stream was on it's own channel, or there was other data being transmitted on the same channel with some other controls in place to keep the data segregated?

If it was on it's own channel, having an analogue(time domain) copy of the original transmission, perhaps even with time code synced to the video file provided, there might be a lot more that could be done to extract the information.  Having the the time domain data stream it would also be much easier to tell where and when data was dropping out.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: djellison on 05/05/2014 10:52 pm
I asked Doug Ellison, the JPL guy who did the MSL descent video, if he had any tips

While I did some MSL MARDI vid stuff - I was never involved in any of the gorgeous interpolated and rotoscoped versions that some really creative people came up with. 

Impressed people are making some progress with the F9 1.1 feed - that's some seriously junky data!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: lgjy98d on 05/06/2014 12:13 am
Just a question: Is anybody collecting/monitoring the progress on this task? As several of you are working on parallel, I believe a monitoring tool would be useful in order to lower the risk of having several individuals doing the same thing. 

To start, may I suggest on keeping a table like the one originally describing the I-Frames (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1193316#msg1193316) in order for everyone to know who is working on each one?

There is wiki at http://spacexlanding.wikispaces.com/ with current state of affairs.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: apollolanding on 05/06/2014 12:45 am
Just when I thought I'd seen it all from the NSF crowd, this thread shows up.  Amazing, amazing work!  I don't understand 99% of what you guys are sharing, but I find the labor and results fascinating nonetheless!  Bravo Zulu to you all!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/06/2014 12:56 am
mlindner was asking for a non-corrupted image, so this one comes from the reentry burn of Cassiope launch

That's a different camera unfortunately.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/06/2014 12:59 am
I updated my Transport stream tool.  It's probably not going to help much at this point but it does have a hex editor and powerful search feature with a bit level wildcard filter, ability to return results with a specified tolerated number of single bit errors and the results are easy to visualize and locate. 

To do:
Disable controls until a ts file is opened.
Make the search capable of finding results that are not byte aligned. 
Add a replace feature with a wildcard filter.
Enable bit level insertion and deletion ability. (would bit shift everything after that point in the file)


I think its a good companion tool for this transport stream editor. 
http://www.videohelp.com/tools/TS-Packet-Editor

Do you have something that's not for windows?

Also, a tool that could search by bits rather than hex would be even more helpful. Hardly anything in these video blocks appears to be byte aligned so hex searching doesn't help much.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: michaelni on 05/06/2014 01:01 am
Ive added 2 features to https://github.com/michaelni/FFmpeg/tree/spacexdebug1
first, you can now use syntax like
-mmb 40:03:-1
to fill the image with "blank" blocks starting from 40:03 and until the next specified command
and
-mmb X:20009:1B
to xor 0x1B at bit position 20009 of the frame, that is this allows to flip some bits without the need to edit the file
(atm this is limited to max 8bit per command)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/06/2014 01:02 am
I feel like we're missing an important piece of this puzzle.

Was the video data stream was on it's own channel, or there was other data being transmitted on the same channel with some other controls in place to keep the data segregated?

If it was on it's own channel, having an analogue(time domain) copy of the original transmission, perhaps even with time code synced to the video file provided, there might be a lot more that could be done to extract the information.  Having the the time domain data stream it would also be much easier to tell where and when data was dropping out.

Someone was mentioning over on reddit they wish they had gotten an SDR (software defined radio) recording of the signal then we could dive into the signal processing layer to recover the signal.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/06/2014 01:05 am
Ive added 2 features to https://github.com/michaelni/FFmpeg/tree/spacexdebug1
first, you can now use syntax like
-mmb 40:03:-1
to fill the image with "blank" blocks starting from 40:03 and until the next specified command
and
-mmb X:20009:1B
to xor 0x1B at bit position 20009 of the frame, that is this allows to flip some bits without the need to edit the file
(atm this is limited to max 8bit per command)

Oh awesome that helps a lot. I was trying to edit things via a hexeditor but was making little progress because I couldn't find the alignments of the data.

Quick question. When you say "xor 0x1B at bit position" does that mean it takes the next 8 bits starting at 20009 and xor's them with the 0x1B value?
Edit: He said "yes."
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/06/2014 01:23 am
Just a question: Is anybody collecting/monitoring the progress on this task? As several of you are working on parallel, I believe a monitoring tool would be useful in order to lower the risk of having several individuals doing the same thing. 

To start, may I suggest on keeping a table like the one originally describing the I-Frames (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1193316#msg1193316) in order for everyone to know who is working on each one?

There is wiki at http://spacexlanding.wikispaces.com/ with current state of affairs.

Thanks for setting that up.

Just set up an IRC channel here on freenode for us. Use your favorite IRC client or use https://webchat.freenode.net/

Channel is #nsf-spacex-video
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: michaelni on 05/06/2014 02:03 am
Ive added 2 features to https://github.com/michaelni/FFmpeg/tree/spacexdebug1
first, you can now use syntax like
-mmb 40:03:-1
to fill the image with "blank" blocks starting from 40:03 and until the next specified command
and
-mmb X:20009:1B
to xor 0x1B at bit position 20009 of the frame, that is this allows to flip some bits without the need to edit the file
(atm this is limited to max 8bit per command)

Added another feature, now you can adjust DC values of the blank blocks
each MB has 6 blocks 4 luma and 2 chroma
example to adjust the 4th luma DC of (blank) MB 40:3 by +100
-mmb 40:03:-1:0:0:0:100:0:0
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Adaptation on 05/06/2014 02:04 am
Do you have something that's not for windows?

Also, a tool that could search by bits rather than hex would be even more helpful. Hardly anything in these video blocks appears to be byte aligned so hex searching doesn't help much.

Pretty sure mine would work in wine, if it dosent let me know and I can fix it.  Not certain about the other one. 

My tool does have a bit shift, with auto search selected you just press the bit shift button 7 times and it will make the result pop out at their respective offset. 

React OS in a vm is a good alternative should wine fail you.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: knotnic on 05/06/2014 03:10 am
The video seems simple profile level 3 mpeg4 video in mpeg TS.
It seems none of the error resilience features of mpeg4 have been used when encoding it. Which is a pitty, had slices been used then the decoder could resume decoding of a frame at the next slice start, had data partitioning been used then the more important low resolution information and motion vectors would have been coded first in each slice making errors less likely to damage them. And had rvlcs been used then slices could have been decoded from both the start and end again, limiting the impact of bit errors.

I know nothing about how the video was generated or how it was transmitted, but if there was some FEC in there then then it should be possible in principle to re-run FEC decoding after manual fixing up all mpeg-TS and mpeg4-ES headers. And as such manual fixing would decrease the errors, FEC would then have fewer errors to deal with and might in a few rare cases be able to fix a few more errors.

Also if some kind of CRCs have been used, CRCs can also be used to correct bit errors as long as the number of errors is sufficiently small, which each CRC would need to correct, the exact number that could be corrected that way depends on the packet size and  crc polynomial being used.

My [outside the industry, and not a video expert either] guess is that the the reason they didn't enable them here is that the mpeg data has been extracted from a telemetry stream, and what we are seeing is after their own decryption/forward error correction/CRC was performed.  For minimizing data rate or other reasons perhaps they decided not to double up error-correction, since it was already being done.  I suspect they are probably required to encrypt the data stream for ITAR reasons and therefore couldn't release pre-error-corrected raw telemetry even if they wanted to.


And by the way-- amazing effort so far, by all involved. 
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/06/2014 03:42 am
Ive added 2 features to https://github.com/michaelni/FFmpeg/tree/spacexdebug1
first, you can now use syntax like
-mmb 40:03:-1
to fill the image with "blank" blocks starting from 40:03 and until the next specified command
and
-mmb X:20009:1B
to xor 0x1B at bit position 20009 of the frame, that is this allows to flip some bits without the need to edit the file
(atm this is limited to max 8bit per command)

Added another feature, now you can adjust DC values of the blank blocks
each MB has 6 blocks 4 luma and 2 chroma
example to adjust the 4th luma DC of (blank) MB 40:3 by +100
-mmb 40:03:-1:0:0:0:100:0:0

Can you explain more how to use this feature and what it does and why its useful? We're getting really far down in the trenches here.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: michaelni on 05/06/2014 04:03 am
Added another feature, now you can adjust DC values of the blank blocks
each MB has 6 blocks 4 luma and 2 chroma
example to adjust the 4th luma DC of (blank) MB 40:3 by +100
-mmb 40:03:-1:0:0:0:100:0:0

Can you explain more how to use this feature and what it does and why its useful? We're getting really far down in the trenches here.

Some of the decoded pictures show rectangular areas extending to the right and down with wrong color/brightness, adjusting the DC value of the MB in the left top corner of such rectangle would allow to workaround the artifact
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/06/2014 04:08 am
Using michaelni's changes I have made the best version to date I think of the upper portion of iframe8. Really works amazingly well. Now to start into the mess at the bottom.

-mmb 08:14:-1:15:-15:1:14:-2:1,09:14:56683

Edit: fixed to better version
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/06/2014 05:16 am
Hi guys,

Great stuff all! Like the wiki and the new "empty"-block commands!

Btw I updated the install.txt so you can download the new version:

http://forum.nasaspaceflight.com/index.php?topic=34597.msg1193442#msg1193442

Regards and have fun, :)

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/06/2014 07:08 am
I think I got a bit more of iframe 8, hard to tell what the correct alignment is.

ffmpeg -mmb 07:14:17233,14:14:57111,08:15:17233,09:15:63546,19:16:17233,21:16:73568,08:21:17233,00:22:111664 -debug mb_pos_size -err_detect ignore_err -s:0 704:480 -i iframe_8.mpg4-img -f image2 img-%03d.png

Working from yours combining with mine I have this so far.

-mmb 08:14:-1:15:-15:1:14:-2:1,09:14:56683,21:16:73568,08:21:17233,00:22:111664,01:28:139511,05:28:143051

I need more information on what's at the bottom so going to work on one of the other iframes now. I'm moving on to working on iframe7.

Edit: I think part of the problem is that we're seeing remnants of that darn dirty water that they splashed all over the rocket... So we have these dark and light splotches.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/06/2014 10:01 am
Hack job on iframe7. I can do better, but probably am calling quits for tonight.

-mmb X:18494:01,19:05:-1,19:06:20616,15:12:-1,17:12:44357
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: gospacex on 05/06/2014 11:28 am
In order to fix this you could try to restore the bitstream itself. But this is very hard to do and extremely time consuming (fixing compressed data is no joke!). The worse is: you don't know when the bad bits stop.

If data errors are random, if garbled bits are not occurring in bursts, but randomly, then fixing garbled block entails inverting one bit and looking whether image got better in the block and after it. If it got worse, you inverted a "good" bit. If it got better, you inverted a corrupted one.

Thankfully, there aren't than many bits in, say, 32 bytes - only 256.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/06/2014 02:29 pm
In order to fix this you could try to restore the bitstream itself. But this is very hard to do and extremely time consuming (fixing compressed data is no joke!). The worse is: you don't know when the bad bits stop.

If data errors are random, if garbled bits are not occurring in bursts, but randomly, then fixing garbled block entails inverting one bit and looking whether image got better in the block and after it. If it got worse, you inverted a "good" bit. If it got better, you inverted a corrupted one.

Thankfully, there aren't than many bits in, say, 32 bytes - only 256.

Nope. Errors tend to come in bursts I've seen, which makes sense. Also, inverting a bad bit can make the entire image worse because a following bad bit could have been partially correcting for the changes the previous change did.

Also, its better to talk about bits here because, to save space, almost nothing is byte aligned. Everything is packed together.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: gospacex on 05/06/2014 05:15 pm
In order to fix this you could try to restore the bitstream itself. But this is very hard to do and extremely time consuming (fixing compressed data is no joke!). The worse is: you don't know when the bad bits stop.

If data errors are random, if garbled bits are not occurring in bursts, but randomly, then fixing garbled block entails inverting one bit and looking whether image got better in the block and after it. If it got worse, you inverted a "good" bit. If it got better, you inverted a corrupted one.

Thankfully, there aren't than many bits in, say, 32 bytes - only 256.

Nope. Errors tend to come in bursts I've seen, which makes sense.

It makes sense if no bit interleaving encoding was employed on the data link.

I hoped it *has been* used, making error bit flips spread instead of bunching up. But maybe SpaceX didn't go for a more expensive hardware with those capabilities...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: lgjy98d on 05/06/2014 07:45 pm
I've started experimenting with I-Frame 12 and doing some automated analysis.

better.png was produced with -mmb 0:0:-1,1:0:583

Legend for -annotated images:
* Red circles annotate the MBs that disappeared from base.png after the modification
* Green circles annotate the MBs that appeared in the better.png after -mmb modification
* Crossed MBs are ones that considered corrupted by ffmpeg

I'd be glad to get feedack if anyone sees any additional information from the annotated images.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jakusb on 05/06/2014 07:56 pm
I've started experimenting with I-Frame 12 and doing some automated analysis.

better.png was produced with -mmb 0:0:-1,1:0:583

Legend for -annotated images:
* Red circles annotate the MBs that disappeared from base.png after the modification
* Green circles annotate the MBs that appeared in the better.png after -mmb modification
* Crossed MBs are ones that considered corrupted by ffmpeg

I'd be glad to get feedack if anyone sees any additional information from the annotated images.

Just curious, but why is the time stamp so clear? And does it help to know it should be much further to lower left corner?
It is the one big constant in a frames and pretty predictable what it should be.

Ps: btw, amazing what work is being put into this. Kudos! :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/06/2014 08:32 pm
Just curious, but why is the time stamp so clear? And does it help to know it should be much further to lower left corner?

Luck. Yes.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/06/2014 08:37 pm
Hi Guys,

My 2 CC. A little improvement on Frame 11:

mmb=0:0:10008

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/06/2014 08:37 pm
Hack job on iframe7. I can do better, but probably am calling quits for tonight.

-mmb X:18494:01,19:05:-1,19:06:20616,15:12:-1,17:12:44357

I could improve a bit the image with the command:

-mmb X:18494:01,20:05:-1,02:06:18794,15:12:-1,19:12:45121,19:15:-1,20:15:62858,21:15:-1,29:15:64161

I could compile ffmpeg with windows using MinGW (had to add some packages: yasm, gettext, glib, pkg-config), the only thing that does not work is to write the png, so I had to write a jpg.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: John_L on 05/06/2014 08:45 pm
Not sure if anyone's  interested or not but I wrote an MS-DOS batch file to automate the process a bit if anyone's interested in running multiple MMB values:

set a=1
for /f "tokens=1 delims=" %%i in (file.txt) do call :part2 %%i
goto :fin
:part2
ffmpeg -mmb %%i -debug mb_pos_size -err_detect ignore_err -s:0 704:480 -i iframe_1.mpg4-img -f image2 img-%a%.png
set /a a+=1
goto :eof
:fin

copy the above text into a text file and save as "name.bat" where name is the name you want to give it, mine's simply named "doit.bat"

then create a text file with just the mmb values... i.e.:

0:0:-1,0:24:19233
0:0:-1,0:24:19234
0:0:-1,0:24:19235

... or whatever values you want after the -mmb in the ffmpeg command line.  Save this out as "file.txt".

Then, when you type "doit" from the command line the batch file will read in the first value from file.txt and plug it into the ffmpeg command and execute it, the resultant output files will be named "img-x.png" where x is a number.  Don't forget to edit which iframe you're working on as well.

I also installed a program called ImageMagick-6.8.9-Q16.  It has a display program that can be called from the command line.  If you install it, you can copy imdisplay.exe to the directory that contains ffmpeg and the batch files (and iframe of course), and you can edit the batch file to include "imdisplay img-%a%.png" after the ffmpeg command line in the call and it will display them as well.  Like so:

set a=1
for /f "tokens=1 delims=" %%i in (file.txt) do call :part2 %%i
goto :fin
:part2
ffmpeg -mmb %%i -debug mb_pos_size -err_detect ignore_err -s:0 704:480 -i iframe_1.mpg4-img -f image2 img-%a%.png
imdisplay img-%a%.png
set /a a+=1
goto :eof
:fin


Not sure if anyone wants this or not, but thought I'd throw it out there if anyone wishes to use it.

John.

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/06/2014 09:10 pm
I was also able to recover the timestamp on frame 6  :)

mmb=0:0:61341
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: grythumn on 05/06/2014 09:25 pm
There was a request for updated cygwin (windows) binary. I split the DLLs out from the executable; if you grabbed the last package, you don't need the DLLs again.

-Bob
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/06/2014 09:32 pm
Awesome, thanks for the update :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/06/2014 09:50 pm
We created a way of doing some of the work through a mini web app. You can find it here: http://spacexlanding.wikispaces.com/Online+Editor
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/06/2014 09:53 pm
We created a way of doing some of the work through a mini web app. You can find it here: http://spacexlanding.wikispaces.com/Online+Editor

Which is really cool and awesome. People don't have to compile anything now.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/06/2014 09:57 pm
We created a way of doing some of the work through a mini web app. You can find it here: http://spacexlanding.wikispaces.com/Online+Editor

Which is really cool and awesome. People don't have to compile anything now.

Working like a charm! I got this with iframe 12 and 0:0:1001:


 
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/06/2014 10:11 pm
We created a way of doing some of the work through a mini web app. You can find it here: http://spacexlanding.wikispaces.com/Online+Editor

Which is really cool and awesome. People don't have to compile anything now.

Working like a charm! I got this with iframe 12 and 0:0:1001:
Great work!

Ok. I will say this now. This is a picture of an intact falcon 9 first stage slightly embedded into water. You can't see the full legs because the tips are under water. You can see small parts of them: only the top. They look "thinner" and shorter that way.

That' s my take on it now. I really do think we have given SpaceX visual proof they did it. :) I am sure they are enjoying this :) :)

And it would only make sense that they would give out video until right after spashdown, where they know (from telemetry) that everything was fine.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/06/2014 10:12 pm
iframe_11 with -mmb 0:0:-1,0:13:32118,0:25:-1,1:25:82578,7:29:-1

(edited)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/07/2014 12:00 am
Hi people.

I got a bit of sea from frame 6 and moved the legs and the clock from previous post.

edit: forgot the main bit -mmb 0:0:583,00:01:6129,17:14:64694,2:25:202509
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/07/2014 12:19 am
I tried to recenter iframe 12 from previous solution, difficult...

-mmb 0:0:31664,40:03:-1,41:03:43222,39:6:-1,40:06:53586,39:8:-1,0:9:59654,36:9:-1,0:10:62314,
36:10:-1,40:10:64640,40:12:-1,0:13:69892,0:15:-1,1:15:74005,39:16:-1,0:17:77748,39:17:-1,0:18:79454

(edit: see http://forum.nasaspaceflight.com/index.php?topic=34597.msg1194745#msg1194745 for a better version)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: grythumn on 05/07/2014 12:30 am
We created a way of doing some of the work through a mini web app. You can find it here: http://spacexlanding.wikispaces.com/Online+Editor

Very cool. You may want to add the debug log output to the page somehow... you're kind of shooting blind without it. Apologies if it is there and I missed it...

Hope the hosting doesn't cost you an arm and a leg...

-Bob
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: grythumn on 05/07/2014 01:02 am
Improvement on Frame 9. You can sort of see part of the rocket body now, but if I try to shift it left it goes all green blocks on me again.

The null MMBs help a lot.

20:04:-1:0:0:-5:0:0:0,26:04:16025,21:07:-1,22:07:27346,05:15:-1,09:15:73077,
12:15:-1,20:15:73574,24:15:-1,32:15:74745,1:16:-1,4:16:77775,
8:16:-1,9:16:78463,26:16:-1,27:16:81473,28:16:-1,29:16:82604,31:16:82767,
36:16:-1,37:16:83144

-Bob
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mmeijeri on 05/07/2014 01:28 am
It is a bit early for this, but I notice there are many slightly discoloured blocks in the otherwise seemingly correctly restored areas. Could these be single bit errors in the lowest order DCT coefficients? If so, we may eventually be able to correct them by hand. Maybe we can even semi-automate the process.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/07/2014 04:54 am
Improvement on Frame 9. You can sort of see part of the rocket body now, but if I try to shift it left it goes all green blocks on me again.

The null MMBs help a lot.

20:04:-1:0:0:-5:0:0:0,26:04:16025,21:07:-1,22:07:27346,05:15:-1,09:15:73077,
12:15:-1,20:15:73574,24:15:-1,32:15:74745,1:16:-1,4:16:77775,
8:16:-1,9:16:78463,26:16:-1,27:16:81473,28:16:-1,29:16:82604,31:16:82767,
36:16:-1,37:16:83144

-Bob

ffmpeg -mmb 20:04:-1:0:0:-5:0:0:0,26:04:16025,21:07:-1,22:07:27346,01:12:-1,02:12:50994,
03:15:-1,04:15:72834,05:15:-1,09:15:73077,10:15:-1,11:15:73260,12:15:-1,21:15:73843,
24:15:-1,4:16:77775,8:16:-1,16:16:79528,26:16:-1,27:16:81473,28:16:-1,29:16:82604,
31:16:82767,36:16:-1,37:16:83144 -debug mb_pos_size -err_detect ignore_err -s:0 704:480 -i iframe_9.mpg4-img -f image2 img-%03d.png

Matt
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: John_L on 05/07/2014 05:29 am
I think I've made some headway with iframe 2.

-mmb 40:3:1424,12:19:18210,20:6:2006,25:8:18680,2:20:35080


Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/07/2014 05:47 am
I think I got a bit more of iframe 8, hard to tell what the correct alignment is.

ffmpeg -mmb 07:14:17233,14:14:57111,08:15:17233,09:15:63546,19:16:17233,21:16:73568,08:21:17233,00:22:111664 -debug mb_pos_size -err_detect ignore_err -s:0 704:480 -i iframe_8.mpg4-img -f image2 img-%03d.png

Working from yours combining with mine I have this so far.

-mmb 08:14:-1:15:-15:1:14:-2:1,09:14:56683,21:16:73568,08:21:17233,00:22:111664,01:28:139511,05:28:143051

I need more information on what's at the bottom so going to work on one of the other iframes now. I'm moving on to working on iframe7.

Edit: I think part of the problem is that we're seeing remnants of that darn dirty water that they splashed all over the rocket... So we have these dark and light splotches.

More of frame 8:

08:14:-1:15:-15:1:14:-2:1,09:14:56683,21:16:73568,08:21:17233,00:22:111664,
42:24:-1,05:25:127302,06:25:-1,07:25:127642,08:25:-1,09:25:127920,10:25:-1,
18:25:128500,20:25:-1,22:25:128837,25:25:-1,29:25:129306,10:27:-1,19:27:136208,
21:27:-1,24:27:136771,01:28:139511,05:28:143051

Frame 12 calls me.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/07/2014 06:04 am
Frame 12 calls me.

Yes. I think iframe12 contains the "money shot". :)

If you look at the version from SwissCheese you only see the pistons that extend the legs. You can't see the legs themselves anymore. They have become very "thin". This is perfectly consistent with the great simulations done by meadows.st ( here (http://forum.nasaspaceflight.com/index.php?topic=34065.msg1165692#msg1165692) ).

I've attached both the improved iframe 12 and the simulation of a submerged first stage. Look closely at the the legs in our frame and then look at the drawing from meadows.st. It fits.

This is a first stage in the water! I'm telling you :) :)

You guys all rock!

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: aero on 05/07/2014 06:16 am
Quote
[edit] Is that a piece of the leg broken off right-above the right leg? Is that the piece of the leg they found? That would be cool  8)

My guess is that it is a splotch on the lens from the dirty water bath at launch. Maybe it is from some other source but it shows up in the same spot in a lot of the frames so I think it is on the lens.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/07/2014 06:27 am
Quote
[edit] Is that a piece of the leg broken off right-above the right leg? Is that the piece of the leg they found? That would be cool  8)

My guess is that it is a splotch on the lens from the dirty water bath at launch. Maybe it is from some other source but it shows up in the same spot in a lot of the frames so I think it is on the lens.
You're right.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/07/2014 06:34 am
Good morning chaps. This is such an impressive effort - and a big welcome to the new people who have helped push this forward.

We've got SpaceX people watching this thread, but I think we're close to asking PAO to make Elon and the top brass aware of the progress. Will send out some e-mails today.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/07/2014 06:44 am
Good morning chaps. This is such an impressive effort - and a big welcome to the new people who have helped push this forward.

We've got SpaceX people watching this thread, but I think we're close to asking PAO to make Elon and the top brass aware of the progress. Will send out some e-mails today.

It would be great if you could get someone, anyone, to give us any screenshot from this camera of the stage from when they had signal from it, like just after separation. It would help greatly to know the dark/light patterns on the lower part of the image.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/07/2014 06:49 am
One more note, to people working on the images. It's very important that the _positions_ of the blocks be in the correct locations. I notice some people just chopping off the top part of the image. That really needs to be avoided. Compare to iframe8 for example and get the legs into the correct positions at the least. If they're not in the right positions they're useless as iframes.

Great work though everyone!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/07/2014 07:00 am
Good morning chaps. This is such an impressive effort - and a big welcome to the new people who have helped push this forward.

We've got SpaceX people watching this thread, but I think we're close to asking PAO to make Elon and the top brass aware of the progress. Will send out some e-mails today.

It would be great if you could get someone, anyone, to give us any screenshot from this camera of the stage from when they had signal from it, like just after separation. It would help greatly to know the dark/light patterns on the lower part of the image.

That's what I see in iframe 12. The water is buggy but the bottom half of the frame is really clear.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/07/2014 07:48 am
We created a way of doing some of the work through a mini web app. You can find it here: http://spacexlanding.wikispaces.com/Online+Editor
Hi,

What would be really useful is when you click somewhere on the generated pixel it would put the x and y of the corresponding block in your clipboard (or show it somewhere on the page so yuu can copy-paste it from there.

Even more useful would be if you could parse the log on the serverside and when you click on the image you get x,y and bitposition in your clipboard! Or maybe simply the line from the log corresponding with the x and y you just clicked.

Just an idea.

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/07/2014 07:51 am
All right, I have to give up for the night. This is my best run for iframe 12:

X:598:1,0:0:550,1:0:-1,14:09:29489,0:10:31664,0:11:34522,0:12:37439,
0:13:40079,0:14:43312:0:15:46476,0:16:49752,0:17:53744,0:18:57020,
0:19:59654,5:19:-1,06:19:60089,0:20:62314,0:21:64864,0:22:67531,
0:23:69892,0:24:72120,0:25:73937,0:26:75964,0:27:77748,0:28:79454,
0:29:86580
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/07/2014 07:53 am
Good morning chaps. This is such an impressive effort - and a big welcome to the new people who have helped push this forward.

We've got SpaceX people watching this thread, but I think we're close to asking PAO to make Elon and the top brass aware of the progress. Will send out some e-mails today.

Send them this link too:

http://spacexlanding.wikispaces.com/ (http://spacexlanding.wikispaces.com/)

Good overview of the findings in this thread.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/07/2014 09:27 am
iframe 10 with:

-mmb 2:5:-1,0:7:18695,23:13:37251,24:13:-1,7:17:47677
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/07/2014 09:53 am
We created a way of doing some of the work through a mini web app. You can find it here: http://spacexlanding.wikispaces.com/Online+Editor
Hi,

What would be really useful is when you click somewhere on the generated pixel it would put the x and y of the corresponding block in your clipboard (or show it somewhere on the page so yuu can copy-paste it from there.

Even more useful would be if you could parse the log on the serverside and when you click on the image you get x,y and bitposition in your clipboard! Or maybe simply the line from the log corresponding with the x and y you just clicked.

Just an idea.

Regards,

arnezami

I updated the image handler to do this (http://dz0bwiwndcjbh.cloudfront.net/info/<frame>?mmb=<mmb>) but I haven't got time to put it into the web app atm, I'll do that later unless someone else wants to
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/07/2014 09:54 am
Ive added 2 features to https://github.com/michaelni/FFmpeg/tree/spacexdebug1
first, you can now use syntax like
-mmb 40:03:-1
to fill the image with "blank" blocks starting from 40:03 and until the next specified command
and
-mmb X:20009:1B
to xor 0x1B at bit position 20009 of the frame, that is this allows to flip some bits without the need to edit the file
(atm this is limited to max 8bit per command)
Hi Michael,

I've been looking at a lot of frames that were "fixed" with the blank blocks. I think they are useful. But what would be better is a block that cancels/negates all information coming from the left and upper block.

In other words a block that if you would place one of its kind above (and one its kind to the left) of a block with content it would be as if that block with content would be at 0,0: no disturbance at all. Of course this would also remove all good information from the left and top but I think that might in many cases to better than the blank-block. Maybe choose to cancel only the top information or only the left information.

Do you think this can be done?

So like this:

-1 : let info through from left and top
-2 : let info through from left, cancel from top
-3 : cancel info from left, let it through from top
-4 : cancel info from left and top

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/07/2014 10:16 am
And a nice iframe 9  :)

-mmb 20:04:-1:0:0:-5:0:0:0,24:04:16025,1:15:-1,3:16:79137,42:19:-1,8:20:96260
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/07/2014 10:52 am
Ive added 2 features to https://github.com/michaelni/FFmpeg/tree/spacexdebug1
first, you can now use syntax like
-mmb 40:03:-1
to fill the image with "blank" blocks starting from 40:03 and until the next specified command
and
-mmb X:20009:1B
to xor 0x1B at bit position 20009 of the frame, that is this allows to flip some bits without the need to edit the file
(atm this is limited to max 8bit per command)
Hi Michael,

I've been looking at a lot of frames that were "fixed" with the blank blocks. I think they are useful. But what would be better is a block that cancels/negates all information coming from the left and upper block.

In other words a block that if you would place one of its kind above (and one its kind to the left) of a block with content it would be as if that block with content would be at 0,0: no disturbance at all. Of course this would also remove all good information from the left and top but I think that might in many cases to better than the blank-block. Maybe choose to cancel only the top information or only the left information.

Do you think this can be done?

So like this:

-1 : let info through from left and top
-2 : let info through from left, cancel from top
-3 : cancel info from left, let it through from top
-4 : cancel info from left and top

Regards,

arnezami

Don't think that's simple. You'd have to store the id of the block below and to the right of the block you are placing. Also every block AFAIK _HAS_ to reference either the top or the left, so you could only change the block to reference the other direction which would probably mess things up worse.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/07/2014 11:12 am
iframe 2 at correct position:

-mmb 40:3:-1,16:10:18210,4:28:-1,40:28:76955
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/07/2014 11:17 am
I created a new set of iframes from michaelni's try1.ts file he linked on IRC. mpg4-img and pngs are in the zip. These supposedly contain more information because they are larger, whether its recoverable information or not is another question. mmb's from other iframes may require adapting to work

http://ffmpeg.org/~michael/space-x-video/try1.ts
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/07/2014 11:20 am
Don't think that's simple. You'd have to store the id of the block below and to the right of the block you are placing. Also every block AFAIK _HAS_ to reference either the top or the left, so you could only change the block to reference the other direction which would probably mess things up worse.
No what I mean is that you change its "transformation" data so that the result is that it is the "opposite" of what it receives from its left and top. That way no matter what block is placed right or bottom the will have data from left and top that is effectively the same as if that block is placed at 0,0.

I'm not sure changing the "transformation" data this way is possible (I guess its a mathematical problem). But I did'nt mean to solve this by making pointers to blocks that are right or bottom.

Another way of doing this is to say to a block: don't "listen" to your left block, instead "listen" to the left "block" of 0,0 (initial state I guess). I believe these things are called "predictions" in mepg-4.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/07/2014 11:56 am
We created a way of doing some of the work through a mini web app. You can find it here: http://spacexlanding.wikispaces.com/Online+Editor
Hi,

What would be really useful is when you click somewhere on the generated pixel it would put the x and y of the corresponding block in your clipboard (or show it somewhere on the page so yuu can copy-paste it from there.

Even more useful would be if you could parse the log on the serverside and when you click on the image you get x,y and bitposition in your clipboard! Or maybe simply the line from the log corresponding with the x and y you just clicked.

Just an idea.

Regards,

arnezami

I updated the image handler to do this (http://dz0bwiwndcjbh.cloudfront.net/info/<frame>?mmb=<mmb>) but I haven't got time to put it into the web app atm, I'll do that later unless someone else wants to

I've now updated the app to allow the selection of MB, I "think" it's right but let me know if it isn't
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/07/2014 12:56 pm
And a big high five and a welcome to you Iain!

Note from the SpaceX side (paraphrased) says that a request has been sent to provide a 10 second shot from the camera while the vehicle was still on the pad. This should go up on the page where the videos were posted on the SpaceX site, hopefully soon.

I'm told the pad shot is the most error free telem they have, but it's recognized the need for a shot later in flight that has the muddy streaks for reference. That is being inquired about, per the note I've been sent.

The main issue is everyone is busy with the upcoming flights, as can be expected.

We have a lot of SpaceX folk watching this thread and we'll get some additional eyeballs via an official PAO approach today. It's noted that people are impressed with the work going on here, so well done everyone!

PS Per the PAO approach, I think this process deserves a news article. It's fascinating enough and of interest to the readership. It may also shake the tree for a few more video experts, but we're clearly doing well on that score already. Problem with that is I pretty much know next to nothing about what you guys are doing from a tech standpoint, so if one of you guys is confident at being able to summarize, send me a message, as I can write around it and subedit. No rush on that.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Okie_Steve on 05/07/2014 01:49 pm
Chris,

If SpaceX is willing to provide a bit of information about the stream format it would be very helpful to know the location of any ECC/CRC errors and the packet size and any type of cross-interleave etc. used in the telemetry stream to help identify "possibly corrupt"and "probably correct" portions of the data.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/07/2014 02:16 pm
Chris,

If SpaceX is willing to provide a bit of information about the stream format it would be very helpful to know the location of any ECC/CRC errors and the packet size and any type of cross-interleave etc. used in the telemetry stream to help identify "possibly corrupt"and "probably correct" portions of the data.

I'll ask....as much as you have already asked by posting it (such people watching the thread).
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Untribium on 05/07/2014 03:07 pm
Amazing work! A few days ago I was worried this wouldn't go anywhere, especially considering that we're most likely (hopefully) going to get new footage in a few days :)
I put a little bit of work into iframe 7. Basically, I started with the line by mlindner and modified it slightly to fix the color issue around the legs.
X:18494:01, 19:05:-1, 20:06:20716,15:12:-1, 22:12:45309, 22:15:-1, 23:15:63096, 26:15:-1, 30:15:64161
(http://dz0bwiwndcjbh.cloudfront.net/7?v=1&mmb=X:18494:01,%2019:05:-1,%2020:06:20716,15:12:-1,%2022:12:45309,%2022:15:-1,%2023:15:63096,%2026:15:-1,%2030:15:64161)
Instead of taking the first "nice" block after some garbage, I take one further to the right and if necessary down, so that all the blocks that influence this block are non-garbage. Unfortunately, in this case the first nice block contains part of the left leg, so there's some detail lost...any ideas on how to fix the colors while keeping the good blocks?
Oh, could someone quickly explain what the first part of the line does (X:18494:01)? I read through all the posts, but I didn't find an answer (may have missed it...)

Edit: Found an improvement right after posting...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/07/2014 03:32 pm
All right, I have to give up for the night. This is my best run for iframe 12:

X:598:1,0:0:550,1:0:-1,14:09:29489,0:10:31664,0:11:34522,0:12:37439,
0:13:40079,0:14:43312:0:15:46476,0:16:49752,0:17:53744,0:18:57020,
0:19:59654,5:19:-1,06:19:60089,0:20:62314,0:21:64864,0:22:67531,
0:23:69892,0:24:72120,0:25:73937,0:26:75964,0:27:77748,0:28:79454,
0:29:86580

This one is a bit better for iframe 12, still not great:

X:552:1,X:553:1,0:0:550,1:0:-1,14:09:29489,0:10:31664,0:11:34522,
0:12:37439,0:13:40079,0:14:43312:0:15:46476,0:16:49752,0:17:53744,
0:18:57020,0:19:59654,5:19:-1,06:19:60089,0:20:62314,0:21:64864,
0:22:67531,0:23:69892,0:24:72120,0:25:73937,0:26:75964,0:27:77748,
0:28:79454,0:29:86580
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Smokablecheese on 05/07/2014 03:37 pm
Just a small improvement on IFrame 9, I made a modification to Swisscheeses last edit


mmb 20:04:-1:0:0:-5:0:0:0,24:04:16023,1:15:-1,3:16:79127,41:19:-1,8:20:96220,15:04:16023
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/07/2014 03:45 pm
Amazing work everyone.
Remember we have a wiki space for the project (http://spacexlanding.wikispaces.com/) for you to add your versions.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/07/2014 03:52 pm
How are the MB's ordered in the file?

Is it
a) left->right, top->bottom
b) zigzag like (see fig 7-4-c in the ISO file (attached))
c) something else?

I ask because AFAIUI we need the startbit of every MB in an Iframe so that if one MB is somehow broken the decoder still knows where the next begins.
It would be nice to know in which order one should work on the MB's.

Cheers,
Shanuson
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/07/2014 04:07 pm
How are the MB's ordered in the file?

Is it
a) left->right, top->bottom
b) zigzag like (see fig 7-4-c in the ISO file (attached))
c) something else?

I ask because AFAIUI we need the startbit of every MB in an Iframe so that if one MB is somehow broken the decoder still knows where the next begins.
It would be nice to know in which order one should work on the MB's.

Cheers,
Shanuson

Left to right, top to bottom. Not like the attached diagram. However, blocks can affect others to the right and below.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/07/2014 04:20 pm
How are the MB's ordered in the file?

Is it
a) left->right, top->bottom
b) zigzag like (see fig 7-4-c in the ISO file (attached))
c) something else?

I ask because AFAIUI we need the startbit of every MB in an Iframe so that if one MB is somehow broken the decoder still knows where the next begins.
It would be nice to know in which order one should work on the MB's.

Cheers,
Shanuson

First block is top left corner, and the blocks continue going left to right until they hit end of row, then go to next row and repeat.

Every block has 6 sub-blocks, 4 luma blocks (brightness) and 2 chroma blocks (color). The 4 luma blocks are the 4 squares in a block you often see. Both color blocks cover the entire block and are two different color planes. https://en.wikipedia.org/wiki/YUV There are 4 Y blocks and 1 U and 1 V block. Each one of those sub blocks references either the block above it or the block to the left of it (they all don't have to reference the same block). Which block they reference is determined on-the-fly (not hardcoded) so if a block is screwed up then it could cause blocks that would reference it, to not reference it or vice versa.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: R7 on 05/07/2014 04:34 pm
The zigzag image is how DCT (discrete cosine transformation (http://en.wikipedia.org/wiki/Discrete_cosine_transform)) coefficients of a 8x8 block are ordered in the data stream. A macroblock has 6 of these blocks, four for 16x16 luminance information and two for 8x8 color information.

The compression trickery comes from the fact that often most of those coefficients are zero so they are written using variable length encoding, like almost anything in the protocol. The downside royal PITA is that single bit error can make the decoder go haywire, not only read wrong values but read too many or too few bits.

It's a wonder the pictures are as good as they are.

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: John_L on 05/07/2014 05:05 pm
How would one go about re-encoding the video with the replacement iframes, what's the command line for ffmpeg?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: aero on 05/07/2014 05:35 pm
How soon will we see a .gif of the improved frames?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Swatch on 05/07/2014 05:37 pm
This has been some amazing work to watch develop fellas, keep up the good work!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/07/2014 05:43 pm
How would one go about re-encoding the video with the replacement iframes, what's the command line for ffmpeg?

This is a good point, and it got me thinking. If everything goes as planned, SpaceX will launch again in a few days, and I think they're planning to attempt recovery again. This could upstage the (arguably more historic) first landing video. We need to put together a video with what we've got today, tomorrow at the lastest. If necessary, I can hand-assemble something from the iframes and pieces of the pframes. It wouldn't be as pretty as what arnezami and mlindner can do, so I'll wait until I hear from them.

Matt
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/07/2014 05:48 pm
How would one go about re-encoding the video with the replacement iframes, what's the command line for ffmpeg?

This is a good point, and it got me thinking. If everything goes as planned, SpaceX will launch again in a few days, and I think they're planning to attempt recovery again. This could upstage the (arguably more historic) first landing video. We need to put together a video with what we've got today, tomorrow at the lastest. If necessary, I can hand-assemble something from the iframes and pieces of the pframes. It wouldn't be as pretty as what arnezami and mlindner can do, so I'll wait until I hear from them.

Matt

It's actually a lot easier than that. If no one else does it by Friday Eastern time then I'll be modifying ffmpeg to add a frame specifier to the mmb options. Then we simply pile all the mmb options into one long entry and run ffmpeg with it. viola, produced video. This would also allow us to try tweaking some p-frames if we so wish.

Producing a video now with what we have won't be very attractive yet. We need to fine tune a lot of the frames still. I'm doing that now for iframe8, which should hopefully look really really good. This means doing things like whenever you use -1 and replace the block, fine tune all the luma and chroma settings for it to have the least effect on surrounding blocks.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: John_L on 05/07/2014 06:03 pm
It's actually a lot easier than that. If no one else does it by Friday Eastern time then I'll be modifying ffmpeg to add a frame specifier to the mmb options. Then we simply pile all the mmb options into one long entry and run ffmpeg with it. viola, produced video. This would also allow us to try tweaking some p-frames if we so wish.

Producing a video now with what we have won't be very attractive yet. We need to fine tune a lot of the frames still. I'm doing that now for iframe8, which should hopefully look really really good. This means doing things like whenever you use -1 and replace the block, fine tune all the luma and chroma settings for it to have the least effect on surrounding blocks.

So you're saying at this point, ffmpeg doesn't have the ability to re-encode with replacement iframes?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: AncientU on 05/07/2014 06:07 pm
This has been some amazing work to watch develop fellas, keep up the good work!
Yes, truly incredible effort!
So cool to see the first ever soft landing emerge frame by frame.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/07/2014 06:14 pm
It's actually a lot easier than that. If no one else does it by Friday Eastern time then I'll be modifying ffmpeg to add a frame specifier to the mmb options. Then we simply pile all the mmb options into one long entry and run ffmpeg with it. viola, produced video. This would also allow us to try tweaking some p-frames if we so wish.

Producing a video now with what we have won't be very attractive yet. We need to fine tune a lot of the frames still. I'm doing that now for iframe8, which should hopefully look really really good. This means doing things like whenever you use -1 and replace the block, fine tune all the luma and chroma settings for it to have the least effect on surrounding blocks.

So you're saying at this point, ffmpeg doesn't have the ability to re-encode with replacement iframes?

You might be able to manually have it use those iframes and re-encode it that way. I'm not sure how to use ffmpeg to do so. We haven't been producing replacement iframes anyway, only modifications to iframes that need to be applied every time.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Sohl on 05/07/2014 06:40 pm
Excellent work people!  I agree the historical significance of this video makes restoration efforts worthwhile even if the Orbcom flight provides a much cleaner landing video.  I'd probably enjoy this bit-hacking myself if I wasn't up to my neck in my day-job programming work!   :-[
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: John_L on 05/07/2014 06:44 pm
You might be able to manually have it use those iframes and re-encode it that way. I'm not sure how to use ffmpeg to do so. We haven't been producing replacement iframes anyway, only modifications to iframes that need to be applied every time.

Right, well, I'm nowhere near an expert at mpeg encoding, but it was my assumption that once the iframes were done, we'd re-encode the other frames using the iFrames as a reference, or would reinserting the fixed iframes automatically  fix the other frames (because they're now referencing the fixed iframes), which would mean we would just need a way to reinsert the iframes into the video.  Luckily, we have michaelni onboard helping out... nice to have the curator of ffmpeg onboard... ;)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: grythumn on 05/07/2014 06:45 pm
Frame 9, cleaned up some single-bit errors up top.

20:04:-1,24:04:16023,X:16410:01,16:5:-1,17:05:18692,X:19903:1,X:27244:1,
1:15:-1,3:16:79127,41:19:-1,8:20:96220,15:04:16023

-Bob
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/07/2014 06:54 pm
Frame 9, cleaned up some single-bit errors up top.

20:04:-1,24:04:16023,X:16410:01,16:5:-1,17:05:18692,X:19903:1,X:27244:1,
1:15:-1,3:16:79127,41:19:-1,8:20:96220,15:04:16023

-Bob

Nice work! The mixed up part of the right leg is likely also a bit error.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: grythumn on 05/07/2014 07:00 pm
Technique: Find suspect macroblock. Isolate bit range. Plug them into the for loop. Put the string ",X,$i:1," in the middle of your MMB string, where it goes from a bit range point point of view. Run the loop, wait a bit, flip through the resulting pictures. Can usually narrow it down to 4 or 5. If nothing works, try changing the macroblock to the left, or nulling out the macro block... if it's a multibit error, it's a lot more permutations to try to figure out.

For example, my last run:
for i in {32260..32407} ; do ./ffmpeg -debug mb_pos_size -err_detect ignore_err -s:0 704:480 -mmb 20:04:-1,24:04:16023,X:16410:01,16:5:-1,17:05:18692,X:19903:1, X:27244:1,X:$i:1,1:15:-1,3:16:79127,41:19:-1, 8:20:96220,15:04:16023 -i iframe_9.mpg4-img -f image2 img-%03d-$i.png 2> /dev/null ; done

-Bob
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/07/2014 07:04 pm
Technique: Find suspect macroblock. Isolate bit range. Plug them into the for loop. Put the string ",X,$i:1," in the middle of your MMB string, where it goes from a bit range point point of view. Run the loop, wait a bit, flip through the resulting pictures. Can usually narrow it down to 4 or 5. If nothing works, try changing the macroblock to the left, or nulling out the macro block... if it's a multibit error, it's a lot more permutations to try to figure out.

For example, my last run:
for i in {32260..32407} ; do ./ffmpeg -debug mb_pos_size -err_detect ignore_err -s:0 704:480 -mmb 20:04:-1,24:04:16023,X:16410:01,16:5:-1,17:05:18692,X:19903:1, X:27244:1,X:$i:1,1:15:-1,3:16:79127,41:19:-1, 8:20:96220,15:04:16023 -i iframe_9.mpg4-img -f image2 img-%03d-$i.png 2> /dev/null ; done

-Bob

If you start with an error-free image, you can also eliminate any bit-flips that introduce errors by grepping through the output. I have a script that does just that.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: dorkmo on 05/07/2014 07:09 pm
i made a super basic spreadsheet that just concatenates the x:y:bitpos. i was just guna use it for me but heres the link. https://docs.google.com/spreadsheets/d/1sP5lyTbC3ICyca03ZJlWwWaeIn_H6MSeqcp2ACT1HOQ/edit?usp=sharing
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/07/2014 07:18 pm
The University of Wisconsin has a very large HTCondor pool which would probably be very useful to run through this processing. I'll reach out and see if there's anyone over there who can leverage their tens of thousands of available CPU cores to crank out macroblock iterations.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/07/2014 07:54 pm
Technique: Find suspect macroblock. Isolate bit range. Plug them into the for loop. Put the string ",X,$i:1," in the middle of your MMB string, where it goes from a bit range point point of view. Run the loop, wait a bit, flip through the resulting pictures. Can usually narrow it down to 4 or 5. If nothing works, try changing the macroblock to the left, or nulling out the macro block... if it's a multibit error, it's a lot more permutations to try to figure out.

For example, my last run:
for i in {32260..32407} ; do ./ffmpeg -debug mb_pos_size -err_detect ignore_err -s:0 704:480 -mmb 20:04:-1,24:04:16023,X:16410:01,16:5:-1,17:05:18692,X:19903:1, X:27244:1,X:$i:1,1:15:-1,3:16:79127,41:19:-1, 8:20:96220,15:04:16023 -i iframe_9.mpg4-img -f image2 img-%03d-$i.png 2> /dev/null ; done

-Bob

Oh that's an awesome idea. Hadn't thought of doing it for bit flipping. I was already doing that for moving start locations of blocks around.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: CraigLieb on 05/07/2014 08:00 pm
Not really able to follow the technical process being executed, but I have been putting the most recent IFrame images posted into a powerpoint (using it like a flip book). One thing that jumps out is some artifacts that are at the top center and offset to the left of each of the frames. Maybe they can be used like a key to lock in some image locations like the time stamp at the bottom on currently hard interpret frames.

I have placed a few of the frames top strips next to each other here so you can see what I mean.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/07/2014 08:01 pm
Not really able to follow the technical process being executed, but I have been putting the most recent IFrame images posted into a powerpoint (using it like a flip book). One thing that jumps out is some artifacts that are at the top center and offset to the left of each of the frames. They may be like a key that can be used to lock in some image locations like the time stamp at the bottom. I have captured a few of the frames just top strips next to each other.

Yay for dust and dirt on the lens.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/07/2014 08:03 pm
Hi guys,

I would like to explain what we are actually doing. Because some of you might not completely grasp it. And I think I can best explain it by an illustration. Otherwise it gets too technical too quickly. I will do it in terms of rockets. ;)

--

Imagine a book. A thick book. It's a book containing very detailed technical instructions on how to build a new type of rocket.  It contains 13 chapters. One for each rocket part. Each chapter consists of an introductory paragraph explaining what the part is, how it should look like and how it works. None of the following paragraphs can be understood if this paragraph has not been read. The following paragraps are highly dependent on eachother: if you haven't read them in sequence they will be meaningless since they always refer heavely to the previous one to understand.

All sentences are written in a foreign language. And all punctuation is removed. Also all captitals are made lower case. The letters in the words are coded: each letter can only be decoded if you know the previous letter. For example this code: when reading a letter add the letternumber (a=1, b=2 etc) of the previous letter to this one to get the real letter. All whitespaces are removed. So each paragraph is just a string of seemingly random letters.

Ok. Now you're thinking: "This is insane. Who would read such a book?"

But this book was never intended to be read by a human being! It was supposed to be read by a machine that would also automatically build the rocket parts. Here is the real problem though: the printing machine that printed the book had a major error: it sometimes printed the wrong letter! So the book has thousands of random errors in it! Unsuprisingly when the book is now given to the rocket-part-building-machine it says: "Error". And that's basicly it.

Ok it turns out that the rocket-part-building-machine made a few incomplete and defective drawings of parts before giving the error. Mostly useless. So it seems that this is impossible to fix. Trying to figure out which letters are randomized by hand is insanely hard and time consuming.

What we did though was the following: instead of fixing the book we changed the machine so it would (1) give us better errors and (2) allow us to overrule certain instructions. We fed the machine with the first paragraph of a chapter and it started to draw the part. When we thought it went wrong we gave it the instruction to skip that portion of the drawing. Because apparently it had read a wrongly printed letter.

Of course we did not know how far it had to skip. So we guessed several times and hoped it would start drawing something sane again. And each time it was drawing something that looked sensible we knew it was reading uncompromised letters. We got an understanding which parts of the paragraph were wrongly printed and then mostly avoided them. By doing this many times and collaborating with several people we have now been able to let the machine draw several almost complete and fairly accurate parts. We do not know yet if it is going to be possible to let the machine read the other paragraphs in the chapters (they are much harder). But getting these drawings is already very satisfying.

--

I hope this illustration helps you (who might not understand the deeply technical stuff) to understand what the problem really is. How insane it is. And I think it's a quite good representation of what is actually the case.

Regards,

arnezami

---

PS. Below is a translation from the illustration into the real problem.

- Book: Video

- 13 chapters: 13 parts of the video

- First paragraph of each chapter: I-frame (independent picture)

- Other paragraphs: P- and B-frames (all dependent on each other)

- Foreign language: the language of video decoding (things like "discrete cosine transform", "motion vectors", "chrominance and luminance blocks" etc)

- Removal of punctuation and capitals: there are essentially no clear "markers" in the bitstream: no marker to detect the beginning of a horizonal line in the picture.

- Words: macroblocks (16 x 16 pixel blocks)

- Letter coding and removal of whitespaces: the coding of the bits is such that you need every previous bit to know what the current one means withing a word. The words are also variable in length. There is essentially no clear marker between each macroblock.

- Printing errors: random bits that were flipped due to bad reception

- Rocket-part-building-machine: the video decoder (the open source ffmpeg in our case)

- Drawings: the pictures the video decoder makes after trying to decode a frame
 
The transportstream is not mentioned. But effectively many pages of the book were missing. We mostly fixed that: they were actually "hidden" and we discovered most of them.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: AncientU on 05/07/2014 08:07 pm
Thanks, that is perfect even for those of us who don't build rockets!
Ever thought of writing users' manuals?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Adaptation on 05/07/2014 08:11 pm
 

The University of Wisconsin has a very large HTCondor pool which would probably be very useful to run through this processing. I'll reach out and see if there's anyone over there who can leverage their tens of thousands of available CPU cores to crank out macroblock iterations.

This could work, they could run it through an edge detection filter to find ones with good time stamp in the proper location and the dust artifacts on the lens then bias the rest of the results against having strong vertical or horizontal lines.

A little bit of evolutinary code could reduce the power needed drastically, simply did this change make it better or worse.  If it was better use it as the starting point for the next set of changes. 
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/07/2014 08:17 pm
The University of Wisconsin has a very large HTCondor pool which would probably be very useful to run through this processing. I'll reach out and see if there's anyone over there who can leverage their tens of thousands of available CPU cores to crank out macroblock iterations.

This could work, they could run it through an edge detection filter to find ones with good time stamp in the proper location then biase the rest of the results against having strong verticle or horisontal lines.

We could then load "candidate images" in a server and crowdsource it. (like tomnod)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/07/2014 08:17 pm
Hi guys,

I would like to explain what we are actually doing. Because some of you might not completely grasp it. And I think I can best be explained it by an illustration. Otherwise it gets too technical too quickly. I will do it in terms of rockets. ;)

--

Imagine a book. A thick book. It's a book containing very detailed technical instructions on how to build a new type of rocket.  It contains 13 chapters. One for each rocket part. Each chapter consists of an introductory paragraph explaining what the part is, how it should look like and how it works. None of the following paragraphs can be understood if this paragraph has not been read. The following paragraps are highly dependent on eachother: if you haven't read them in sequence they will be meaningless since they always refer heavely to the previous one to understand.

All sentences are written in a foreign language. And all punctuation is removed. Also all captitals are made lower case. The letters in the words are coded: each letter can only be decoded if you know the previous letter. For example this code: when reading a letter add the letternumber (a=1, b=2 etc) of the previous letter from this one to get the real letter. All whitespaces are removed. So each paragraph is just a string of seemingly random letters.

Ok. Now you're thinking: "This is insane. Who would read such a book?"

But this book was never intended to be read by a human being! It was supposed to be read by a machine that would also automatically build the rocket parts. Here is the real problem though: the printing machine that printed the book had a major error: it sometimes printed the wrong letter! So the book has thousands of random errors in it! Unsuprisingly when the book is now given to the rocket-part-building-machine it says: "Error". And that's basicly it.

Ok it turnes out that the rocket-part-building-machine made a few incomplete and defective drawings of parts before giving the error. Mostly useless. So it seems that this is impossible to fix. Trying to figure out which letters are randomized by hand is insanely hard and time consuming.

What we did though was the following: instead of fixing the book we changed the machine so it would (1) give us better errors and (2) allow us to overrule certain instructions. We fed the machine with the first paragraph of a chapter and it started to draw the part. When we thought it went wrong we gave it the instruction to skip that portion of the drawing. Because apparently it had read a wrongly printed letter.

Of course we did not know how far it had to skip. So we guessed several times and hoped it would start drawing something sane again. And each time it was drawing something that looked sensible we knew it was reading uncompromised letters. We got an understanding which parts of the paragraph were wrongly printed and then mostly avoided them. By doing this many times and collaborating with several people we have now been able to let the machine draw several almost complete and fairly accurate parts. We do not know yet if it is going to be possible to let the machine read the other paragraphs in the chapters (they are much harder). But getting these drawings is already very satisfying.

--

I hope this illustration helps you (who might not understand the deeply technical stuff) to understand what the problem really is. How insane it is. And I think it's a quite good representation of what is actually the case.

Regards,

arnezami

---

PS. Below is a translation from the illustration into the real problem.

- Book: Video

- 13 chapters: 13 parts of the video

- First paragraph of each chapter: I-frame (independent picture)

- Other paragraphs: P- and B-frames (all dependent on each other)

- Foreign language: the language of video decoding (things like "discrete cosine transform", "motion vectors", "chrominance and luminance blocks" etc)

- Removal of punctuation and capitals: there are essentially no clear "markers" in the bitstream: no marker to detect the beginning of a horizonal line in the picture.

- Words: macroblocks (16 x 16 pixel blocks)

- Letter coding and removal of whitespaces: the coding of the bits is such that you need every previous bit to know what the current one means withing a word. The words are also variable in length. There is essentially no clear marker between each macroblock.

- Printing errors: random bits that were flipped due to bad reception

- Rocket-part-building-machine: the video decoder (the open source ffmpeg in our case)

- Drawings: the pictures the video decoder makes after trying to decode a frame
 
The transportstream is not mentioned. But effectively many pages of the book were missing. We mostly fixed that: they were actually "hidden" and we discovered most of them.


Such a really good post! :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: John_L on 05/07/2014 08:19 pm
And to continue the analogy... a new book will be written this Saturday....lol
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/07/2014 08:33 pm
Technique: Find suspect macroblock. Isolate bit range. Plug them into the for loop. Put the string ",X,$i:1," in the middle of your MMB string, where it goes from a bit range point point of view. Run the loop, wait a bit, flip through the resulting pictures. Can usually narrow it down to 4 or 5. If nothing works, try changing the macroblock to the left, or nulling out the macro block... if it's a multibit error, it's a lot more permutations to try to figure out.

For example, my last run:
for i in {32260..32407} ; do ./ffmpeg -debug mb_pos_size -err_detect ignore_err -s:0 704:480 -mmb 20:04:-1,24:04:16023,X:16410:01,16:5:-1,17:05:18692,X:19903:1, X:27244:1,X:$i:1,1:15:-1,3:16:79127,41:19:-1, 8:20:96220,15:04:16023 -i iframe_9.mpg4-img -f image2 img-%03d-$i.png 2> /dev/null ; done

-Bob

One more note. X doesn't behave quite as you think. The bitmask is inverted. X:pos:1 actually flips the bit 8 positions away from pos. It reads the next 8 bits starting from pos with pos being the MSB of those 8 bits. If you want to flip just the target bit, do X:pos:80.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: R7 on 05/07/2014 08:39 pm
And to continue the analogy... a new book will be written this Saturday....lol

Let's hope they have changed the page layout to contain additional information to help detect sentence boundaries, errors and such. This macroblock necromancy goes only so far.  :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/07/2014 08:40 pm
And to continue the analogy... a new book will be written this Saturday....lol

Let's hope they have changed the page layout to contain additional information to help detect sentence boundaries, errors and such. This macroblock necromancy goes only so far.  :)

Should let one of us hop on their plane with an SDR with a patchfeed into their radios. :P
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: starsilk on 05/07/2014 08:50 pm
And to continue the analogy... a new book will be written this Saturday....lol

Let's hope they have changed the page layout to contain additional information to help detect sentence boundaries, errors and such. This macroblock necromancy goes only so far.  :)

Should let one of us hop on their plane with an SDR with a patchfeed into their radios. :P

three important things that should be communicated back to SpaceX:

1) turn on the error correction features in their camera/encoder (probably too late for Saturday's launch.. but maybe)

2) turn off interlace

3) make an analog recording of the telemetry stream..
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/07/2014 08:54 pm
3) make an analog recording of the telemetry stream..

Well a digital recording of the analog waveform, but yes.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/07/2014 09:01 pm
3) make an analog recording of the telemetry stream..

Would this be analog enough?  ;D
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: TrueBlueWitt on 05/07/2014 09:06 pm
And to continue the analogy... a new book will be written this Saturday....lol

Let's hope they have changed the page layout to contain additional information to help detect sentence boundaries, errors and such. This macroblock necromancy goes only so far.  :)

Should let one of us hop on their plane with an SDR with a patchfeed into their radios. :P

three important things that should be communicated back to SpaceX:

1) turn on the error correction features in their camera/encoder (probably too late for Saturday's launch.. but maybe)

2) turn off interlace

3) make an analog recording of the telemetry stream..


(2)
Why is anything still recorded using Interlaced processing(OK.. 1080i.. but still)???? 
And my personal favorite.. TV commercials that are recorded in interlaced SD.. 

(3)
High resolution(24bit with high sample rate) "Analog" copy of the data stream with time reference to assess where S/N and coherence are bad would be very helpful. I think I mentioned that many pages ago.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: starsilk on 05/07/2014 09:09 pm
3) make an analog recording of the telemetry stream..

Would this be analog enough?  ;D

nah. if we're going to record something important like this, it needs to at least be a metal tape. they can afford the extra cost  ;)

Edit: just make sure to push the 'metal' button on the tape recorder first.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/07/2014 09:10 pm
Why is anything still recorded using Interlaced processing? ???

(http://www.graphicdesignbasics.com/imagesvr_ce/graphicdesign/uploadedfiles/2009/04/dr-mccoy.jpg)
"Dammit, Jim, I'm a rocket scientist, not a video technician!"
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Turix on 05/07/2014 09:13 pm
I had a go at frame 1, I managed to find most of the timestamp and a bit of the top of the rocket (notice the darker feature bottom left, which appears to match some of the other frames - though I'm not sure how well its aligned).

Done using: 41:00:-1, 33:17:66620, 03:28:112199
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/07/2014 09:30 pm
Someone needs to take a crack at iframe3. Don't know if we can get anything out of that or not.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/07/2014 09:51 pm
Here's a bunch of ocean from iframe4 (the new try1 version). Unfortunately unaligned. No legs to be seen. I think this is the last iframe with legs stowed. They appear to be deployed on iframe5.

-mmb 5:1:4200,0:3:10968,8:3:17911,34:3:23414,19:8:39488,1:9:41549,18:9:42693

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: grythumn on 05/07/2014 10:22 pm
One more note. X doesn't behave quite as you think. The bitmask is inverted. X:pos:1 actually flips the bit 8 positions away from pos. It reads the next 8 bits starting from pos with pos being the MSB of those 8 bits. If you want to flip just the target bit, do X:pos:80.

Ah, boundary conditions/endianness/alignment, how do I love thee. I start doing this manually, 01,02,04,08,10,20,40,80, increment by 8, then decided to automate it without looking at the source. :)

Empirically, it worked fairly well as a semi-targeted brute force search. The correct one should work better. :) I'm playing with a #define TRACE build, but don't understand quite what I'm doing when twiddling bits yet to narrow the search range smaller than the whole macroblock.

-Bob
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/07/2014 10:35 pm
Ok, so no additional help from SpaceX tonight at least I'm told, as they are all busy bees with the next launch (including a lot in transit to the Cape). Again, this thread is being watched, big time, and by one rather well known name at SpaceX I'm reliably informed! ;) So stay focused as much as you can with what we've got.

I'll get more eyeballs on it when I add a few paras and links to here in my F9 v1.1 Static Fire article tomorrow.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/07/2014 10:41 pm
One more note. X doesn't behave quite as you think. The bitmask is inverted. X:pos:1 actually flips the bit 8 positions away from pos. It reads the next 8 bits starting from pos with pos being the MSB of those 8 bits. If you want to flip just the target bit, do X:pos:80.

Ah, boundary conditions/endianness/alignment, how do I love thee. I start doing this manually, 01,02,04,08,10,20,40,80, increment by 8, then decided to automate it without looking at the source. :)

Empirically, it worked fairly well as a semi-targeted brute force search. The correct one should work better. :) I'm playing with a #define TRACE build, but don't understand quite what I'm doing when twiddling bits yet to narrow the search range smaller than the whole macroblock.

-Bob

I pushed up one of michaelni's changes from the other day to the repo if you want to grab it. Adds some warnings when data is possible off. Haven't found it very useful. They print out in yellow.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: John_L on 05/07/2014 10:47 pm
Ok, so no additional help from SpaceX tonight at least I'm told, as they are all busy bees with the next launch (including a lot in transit to the Cape). Again, this thread is being watched, big time, and by one rather well known name at SpaceX I'm reliably informed! ;) So stay focused as much as you can with what we've got.

I'll get more eyeballs on it when I add a few paras and links to here in my F9 v1.1 Static Fire article tomorrow.

Thanks for the heads up Chris... it's understandable with another launch coming up this weekend that everyone's busy with that ... and with that said...

Hey Elon!

(lol)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Okie_Steve on 05/07/2014 11:10 pm
Every block has 6 sub-blocks, 4 luma blocks (brightness) and 2 chroma blocks (color). The 4 luma blocks are the 4 squares in a block you often see. Both color blocks cover the entire block and are two different color planes. https://en.wikipedia.org/wiki/YUV There are 4 Y blocks and 1 U and 1 V block. Each one of those sub blocks references either the block above it or the block to the left of it (they all don't have to reference the same block). Which block they reference is determined on-the-fly (not hardcoded) so if a block is screwed up then it could cause blocks that would reference it, to not reference it or vice versa.
Hmmm, I suspect that color is not really important here, has anyone tried stripping/ignoring the chroma information and just generated "black & white"? The human visual system is *MUCH* more sensitive to intensity then color which is part of why the chroma is sampled at half (or less) of luma in most cases.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/07/2014 11:16 pm
Every block has 6 sub-blocks, 4 luma blocks (brightness) and 2 chroma blocks (color). The 4 luma blocks are the 4 squares in a block you often see. Both color blocks cover the entire block and are two different color planes. https://en.wikipedia.org/wiki/YUV There are 4 Y blocks and 1 U and 1 V block. Each one of those sub blocks references either the block above it or the block to the left of it (they all don't have to reference the same block). Which block they reference is determined on-the-fly (not hardcoded) so if a block is screwed up then it could cause blocks that would reference it, to not reference it or vice versa.
Hmmm, I suspect that color is not really important here, has anyone tried stripping/ignoring the chroma information and just generated "black & white"? The human visual system is *MUCH* more sensitive to intensity then color which is part of why the chroma is sampled at half (or less) of luma in most cases.

The data is there regardless, why not get it for free while we're at it?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: lgjy98d on 05/07/2014 11:23 pm
Is there a sense to create a grid with well-known MB locations?

I.e. 19.37.SS.mm timestamp spans MBs with following coordinates:
* colums 1 and 2 are for "19" hours,
* columns 3 and 4 are for ".35" minutes
* column 5 is for dot between minutes and seconds
* columns 6 and 7 are for SS. seconds
* column 8 and 9 are for mm miliseconds
* row 28 is for upper part of timestamp
* row 29 is for lower part

And once I'm experimenting with the image, I can see the recognizable elements, having seen them the POS in the stream would let me fixate the coordinates of that MB in the mmb string.

This would let some parts of the frame to snap in places even if there are other errors. It look like if whole row of MBs decodes properly the second row "sticks to previous one visually and lets to recognize the picture.

Is there a sense in the approach?

Besides the timestamp there are other notable features to watch for:
* dirt patch in the middle of top part of the picture.
* engine
* legs
* stage body edges (left and right)
* large dark spot above timestamp on the body of the stage
* camera lens edge (left and right bottom corners of the frame)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/07/2014 11:41 pm
iframe 11 a bit better (not much...):

-mmb 0:0:-1:1:0:1,0:12:27692,0:25:-1,1:25:82578,7:29:-1
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: michaelni on 05/07/2014 11:46 pm
How would one go about re-encoding the video with the replacement iframes, what's the command line for ffmpeg?

The following should work
assuming you have extracted  the frames with -vcodec copy into individual mpg4-img files
you could replace one with
ffmpeg -framerate 44999/3003 -i ../tests/lena.pnm -s 704x480  -aspect 22:15 -f image2 -r 44999/3003 image120.mpg4-img
the image this creates seems close enough to the original parameters to work

and the following would put it all back into one file:
ffmpeg -i image%3d.mpg4-img -vcodec copy video.avi
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/08/2014 12:27 am
iframe 7 with some tuning:

-mmb X:18494:01,18:05:-1,02:06:18794,15:12:-1,19:12:45121,19:15:-1,20:15:62858,
21:15:-1,23:15:-1:50:50:50,24:15:-1:-25:-25:-25,25:15:-1,29:15:64161,19:16:-1:-5:0:0,
21:16:68508,30:18:-1,34:18:76643,36:18:-1:20:20:20,37:18:77219,0:19:85528,14:20:-1,1:21:97689

also tried iframe 4 and 5 with no meaningful improvement
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: michaelni on 05/08/2014 01:25 am
Another way of doing this is to say to a block: don't "listen" to your left block, instead "listen" to the left "block" of 0,0 (initial state I guess). I believe these things are called "predictions" in mepg-4.

Implemented, example:
-mmb 40:3:-2
or
-mmb 20:19:-2:32:32:32:32:50:-11
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Okie_Steve on 05/08/2014 01:37 am
Every block has 6 sub-blocks, 4 luma blocks (brightness) and 2 chroma blocks (color). The 4 luma blocks are the 4 squares in a block you often see. Both color blocks cover the entire block and are two different color planes. https://en.wikipedia.org/wiki/YUV There are 4 Y blocks and 1 U and 1 V block. Each one of those sub blocks references either the block above it or the block to the left of it (they all don't have to reference the same block). Which block they reference is determined on-the-fly (not hardcoded) so if a block is screwed up then it could cause blocks that would reference it, to not reference it or vice versa.
Hmmm, I suspect that color is not really important here, has anyone tried stripping/ignoring the chroma information and just generated "black & white"? The human visual system is *MUCH* more sensitive to intensity then color which is part of why the chroma is sampled at half (or less) of luma in most cases.

The data is there regardless, why not get it for free while we're at it?
Because approximately half of the bit stream is chroma information and hence half of the errors needing to be dealt with. Given the way errors cascade color is probably just a distraction in this case.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: John_L on 05/08/2014 01:59 am
More on frame 2...
this is not from the try1.ts file, the older one.

40:3:11229,16:10:18210,40:28:76955,25:8:18680,4:28:-1
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Asmegin on 05/08/2014 02:36 am
Frame 3 is disgusting  >:(

I can't do anything with it (disclaimer: I have no idea what I'm doing)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: HALtheWise on 05/08/2014 02:44 am
Every block has 6 sub-blocks, 4 luma blocks (brightness) and 2 chroma blocks (color). The 4 luma blocks are the 4 squares in a block you often see. Both color blocks cover the entire block and are two different color planes. https://en.wikipedia.org/wiki/YUV There are 4 Y blocks and 1 U and 1 V block. Each one of those sub blocks references either the block above it or the block to the left of it (they all don't have to reference the same block). Which block they reference is determined on-the-fly (not hardcoded) so if a block is screwed up then it could cause blocks that would reference it, to not reference it or vice versa.
Hmmm, I suspect that color is not really important here, has anyone tried stripping/ignoring the chroma information and just generated "black & white"? The human visual system is *MUCH* more sensitive to intensity then color which is part of why the chroma is sampled at half (or less) of luma in most cases.

The data is there regardless, why not get it for free while we're at it?
Because approximately half of the bit stream is chroma information and hence half of the errors needing to be dealt with. Given the way errors cascade color is probably just a distraction in this case.
Unfortunately, the information that is needed to separate the chroma and intensity values has also been corrupted in many cases. AFAIK, until the macro blocks have been correctly identified and placed, separating the streams simply isn't possible. After that, however, it may in fact make sense to focus (at least initially) on bit-level corrections for the intensity alone before worrying about chroma. Most the I-frames haven't yet reached nearly-complete Macro block identification yet, though. It can also be much easier to identify the correct placement of blocks by using the color information.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/08/2014 03:21 am
Frame 3 is disgusting  >:(

I can't do anything with it (disclaimer: I have no idea what I'm doing)

Agreed. It seems to be totally random noise.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: dorkmo on 05/08/2014 03:45 am
Frame 3 is disgusting  >:(

I can't do anything with it (disclaimer: I have no idea what I'm doing)

Agreed. It seems to be totally random noise.

yeah i think its just the fact that it has so few bytes, its got 8927. while the best picture so far (iframe8) has 20428. i think its interesting that iframe 6 has the at most 29622 but so far looks just as garbled as others. might be the one with most room for improvement?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: GregA on 05/08/2014 06:02 am
Amazing work guys, I'm really looking forward to the video, and wish I had knowledge/ability to add.

With something like I Frame 3, would you be able to reconstruct based on I-Frame 2 plus all the P & B frames applied (which would theoretically be one frame before the i-frame)? I haven't seen that kind of thing mentioned here. I would assume you theoretically could - but that if i-frame 3 is gone then the preceding frames are probably also junk.

Similarly, would you replace a junk i-frame with a generic shot (plain blue plus image of rocket?) to then have the change frames act on, or would you leave the junk frame? (Of course, you need a good generic shot of the rocket for that to be possible!)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/08/2014 06:24 am
Frame 9, cleaned up some single-bit errors up top.

20:04:-1,24:04:16023,X:16410:01,16:5:-1,17:05:18692,X:19903:1,X:27244:1,
1:15:-1,3:16:79127,41:19:-1,8:20:96220,15:04:16023

More cleanup on iframe 9. I am less convinced now that the odd shapes on the right leg are bugged bits. They seem to be consistent across multiple blocks and sub-blocks.

X:19164:10,X:32551:80,X:32743:1,20:04:-1,24:04:16023,X:16410:01,16:5:-1,
17:05:18692,X:19903:1,X:27244:1,24:08:-1,25:08:32410,1:15:-1,3:16:79127,
41:19:-1,8:20:96220,15:04:16023
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/08/2014 07:08 am
Frame 9, cleaned up some single-bit errors up top.

20:04:-1,24:04:16023,X:16410:01,16:5:-1,17:05:18692,X:19903:1,X:27244:1,
1:15:-1,3:16:79127,41:19:-1,8:20:96220,15:04:16023

More cleanup on iframe 9. I am less convinced now that the odd shapes on the right leg are bugged bits. They seem to be consistent across multiple blocks and sub-blocks.

X:19164:10,X:32551:80,X:32743:1,20:04:-1,24:04:16023,X:16410:01,16:5:-1,
17:05:18692,X:19903:1,X:27244:1,24:08:-1,25:08:32410,1:15:-1,3:16:79127,
41:19:-1,8:20:96220,15:04:16023

They look a bit like the holding clamps which attach the legs to the body of the rocket.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/08/2014 07:52 am
I was about to suggest that depending on the altitude at that moment, which you could probably estimate knowing the time interval between iframes 9 and 10, perhaps the apparent distortion of the center portion of the right leg could be a splash of brightly-illuminated water between the camera and the leg.

But then after looking at the emerging details of frames 7 and 8, I'm beginning to fear your suspicions may be correct that they are not bugged bits but rather a big gnarly dent in the leg.  :o Maybe a chunk of muddy ice smashed into it in just the wrong way?

Looking at the image of the freshly-installed legs it's hard to picture how something like that could have happened, but then again I've seen the heartbreaking proof of what a mere  chunk of foam can do to composite structures with my own eyes down at the USSRC in in Huntsville. I fervently hope that the meticulous grooming of the iframes you guys are doing will prove this notion wrong. Keep up the utterly amazing, amazing work! You guys deserve medals!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/08/2014 08:07 am
I was about to suggest that depending on the altitude at that moment, which you could probably estimate knowing the time interval between iframes 9 and 10, perhaps the apparent distortion of the center portion of the right leg could be a splash of brightly-illuminated water between the camera and the leg.

But then after looking at the emerging details of frames 7 and 8, I'm beginning to fear your suspicions may be correct that they are not bugged bits but rather a big gnarly dent in the leg.  :o Maybe a chunk of muddy ice smashed into it in just the wrong way?

Looking at the image of the freshly-installed legs it's hard to picture how something like that could have happened, but then again I've seen the heartbreaking proof of what a mere  chunk of foam can do to composite structures with my own eyes down at the USSRC in in Huntsville. I fervently hope that the meticulous grooming of the iframes you guys are doing will prove this notion wrong. Keep up the utterly amazing, amazing work! You guys deserve medals!
I suspect what happened is that one of the bolts attaching the leg to the rocket did not let loose. So that part of the leg snapped off and is still attached to the body of the rocket. This event is somewhere in this video!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: luinil on 05/08/2014 08:09 am
It would be interesting to see if we can determine if the part of the leg is still on the rocket body (that would confirm the hypothesis).
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/08/2014 08:10 am
It would be interesting to see if we can determine if the part of the leg is still on the rocket body (that would confirm the hypothesis).
It could also have damaged the sidewall of the rocket. Maybe bending it outwards or possibly creating a hole.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: ugordan on 05/08/2014 08:40 am
I suspect what happened is that one of the bolts attaching the leg to the rocket did not let loose. So that part of the leg snapped off and is still attached to the body of the rocket.

That sounds plausible, IMHO.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: fatdeeman on 05/08/2014 08:58 am
I suspect what happened is that one of the bolts attaching the leg to the rocket did not let loose. So that part of the leg snapped off and is still attached to the body of the rocket. This event is somewhere in this video!

That sounds very plausible to me, those rams look similar to what you would see on an excavator so it was going to fully deploy whether the leg was partially attached or not, if the ram is as powerful as it looks then causing that damage wouldn't have made it break a sweat!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: eriblo on 05/08/2014 09:12 am
How sure can we be that it's actual damage to the leg? The "cutout" edges matches the video block, which for at least the first and the second image is obviously corrupt in some way. That the same block would be corrupt in all three images is of course a coincidence, but there are a lot of errors and they are most visible in these high contrast blocks (going from water to leg).

Two other things: First, it's hard to see the leg cylinder in these images, shouldn't it show more clearly at least in the last image (although we might be well into sub resolution territory here ;))? Second, I have a hard time believing that the bolts would be designed to be stronger than the leg - if it fails to release the failure modes you want are either failure to release the leg or failure of the bolt, not ripping half the leg and/or chunks of the tank off.

Also remember that even if the pistons are capable of large forces (lets say about 10-15 tonnes) they have a very small leverage during the start of the extension - maybe even negative, see relevant discussion threads.

EDIT to add: Relatively large forces (mass/4 times 1.5 or so for the angle) are only necessary if the legs support the stage on gas pressure alone, which is much less likely than some sort of locking mechanism. Even so they are nowhere close to hydraulic systems that operate at a magnitude higher pressure.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/08/2014 09:20 am
Frame 3 is disgusting  >:(

I can't do anything with it (disclaimer: I have no idea what I'm doing)

Agreed. It seems to be totally random noise.

yeah i think its just the fact that it has so few bytes, its got 8927. while the best picture so far (iframe8) has 20428. i think its interesting that iframe 6 has the at most 29622 but so far looks just as garbled as others. might be the one with most room for improvement?

Can it be that due to bit errors the size of Iframe3 is not correctly determined and part of IF3 is interpreted to be part of the next frame?

Cheers
Shanuson
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Microphobe on 05/08/2014 09:41 am
But then after looking at the emerging details of frames 7 and 8, I'm beginning to fear your suspicions may be correct that they are not bugged bits but rather a big gnarly dent in the leg. 

Hi, I think it might be some sort of splodge of high refractive index stuff on the camera bending light around it (see attached image: red circle in same place on the two frames, aligned them using the black mark above it) Its an oval region without any internal texture (ie blurred) in all other frames AFAICS, bordering another dirty mark, and still appears to be there in frame 12 where the leg is underwater. I reckon the leg was probably OK until hitting the water, but it would be nice to get more clear images of the "splodge".

Great work guys, although I don't understand a word of what you are talking about re: video decoding !!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/08/2014 10:46 am
Aligned iframe 1 from the previous solutions with:

-mmb 40:00:-1,14:02:5342,41:02:-1,7:05:17826,26:12:-1,0:14:50695,43:14:-1,31:16:66620,
7:21:-1,10:21:-1:-5:10:0,11:21:-1,20:21:87382,25:21:-1:0:-20:0,26:21:87827,
22:27:-1,28:27:109726,41:27:-1,03:28:112199,09:29:-1,15:29:131724,20:29:-1,24:29:132485

there are still two small sequences which I +/- randomly put: 14:02:5342,41:02:-1 and 0:14:50695,43:14:-1, and line 21 could be improved
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/08/2014 10:57 am
Aligned iframe 1 from the previous solutions with:

-mmb 40:00:-1,14:02:5342,41:02:-1,7:05:17826,26:12:-1,0:14:50695,43:14:-1,31:16:66620,
7:21:-1,10:21:-1:0:-16:0,11:21:-1,20:21:87382,22:27:-1,28:27:109726,41:27:-1,03:28:112199,
09:29:-1,15:29:131724,20:29:-1,24:29:132485

there are still two small sequences which I +/- randomly put: 14:02:5342,41:02:-1 and 0:14:50695,43:14:-1

Be careful how you pick blocks, you can start to create an image that does not represent reality. The lower portion of the rocket is not visible.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/08/2014 11:01 am
Frame 3 is disgusting  >:(

I can't do anything with it (disclaimer: I have no idea what I'm doing)

Agreed. It seems to be totally random noise.

yeah i think its just the fact that it has so few bytes, its got 8927. while the best picture so far (iframe8) has 20428. i think its interesting that iframe 6 has the at most 29622 but so far looks just as garbled as others. might be the one with most room for improvement?

Can it be that due to bit errors the size of Iframe3 is not correctly determined and part of IF3 is interpreted to be part of the next frame?

Cheers
Shanuson

Possible. Possibly with some better work on the transport stream that can be determined to be the case or ruled out. The try1.ts file's iframe3 is a bit larger than the others at 9005 bytes.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/08/2014 11:02 am
Aligned iframe 1 from the previous solutions with:

-mmb 40:00:-1,14:02:5342,41:02:-1,7:05:17826,26:12:-1,0:14:50695,43:14:-1,31:16:66620,
7:21:-1,10:21:-1:0:-16:0,11:21:-1,20:21:87382,22:27:-1,28:27:109726,41:27:-1,03:28:112199,
09:29:-1,15:29:131724,20:29:-1,24:29:132485

there are still two small sequences which I +/- randomly put: 14:02:5342,41:02:-1 and 0:14:50695,43:14:-1

Be careful how you pick blocks, you can start to create an image that does not represent reality. The lower portion of the rocket is not visible.

No no, I used iframe 2 to align the image features, only the two small mentioned blocks are not where they should be. There might be condensation on the lens or the rocket was in a cloud, reducing a lot the contrast.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/08/2014 11:07 am
In other news. Iframe8 is becoming amazing. I'm still working on the lower part, but if anyone want's to try their hand at blocks 05:21 through 27:21 or fixing the ugly color/brightness cast into the bottom right, it'd be much appreciated. That's bit 107247 to 108877 or 1630 bits spread over 23 blocks for an average of 71 bits per block.

MMB split out into multiple lines to be readable.
08:14:-1:15:-15:1:14:-2:1,
09:14:56683,
05:21:-1:0:-20:-5:2,
06:21:-1:-20:-20:6:5,
07:21:-1:0:0:3:5,
08:21:-1:0:0:0:0,
09:21:-1:10:15:0:0,
10:21:-1:15:15:0:0,
11:21:-1:5:10,
12:21:-1:5:3:0:0:-2,
13:21:-1:4:4,
14:21:-1,
28:21:108878,
X:126932:80,
04:27:-1:0:-4:13:7,
05:27:135412,
X:143273:80,
X:143386:80,
06:28:144012

As a note, in every image there should be two black triangles in the bottom right corners. I suspect this is part of the camera housing. They are good cues for reconstructing the lower part of the image.

Edit: Attached before image.

Edit2: Forgot to mention. I used try1.ts iframe8.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: ugordan on 05/08/2014 11:09 am
Nice!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/08/2014 11:24 am
As to the previous conversation about things breaking off the legs. In this iframe you can clearly see a white lump on the right side that would probably line up with that leg. I'm reasonably certain that that section of the image isn't corrupted, or if it is corrupted would be very minor and something of the leg would still be showing. Good chance something actually did bust off.

Edit: One more thing. If this is actually the case, getting this video showing the break-off point suddenly becomes pretty important for SpaceX if they weren't aware of it. Gives more reason for us to continue work on it after Saturday's launch.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/08/2014 11:38 am
Hi, I think it might be some sort of splodge of high refractive index stuff on the camera bending light around it (see attached image: red circle in same place on the two frames, aligned them using the black mark above it) Its an oval region without any internal texture (ie blurred) in all other frames AFAICS, bordering another dirty mark, and still appears to be there in frame 12 where the leg is underwater. I reckon the leg was probably OK until hitting the water, but it would be nice to get more clear images of the "splodge".

I like that idea best! Phew! Good thinking! And if you peer at frame 10 in the right way you can start to make out a smudge there as well. You'd think that if it was a water droplet doing the lensing that it would move with the wind, so maybe it was some sort of damage to the window of the camera housing, or something that froze at a higher altitude and then didn't melt by the time it landed. We'll probably see more clearly when these heroic bit-wranglers persuade frames 4 and 5 to emerge and we can start to make out the deployment.

Maybe someone should try an alignment of frame 2 with the "smudge" idea in mind, coupling it with the little black quotation mark.

EDIT: Also, smudge or damage, either way it looks worse than it actually is. Take a look at the tip of the leg in the shop-floor photo: it's just a hollow shell of composite. And take a look at Grasshopper - the load-bearing pieces of the legs are the steel (or some sort of highstrengthtoweightium alloy) tubes, and even if the fairing got smashed by ice or ripped away from a stuck latch, it would have absolutely no implications for the business parts of the leg.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: MTom on 05/08/2014 11:45 am
As to the previous conversation about things breaking off the legs. In this iframe you can clearly see a white lump on the right side that would probably line up with that leg. I'm reasonably certain that that section of the image isn't corrupted, or if it is corrupted would be very minor and something of the leg would still be showing. Good chance something actually did bust off.

Edit: One more thing. If this is actually the case, getting this video showing the break-off point suddenly becomes pretty important for SpaceX if they weren't aware of it. Gives more reason for us to continue work on it after Saturday's launch.

Not sure, the waves on the water surface also missing in this section.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: TrueBlueWitt on 05/08/2014 12:02 pm
Rampant speculation about something "damaged" when it could as easily be a video artifact or misaligned video data block is why Elon and SpaceX are leary about releasing such things in the first place. To me the anamoly it looks like it has right angled edges which align with the macro blocks.  Let's not get too crazy about this yet.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Mongo62 on 05/08/2014 12:05 pm
I have to go with the "smudge on the lens" theory as well. If (as appears to be the case) a suspiciously blank patch is present in the same location on the images from before leg deployment, as well as after the leg has been partly submerged, then it is very unlikely to be a physical chunk missing from the leg, in my non-expert opinion.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/08/2014 12:09 pm
I have to go with the "smudge on the lens" theory as well. If (as appears to be the case) a suspiciously blank patch is present in the same location on the images from before leg deployment, as well as after the leg has been partly submerged, then it is very unlikely to be a physical chunk missing from the leg, in my non-expert opinion.

That's very true. Either way, we'll find out as we get the video completed.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/08/2014 12:09 pm
Rampant speculation about something "damaged" when it could as easily be a video artifact or misaligned video data block is why Elon and SpaceX are leary about releasing such things in the first place. To me the anamoly it looks like it has right angled edges which align with the macro blocks.  Let's not get too crazy about this yet.

I wouldn't call it "rampant" - that implies "unbridled" and "widespread," and nobody here seems particularly breathless and it's only been four hours. It's a little more than "idle," yes, but not "rampant." Plus, it's only been four hours and folks may already have come up with another fixed reference point for MB alignment in addition to the quotation mark. Welcome to the wonderful and exciting world of "crowdsourcing."
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/08/2014 12:27 pm
Rampant speculation about something "damaged" when it could as easily be a video artifact or misaligned video data block is why Elon and SpaceX are leary about releasing such things in the first place. To me the anamoly it looks like it has right angled edges which align with the macro blocks.  Let's not get too crazy about this yet.

I wouldn't call it "rampant" - that implies "unbridled" and "widespread," and nobody here seems particularly breathless and it's only been four hours. It's a little more than "idle," yes, but not "rampant." Plus, it's only been four hours and folks may already have come up with another fixed reference point for MB alignment in addition to the quotation mark. Welcome to the wonderful and exciting world of "crowdsourcing."

Any shape in highly compressed MPEG will tend to "blur" because of the reduction of the pixel data to sets of something akin to 2D Fourier transforms (Discrete Cosine Transform (https://en.wikipedia.org/wiki/Discrete_cosine_transform)) resulting in "ringing" on any sharp edges in an image. Further those blurs cannot extend beyond the edge a block (or a sub-block) which can lead to sharp cutoffs.

Every 8x8 pixel sub-block is encoded like this. Example from wikipedia of encoding an image of the letter A.
(https://upload.wikimedia.org/wikipedia/commons/5/5e/Idct-animation.gif)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mikes on 05/08/2014 12:48 pm
Frame 6 got a chunk of the body

0:0:-1,0:19:121762,1:28:202509

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/08/2014 12:54 pm
In other news. Iframe8 is becoming amazing. I'm still working on the lower part, but if anyone want's to try their hand at blocks 05:21 through 27:21 or fixing the ugly color/brightness cast into the bottom right, it'd be much appreciated. That's bit 107247 to 108877 or 1630 bits spread over 23 blocks for an average of 71 bits per block.

MMB split out into multiple lines to be readable.
08:14:-1:15:-15:1:14:-2:1,
09:14:56683,
05:21:-1:0:-20:-5:2,
06:21:-1:-20:-20:6:5,
07:21:-1:0:0:3:5,
08:21:-1:0:0:0:0,
09:21:-1:10:15:0:0,
10:21:-1:15:15:0:0,
11:21:-1:5:10,
12:21:-1:5:3:0:0:-2,
13:21:-1:4:4,
14:21:-1,
28:21:108878,
X:126932:80,
04:27:-1:0:-4:13:7,
05:27:135412,
X:143273:80,
X:143386:80,
06:28:144012

As a note, in every image there should be two black triangles in the bottom right corners. I suspect this is part of the camera housing. They are good cues for reconstructing the lower part of the image.

Edit: Attached before image.

Edit2: Forgot to mention. I used try1.ts iframe8.

That's amazing!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: ugordan on 05/08/2014 12:57 pm
If this is actually the case, getting this video showing the break-off point suddenly becomes pretty important for SpaceX if they weren't aware of it.

They're bound to already have telemetry that would indicate this, if it indeed happened at all. Things like pressure vs. time profile in the cylinder compared to the other 3 indicating more resistance to deployment, etc.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/08/2014 12:58 pm
In other news. Iframe8 is becoming amazing. I'm still working on the lower part, but if anyone want's to try their hand at blocks 05:21 through 27:21 or fixing the ugly color/brightness cast into the bottom right, it'd be much appreciated. That's bit 107247 to 108877 or 1630 bits spread over 23 blocks for an average of 71 bits per block.

MMB split out into multiple lines to be readable.
08:14:-1:15:-15:1:14:-2:1,
09:14:56683,
05:21:-1:0:-20:-5:2,
06:21:-1:-20:-20:6:5,
07:21:-1:0:0:3:5,
08:21:-1:0:0:0:0,
09:21:-1:10:15:0:0,
10:21:-1:15:15:0:0,
11:21:-1:5:10,
12:21:-1:5:3:0:0:-2,
13:21:-1:4:4,
14:21:-1,
28:21:108878,
X:126932:80,
04:27:-1:0:-4:13:7,
05:27:135412,
X:143273:80,
X:143386:80,
06:28:144012

As a note, in every image there should be two black triangles in the bottom right corners. I suspect this is part of the camera housing. They are good cues for reconstructing the lower part of the image.

Edit: Attached before image.

Edit2: Forgot to mention. I used try1.ts iframe8.

That's amazing!

Key thing is that all the information is more or less still there. It's just that the nature of MPEG means that flipping a single bit from 0 to 1 (or reverse) can mean the entire image being decoded wrong. MPEG is designed to be error resilient despite this though (its how we all get our cable TV or satellite TV or over the air digital TV), just that SpaceX didn't turn on many of the error resilience features apparently that could have stopped the propagation of the errors as far as they did.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/08/2014 01:23 pm
Implemented, example:
-mmb 40:3:-2
or
-mmb 20:19:-2:32:32:32:32:50:-11

Is the source for this version available for download? Git does not work for me.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/08/2014 01:46 pm
Implemented, example:
-mmb 40:3:-2
or
-mmb 20:19:-2:32:32:32:32:50:-11

Is the source for this version available for download? Git does not work for me.

I had forgotten about it, just pushed it up to git a few hours ago. If you grab the source from the URL in the install.txt it should be the right version.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Herb Schaltegger on 05/08/2014 02:06 pm
Great work on this, mlinder. Thanks for sharing your efforts here on NSF.

ASIDE: Watching all this amateur Kremlinology on some badly-corrupted individual video frames reminds me somewhat unpleasantly of watching the internets go nuts over similarly-confusing bits of video and tidbits of information following STS-107. I can see why SpaceX didn't rush to release the video immediately. I sincerely hope there is better video (preferably from observation ships/aircraft) following Saturday's launch.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: ugordan on 05/08/2014 02:11 pm
I sincerely hope there is better video (preferably from observation ships/aircraft) following Saturday's launch.

I don't know how much "better" such video would be. On another site I saw a claim that the telemetry plane on CRS-3 was over 20 miles away. The CASSIOPE launch aerial images released also suggest a pretty large distance from the vehicle. I guess safety trumps closeup observations.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mmeijeri on 05/08/2014 02:18 pm
Maybe they can enable some of the resilience features this time round?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: ugordan on 05/08/2014 02:20 pm
If it was that simple and at no additional cost (like increase in b/w requirements) I imagine they'd already have done that. After all, even the video on the way up breaks up fairly badly at times.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Adaptation on 05/08/2014 02:23 pm
Frame 3 is disgusting  >:(

I can't do anything with it (disclaimer: I have no idea what I'm doing)
Agreed. It seems to be totally random noise.
yeah i think its just the fact that it has so few bytes, its got 8927. while the best picture so far (iframe8) has 20428. i think its interesting that iframe 6 has the at most 29622 but so far looks just as garbled as others. might be the one with most room for improvement?
Can it be that due to bit errors the size of Iframe3 is not correctly determined and part of IF3 is interpreted to be part of the next frame?

Cheers
Shanuson
Possible. Possibly with some better work on the transport stream that can be determined to be the case or ruled out. The try1.ts file's iframe3 is a bit larger than the others at 9005 bytes.

Been rather busy this week, I'm going to try to finish the todo list for my program this weekend then start in on a fresh reconstruction of the raw.ts, no guarantees it will yield much but I'm hopeful. 
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/08/2014 02:33 pm
If it was that simple and at no additional cost (like increase in b/w requirements) I imagine they'd already have done that. After all, even the video on the way up breaks up fairly badly at times.

If they'd turn off interlacing they'd get a big boost to compressibility. Compressing interlaced video at the frame level rather than field level gives you nightmare terrible compression.

I'd guess its because they got some stock "space rated" (super old technology) cameras that are old analog NTSC cameras and they transcode that to a digital video signal for downlink.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/08/2014 02:44 pm
ASIDE: Watching all this amateur Kremlinology on some badly-corrupted individual video frames reminds me somewhat unpleasantly of watching the internets go nuts over similarly-confusing bits of video and tidbits of information following STS-107. I can see why SpaceX didn't rush to release the video immediately. I sincerely hope there is better video (preferably from observation ships/aircraft) following Saturday's launch.

I think you're overreacting. The "kremlinology" you deride is in reality a genuine and heartfelt cooperative effort among enthusiastic and obviously highly-capable fans of SpaceX to suss out the precisely correct layout of each iframe, not just some pointless exercise in conspiracy-theory grandstanding. The "secretive organization" and the "limited data points" really do exist, in the form of the scrambled and uncharted bits of each iframe. If there is, in reality, a smudge or dent on the lens or a dent in the leg, then that provides important reference information for untangling the rest of the iframes. If you go grab a cup of coffee and then take another look at the astounding progress that has been made so far, I'm sure you'll feel more pleasant about it.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/08/2014 02:51 pm
iframe 13 from try1:

40:1:-1,0:10:-1:0:-10,1:10:-1:,4:10:49027,8:29:-1

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: AncientU on 05/08/2014 02:54 pm
iframe 13 from try1:

40:1:-1,0:10:-1:0:-10,1:10:-1:,4:10:49027,8:29:-1

Semi-submerged legs?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: fatdeeman on 05/08/2014 03:02 pm
iframe 13 from try1:

40:1:-1,0:10:-1:0:-10,1:10:-1:,4:10:49027,8:29:-1

Oh wow look at that! 
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: ugordan on 05/08/2014 03:02 pm
If they'd turn off interlacing they'd get a big boost to compressibility. Compressing interlaced video at the frame level rather than field level gives you nightmare terrible compression.

Yes, this is actually something I've been whining about ever since they switched to this method (CRS-2 I think) and the video is still 15 fps interlaced. I'm not getting my hopes up anymore.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Joffan on 05/08/2014 03:04 pm
iframe 13 from try1:

40:1:-1,0:10:-1:0:-10,1:10:-1:,4:10:49027,8:29:-1


Looks like the stage moved to the right, compared to the initial splash?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/08/2014 03:16 pm
iframe 13 from try1:

40:1:-1,0:10:-1:0:-10,1:10:-1:,4:10:49027,8:29:-1


Looks like the stage moved to the right, compared to the initial splash?

Hard to tell, remember that color and "brightness" of the blocks are dependent on nearby blocks, often unpredictably so, when there are errors. You should only rely on the texture patterns themselves for determining information here IMO.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Elmar Moelzer on 05/08/2014 03:19 pm
Yeah, interlacing is a terrible idea in pretty much any case, especially in the digital age. I was appalled when they decided to continue that for the HD standard(s).
Definitely bad for compression and when you are watching it on a computer.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Lars_J on 05/08/2014 03:24 pm

But then after looking at the emerging details of frames 7 and 8, I'm beginning to fear your suspicions may be correct that they are not bugged bits but rather a big gnarly dent in the leg. 

Hi, I think it might be some sort of splodge of high refractive index stuff on the camera bending light around it (see attached image: red circle in same place on the two frames, aligned them using the black mark above it) Its an oval region without any internal texture (ie blurred) in all other frames AFAICS, bordering another dirty mark, and still appears to be there in frame 12 where the leg is underwater. I reckon the leg was probably OK until hitting the water, but it would be nice to get more clear images of the "splodge".

Great work guys, although I don't understand a word of what you are talking about re: video decoding !!

Yes, I think the "dirt on lens" is the more likely explanation.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mmeijeri on 05/08/2014 03:28 pm
I'd guess its because they got some stock "space rated" (super old technology) cameras that are old analog NTSC cameras and they transcode that to a digital video signal for downlink.

If they're using a COTS encoder then they can likely turn off interlacing and maybe enable some resilience features. That either means they're not using a COTS encoder, or getting video footage wasn't a priority before.

Having worked for Siqura BV, I naturally recommend their EVE (http://www.eve.nl/) line of encoders. :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/08/2014 03:44 pm
More work on iframe8. Will post the mmb later.

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/08/2014 03:48 pm
More work on iframe8. Will post the mmb later.



Super impressive.

Now a quick task for anyone able.

Can we create a .gif, showing the original frame8 and the one above, with before and after written on them?

The .gif needs to be 350 pixels wide and whatever you require in length and then I'll use that in the article I'm going to put on shorty.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/08/2014 03:48 pm
This is a good point, and it got me thinking. If everything goes as planned, SpaceX will launch again in a few days, and I think they're planning to attempt recovery again. This could upstage the (arguably more historic) first landing video. We need to put together a video with what we've got today, tomorrow at the lastest. If necessary, I can hand-assemble something from the iframes and pieces of the pframes. It wouldn't be as pretty as what arnezami and mlindner can do, so I'll wait until I hear from them.

It's actually a lot easier than that. If no one else does it by Friday Eastern time then I'll be modifying ffmpeg to add a frame specifier to the mmb options. Then we simply pile all the mmb options into one long entry and run ffmpeg with it. viola, produced video. This would also allow us to try tweaking some p-frames if we so wish.

Producing a video now with what we have won't be very attractive yet. We need to fine tune a lot of the frames still. I'm doing that now for iframe8, which should hopefully look really really good. This means doing things like whenever you use -1 and replace the block, fine tune all the luma and chroma settings for it to have the least effect on surrounding blocks.

Are you still planning on doing the frame specifier modifications tonight? Right now we have mods for two different video streams. Are you going to be able to integrate those, or do we need to start a porting effort to a newer reconstruction?

I think it is likely that it could take weeks or months to get the video to a level of quality that is going to satisfy everyone working on it. Chris, can you ask PAO what timeframe would be most useful to them?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Zach121k on 05/08/2014 03:48 pm
I just started reading this thread today, and I want to say... this all looks beautiful.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SpaceX_MS on 05/08/2014 03:56 pm

I think it is likely that it could take weeks or months to get the video to a level of quality that is going to satisfy everyone working on it. Chris, can you ask PAO what timeframe would be most useful to them?

You guys are ahead of the game, the only real game in town actually. Provide what you can, when you can. This is an unbelievable effort on show here.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/08/2014 04:01 pm
This is a good point, and it got me thinking. If everything goes as planned, SpaceX will launch again in a few days, and I think they're planning to attempt recovery again. This could upstage the (arguably more historic) first landing video. We need to put together a video with what we've got today, tomorrow at the lastest. If necessary, I can hand-assemble something from the iframes and pieces of the pframes. It wouldn't be as pretty as what arnezami and mlindner can do, so I'll wait until I hear from them.

It's actually a lot easier than that. If no one else does it by Friday Eastern time then I'll be modifying ffmpeg to add a frame specifier to the mmb options. Then we simply pile all the mmb options into one long entry and run ffmpeg with it. viola, produced video. This would also allow us to try tweaking some p-frames if we so wish.

Producing a video now with what we have won't be very attractive yet. We need to fine tune a lot of the frames still. I'm doing that now for iframe8, which should hopefully look really really good. This means doing things like whenever you use -1 and replace the block, fine tune all the luma and chroma settings for it to have the least effect on surrounding blocks.

Are you still planning on doing the frame specifier modifications tonight? Right now we have mods for two different video streams. Are you going to be able to integrate those, or do we need to start a porting effort to a newer reconstruction?

I think it is likely that it could take weeks or months to get the video to a level of quality that is going to satisfy everyone working on it. Chris, can you ask PAO what timeframe would be most useful to them?

The mods I described would probably only support a single transport stream and changes to iframes in that transport stream. Ideally people can port their changes to try1.ts as it has more data in the frames. If not, I'm not too worried about just waiting until after the next flight before getting something out unless someone wants to work on modifying the iframes with the changes and combining them with the transport streams that way.

I won't get the changes done tonight (Eastern) will probably have to do it tomorrow.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/08/2014 04:02 pm

I think it is likely that it could take weeks or months to get the video to a level of quality that is going to satisfy everyone working on it. Chris, can you ask PAO what timeframe would be most useful to them?

You guys are ahead of the game, the only real game in town actually. Provide what you can, when you can. This is an unbelievable effort on show here.

Awesome. Thanks! Glad we could be of use. I hope this helps SpaceX to consider releasing more information to the knowledgable public in the future! We're always ready to get more candy, even if it requires some cleaning first.

The world is on SpaceX's side even if a few companies/corporations/governments aren't.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: ugordan on 05/08/2014 04:11 pm
Can we create a .gif, showing the original frame8 and the one above, with before and after written on them?

The .gif needs to be 350 pixels wide and whatever you require in length and then I'll use that in the article I'm going to put on shorty.

Something like this?

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/08/2014 04:12 pm
Can we create a .gif, showing the original frame8 and the one above, with before and after written on them?

The .gif needs to be 350 pixels wide and whatever you require in length and then I'll use that in the article I'm going to put on shorty.

Something like this?



Something like that! :)

Splendid. Thanks much!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/08/2014 04:41 pm
Ok, so this thread is now fully linked up on the news site via the Static Fire article - in which I added some paras to the end per the video repair effort.

http://www.nasaspaceflight.com/2014/05/spacex-falcon-9-static-fire-test/

Had to keep it light, given the wider audience on the news site, but linked the thread and linked Arnezami's great post on how this process is taking place.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: AncientU on 05/08/2014 04:54 pm
Nice article Chris!
The ongoing video recovery effort on NSF is wonderful and the subject even more so.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: grythumn on 05/08/2014 05:08 pm
Another day, another cygwin build. This is from today's git and adds the -2 parameter to mmb. As usual, only grab the DLLs if you don't already have them.

-Bob
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: dorkmo on 05/08/2014 05:15 pm
frame 5 so far

0:0:-1,26:12:35375,12:13:-1,16:13:38823,17:13:-1,19:13:39153,22:13:-1,27:13:40118,29:13:-1,33:13:40830,0:15:50056

2 or 3 clear blocks of leg. some of the stage, a row of it is miss aligned.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: John_L on 05/08/2014 05:32 pm
Rampant speculation about something "damaged" when it could as easily be a video artifact or misaligned video data block is why Elon and SpaceX are leary about releasing such things in the first place. To me the anamoly it looks like it has right angled edges which align with the macro blocks.  Let's not get too crazy about this yet.

What gives you the impression that SpaceX and/or Elon Musk "are leary" about releasing the video.  They specifically released the video to the public in the hopes that someone (like this band of forum members) would take up the challenge of reconstructing the video.  Alot of us have been working hard to learn mpeg decoding better, have worked directly with the curator of FFMpeg to get new functionality created within FFMpeg, creating new tools on the fly so we can edit parts of the video that have never been editable before. 

I seriously doubt that a few people speculating about the video in a forum that's read mainly by it's members, and, if I may add, SpaceX themselves, would be damaging, and, they are also following our work closely, and are happy that we are putting for the effort for nothing more than the enjoyment we're getting from positive results, and the desire for us to HELP SpaceX gain extremely useful data regarding the last launches attempt to land the stage.

We're all SpaceX fans here...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/08/2014 05:39 pm

I think it is likely that it could take weeks or months to get the video to a level of quality that is going to satisfy everyone working on it. Chris, can you ask PAO what timeframe would be most useful to them?

You guys are ahead of the game, the only real game in town actually. Provide what you can, when you can. This is an unbelievable effort on show here.

This is truly an d unbelievable effort. And is amazing to see the amount of details that have been recovered form the initial frames, and the conviction and talent of all people that have been involved....

This is what we have so far guys! And let's keep with the good job:


Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Swatch on 05/08/2014 05:44 pm
Chris, may I recommend you add links to the first post of this thread detailing how to start helping with this effort and some of the better posts in the thread describing what's going on?  Also, a link to the wikipage they setup?

I think it will help orient people without going through 24 pages of stuff.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: John_L on 05/08/2014 05:46 pm
I don't know how much "better" such video would be. On another site I saw a claim that the telemetry plane on CRS-3 was over 20 miles away. The CASSIOPE launch aerial images released also suggest a pretty large distance from the vehicle. I guess safety trumps closeup observations.

There was really bad weather, so ships and planes that were slotted to be much closer to the "action" weren't able to get there for safety reasons.  If this were a NASA launch, NASA would have likely had a mission rule stating that they wouldn't launch due to the fact that the weather (at the stage landing site) was bad, and might have scrubbed the launch.  In this case however, NASA is SpaceX's customer, and SpaceX is using the launch to test the recovery.  NASA's objective here is to get goods up to the ISS, so SpaceX had to go with things as they were, and hoped they could what data they could.  Bad weather at the 1st stage landing site wasn't going to stop the launch, but it did prevent planes and ships from getting close enough to view/better record the data.

The weather this Saturday will hopefully be better so that they can get close enough to A) record video directly from a 3rd person perspective (planes or ships or both), and B) get better data recorded from the stage itself.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: AncientU on 05/08/2014 05:52 pm

I think it is likely that it could take weeks or months to get the video to a level of quality that is going to satisfy everyone working on it. Chris, can you ask PAO what timeframe would be most useful to them?

You guys are ahead of the game, the only real game in town actually. Provide what you can, when you can. This is an unbelievable effort on show here.

This is truly an d unbelievable effort. And is amazing to see the amount of details that have been recovered form the initial frames, and the conviction and talent of all people that have been involved....

This is what we have so far guys! And let's keep with the good job:

Just curious...
The original releases (raw and 'repaired') had frames as the stage was descending through the cloud layer which fit before this sequence, and the stage falling over at the end which come after -- both best seen in raw IMO.  Are those frames somewhere in this amazing recovery effort?

Link back to OP:
http://forum.nasaspaceflight.com/index.php?topic=34597.msg1191316#msg1191316
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jakusb on 05/08/2014 05:52 pm


I think it is likely that it could take weeks or months to get the video to a level of quality that is going to satisfy everyone working on it. Chris, can you ask PAO what timeframe would be most useful to them?

You guys are ahead of the game, the only real game in town actually. Provide what you can, when you can. This is an unbelievable effort on show here.

This is truly an d unbelievable effort. And is amazing to see the amount of details that have been recovered form the initial frames, and the conviction and talent of all people that have been involved....

This is what we have so far guys! And let's keep with the good job:
Great! I was about to ask for something like this. Or make it myself. :)
One suggestion/request: a bit slower sequence? Either more realistic in terms of actual movie speed or even slower to be able to find common blocks?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/08/2014 05:59 pm
Chris, may I recommend you add links to the first post of this thread detailing how to start helping with this effort and some of the better posts in the thread describing what's going on?

I think it will help orient people without going through 24 pages of stuff.

I'd rather people read through what is only 24 pages. It takes 5-10 minutes to read and new people to the thread will gain an appreciation of the work and the processes involved. :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/08/2014 06:06 pm


I think it is likely that it could take weeks or months to get the video to a level of quality that is going to satisfy everyone working on it. Chris, can you ask PAO what timeframe would be most useful to them?

You guys are ahead of the game, the only real game in town actually. Provide what you can, when you can. This is an unbelievable effort on show here.

This is truly an d unbelievable effort. And is amazing to see the amount of details that have been recovered form the initial frames, and the conviction and talent of all people that have been involved....

This is what we have so far guys! And let's keep with the good job:
Great! I was about to ask for something like this. Or make it myself. :)
One suggestion/request: a bit slower sequence? Either more realistic in terms of actual movie speed or even slower to be able to find common blocks?

Sure. How about this:

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/08/2014 06:17 pm
I began to rebuild the iframe 6 (normal version); the MB containing the legs are visible but lots of the neighboring blocks are corrupted, so the reconstruction is not easy. I am not completely sure of the exact position of 2-3 MB sequences over the water. Maybe it works better with try1.ts?

Is it me or are the legs still not completely deployed? It seems not to be exactly the same as in the following frames.

0:0:-1,
2:1:6129,17:1:-1,
1:02:11561,6:3:-1
3:12:56455,18:12:-1,
26:12:61670,27:12:-1,16:13:-1:50,
17:13:64694,28:13:-1,
32:13:71022,4:14:-1,
27:14:78837,5:15:-1,
19:15:92898,23:15:-1,20:16:-1:0:0:0:-12,22:16:-1,
21:17:106645,27:18:-1,29:18:-1:0:0:0:-25,30:18:-1,
0:19:121762,04:21:-1:0:0:0:20,05:21:-1,
31:22:159772,0:24:-1,
30:25:184570,07:27:-1,
35:27:200466

(edit: top of frame added)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mme on 05/08/2014 06:40 pm
...
Just curious...
The original releases (raw and 'repaired') had frames as the stage was descending through the cloud layer which fit before this sequence, and the stage falling over at the end which come after -- both best seen in raw IMO.  Are those frames somewhere in this amazing recovery effort?

Link back to OP:
http://forum.nasaspaceflight.com/index.php?topic=34597.msg1191316#msg1191316
The the wiki (http://spacexlanding.wikispaces.com) has a column for a frame 0 that reads:
Quote
#IFPosition in fileSize in bytesFrame #mp4-img linkimageContributors
0~254956n/an/anot extracted yet, transport stream issue?
I'm not involved, and knew nothing about MPEG a few days ago. But for the early frames, I guess there is an issue in extracting the data for that i-frame. Of course I don't know how it's playable at all if that's the case.

As for the falling over, the repair effort is on the i-frames from which all subsequent frames are deltas. So I assume the "falling over" is after the last i-frame.  Assuming the i-frames can be reconstructed, the playback of the falling over will be better.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/08/2014 06:48 pm
frame 5 so far

0:0:-1,26:12:35375,12:13:-1,16:13:38823,17:13:-1,19:13:39153,22:13:-1,27:13:40118,29:13:-1,33:13:40830,0:15:50056

2 or 3 clear blocks of leg. some of the stage, a row of it is miss aligned.

The rocket is 2 MB rows to far up compared to IF7 or 8
otherwise nice pieces of the leg.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Swatch on 05/08/2014 07:12 pm
Chris, may I recommend you add links to the first post of this thread detailing how to start helping with this effort and some of the better posts in the thread describing what's going on?

I think it will help orient people without going through 24 pages of stuff.

I'd rather people read through what is only 24 pages. It takes 5-10 minutes to read and new people to the thread will gain an appreciation of the work and the processes involved. :)

Maybe just a link to the wiki page then?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: MarsInMyLifetime on 05/08/2014 07:25 pm
Chris, may I recommend you add links to the first post of this thread detailing how to start helping with this effort and some of the better posts in the thread describing what's going on?

I think it will help orient people without going through 24 pages of stuff.

I'd rather people read through what is only 24 pages. It takes 5-10 minutes to read and new people to the thread will gain an appreciation of the work and the processes involved. :)

Maybe just a link to the wiki page then?
I think he's saying that if you read the pages, you'll come across the link anyway, but by then you'll have much better context and appreciation for the incredible effort going on. As a Web publisher myself, I have to keep in mind (and be reminded) that I want to try to keep visitors ON my site for as long as I can. Besides, the best exhilaration is when you keep coming across ever new news here, right?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: ugordan on 05/08/2014 07:33 pm
I don't know how much "better" such video would be. On another site I saw a claim that the telemetry plane on CRS-3 was over 20 miles away. The CASSIOPE launch aerial images released also suggest a pretty large distance from the vehicle. I guess safety trumps closeup observations.

There was really bad weather, so ships and planes that were slotted to be much closer to the "action" weren't able to get there for safety reasons. 

SpaceX couldn't have possibly asked for *better* weather for the CASSIOPE launch and, yet, the photography of the 1st stage was still very distant and blurry.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Swatch on 05/08/2014 07:55 pm
...
Maybe just a link to the wiki page then?
I think he's saying that if you read the pages, you'll come across the link anyway, but by then you'll have much better context and appreciation for the incredible effort going on. As a Web publisher myself, I have to keep in mind (and be reminded) that I want to try to keep visitors ON my site for as long as I can. Besides, the best exhilaration is when you keep coming across ever new news here, right?

Duly noted... but I was trying to be lazy and have had trouble finding the link again for myself with it buried in the thread. :P 

Can't always spend a lot of time digging through the thread every time I want to see the updated pics when  there are more rockets to build!  ;)  Things are hectic around here, but it's amazing following the stuff you guys are doing with this video.  :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/08/2014 08:28 pm
Hi guys,

This is just for inspiration:

https://www.youtube.com/watch?v=G9933rIJEVw

"We've always defined ourselves by the ability to overcome the impossible"

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: CraigLieb on 05/08/2014 08:48 pm
Hi guys,

This is just for inspiration:

"We've always defined ourselves by the ability to overcome the impossible"

Regards,

arnezami

Terrific video.

And also useful to note the side view of the rocket descending into the smoke,
 much like the stage returning as it hits the surface of the water.

I suppose this is obvious to the rocket scientists that we would see just parts of the struts.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: grythumn on 05/08/2014 08:51 pm
More cleanup of iframe 9. These are from try1, but not much changed at least in the top of the frame...

Now that iframe 8 is almost done, pulled the clock position from it and set the top of this clock to that... gives us a reference point for the bottom of the frame.

X:16410:01,X:19164:10,X:19903:1,X:20037:80,X:27244:1,X:32743:1,X:32551:80,20:04:-1,
24:04:16023,16:5:-1,17:05:18692,24:08:-1,25:08:32410,1:15:-1,3:16:79127,41:19:-1,8:20:96220,
15:04:16023,4:28:112147

Bonus animation from when I was testing individual fixes against try1.ts from the old frame.

-Bob
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: John_L on 05/08/2014 08:59 pm
Just curious...
The original releases (raw and 'repaired') had frames as the stage was descending through the cloud layer which fit before this sequence, and the stage falling over at the end which come after -- both best seen in raw IMO.  Are those frames somewhere in this amazing recovery effort?

The answer is yes and no...

MPEG encoding has different types of frames... I-Frames, which contain All of the information needed to reconstruct that frame, the other types B and P frames only contain partial information, just information about what's changed since the last i-frame.  In our example here, we have a 30 second video with about 307 frames (I think), and of those 307 frames, only about a dozen or so are I-Frames.  Those are the frames we are working on, because the rest of the frames will be generated from data within the I-Frames. A given block of pixels on the screen can rely on blocks before it, after it, or from the I-Frame that was several frames back.

When you see us talking about frame 1, frame 2, frame 3, etc, we're not talking about the first 3 frames of the video, we're talking about just the iframes.  There are (by my guess) about 20-25 intermediate frames between each iframe.  We'll be getting to those once the iframes are fixed, at that point, we'll re-encode the video replacing the bad iframes with the good ones, and once the video is re-encoded, the intermediate frame data will be regenerated from the repaired iframes, and hopefully we'll have a much cleaner video.

Moral of the story... MPEG cheats... ALOT !

I tried not to get too technical with the explanation, hope that clears it up for you though.

John
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/08/2014 09:00 pm
More cleanup of iframe 9. These are from try1, but not much changed at least in the top of the frame...

Now that iframe 8 is almost done, pulled the clock position from it and set the top of this clock to that... gives us a reference point for the bottom of the frame.

X:16410:01,X:19164:10,X:19903:1,X:20037:80,X:27244:1,X:32743:1,X:32551:80,20:04:-1,
24:04:16023,16:5:-1,17:05:18692,24:08:-1,25:08:32410,1:15:-1,3:16:79127,41:19:-1,8:20:96220,
15:04:16023,4:28:112147

Bonus animation from when I was testing individual fixes against try1.ts from the old frame.

-Bob

One key thing to note about the clock. The blocks for it are abnormally large so corruption chance increases. They average around 1000 bits each for me.

Also, every timestamp in the video is of the form: 19:35:XX.XX
Importantly, the lower portion of the colons are entirely in the lower two sub-blocks of the bottom row and the upper portions are entirely in the lower two sub-blocks of the second from bottom row. Further, for the "." character its slightly wider than the lower portion of the ":" meaning you can recognize where to put it easier.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Alpha Control on 05/08/2014 09:02 pm


I think it is likely that it could take weeks or months to get the video to a level of quality that is going to satisfy everyone working on it. Chris, can you ask PAO what timeframe would be most useful to them?

You guys are ahead of the game, the only real game in town actually. Provide what you can, when you can. This is an unbelievable effort on show here.

This is truly an d unbelievable effort. And is amazing to see the amount of details that have been recovered form the initial frames, and the conviction and talent of all people that have been involved....

This is what we have so far guys! And let's keep with the good job:
Great! I was about to ask for something like this. Or make it myself. :)
One suggestion/request: a bit slower sequence? Either more realistic in terms of actual movie speed or even slower to be able to find common blocks?

Sure. How about this:

Moralec, that's terrific. Thanks for adding the frame numbers and slowing down the playback.

Really great summary of where this amazing effort is at this point! It's not my field, but it is wonderful to follow (if only at a high level) the great progress everyone is making. 
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/08/2014 09:07 pm
Hi guys,

This is just for inspiration:

"We've always defined ourselves by the ability to overcome the impossible"

Regards,

arnezami

Terrific video.

And also useful to note the side view of the rocket descending into the smoke,
 much like the stage returning as it hits the surface of the water.

I suppose this is obvious to the rocket scientists that we would see just parts of the struts.

The only problem is that in iframe 13, you can see both being the same size again....


Weird....

http://spacexlanding.wikispaces.com/
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Asmegin on 05/08/2014 09:07 pm
A bit more of the Frame 9 clock;

5:28:110675,6:29:120015,4:29:117503
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: AncientU on 05/08/2014 09:08 pm
Just curious...
The original releases (raw and 'repaired') had frames as the stage was descending through the cloud layer which fit before this sequence, and the stage falling over at the end which come after -- both best seen in raw IMO.  Are those frames somewhere in this amazing recovery effort?

The answer is yes and no...

MPEG encoding has different types of frames... I-Frames, which contain All of the information needed to reconstruct that frame, the other types B and P frames only contain partial information, just information about what's changed since the last i-frame.  In our example here, we have a 30 second video with about 307 frames (I think), and of those 307 frames, only about a dozen or so are I-Frames.  Those are the frames we are working on, because the rest of the frames will be generated from data within the I-Frames. A given block of pixels on the screen can rely on blocks before it, after it, or from the I-Frame that was several frames back.

When you see us talking about frame 1, frame 2, frame 3, etc, we're not talking about the first 3 frames of the video, we're talking about just the iframes.  There are (by my guess) about 20-25 intermediate frames between each iframe.  We'll be getting to those once the iframes are fixed, at that point, we'll re-encode the video replacing the bad iframes with the good ones, and once the video is re-encoded, the intermediate frame data will be regenerated from the repaired iframes, and hopefully we'll have a much cleaner video.

Moral of the story... MPEG cheats... ALOT !

I tried not to get too technical with the explanation, hope that clears it up for you though.

John
Thanks for the generous explanation and all effort on this reconstruction.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: John_L on 05/08/2014 09:09 pm
Interesting video about how MPEG works...

https://www.youtube.com/watch?v=P7abyWT4dss

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: AncientU on 05/08/2014 09:10 pm
Hi guys,

This is just for inspiration:

"We've always defined ourselves by the ability to overcome the impossible"

Regards,

arnezami

Terrific video.

And also useful to note the side view of the rocket descending into the smoke,
 much like the stage returning as it hits the surface of the water.

I suppose this is obvious to the rocket scientists that we would see just parts of the struts.

The only problem is that in iframe 13, you can see both being the same size again....


Weird....

http://spacexlanding.wikispaces.com/
The stage has bobbed back up to the surface (mostly) and just has the thrust structure underwater.
The thin 'legs' are actually just the pistons.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/08/2014 09:29 pm
Hi guys,

This is just for inspiration:

"We've always defined ourselves by the ability to overcome the impossible"

Regards,

arnezami

Terrific video.

And also useful to note the side view of the rocket descending into the smoke,
 much like the stage returning as it hits the surface of the water.

I suppose this is obvious to the rocket scientists that we would see just parts of the struts.

The only problem is that in iframe 13, you can see both being the same size again....


Weird....

http://spacexlanding.wikispaces.com/
The stage has bobbed back up to the surface (mostly) and just has the thrust structure underwater.
The thin 'legs' are actually just the pistons.

Yep, that's possible...

The answer lies in the b and p frames :P
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/08/2014 09:34 pm
Ocean from iframe 6 (try1). Unfortunately I have yet to think of a straightforward way to convert between the two framesets. So I can't combine this with other peoples progress. A chain file to tell which bit ranges in original frames correspond to which ranges in try1 would be of great value.
The data in this frame seems to be in fairly good condition. I think most of the frame can be perfectly recovered.

mmb 0:0:583,39:0:5496,43:0:-1,2:1:6129,21:1:8584,22:1:8796,6:3:
18105,7:3:18276,9:3:29527,30:3:32359,31:3:-1,43:3:33591,2:6:44738,4:6:46341,5:6:48794
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: MTom on 05/08/2014 09:51 pm
Hi guys,

This is just for inspiration:

"We've always defined ourselves by the ability to overcome the impossible"

Regards,

arnezami

Terrific video.

And also useful to note the side view of the rocket descending into the smoke,
 much like the stage returning as it hits the surface of the water.

I suppose this is obvious to the rocket scientists that we would see just parts of the struts.

The only problem is that in iframe 13, you can see both being the same size again....


Weird....

http://spacexlanding.wikispaces.com/

IMHO:
On frames 10-11-12 you see nearly anything about the legs because of the vapor, likely not because the legs are still partially under the water.
On frame 13 the pistons are much clearer to see - the vapor disappeared.
About from this point are the legs only partially under water (or a little bit earlier, while the rocket should cause also a big wave - it has about 4.5m diameter - this delays also a little bit that the pistons go under water).


Edit: the explanation from AncientU is also logical - the video will hopefully give an answer.

Edit2: If I see the timeline, frame13 is min. 3 seconds after engine shut down. For my theory this is too long time (and in 8 seconds it went horizontal)...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: grythumn on 05/08/2014 09:54 pm
More cleanup of iframe 9. These are from try1, but not much changed at least in the top of the frame...

Now that iframe 8 is almost done, pulled the clock position from it and set the top of this clock to that... gives us a reference point for the bottom of the frame.

X:16410:01,X:19164:10,X:19903:1,X:20037:80,X:27244:1,X:32743:1,X:32551:80,20:04:-1,
24:04:16023,16:5:-1,17:05:18692,24:08:-1,25:08:32410,1:15:-1,3:16:79127,41:19:-1,8:20:96220,
15:04:16023,4:28:112147

Bonus animation from when I was testing individual fixes against try1.ts from the old frame.

-Bob

One key thing to note about the clock. The blocks for it are abnormally large so corruption chance increases. They average around 1000 bits each for me.

Also, every timestamp in the video is of the form: 19:35:XX.XX
Importantly, the lower portion of the colons are entirely in the lower two sub-blocks of the bottom row and the upper portions are entirely in the lower two sub-blocks of the second from bottom row. Further, for the "." character its slightly wider than the lower portion of the ":" meaning you can recognize where to put it easier.

Add ,3:29:118925 for the damaged bottom portion of the clock. The 9 comes and goes, it's pretty damaged.

-Bob
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/08/2014 10:02 pm
For those that aren't aware there's a new improved online editor that has all the UI for the various options and the ability to use different sources ("coerced.ts", "try1.ts") you can get to it via the wiki here http://spacexlanding.wikispaces.com/ (Online Editor v2)

If anyone has any other .ts they are using I can upload that too, just let me know (I just need the .mp4-img files and a name for it)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/08/2014 10:35 pm
Ok....peer pressure! ;D

So if someone wants to take my Opening Post, throw in some links like Iain's Wiki page (which looks a lot better now) and maybe some highlight links to good posts throughout the thread, and put it all together.....send me the updated post via PM so I can edit it in.

And apologies to Swatch seen as it was his idea...... :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/08/2014 10:45 pm
Could someone write a detailed instruction how to update the table with the images in the wiki http://spacexlanding.wikispaces.com/home without messing everything?

Or could someone simply put the latest image versions of:
- iframe 1 (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195314#msg1195314)
- iframe 4 (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195103#msg1195103)
- iframe 5 (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195546#msg1195546)
- iframe 6 (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195584#msg1195584 and/or http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195704#msg1195704)
- iframe 8 (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195321#msg1195321 and http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195464#msg1195464)
- iframe 9 (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195709#msg1195709)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Untribium on 05/08/2014 10:48 pm
For those that aren't aware there's a new improved online editor that has all the UI for the various options and the ability to use different sources ("coerced.ts", "try1.ts") you can get to it via the wiki here http://spacexlanding.wikispaces.com/ (Online Editor v2)

If anyone has any other .ts they are using I can upload that too, just let me know (I just need the .mp4-img files and a name for it)

nice work! now I no longer have to make sure I don't accidentally modify the same block twice :D One thing though: I think it would make more sense to sort the modified blocks by y and then x as opposed to the other way around, since this is the order in which they are traversed
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/08/2014 11:00 pm
For those that aren't aware there's a new improved online editor that has all the UI for the various options and the ability to use different sources ("coerced.ts", "try1.ts") you can get to it via the wiki here http://spacexlanding.wikispaces.com/ (Online Editor v2)

If anyone has any other .ts they are using I can upload that too, just let me know (I just need the .mp4-img files and a name for it)

nice work! now I no longer have to make sure I don't accidentally modify the same block twice :D One thing though: I think it would make more sense to sort the modified blocks by y and then x as opposed to the other way around, since this is the order in which they are traversed

Done :) Thanks for the feedback
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Asmegin on 05/08/2014 11:11 pm
For those that aren't aware there's a new improved online editor that has all the UI for the various options and the ability to use different sources ("coerced.ts", "try1.ts") you can get to it via the wiki here http://spacexlanding.wikispaces.com/ (Online Editor v2)

If anyone has any other .ts they are using I can upload that too, just let me know (I just need the .mp4-img files and a name for it)

Would some sort of toggle to highlight modified macroblocks be useful here? I find myself getting a bit confused sometimes (although my knowledge level is 1/10 so it may just be me)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/08/2014 11:13 pm
Looks like the news site gave this thread a bit of a boost. Gone from 67,000 to 82,000 views in 12 hours.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/08/2014 11:15 pm
For those that aren't aware there's a new improved online editor that has all the UI for the various options and the ability to use different sources ("coerced.ts", "try1.ts") you can get to it via the wiki here http://spacexlanding.wikispaces.com/ (Online Editor v2)

If anyone has any other .ts they are using I can upload that too, just let me know (I just need the .mp4-img files and a name for it)

Would some sort of toggle to highlight modified macroblocks be useful here? I find myself getting a bit confused sometimes (although my knowledge level is 1/10 so it may just be me)

A mouseover pop-up for which block you are hovering over might be better (you can compare it to your list of changes anyway).
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: dorkmo on 05/08/2014 11:34 pm
frame 5
0:0:-1,27:10:24142,17:12:30040,25:12:-1,18:13:39153,20:13:-1,25:13:35375,29:13:-1,27:15:47754,3:16:50034

added a few things from Shanuson's
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/09/2014 12:08 am
Could someone write a detailed instruction how to update the table with the images in the wiki http://spacexlanding.wikispaces.com/home without messing everything?

Or could someone simply put the latest image versions of:
- iframe 1 (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195314#msg1195314)
- iframe 4 (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195103#msg1195103)
- iframe 5 (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195546#msg1195546)
- iframe 6 (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195584#msg1195584 and/or http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195704#msg1195704)
- iframe 8 (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195321#msg1195321 and http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195464#msg1195464)
- iframe 9 (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195709#msg1195709)

Very simple thing to do.
1. click on edit on top.
2. Scroll down to the table, and find the frame you want to update.
3. Manually delete the old image
4. Add a new image (there is a button for that).  There is no need to upload anything, just make a reference to the image that is stored here in nsf.
5. Add the name of the author and new code.
6. Click save.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/09/2014 12:09 am
frame 5
0:0:-1,27:10:24142,17:12:30040,25:12:-1,18:13:39153,20:13:-1,25:13:35375,29:13:-1,27:15:47754,3:16:50034

added a few things from Shanuson's

Very nice. Don't forget to add it to the wiki!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: CraigLieb on 05/09/2014 12:30 am
Hi guys,

This is just for inspiration:

"We've always defined ourselves by the ability to overcome the impossible"

Regards,

arnezami

Terrific video.

And also useful to note the side view of the rocket descending into the smoke,
 much like the stage returning as it hits the surface of the water.

I suppose this is obvious to the rocket scientists that we would see just parts of the struts.

The only problem is that in iframe 13, you can see both being the same size again....


Weird....

http://spacexlanding.wikispaces.com/

Waves pass,  how much time between iframes?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/09/2014 12:40 am
Hi guys,

This is just for inspiration:

"We've always defined ourselves by the ability to overcome the impossible"

Regards,

arnezami

Terrific video.

And also useful to note the side view of the rocket descending into the smoke,
 much like the stage returning as it hits the surface of the water.

I suppose this is obvious to the rocket scientists that we would see just parts of the struts.

The only problem is that in iframe 13, you can see both being the same size again....


Weird....

http://spacexlanding.wikispaces.com/

Waves pass,  how much time between iframes?

About one second
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: watermod on 05/09/2014 12:55 am
I haven't done much with mpeg-4 but have done work with other forms of imaging (3D).   To me, one fact that sticks in the mind is that the actual upper part of the rocket should be the same (excepting albedo) from the beginning of the flight  until the end.  That should be able to provide clues or hints to reconstructing the images.  I realize that being a 2D and not a 3D image the albedo is not as easy to factor out of the image.  It would be interesting to have a 3D image of the first stage, layer the good frames early in flight  upon it's surface as an albedo cutting out the sky, ocean and such.  Then factor that back into the MPEG-4 images (both as direct missing bits and sunlight and sun angle hints).  Other then bits like the legs as they disappear into the ocean everything else should be the background sky and direct light on the first stage that is different from the light at launch.  Same for the static dirt on the lens.   It's a hint.  They are all hints to the missing bits.  From those missing bits you might get enough hints to reconstruct the ocean and remaining scenery.
(example of albedo being manipulated as a 3D skin to recover useful information: http://www.engr.uky.edu/~lgh/data/dataGraveStones2010.htm (http://www.engr.uky.edu/~lgh/data/dataGraveStones2010.htm))




Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: seanpg71 on 05/09/2014 12:59 am
frame 5
0:0:-1,27:10:24142,17:12:30040,25:12:-1,18:13:39153,20:13:-1,25:13:35375,29:13:-1,27:15:47754,3:16:50034

added a few things from Shanuson's


That's helpful and should save me a bit of time.  I've been working from the bottom up on 5.  It's a bit of a mess.  Mostly I've just been trying to fix the bit alignment issues so that a picture shows up so far.  This is as far as I've gotten thus far.

0:0:-1,40:17:56308,43:19:69441,0:21:106970,1:21:76873,11:22:85251,12:22:85489,26:22:86772,
27:22:87056,37:22:87982,38:22:88152,10:23:90705,11:23:91098,16:23:91462,17:23:91682,
22:24:97633,23:24:97869,0:25:100325,0:27:106003,2:27:105388,22:27:110276,30:28:121538,
31:28:121730,1:29:123050,2:29:123876

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Asmegin on 05/09/2014 01:05 am
frame 5
0:0:-1,27:10:24142,17:12:30040,25:12:-1,18:13:39153,20:13:-1,25:13:35375,29:13:-1,27:15:47754,3:16:50034

added a few things from Shanuson's


That's helpful and should save me a bit of time.  I've been working from the bottom up on 5.  It's a bit of a mess.  Mostly I've just been trying to fix the bit alignment issues so that a picture shows up so far.  This is as far as I've gotten thus far.

0:0:-1,40:17:56308,43:19:69441,0:21:106970,1:21:76873,11:22:85251,12:22:85489,26:22:86772,
27:22:87056,37:22:87982,38:22:88152,10:23:90705,11:23:91098,16:23:91462,17:23:91682,
22:24:97633,23:24:97869,0:25:100325,0:27:106003,2:27:105388,22:27:110276,30:28:121538,
31:28:121730,1:29:123050,2:29:123876

Great start. Added to Wiki
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: dorkmo on 05/09/2014 01:17 am
frame 5
0:0:-1,27:10:24142,17:12:30040,25:12:-1,18:13:39153,20:13:-1,25:13:35375,29:13:-1,27:15:47754,3:16:50034

added a few things from Shanuson's


That's helpful and should save me a bit of time.  I've been working from the bottom up on 5.  It's a bit of a mess.  Mostly I've just been trying to fix the bit alignment issues so that a picture shows up so far.  This is as far as I've gotten thus far.

0:0:-1,40:17:56308,43:19:69441,0:21:106970,1:21:76873,11:22:85251,12:22:85489,26:22:86772,
27:22:87056,37:22:87982,38:22:88152,10:23:90705,11:23:91098,16:23:91462,17:23:91682,
22:24:97633,23:24:97869,0:25:100325,0:27:106003,2:27:105388,22:27:110276,30:28:121538,
31:28:121730,1:29:123050,2:29:123876

heres my most recent

0:0:-1,27:10:24142,16:12:30040,24:12:-1,17:13:39153, 20:13:-1,25:13:35375,29:13:-1,28:16:47754,0:17:50034, 43:19:-1,0:26:103820,13:27:-1,17:27:109631,25:27:-1, 27:27:110752,28:28:-1,33:28:121771,2:29:124970
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Asmegin on 05/09/2014 01:23 am
Is anyone working on 3 or 4? Those are the two bad ones. 5 was bad but the last few posts show progress!

I searched Frame 3 for a while but couldn't find anything. Unrecoverable?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: dorkmo on 05/09/2014 01:46 am
Is anyone working on 3 or 4? Those are the two bad ones. 5 was bad but the last few posts show progress!

I searched Frame 3 for a while but couldn't find anything. Unrecoverable?

judging from whats become of frame 5, i think frame 4 is going to be the most important for getting a nice shot of the leg deployment. and im afraid 3 is a lost cause. :(

i noticed that some are using a different ts file, try1? are we going to be in a world of hurt if we have mixed data or will it be okay?

ps this online editor on the wiki is SPECTACULAR! i wouldnt have been able to figure out doing it the hard way. kudos!


--edit--
added 22:24:97869 and changed some around the bottom:

0:0:-1,27:10:24142,16:12:30040,24:12:-1,17:13:39153,20:13:-1, 25:13:35375,29:13:-1,28:16:47754,0:17:50034,43:19:-1, 22:24:97869,02:27:-1,24:27:110437,13:27:-1,17:27:109631, 25:27:-1,27:27:110752,28:28:-1,33:28:121771,2:29:124970

Edit/Lar: please put spaces in every once in a while so your lines wrap. I think the parms are space agnostic (if they are after the comma but not anywhere else anyway?). thanks
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: grythumn on 05/09/2014 03:44 am
Minor tweak to iframe13, try1: X:90932:80,40:1:-1,0:10:-1:0:-10,1:10:-1:,4:10:49027,8:29:-1
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: dorkmo on 05/09/2014 03:48 am
Is anyone working on 3 or 4? Those are the two bad ones. 5 was bad but the last few posts show progress!

I searched Frame 3 for a while but couldn't find anything. Unrecoverable?

frame 4, so far only found a little bit of the left side and maybe a few blocks of the right
00:14:-1,05:01:-1,35:05:-1,19:05:20944,35:05:-1,43:06:27582,0:10:-1,21:13:50333,25:14:57756 ,08:15:61126,36:15:-1,00:17:71044,0:24:-1,00:27:114198

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/09/2014 04:24 am
Wow just read through the 27 pages and am very impressed.

Trying to work on iframe 4, as a scam guy, I was thinking of trying to develop a brute-force script to match the 19:35 timestamp. From what I have (quickly) counted, the first four blocks in the bottom row have known data. I don't have mpeg experience, but from what I've gathered from this thread, due to the down-right cascade this can only directly check the first four blocks in each row. AFAIU, corrections in those blocks could potentially help resolve the entire frame.

From the 704x480 PNG output of the custom ffmpeg command (using MMB on frames 6, 11, 12, and 13), the known pixel counts are:

0: 36/576
1: 148/576
2: 130/576
3: 154/576

Total: 468/2304 = ~.203

or about 20% which is huge in terms of a known-plaintext attack. Add in that ffmpeg errors should probably decrease with a valid flip, I think a brute force is computationally feasible. A tremendous speedup would be possible with a heuristic search algorithm, which I assume those of you that have been playing manually probably have a mental version of.

Anyone care to take a swing at describing your process of coming up with the next MMB token given the existing tokens and the next error?



Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/09/2014 05:39 am
Wow just read through the 27 pages and am very impressed.

Trying to work on iframe 4, as a scam guy, I was thinking of trying to develop a brute-force script to match the 19:35 timestamp. From what I have (quickly) counted, the first four blocks in the bottom row have known data. I don't have mpeg experience, but from what I've gathered from this thread, due to the down-right cascade this can only directly check the first four blocks in each row. AFAIU, corrections in those blocks could potentially help resolve the entire frame.

From the 704x480 PNG output of the custom ffmpeg command (using MMB on frames 6, 11, 12, and 13), the known pixel counts are:

0: 36/576
1: 148/576
2: 130/576
3: 154/576

Total: 468/2304 = ~.203

or about 20% which is huge in terms of a known-plaintext attack. Add in that ffmpeg errors should probably decrease with a valid flip, I think a brute force is computationally feasible. A tremendous speedup would be possible with a heuristic search algorithm, which I assume those of you that have been playing manually probably have a mental version of.

Anyone care to take a swing at describing your process of coming up with the next MMB token given the existing tokens and the next error?
Hi theshadow27!

Welcome to the forum! :)

What is mainly done is realigning the visual macroblocks to the data. Essentially trying to guess the starting bitposition of a visual block and see if it starts producing a sensible visual block. Apart from that there is also many cases where visual blocks have correct data but this data was meant for a different block: this "simply" needs to be reassigned. But its a lot of work. Due to errors in the data the visual blocks start reading at the wrong bit-position. And this happens for blocks all over the frame (not just in the lower left corner). The clock is usually easy to find and place in the right position (in many of the I-frames they have already been found). But if you have the clock in the right position it won't help you with the rest of the image.

Also since the clock is mostly static (apart from the last two digits) the P- and B-frames will have no data there (since nothing changed and these kinds of frames only contain delta information). It might be possible to use the second to last digit in order to check whether the macroblocks are alighted correctly there. But that's about it I think.

I've been looking at ways to automate the process for quite a while now. The reason this is so hard is because only humans seem to be able to decide what is a good rendered block versus a bad rendered block (apart from blocks with clear errors which could of course be used for a search algo). Without some kind of feedback (not just in the lower left part but all over the image) it doesn't seem feasible to automate the aligment of the blocks to the bitpositions. This is also the reason I decided to make it possible to let the humans be "the feedback" by creating the -mmb option in the first place.

Where I've been looking at is how to detect where macroblocks begin and end. And because mpeg-4 is so damned efficient (they have essentially no overhead bits anywhere) this turns out to be very hard. My best candidates are:

- cbpc should always be below 4. (this is sort of a 2-bit "marker")
- dquant should always be 0 (sort of a 1-bit "marker"). But I am not sure if this is true.
- other errors in general (I haven't assembled all possible errors yet, but they are probably related to the above two)

This is quite meager though. But it's something.

My best idea is so far is to use a backwards search: start at the end of the data and assume the bit before it is the start of a macroblock and decode it: if it doesn't stop reading bits exactly at the end-bit then its not a valid starting bit for a macroblock. So try the previous bit. And repeat this. If you do end correctly then use the start of that new macroblock as the new end-bit. Etc.

I've experimented with this but I ran into the trouble of getting false positives (which might cause a way too big search-space, not sure yet). And there is also no way of knowing whether I'm on the right track: you basicly need a human being for that! (well maybe statistically you can do something). Still looking into that though.

Also there are real errors in the bits so I'm not sure what to decide sometimes: is it the wrong starting position or is there an error in the data? This backwards search idea is a mechanism that doesn't work as good as humans can do it now I think. With I-frames at least. But it might be the only way to do the P- and B-frames: I think it's going to be quite hard for humans to decide between an error-block and a misplaced-block in those kinds of frames. But then again another way of determining whether the blocks are aligned right in P- and B- frames are the borders of the rocket: the blocks right outside the rocket are always changing while the blocks next to them never change. So I'm still hopeful it's still mostly doable by hand.

I have quite a lot more thoughts/ideas on this subject but it's hard to express them all :P

Regards,

arnezami

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: ctillier on 05/09/2014 05:46 am
My meager contribution for iframe 10
0:3:8818,0:4:11564

This reveals another two rows of blocks showing sea spray, but corrupts the rest of the image in a way that I haven't figured out how to fix.  Help me SwissCheese, you're my only hope!

Fantastic effort everyone.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/09/2014 06:11 am
Hi guys,

There is another idea I would like to share regarding P- and B-frames and how to determine whether a block is placed in the right position:

The rocket is always moving downwards (apart from the last few seconds). And the camera is looking downwards. So all visual particles that are not part of the rocket always move from the middle to the outside: their movement vector is quite predictable. Furthermore: when a particle (like a little wave in the sea) moves outwards it speeds up. Right? (pixels per second I mean).

So if mpeg-4 is using movement prediction we should be able to see the change in speed (and the direction of this change in speed) in the blocks of the P- and B-frames. Is this sound logic?

Because if so we can (roughly) see in what line (from the middle part of the image to the outside of the image) the block should be placed. Assuming we can extract this information from the data of course.

Just my 2 cents :)

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/09/2014 08:20 am
Here's what I have so far. It's about 2-3 seconds of video almost up to splashdown:

[youtube]<object width="425" height="350"><param name="movie" value="http://www.youtube.com/v/CxheyOUAlG8"></param><param name="wmode" value="transparent"></param><embed src="http://www.youtube.com/v/CxheyOUAlG8" type="application/x-shockwave-flash" wmode="transparent" width="425" height="350"></embed></object>[/youtube]
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: MTom on 05/09/2014 08:34 am

The rocket is always moving downwards (apart from the last few seconds)...

I would guess on the Frame 13 the rocket is not vertical yet (kipped some to right on the screen).
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jakusb on 05/09/2014 08:45 am
Is anyone working on 3 or 4? Those are the two bad ones. 5 was bad but the last few posts show progress!

I searched Frame 3 for a while but couldn't find anything. Unrecoverable?

judging from whats become of frame 5, i think frame 4 is going to be the most important for getting a nice shot of the leg deployment. and im afraid 3 is a lost cause. :(

i noticed that some are using a different ts file, try1? are we going to be in a world of hurt if we have mixed data or will it be okay?

ps this online editor on the wiki is SPECTACULAR! i wouldnt have been able to figure out doing it the hard way. kudos!


--edit--
added 22:24:97869 and changed some around the bottom:

0:0:-1,27:10:24142,16:12:30040,24:12:-1,17:13:39153,20:13:-1, 25:13:35375,29:13:-1,28:16:47754,0:17:50034,43:19:-1, 22:24:97869,02:27:-1,24:27:110437,13:27:-1,17:27:109631, 25:27:-1,27:27:110752,28:28:-1,33:28:121771,2:29:124970

Edit/Lar: please put spaces in every once in a while so your lines wrap. I think the parms are space agnostic (if they are after the comma but not anywhere else anyway?). thanks

My first attempt to pitch in on this amazing effort... :)
I checked frame 5 with above settings and noticed the piece of mud on camera seems on position off:
27:10:24142 should be 26:10:24142
I compared with frames 8 en 9 and they are consistent in having the mud on 26:10

PS: I am having trouble figuring out the best progress per frame on spacexlanding.wikispaces.com (http://spacexlanding.wikispaces.com). It would be great if it could be made to easy copy paste the values, but also to maybe have someone judge which progress is best, or if multiple routes are chosen to separate them.
Also I guess we are getting into some trouble with people working on 2 different ts files?!?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jakusb on 05/09/2014 08:51 am
Is anyone working on 3 or 4? Those are the two bad ones. 5 was bad but the last few posts show progress!

I searched Frame 3 for a while but couldn't find anything. Unrecoverable?

judging from whats become of frame 5, i think frame 4 is going to be the most important for getting a nice shot of the leg deployment. and im afraid 3 is a lost cause. :(

i noticed that some are using a different ts file, try1? are we going to be in a world of hurt if we have mixed data or will it be okay?

ps this online editor on the wiki is SPECTACULAR! i wouldnt have been able to figure out doing it the hard way. kudos!


--edit--
added 22:24:97869 and changed some around the bottom:

0:0:-1,27:10:24142,16:12:30040,24:12:-1,17:13:39153,20:13:-1, 25:13:35375,29:13:-1,28:16:47754,0:17:50034,43:19:-1, 22:24:97869,02:27:-1,24:27:110437,13:27:-1,17:27:109631, 25:27:-1,27:27:110752,28:28:-1,33:28:121771,2:29:124970

Edit/Lar: please put spaces in every once in a while so your lines wrap. I think the parms are space agnostic (if they are after the comma but not anywhere else anyway?). thanks

My first attempt to pitch in on this amazing effort... :)
I checked frame 5 with above settings and noticed the piece of mud on camera seems on position off:
27:10:24142 should be 26:10:24142
I compared with frames 8 en 9 and they are consistent in having the mud on 26:10

PS: I am having trouble figuring out the best progress per frame on spacexlanding.wikispaces.com (http://spacexlanding.wikispaces.com). It would be great if it could be made to easy copy paste the values, but also to maybe have someone judge which progress is best, or if multiple routes are chosen to separate them.
Also I guess we are getting into some trouble with people working on 2 different ts files?!?

When I try to align the landing legs, I do not seem to get a matching block in frames 8 and 9. Can I conclude the landing legs not yet fully deployed in frame 5?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/09/2014 08:57 am
I posted a gif in the IRC yesterday comparing my IF5 with IF8.
I could align the lower edge of the left leg, but the right leg was a few pixel to low.
The Rocket body and the smudge above the right leg was also aligned.
My interpretation is that the right leg is almost but not fully extended yet.


Cheers
Shanuson
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jakusb on 05/09/2014 09:41 am
I posted a gif in the IRC yesterday comparing my IF5 with IF8.
I could align the lower edge of the left leg, but the right leg was a few pixel to low.
The Rocket body and the smudge above the right leg was also aligned.
My interpretation is that the right leg is almost but not fully extended yet.


Cheers
Shanuson

Regarding Frame 5:
I tried to combine the work of both dorkmo (upper half) with seanpg71 (lower part).
Also moved the 0:0:-1 to 14:0:-1 as first blocks do appear to be ok.
I shifted a number of blocks from dorkmo to left and the richt part of body a bit down and aligns now better with frame 8

00:21:106970,00:25:100325,00:27:106003,01:21:76873,01:29:123050,02:27:105388, 02:29:123876,03:16:50034,
10:23:90705,11:22:85251,11:23:91098,12:22:85489,14:00:-1, 16:12:30040,16:23:91462,17:13:39153,17:23:91682,
20:13:-1,22:27:110276,23:24:97869,24:13:35375,25:12:-1, 26:10:24142,26:22:86772,27:16:47754,29:13:-1,
30:28:121538,37:22:87982,38:22:88152,40:17:56308,43:19:69441

PS: I simply use the Online Tool v2 in Firefox and have 2 tabs open. One of Frame 5 (working on it) and one of Frame 8 (for comparison). I simply switch often (ctrl-tab) to keep comparing alignment. Works like a charm.. You could also have a second Firefox open and use alt-tab to compare against yet another frame...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/09/2014 10:22 am
Here is a give comparing IF5 and IF8,
legs lookl like they are not yet in their final position.

Created with via Imgflip GIF Maker (https://imgflip.com/gifgenerator)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: MP99 on 05/09/2014 10:52 am
PS: I am having trouble figuring out the best progress per frame on spacexlanding.wikispaces.com (http://spacexlanding.wikispaces.com). It would be great if it could be made to easy copy paste the values, but also to maybe have someone judge which progress is best, or if multiple routes are chosen to separate them.
Also I guess we are getting into some trouble with people working on 2 different ts files?!?

I wonder if this would work under GIT?

Separate source for the parms for each iframe.

Each person working in their own branch(es), making trial commits, and then someone combining them into master if they look to be progressing the overall goal?

cheers, Martin
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jakusb on 05/09/2014 10:58 am

Here is a give comparing IF5 and IF8,
legs lookl like they are not yet in their final position.

Created with via Imgflip GIF Maker (https://imgflip.com/gifgenerator)

If you use the Online Tool v2, and 2 tabs in Firefox, you can check the alignment on block level. Simply select the same block in both tabs and ctrl-tab a lot. :)
Legtips seem to be in place, but in process of getting deployed still. If so, impossible to exactly align. The tips should be both a little bit inward aligned.
I do think de upper (wider) part of legs need some work still.
The rocket body seems perfectly aligned al the way to the to corners where the camera housing cuts the view.
Time is also in correct place.
If I can find time, I will try to improve the legs a bit more. Maybe discover some hidden blocks there.
If someone else would like to try, go for it. :)
If someone could recover some sea, that would be sweet too. There seem to be lots of data available for it, although likely pretty hard to discover.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: seanpg71 on 05/09/2014 11:22 am
I posted a gif in the IRC yesterday comparing my IF5 with IF8.
I could align the lower edge of the left leg, but the right leg was a few pixel to low.
The Rocket body and the smudge above the right leg was also aligned.
My interpretation is that the right leg is almost but not fully extended yet.


Cheers
Shanuson

Regarding Frame 5:
I tried to combine the work of both dorkmo (upper half) with seanpg71 (lower part).
Also moved the 0:0:-1 to 14:0:-1 as first blocks do appear to be ok.
I shifted a number of blocks from dorkmo to left and the richt part of body a bit down and aligns now better with frame 8

00:21:106970,00:25:100325,00:27:106003,01:21:76873,01:29:123050,02:27:105388, 02:29:123876,03:16:50034,
10:23:90705,11:22:85251,11:23:91098,12:22:85489,14:00:-1, 16:12:30040,16:23:91462,17:13:39153,17:23:91682,
20:13:-1,22:27:110276,23:24:97869,24:13:35375,25:12:-1, 26:10:24142,26:22:86772,27:16:47754,29:13:-1,
30:28:121538,37:22:87982,38:22:88152,40:17:56308,43:19:69441

PS: I simply use the Online Tool v2 in Firefox and have 2 tabs open. One of Frame 5 (working on it) and one of Frame 8 (for comparison). I simply switch often (ctrl-tab) to keep comparing alignment. Works like a charm.. You could also have a second Firefox open and use alt-tab to compare against yet another frame...

I've gotten to here with 5:

0:0:-1,26:10:24142,16:12:30040,24:12:-1,25:12:32001,28:12:-1,17:13:39153,18:13:34192,29:13:-1,
18:14:39177,23:14:-1,24:15:41044,23:16:46465,28:16:47954,40:17:56308,43:19:69441,0:21:106970,
1:21:76873,11:22:85251,12:22:85489,26:22:86772,27:22:87056,37:22:87982,38:22:88152,10:23:90705,
11:23:91098,16:23:91462,17:23:91682,22:24:97633,23:24:97869,0:25:100325,0:27:106003,2:27:105388,
22:27:110276,30:28:121538,31:28:121730,1:29:123050,2:29:123876

There's also some largish good hunks of water starting at 5895, 9085, and 22377 that I gotten around to trying to jam in there yet.


I think part of the problem I'm having getting things to line up neatly is that is seems like there's a bunch of data that's just missing - unless I'm just trying to put things in the wrong spot, it seems like there should be around 7000 bits between 40500 and 41000 for the end of row 14 and the beginning of 15.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jakusb on 05/09/2014 11:54 am
A quick attempt at Frame 4 (try1.ts), based on earlier work from saliva_sweet.

00:03:10968,01:28:129620,05:01:4200,08:03:17911,26:10:36805,31:07:-1,34:03:23414

I see some reoccurring dirt spots on lens that help align/confirm they are consistent in frame 4,8 & 9

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/09/2014 11:55 am
So I could align better iframe 5 (coerced.ts) with the other ones and add a few sequences, still room for improvement.

We clearly see that the legs are not completely deployed!

0:0:550,14:0:-1,
0:5:9736,26:7:-1,
4:10:22722,25:12:32001,28:12:-1,16:13:-1:0:-25:0:0,17:13:-1:-15:0:0:0,
19:13:34518,10:14:-1,
17:14:39053,24:14:-1,21:15:-1:-25:-10,22:15:-1,
34:15:42349,24:16:-1,
27:16:47754,42:19:-1,
43:19:69441,43:20:-1,
01:21:76873,11:22:-1,
12:22:85489,26:22:-1,
17:23:91682,22:24:-1,
23:24:97869,00:27:-1,
16:27:109631,24:27:-1,
25:27:110437,28:28:-1,
3:29:124970
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jakusb on 05/09/2014 12:02 pm
3 consistent dirt spots that should help to align ocean in top part
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/09/2014 12:53 pm
We clearly see that the legs are not completely deployed!

And what's more, we clearly see that the right leg is intact and that there's actually a smudge on the lens to the lower left of the black mark. Woo!  ;D Way to go!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/09/2014 12:56 pm

What is mainly done is realigning the visual macroblocks to the data. Essentially trying to guess the starting bitposition of a visual block and see if it starts producing a sensible visual block. Apart from that there is also many cases where visual blocks have correct data but this data was meant for a different block: this "simply" needs to be reassigned. But its a lot of work. Due to errors in the data the visual blocks start reading at the wrong bit-position. And this happens for blocks all over the frame (not just in the lower left corner). The clock is usually easy to find and place in the right position (in many of the I-frames they have already been found). But if you have the clock in the right position it won't help you with the rest of the image.

Also since the clock is mostly static (apart from the last two digits) the P- and B-frames will have no data there (since nothing changed and these kinds of frames only contain delta information). It might be possible to use the second to last digit in order to check whether the macroblocks are alighted correctly there. But that's about it I think.


Interesting… Do we have a model for the noise in this channel? 

At the very least, are all errors bit flips or are there dropped bits (or dropped bytes) and added bits?

If all errors are flips, and P(flip) is low and nominally time invariant, the task is several orders of magnitude easier.

I've been looking at ways to automate the process for quite a while now. The reason this is so hard is because only humans seem to be able to decide what is a good rendered block versus a bad rendered block (apart from blocks with clear errors which could of course be used for a search algo). Without some kind of feedback (not just in the lower left part but all over the image) it doesn't seem feasible to automate the aligment of the blocks to the bitpositions. This is also the reason I decided to make it possible to let the humans be "the feedback" by creating the -mmb option in the first place.


I am approaching it from a post-processing standpoint, so use ffmpeg to create a PNG, then consume the PNG with a separate application. My biggest limitation here is that the output has to go to file (due to image2 spec), so I have it writing to a ramdisk. Is there any way to have to write PNG data to stdout instead?



Just a couple of observations playing with ffmpeg more.

First, it is very sensitive to errors - most (guessing 95%+) changes will result in one or more solid (usually green) blocks showing up after the shifted block. This is trivial to detect (though time consuming, might be faster to just parse the stderr).

Second, to see which pieces "fit" due to a smooth intersection, I can get a fairly high success rate in marking bad blocks by differencing pixels that straddle N, S, E, W block boundaries using just sqrt( (r2-r1)^2 + (g2-g1)^2 + (b2-b1)^2 ) summed 22 x 4 times (pixels x each boundary), with a threshold of 0.7. This is my starting point for finding misaligned blocks, but it can't be used to find the correct alignment, since all of the ocean blocks for example will easily pass this test regardless of where they are.

Third, there when a block from a static area (faring, clock, smudge) is out of place but parsed correctly, it should have a high correlation with the same block in a "known good" frame using an appropriate fourier transform. Since there are relatively few blocks (600) a complete distance matrix could be constructed with 380k convolutions (or half that with a good mask and bad-block detection). Main problem here is determining the fastest/most accurate transform for block difference. Maybe the ffmpeg guys have some advice here?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/09/2014 01:07 pm
Hey Theshadow27! Welcome to the site's forum and this particular effort.

Incredible to see a number of new members turn up for this task - in addition to people we already had on site - members who are clearly superb at what is required here.

The best thing about the site is its community and as the fella who started this site back in 2005 - with a couple of STS-114 RTF articles and one forum section frequented by a handful Shuttle huggers, a NASA guy and a United Space Alliance guy - it's incredible how it's grown into what we have today. :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/09/2014 01:31 pm
My 2 cents on frame 3. Because of how garbled it is, its possible that the corruption is not in the data itself but in the frame header. The iframe header has lots of specifications that define how even numbers are defined (whether to use reversible or non-reversible codings). That would cause the entire frame to be decoded wrong.

Edit: Just checked, yeah the frame header is corrupted. If we fix it, large parts of the image may pop out. Other frames may also have corrupted headers but not bad enough to block header decoding. This would result in odd settings for the image still though.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/09/2014 01:33 pm
Also, for completeness, this guy commented on the wiki page but has not posted in this thread:

http://aeroquartet.com/wordpress/2014/05/08/spacex-falcon-first-stage-landing-part-ii/

Seems to know his stuff. From his page,

Quote
I have seen in some forums that people is trying to improve the video by fixing the TSHs.
This doesn’t hurt, but it will not improve the video, because while TSHs are easy to fix, a bad TSH indicates that probably several bits in the neighborhood are also bad. And those are the critical ones, and video won’t improve until you fix them all.

As mentioned previously, an error rate higher than 1/50,000 is impossible to fix. And if you have bad TSHs you probably have an order of magnitude more errors than that in the neighborhood.

Sound's pretty pessimistic to me :/
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/09/2014 01:36 pm
Also, for completeness, this guy commented on the wiki page but has not posted in this thread:

http://aeroquartet.com/wordpress/2014/05/08/spacex-falcon-first-stage-landing-part-ii/

Seems to know his stuff. From his page,

Quote
I have seen in some forums that people is trying to improve the video by fixing the TSHs.
This doesn’t hurt, but it will not improve the video, because while TSHs are easy to fix, a bad TSH indicates that probably several bits in the neighborhood are also bad. And those are the critical ones, and video won’t improve until you fix them all.

As mentioned previously, an error rate higher than 1/50,000 is impossible to fix. And if you have bad TSHs you probably have an order of magnitude more errors than that in the neighborhood.

Sound's pretty pessimistic to me :/

Quote
I’ve been repairing videos full time for the last 7 years, so I immediately knew that the bitstream was corrupt and that at best we could hope to partially fix some frames. Maybe a 50% chance of recovering one or two frames.

I think he's referring to restoring it to 100% working video. If he wasn't, we already vastly beat his predictions. Not sure whether to trust his posts or not. Either that or his experience isn't as much as he claims it is as I'm apparently doing better than him with 1 week of experience and possibly superior tools.

Repairing the TSHs is important for getting access to the frames, that was already done. We're going beyond his anticipations and repairing the iframes which in a properly repaired transport stream (or newly created transport stream) will produce a possibly working video.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/09/2014 01:56 pm
Here's what I have so far. It's about 2-3 seconds of video almost up to splashdown:

[youtube]<object width="425" height="350"><param name="movie" value="http://www.youtube.com/v/CxheyOUAlG8"></param><param name="wmode" value="transparent"></param><embed src="http://www.youtube.com/v/CxheyOUAlG8" type="application/x-shockwave-flash" wmode="transparent" width="425" height="350"></embed></object>[/youtube]

This looks good. Was this only adding the improved iframes to the video as described earlier?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/09/2014 02:26 pm
My meager contribution for iframe 10
0:3:8818,0:4:11564

This reveals another two rows of blocks showing sea spray, but corrupts the rest of the image in a way that I haven't figured out how to fix.  Help me SwissCheese, you're my only hope!

Fantastic effort everyone.

It improved a bit the image, I also added a line with the new option -2 to avoid having the flame "contaminating" the image bottom:

43:2:-1,
0:3:8818,4:5:-1,
7:5:14847,18:05:-1,
0:7:18695,23:13:-1:-30:-30:-30:-30,24:13:-1,
0:16:-2:50:50:50:50,0:17:-1,
7:17:47677

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: AncientU on 05/09/2014 02:30 pm
Here is a give comparing IF5 and IF8,
legs lookl like they are not yet in their final position.

Created with via Imgflip GIF Maker (https://imgflip.com/gifgenerator)
Agree.  You can also see the end of the (left) leg in 5 where it mates to the tip fairing -- this piece would only be visible if the legs were basically still pointing up (at say 30-60 degrees from fully stowed).  The longest extent of the legs away from the stage axis would be just before reaching 90 degrees.

Edit: typos (need more coffee)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/09/2014 02:55 pm
My 2 cents on frame 3. Because of how garbled it is, its possible that the corruption is not in the data itself but in the frame header. The iframe header has lots of specifications that define how even numbers are defined (whether to use reversible or non-reversible codings). That would cause the entire frame to be decoded wrong.

Edit: Just checked, yeah the frame header is corrupted. If we fix it, large parts of the image may pop out. Other frames may also have corrupted headers but not bad enough to block header decoding. This would result in odd settings for the image still though.

I've been doing some inspection of the transport stream and iframe3 in a hex editor. iframe3 bears no resemblance to any other iframe in its data content. I don't think its actually an iframe or even a fame. I think the decoder got confused and tried to extract the data as an iframe. There are indeed 13 iframes though, we're just not extracting the first one for whatever reason.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/09/2014 02:58 pm
I think he's referring to restoring it to 100% working video. If he wasn't, we already vastly beat his predictions. Not sure whether to trust his posts or not. Either that or his experience isn't as much as he claims it is as I'm apparently doing better than him with 1 week of experience and possibly superior tools.

MLindner, I think the key advantage which you and your one week of experience bring to the table is that you have no idea what's supposed to be impossible.  :D

Perhaps because he does video recovery as a business, he never considered the possibility of a team of people staying up until the wee hours of the morning fueled by Red Bull and Doritos twiddling bit combinations in a single macro block of a single iframe looking for the best incremental improvement in a nearly-uniform stretch of blue ocean among hundreds of variations, because no customer could ever afford that, even ULA. (I hope he's not wasting a bunch of time trying to suss out more of iframe 8 on his own.)

Since the remuneration for the folks here, and your cheering section, is not mere cash but the excitement of seeing these historic images emerge from the tangled snarl of the transport stream, it becomes an affordable effort. I wonder if they'll have this discussion thread on display next to the restored video that visitors will find looping at the kiosk in front of the first recovered Falcon 9R at the Smithsonian?  ;D
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Swatch on 05/09/2014 03:23 pm
More work on iframe8. Will post the mmb later.


Did you ever post the mmb for this?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/09/2014 03:45 pm
My 2 cents on frame 3. Because of how garbled it is, its possible that the corruption is not in the data itself but in the frame header. The iframe header has lots of specifications that define how even numbers are defined (whether to use reversible or non-reversible codings). That would cause the entire frame to be decoded wrong.

Edit: Just checked, yeah the frame header is corrupted. If we fix it, large parts of the image may pop out. Other frames may also have corrupted headers but not bad enough to block header decoding. This would result in odd settings for the image still though.

I've been doing some inspection of the transport stream and iframe3 in a hex editor. iframe3 bears no resemblance to any other iframe in its data content. I don't think its actually an iframe or even a fame. I think the decoder got confused and tried to extract the data as an iframe. There are indeed 13 iframes though, we're just not extracting the first one for whatever reason.

Also the Postion in the Video suggest that interpretation:
iFrame2 at 1146612
iFrame3 (which probably is no iFrame) at 1207712
iFrame 4 at 1446284

all other iFrames are also separeted by around 300000 bits.

Exceptions: between IF 6 and 7 it is twice as much, Are we sure there is no Iframe between these 2?
Also IF1 is at 847880. Could this mean there are 3 more iframes before that? 85/30 is ~ 3.

Cheers,
Shanuson

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/09/2014 04:07 pm
My 2 cents on frame 3. Because of how garbled it is, its possible that the corruption is not in the data itself but in the frame header. The iframe header has lots of specifications that define how even numbers are defined (whether to use reversible or non-reversible codings). That would cause the entire frame to be decoded wrong.

Edit: Just checked, yeah the frame header is corrupted. If we fix it, large parts of the image may pop out. Other frames may also have corrupted headers but not bad enough to block header decoding. This would result in odd settings for the image still though.

I've been doing some inspection of the transport stream and iframe3 in a hex editor. iframe3 bears no resemblance to any other iframe in its data content. I don't think its actually an iframe or even a fame. I think the decoder got confused and tried to extract the data as an iframe. There are indeed 13 iframes though, we're just not extracting the first one for whatever reason.

Also the Postion in the Video suggest that interpretation:
iFrame2 at 1146612
iFrame3 (which probably is no iFrame) at 1207712
iFrame 4 at 1446284

all other iFrames are also separeted by around 300000 bits.

Exceptions: between IF 6 and 7 it is twice as much, Are we sure there is no Iframe between these 2?
Also IF1 is at 847880. Could this mean there are 3 more iframes before that? 85/30 is ~ 3.

Cheers,
Shanuson

I do not know for the beginning, but for iframe 6 the blocks are "heavier"  (more bits)  than in the other iframes. The file is roughly twice larger than the other ones.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/09/2014 04:08 pm
More work on iframe8. Will post the mmb later.


Did you ever post the mmb for this?

That was a hackjob i threw together in time for Chris's article. I wasn't exactly happy with how I did some of the erasing of parts of the image. I'll release something better later. Right now I'm fussing with the transport stream to see if we can get the actual first iframe.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/09/2014 04:09 pm
My 2 cents on frame 3. Because of how garbled it is, its possible that the corruption is not in the data itself but in the frame header. The iframe header has lots of specifications that define how even numbers are defined (whether to use reversible or non-reversible codings). That would cause the entire frame to be decoded wrong.

Edit: Just checked, yeah the frame header is corrupted. If we fix it, large parts of the image may pop out. Other frames may also have corrupted headers but not bad enough to block header decoding. This would result in odd settings for the image still though.

I've been doing some inspection of the transport stream and iframe3 in a hex editor. iframe3 bears no resemblance to any other iframe in its data content. I don't think its actually an iframe or even a fame. I think the decoder got confused and tried to extract the data as an iframe. There are indeed 13 iframes though, we're just not extracting the first one for whatever reason.

Also the Postion in the Video suggest that interpretation:
iFrame2 at 1146612
iFrame3 (which probably is no iFrame) at 1207712
iFrame 4 at 1446284

all other iFrames are also separeted by around 300000 bits.

Exceptions: between IF 6 and 7 it is twice as much, Are we sure there is no Iframe between these 2?
Also IF1 is at 847880. Could this mean there are 3 more iframes before that? 85/30 is ~ 3.

Cheers,
Shanuson

I do not know for the beginning, but for iframe 6 the blocks are "heavier"  (more bits)  than in the other iframes. The file is roughly twice larger than the other ones.

I have the feeling that there is at least two additional iframes before IF1. In the original video there is a section of the rocket surrounded by clouds, with no sea visible.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mikes on 05/09/2014 04:34 pm
I am approaching it from a post-processing standpoint, so use ffmpeg to create a PNG, then consume the PNG with a separate application. My biggest limitation here is that the output has to go to file (due to image2 spec), so I have it writing to a ramdisk. Is there any way to have to write PNG data to stdout instead?
Have you tried using "-" as the output file specifier?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: david1971 on 05/09/2014 04:35 pm
The first thing that went though my head when I read about the ORBCOMM launch delay was "this means the video folks will get a few extra days to get their clean version of a landing done before SpaceX has new video."
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: rsnellenberger on 05/09/2014 04:49 pm

The first thing that went though my head when I read about the ORBCOMM launch delay was "this means the video folks will get a few extra days to get their clean version of a landing done before SpaceX has new video."

Same here...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/09/2014 04:58 pm
My 2 cents on frame 3. Because of how garbled it is, its possible that the corruption is not in the data itself but in the frame header. The iframe header has lots of specifications that define how even numbers are defined (whether to use reversible or non-reversible codings). That would cause the entire frame to be decoded wrong.

Edit: Just checked, yeah the frame header is corrupted. If we fix it, large parts of the image may pop out. Other frames may also have corrupted headers but not bad enough to block header decoding. This would result in odd settings for the image still though.

I've been doing some inspection of the transport stream and iframe3 in a hex editor. iframe3 bears no resemblance to any other iframe in its data content. I don't think its actually an iframe or even a fame. I think the decoder got confused and tried to extract the data as an iframe. There are indeed 13 iframes though, we're just not extracting the first one for whatever reason.

Also the Postion in the Video suggest that interpretation:
iFrame2 at 1146612
iFrame3 (which probably is no iFrame) at 1207712
iFrame 4 at 1446284

all other iFrames are also separeted by around 300000 bits.

Exceptions: between IF 6 and 7 it is twice as much, Are we sure there is no Iframe between these 2?
Also IF1 is at 847880. Could this mean there are 3 more iframes before that? 85/30 is ~ 3.

Cheers,
Shanuson

I do not know for the beginning, but for iframe 6 the blocks are "heavier"  (more bits)  than in the other iframes. The file is roughly twice larger than the other ones.

I have the feeling that there is at least two additional iframes before IF1. In the original video there is a section of the rocket surrounded by clouds, with no sea visible.

Further investigation.

I think there is only a single iframe not being decoded. I found all the relevant frame headers and everything at 0x03E3EC position in the file.

0x000001B0 = visual_object_sequence_start_code
0x03       = profile and level indication
0x000001B5 = visual_object_start_code

There are exactly 13 of these patterns in this sequence in the transport stream. I did some copy and paste searching of the starts of the various iframe file dumps and each one lines up to one of these code sets. Importantly, we entirely skip the first set of these codes. The first iframe is the second set of these. I'm trying to figure out why its not decoding, but not there yet.

We also need to somehow disable iframe3 from being used as that will screw up any video that could possibly be correct that would otherwise reference iframe2.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/09/2014 05:31 pm
Further investigation.

I think there is only a single iframe not being decoded. I found all the relevant frame headers and everything at 0x03E3EC position in the file.

0x000001B0 = visual_object_sequence_start_code
0x03       = profile and level indication
0x000001B5 = visual_object_start_code

There are exactly 13 of these patterns in this sequence in the transport stream. I did some copy and paste searching of the starts of the various iframe file dumps and each one lines up to one of these code sets. Importantly, we entirely skip the first set of these codes. The first iframe is the second set of these. I'm trying to figure out why its not decoding, but not there yet.

We also need to somehow disable iframe3 from being used as that will screw up any video that could possibly be correct that would otherwise reference iframe2.

Here's the current iframe frame numbers and timestamps as best as I can make them out:

 1  33 19:35:03.6
 2  53 19:35:04.?
 3  57 ?
 4  73 19:35:??.?
 5  93 19:35:07.6
 6 113 19:35:08.9
 7 150 ?
 8 169 19:35:12.9
 9 188 19:35:14.3
10 207 19:35:??.?
11 227 19:35:1?.?
12 246 19:35:18.2

Assuming a few of the sequences are contiguous, we can reconstruct some timestamps.

 1  33 19:35:03.6
 2  53 19:35:04.9
 3  57 ?
 4  73 19:35:06.3
 5  93 19:35:07.6
 6 113 19:35:08.9
 7 150 ?
 8 169 19:35:12.9
 9 188 19:35:14.3
10 207 19:35:15.6
11 227 19:35:16.9
12 246 19:35:18.2

So, here are the gaps:

 2   53 19:35:04.9
-3   ?? 19:35:05.1
 4   73 19:35:06.3
 5   93 19:35:07.6
 6  113 19:35:08.9
-7a 13? 19:35:10.2
-7b 150 19:35:11.5
 8  169 19:35:12.9

It's possible the data for iframe 3 isn't even in the stream. We are clearly missing some p and b frames too.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: seanpg71 on 05/09/2014 05:37 pm
So I could align better iframe 5 (coerced.ts) with the other ones and add a few sequences, still room for improvement.

We clearly see that the legs are not completely deployed!

0:0:550,14:0:-1,
0:5:9736,26:7:-1,
4:10:22722,25:12:32001,28:12:-1,16:13:-1:0:-25:0:0,17:13:-1:-15:0:0:0,
19:13:34518,10:14:-1,
17:14:39053,24:14:-1,21:15:-1:-25:-10,22:15:-1,
34:15:42349,24:16:-1,
27:16:47754,42:19:-1,
43:19:69441,43:20:-1,
01:21:76873,11:22:-1,
12:22:85489,26:22:-1,
17:23:91682,22:24:-1,
23:24:97869,00:27:-1,
16:27:109631,24:27:-1,
25:27:110437,28:28:-1,
3:29:124970

I added a few more sections to that.  Definitely still some more areas to get a bit of improvement from though.

0:0:550,14:0:-1,
32:4:9085,25:7:-1,
4:10:22722,25:12:32001,28:12:-1,
16:13:-1:0:-25:0:0,17:13:-1:-15:0:0:0,
18:13:34192,10:14:-1
17:14:39053,23:14:-1,
21:15:-1:-25:-10,
25:15:41044,
23:16:46465,
28:16:47954,
41:19:69012,
43:19:69441,43:20:-1,
1:21:76873,11:22:-1,
12:22:85489,26:22:-1,
27:22:87056,42:22:-1,
38:22:88152,10:23:-1,
11:23:91098,16:23:-1,
17:23:91682,22:24:-1,
23:24:97869,
0:25:100325,0:27:-1,
2:27:105388,
22:27:110276,30:28:-1,
31:28:121730,1:29:-1,
2:29:123876








Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: xcg001 on 05/09/2014 05:44 pm
Here's the current iframe frame numbers and timestamps as best as I can make them out:

 1  33 19:35:03.6
 2  53 19:35:04.?
 3  57 ?
 4  73 19:35:??.?
 5  93 19:35:07.6
 6 113 19:35:08.9
 7 150 ?
 8 169 19:35:12.9
 9 188 19:35:14.3
10 207 19:35:??.?
11 227 19:35:1?.?
12 246 19:35:18.2

Assuming a few of the sequences are contiguous, we can reconstruct some timestamps.

 1  33 19:35:03.6
 2  53 19:35:04.9
 3  57 ?
 4  73 19:35:06.3
 5  93 19:35:07.6
 6 113 19:35:08.9
 7 150 ?
 8 169 19:35:12.9
 9 188 19:35:14.3
10 207 19:35:15.6
11 227 19:35:16.9
12 246 19:35:18.2

So, here are the gaps:

 2   53 19:35:04.9
-3   ?? 19:35:05.1
 4   73 19:35:06.3
 5   93 19:35:07.6
 6  113 19:35:08.9
-7a 13? 19:35:10.2
-7b 150 19:35:11.5
 8  169 19:35:12.9

It's possible the data for iframe 3 isn't even in the stream. We are clearly missing some p and b frames too.

I am not a video specialist but looking at these timestamps there is ~1.3 seconds difference between each I frame so most likely there should be no I frame between frame 2 and 4.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/09/2014 05:50 pm
I am not a video specialist but looking at these timestamps there is ~1.3 seconds difference between each I frame so most likely there should be no I frame between frame 2 and 4.

Gah! You're right. It's too early in the morning for such advanced math.


 2   53 19:35:04.9
-3   does not exist
 4   73 19:35:06.3
 5   93 19:35:07.6
 6  113 19:35:08.9
-7a 13? 19:35:10.2
-7b 150 19:35:11.5
 8  169 19:35:12.9
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: AncientU on 05/09/2014 06:08 pm
I am not a video specialist but looking at these timestamps there is ~1.3 seconds difference between each I frame so most likely there should be no I frame between frame 2 and 4.

Gah! You're right. It's too early in the morning for such advanced math.


 2   53 19:35:04.9
-3   does not exist
 4   73 19:35:06.3
 5   93 19:35:07.6
 6  113 19:35:08.9
-7a 13? 19:35:10.2
-7b 150 19:35:11.5
 8  169 19:35:12.9

Would be consistent number-sequence wise if the first column was incremented by 20 (33, 53, 73, 93, 113, 133-7a, 153-7b, etc.) and the seconds incremented by 1.3.  This drops i frame 3 as suggested by many and also fixes7a/b.  There are also 20-ish gaps in the higher frames as well as 1.3-1.4s increments.  Somewhere in the 150-170 range, an extra DC offset  of about 6 was added.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/09/2014 06:13 pm
Ok, first try at auto-marking bad MBs based on ffmpeg log output and primitive analysis.

Blocks which were reported as errors by ffmpeg (modified, with debug on) are marked black. Internally, this tracks:
  --  the -1 and -2 override blocks from MBM,
  -- dc clipped at
  -- ac-tex at
  -- dquant at
  -- stuffing at
  -- Error at MB
although these are all painted black as the colors were confusing.

Blocks that were NOT reported as errors but probably are bad are translucent red.
   -- currently tests for the block having less than 5 individual colors.

(http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=582552)

Next step is to begin (hopefully) intelligent mutation of the MBM string to achieve a better overall score. Sample image is of frame 4 with saliva_sweet MBMs (from wiki).


Edit: Here's the Java source if anyone else wants to play. http://pastebin.com/bKgzCRqD The mammoth inner class at the bottom is for writing the animated gif :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Swatch on 05/09/2014 06:13 pm
There also appears to be, on average, 20 frames between each I-frame.  Like xcg001, I'm no video specialist, so I'm not sure if that should stay relatively consistant or not.

Edit:  What AncientU said...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/09/2014 06:17 pm
Here's what I have so far. It's about 2-3 seconds of video almost up to splashdown:

[youtube]<object width="425" height="350"><param name="movie" value="http://www.youtube.com/v/CxheyOUAlG8"></param><param name="wmode" value="transparent"></param><embed src="http://www.youtube.com/v/CxheyOUAlG8" type="application/x-shockwave-flash" wmode="transparent" width="425" height="350"></embed></object>[/youtube]

This looks good. Was this only adding the improved iframes to the video as described earlier?

I took raw.ts-nomc-ignoreerr.mkv from way back at the beginning of this thread and broke it up in to individual frames. With motion turned off, what is left is block updates and something approximating a delta between the current frame and nearby i- and p-frames. A fraction of the intermediate frames are useable with a bit of shifting around. I added in the fixed up i-frames and cut out the bad blocks. I then went sequentially through the frames adding deltas and alpha-blending the block updates.

I think the video could be vastly improved with some more tweaking, more intermediate frames, and a longer run time. However, editing the frames is obviously rather time consuming. If anyone is interested in helping out, I can post my project files to the wiki.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/09/2014 06:30 pm
There also appears to be, on average, 20 frames between each I-frame.  Like xcg001, I'm no video specialist, so I'm not sure if that should stay relatively consistant or not.

Edit:  What AncientU said...

A bit off topic, but I updated the Iframe sequence that was on the wiki, to match this speed.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Y Combinator on 05/09/2014 06:52 pm
There also appears to be, on average, 20 frames between each I-frame.  Like xcg001, I'm no video specialist, so I'm not sure if that should stay relatively consistant or not.

Edit:  What AncientU said...

A bit off topic, but I updated the Iframe sequence that was on the wiki, to match this speed.

I believe this is 15 FPS video; it looks like your animated GIF is assuming 30 FPS and as such is too fast.  Check the timestamps against a stopwatch.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/09/2014 07:03 pm
There also appears to be, on average, 20 frames between each I-frame.  Like xcg001, I'm no video specialist, so I'm not sure if that should stay relatively consistant or not.

Edit:  What AncientU said...

A bit off topic, but I updated the Iframe sequence that was on the wiki, to match this speed.

I believe this is 15 FPS video; it looks like your animated GIF is assuming 30 FPS and as such is too fast.  Check the timestamps against a stopwatch.

Yep, you are right. I updated it to a new version.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Swatch on 05/09/2014 07:32 pm
Dang... Frame 4 is tantalizing because of the leg deploy.

Also, that pace feels more accurate moralec.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: AncientU on 05/09/2014 07:34 pm
There also appears to be, on average, 20 frames between each I-frame.  Like xcg001, I'm no video specialist, so I'm not sure if that should stay relatively consistant or not.

Edit:  What AncientU said...

A bit off topic, but I updated the Iframe sequence that was on the wiki, to match this speed.

I believe this is 15 FPS video; it looks like your animated GIF is assuming 30 FPS and as such is too fast.  Check the timestamps against a stopwatch.

Yep, you are right. I updated it to a new version.
Might also want to drop i frame 3.  Leg deployment starts just after 2 and is about a quarter of the way done by frame 4, 1.3s later. Engine exhaust hitting sea surface happens between 8 and 9.  Splashdown at around 10. "Settled" vertically in sea at 13.  Not sure when engine start happens/happened.  Leg deployment was supposed to be half way through a 20ish second burn IIRC -- leg deployment to splash (about 2 -10) is 7 or 8 x 1.3s = 9.1s-10.4s which checks.

Ed: Note -- 3 could be already dropped
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/09/2014 07:48 pm
My 2 cents on frame 3. Because of how garbled it is, its possible that the corruption is not in the data itself but in the frame header. The iframe header has lots of specifications that define how even numbers are defined (whether to use reversible or non-reversible codings). That would cause the entire frame to be decoded wrong.

Edit: Just checked, yeah the frame header is corrupted. If we fix it, large parts of the image may pop out. Other frames may also have corrupted headers but not bad enough to block header decoding. This would result in odd settings for the image still though.

I've been doing some inspection of the transport stream and iframe3 in a hex editor. iframe3 bears no resemblance to any other iframe in its data content. I don't think its actually an iframe or even a fame. I think the decoder got confused and tried to extract the data as an iframe. There are indeed 13 iframes though, we're just not extracting the first one for whatever reason.

Also the Postion in the Video suggest that interpretation:
iFrame2 at 1146612
iFrame3 (which probably is no iFrame) at 1207712
iFrame 4 at 1446284

all other iFrames are also separeted by around 300000 bits.

Exceptions: between IF 6 and 7 it is twice as much, Are we sure there is no Iframe between these 2?
Also IF1 is at 847880. Could this mean there are 3 more iframes before that? 85/30 is ~ 3.

Cheers,
Shanuson

I do not know for the beginning, but for iframe 6 the blocks are "heavier"  (more bits)  than in the other iframes. The file is roughly twice larger than the other ones.

I have the feeling that there is at least two additional iframes before IF1. In the original video there is a section of the rocket surrounded by clouds, with no sea visible.

Further investigation.

I think there is only a single iframe not being decoded. I found all the relevant frame headers and everything at 0x03E3EC position in the file.

0x000001B0 = visual_object_sequence_start_code
0x03       = profile and level indication
0x000001B5 = visual_object_start_code

There are exactly 13 of these patterns in this sequence in the transport stream. I did some copy and paste searching of the starts of the various iframe file dumps and each one lines up to one of these code sets. Importantly, we entirely skip the first set of these codes. The first iframe is the second set of these. I'm trying to figure out why its not decoding, but not there yet.

We also need to somehow disable iframe3 from being used as that will screw up any video that could possibly be correct that would otherwise reference iframe2.

I think the first iframe is a lost cause. Where the user data section should start there is instead a "video_session_error_code" which according to the spec: "The video_session_error_code has been allocated for use by a media interface to indicate where uncorrectable errors have been detected."

Even replacing the code with the user data start code and the data itself with another frame's user data, the output is garbage. The decoder barfs everywhere and can't even get to decoding macroblocks.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Digitalchromakey on 05/09/2014 08:04 pm
I-Frame 6

Looks like the RHS Leg is being deployed with the LHS leg already locked in position.

Needs a lot more work to be more certain.

0:0:583,43:0:5530,
00:01:6129,02:01:6129,21:1:8584,22:1:8796,
4:3:12577,14:3:19199,15:03:-1,18:03:19433,20:3:14083,23:3:14220,42:3:21474,
0:04:-1,31:04:24629,32:4:15280,
6:5:17702,7:5:-1,13:05:27662,14:05:-1,42:0:5530,17:05:28599,18:5:13888,24:05:-1,25:05:29375,
28:7:40520,29:7:40520,32:07:-1,37:07:41235,
1:8:37105,
1:9:37105,
26:10:61670,27:10:40399,
17:11:64694,
0:13:56967'
0:16:71551,1:10:11678,
1:11:11678,
5:12:12185,18:12:58656,
07:13:-1,13:13:60221,14:13:13373,15:13:60221,
19:14:59308,
12:15:72126,
0:17:71551,2:17:71610,
0:18:71551,7:18:72226,
0:19:121762,
1:28:202509




Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/09/2014 08:11 pm
So I tried to get something with iframe 4, but still a lot to do. I have to stop now for a few days so good luck to those who wish to improve it ;)

From try1.ts

5:1:4200,00:03:-1,
02:03:10968,10:03:-1,
17:06:22427,14:11:-1,
38:16:65876,19:17:-1,
00:19:78514,28:20:-1,
0:28:129509

At 17911 and 55810 there are also valid sequences, which I do not know exactly where to put, and I did not really try to improve the bottom of the frame.

Also some sequences from http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195103#msg1195103 could help if put at the accurate place.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: cscott on 05/09/2014 08:11 pm
I think the first iframe is a lost cause.

Perhaps you could turn your attention to the missing iframe "7a"?  Maybe there's a bit flipped somewhere which is causing your search for 0x000001B0 / 0x03 / 0x000001B5 not to turn it up?  (I'm assuming the existing iframe 7 is "7b" based on the timestamp image, someone correct me if i'm wrong.)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/09/2014 08:17 pm
Have the brute-force code running on frame 4 checking 1-bit modifications, with a threshold of eliminating 4 or more erroneous blocks (starting with 00:10:-1,00:14:-1,00:17:71044,00:24:-1,00:27:114198,08:15:61126,19:05
:20944,21:13:50333,25:14:57756,35:05:-1,36:15:-1,43:06:27582). It is running pretty slowly now, about 77ms per bit (and there are a lot of bits!) but it has produced some interesting results (31 solutions and counting for a 1-bit flip).

I'm not very good at playing with the numbers manually, perhaps someone else wants to see if any of these can be improved? These are the best so far:

00:10:-1,00:14:-1,00:17:71044,00:24:-1,00:27:114198,08:
15:61126,19:05:20944,21:13:50333,25:14:57756,35:05:-1,36:15:-1,43:06:27582,X:10838:32

00:10:-1,00:14:-1,00:17:71044,00:24:-1,00:27:114198,08:
15:61126,19:05:20944,21:13:50333,25:14:57756,35:05:-1,36:15:-1,43:06:27582,X:9308:16

00:10:-1,00:14:-1,00:17:71044,00:24:-1,00:27:114198,08:
15:61126,19:05:20944,21:13:50333,25:14:57756,35:05:-1,36:15:-1,43:06:27582,X:9278:128

00:10:-1,00:14:-1,00:17:71044,00:24:-1,00:27:114198,08:
15:61126,19:05:20944,21:13:50333,25:14:57756,35:05:-1,36:15:-1,43:06:27582,X:4548:32

00:10:-1,00:14:-1,00:17:71044,00:24:-1,00:27:114198,08:
15:61126,19:05:20944,21:13:50333,25:14:57756,35:05:-1,36:15:-1,43:06:27582,X:4416:32
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Swatch on 05/09/2014 08:18 pm
I think the first iframe is a lost cause.

Perhaps you could turn your attention to the missing iframe "7a"?  Maybe there's a bit flipped somewhere which is causing your search for 0x000001B0 / 0x03 / 0x000001B5 not to turn it up?  (I'm assuming the existing iframe 7 is "7b" based on the timestamp image, someone correct me if i'm wrong.)

Not sure how long it would take to brute-force through the file in the area of concern (from pos 2054276 to 2940884), performing a bit-flip at each bit along the way and performing a search for 0x000001B0 / 0x03 / 0x000001B5 within the bits +/- the total message length.  If you don't see anything, flip the bit back and move onto the next bit.  When you see a new hit for 0x000001B0 / 0x03 / 0x000001B5, record the file location and then go back later to see if maybe a new iframe materializes.

Frustrating because I know I used to be able to do all I said above, but I've been away from coding for too long to do it in any expedient manner.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/09/2014 08:20 pm
Not sure how long it would take to brute-force through the file in the area of concern, performing a bit-flip at each bit along the way and performing a search for 0x000001B0 / 0x03 / 0x000001B5 within the bits +/- the total message length.  If you don't see anything, flip the bit back and move onto the next bit.  When you see a new hit for 0x000001B0 / 0x03 / 0x000001B5, record the file location and then go back later to see if maybe a new iframe materializes.

Frustrating because I know I used to be able to do all I said above, but I've been away from coding for too long to do it in any expedient manner.

The brute-force I just posted is for frame 4, not the mysterious missing frame. I'm just flipping bits in the MBs that ffmpeg identified.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Oersted on 05/09/2014 08:23 pm
How much intimate knowledge of video compression and codecs and whatnot would have never been discovered if it wasn't for this garbled video! :-)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/09/2014 08:34 pm
Careful with the long number strings folks. So long they break the site width. A split point half way works.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: michaelni on 05/09/2014 08:39 pm
Third, there when a block from a static area (faring, clock, smudge) is out of place but parsed correctly, it should have a high correlation with the same block in a "known good" frame using an appropriate fourier transform. Since there are relatively few blocks (600) a complete distance matrix could be constructed with 380k convolutions (or half that with a good mask and bad-block detection). Main problem here is determining the fastest/most accurate transform for block difference. Maybe the ffmpeg guys have some advice here?

Comparing blocks this way can be a bit tricky, as the DC values could be wrong as they depend on previous blocks and one row or column of AC values could be wrong when ac prediction is used. Also if the QP is wrong the block could be dequantized with a wrong scale factor. Also the blocks could be subjected to cliping when they are too bright or dark. The cliping issue could be avoided by comparing the coefficients before the IDCT.
If one did some block brute force comparisson, one could for each pair of blocks to compare rescale one to compensate for a possible QP error, adjust its DC value and if it uses ac prediction maybe use the block above or to he left if its static too to remove its influence, whats left could be compared by some means like sum of squared differences or sum of absolute differences of the coefficients.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/09/2014 08:48 pm
I-Frame 6

Looks like the RHS Leg is being deployed with the LHS leg already locked in position.

Needs a lot more work to be more certain.

0:0:583,43:0:5530,
00:01:6129,02:01:6129,21:1:8584,22:1:8796,
4:3:12577,14:3:19199,15:03:-1,18:03:19433,20:3:14083,23:3:14220,42:3:21474,
0:04:-1,31:04:24629,32:4:15280,
6:5:17702,7:5:-1,13:05:27662,14:05:-1,42:0:5530,17:05:28599,18:5:13888,24:05:-1,25:05:29375,
28:7:40520,29:7:40520,32:07:-1,37:07:41235,
1:8:37105,
1:9:37105,
26:10:61670,27:10:40399,
17:11:64694,
0:13:56967'
0:16:71551,1:10:11678,
1:11:11678,
5:12:12185,18:12:58656,
07:13:-1,13:13:60221,14:13:13373,15:13:60221,
19:14:59308,
12:15:72126,
0:17:71551,2:17:71610,
0:18:71551,7:18:72226,
0:19:121762,
1:28:202509

Please note the solution from Gnonthgol in http://spacexlanding.wikispaces.com/ (did not find his post):

0:0:583,
39:0:5496,43:0:-1,
02:01:6129,
21:1:8584,
22:1:8796,6:3:-1,
02:06:29800,2:8:-1,
12:10:46767,00:11:-1,
3:12:56455,18:12:-1,
26:12:61670,27:12:-1,
17:13:64694,28:12:-1,
32:13:71022,4:14:-1,
27:14:78837,5:15:-1,
19:15:92898,23:15:-1,
27:17:107825,27:18:-1,
0:19:121762,04:21:-1,
31:22:159772,0:24:-1,
30:25:184570,07:27:-1,
35:27:200466
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/09/2014 09:04 pm
Here's the new try1-fixed.ts Iframe3 has been converted to a b-frame.

Edit: It apparently doesn't decode at all now though. It obviously wasn't an I-frame though. We should probably remove I-frame 3 from the wiki.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/09/2014 09:25 pm
(http://content3.viralnova.com/wp-content/uploads/2014/05/25-pA0T7Mc.jpg) (http://www.viralnova.com/weekly-chalkboard-art/)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: James (Lockheed) on 05/09/2014 10:57 pm
Amazing work guys. We could have used you when we tried to get some footage down from the ET "Death Cam"!

http://www.nasaspaceflight.com/2011/07/sts-135-et-camera-ascent-no-usable-video-reentry/
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/09/2014 10:59 pm
Amazing work guys. We could have used you when we tried to get some footage down from the ET "Death Cam"!

http://www.nasaspaceflight.com/2011/07/sts-135-et-camera-ascent-no-usable-video-reentry/

Wasn't that analog? Analog = black magic.

Edit: Yeah. Analog NTSC over FM radio. If you don't have an SDR recording of it there's nothing better you'll be able to process it into AFAIK.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/10/2014 12:17 am
Ok, I looked into try1.ts

found the postion of an I-Frame around 0x23bdd6 in try1.ts.
most bit patterns are there, but somehow set of by 4bit (half a byte, so one does not find it easily in a hex editor) and also that is probably the reason why it is not found by the decoder. The decoder wants stuff to start at the beginning of a byte not in the middle.

A typical header part, Coloured bits are similar in I-Frame headers 2,3,4,5,6,8,9,10,11,12,13,14 (numbered not as in the wiki, but by appearance in try1.ts)
This could be due to some manual editing by someone else, have not checked how the look in other .ts files.
The position mentioned is the start of sequence 00 00 01 B6 1

IF02:  0x0CEEE9;
00 00 00 01 E0 00 00 81 80 07 21 01 67 CE 5D FF FF 00 00 01 B0 03 00 00 01 B5 0D 0F 00 00 01 00 00 00 01 20 00 C4 8D C0 00 46 56 C0 1E C0 01 F2 46 9A FC
7C 2E EE 2C 08 78 28 C7 00 00 01 B2 65 6D 34 76 20 34 2E 33 2E 30 2E 30 00 C3 FF 00 00 01 B6 14

Now Frame1 (frame0 of wiki), red are the bytes that are different to every other I-Frame
IF01:  0x03E429; 
00 00 00 01 E0 00 00 81 80 07 21 01 59 79 7D FF FF 00 00 01 B0 03 00 00 01 B5 0D 0F 00 00 01 00 00 00 01 20 00 C4 8D C0 00 46 56 C0 1E C0 01 A1 62 9A FC
7C 2E EE 2C 08 78 28 C7 00 00 01 B4 7B 6D 75 76 32 34 02 27 26 5C 4E 5B 82 2A F8 00 31 30 B8 3A

Same for the frame between frame 6 and 7 of the wiki:
Also bits are shifted here by 4, compared to the ts file, therefore the .5 at the position:
IF07:  0x23BDEC.5;
40 C2 82 13 B0 01 81 8B 50 07 21 01 8D 22 8D FF FF 00 00 01 B0 03 00 00 01 B5 0D 0F 00 00 01 00 00 00 01 20 00 C4 85 C0 00 46 54 80 13 40 09 EF DE 9A FE
7C 22 EE 2C 08 78 68 C2 BC 10 49 80 6C 8D 34 76 20 34 2E 33 2E 30 2E 30 00 C3 FF 00 00 01 B6 10

You will find those parts for all 14 I-Frames in the file attached.
I dont know how one could extract that shifted frame, hexeditors do not allow to insert only 4 bits, always wants to add a byte.

Edit: I should really go to bed, but found also one at the start: to tired to colour it now compared to IF2.

IF1.5: 0x085c51;
00 00 00 01 E0 00 00 81 80 07 21 01 61 23 ED FF FF 15 38 13 20 03 00 00 41 A8 8C C7 05 06 01 14 00 00 80 80 06 C4 8D C0 40 C7 D5 C0 1E D0 01 93 1D 9A FC
7C 2E EE 2D 08 7E 28 C7 C0 00 A9 B1 7D 6D 64 76 20 34 4A 32 76 30 2E 30 00 C3 7F 03 08 01 86 28
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Swatch on 05/10/2014 01:21 am
Attempted to color them for Shanuson... not sure if I got it right, but it does show the differences...
I realize there are some 'half-hex' pairs that are identical, but I chose to color the complete hex pair if there's any discrepancy.


Shanuson's Note: Now Frame1 (frame0 of wiki), red are the bytes that are different to every other I-Frame
IF01:  0x03E429;  (POS: 255017)
00 00 00 01 E0 00 00 81 80 07 21 01 59 79 7D FF FF 00 00 01 B0 03 00 00 01 B5 0D 0F 00 00 01 00 00 00 01 20 00 C4 8D C0 00 46 56 C0 1E C0 01 A1 62 9A FC
7C 2E EE 2C 08 78 28 C7 00 00 01 B4 7B 6D 75 76 32 34 02 27 26 5C 4E 5B 82 2A F8 00 31 30 B8 3A

IF1.5: 0x085c51;  (POS: 547921)
00 00 00 01 E0 00 00 81 80 07 21 01 61 23 ED FF FF 15 38 13 20 03 00 00 41 A8 8C C7 05 06 01 14 00 00 80 80 06 C4 8D C0 40 C7 D5 C0 1E D0 01 93 1D 9A FC
7C 2E EE 2D 08 7E 28 C7 C0 00 A9 B1 7D 6D 64 76 20 34 4A 32 76 30 2E 30 00 C3 7F 03 08 01 86 28

IF02:  0x0CEEE9;  (POS: 847593)
00 00 00 01 E0 00 00 81 80 07 21 01 67 CE 5D FF FF 00 00 01 B0 03 00 00 01 B5 0D 0F 00 00 01 00 00 00 01 20 00 C4 8D C0 00 46 56 C0 1E C0 01 F2 46 9A FC
7C 2E EE 2C 08 78 28 C7 00 00 01 B2 65 6D 34 76 20 34 2E 33 2E 30 2E 30 00 C3 FF 00 00 01 B6 14

Shanuson's Note: Also bits are shifted here by 4, compared to the ts file, therefore the .5 at the position:
IF07:  0x23BDEC.5;  (POS:  ~2342380)
40 C2 82 13 B0 01 81 8B 50 07 21 01 8D 22 8D FF FF 00 00 01 B0 03 00 00 01 B5 0D 0F 00 00 01 00 00 00 01 20 00 C4 85 C0 00 46 54 80 13 40 09 EF DE 9A FE
7C 22 EE 2C 08 78 68 C2 BC 10 49 80 6C 8D 34 76 20 34 2E 33 2E 30 2E 30 00 C3 FF 00 00 01 B6 10
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/10/2014 02:43 am
I dont know how one could extract that shifted frame, hexeditors do not allow to insert only 4 bits, always wants to add a byte.

It should be pretty straightforward in Perl with unpack/pack. Just unpack with H*, then pack substr($str, 1) back to H. I'll take a look.

Edit: Ok, look taken.

Here's what I did:

1. Padded out my try1.ts to get it lined up with the offsets in header.txt, which may or may not match what I'm seeing in my hex editor, as it's late.
2. Cut out everything up to the 040 C2 82 13 B0 01 81 shifted region, and then searched for the next 00000001E000008180 marker. This marker turned out to be properly byte-aligned, as it happens. I then cut that out as well, leaving only the bit-shifted frame at 299297 bytes.
3. Quick and dirty Perl:
#!/usr/bin/perl
open(IN, "< bitshiftframe") or die; binmode(IN); $/=undef;

$iframe = <IN>;
$hex = substr(unpack('H*', $iframe), 1);

open(OUT, "> shanusons-bitshifted.ts") or die; binmode(OUT);
print(OUT pack('H*', $hex));


This produced a byte-aligned version of 0x23BDEC.5. I attached it here as an mp4, but it's not an MP4 file - it's a straight excerpt of the bytes of interest from the try1.ts transport stream file - the TS extension is not allowed for attachments here.

Edit: removed the bitshifted ts file because the packet headers were mangled by the operation.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/10/2014 03:25 am
I dont know how one could extract that shifted frame, hexeditors do not allow to insert only 4 bits, always wants to add a byte.

It should be pretty straightforward in Perl with unpack/pack. Just unpack with H*, then pack substr($str, 1) back to H. I'll take a look.

PS: where did you get your try1.ts file? The one I pulled from the very bottom of the wikspaces.com page has IF01 which you list at 0x3E429 showing up in my bvi session starting at 0x3E3DB, a 30-byte offset to the left. I'll pad it out at the beginning to make it line up with your numbers, but I want to make sure we're on the same page.

There's a bigger issue is that when edited.ts and try1.ts were created they had "fixed" the transport stream headers by adding the 4 byte headers to them every 188 bytes throughout the stream. If you simply shift the bytes at the moment then all the transport stream headers will become unaligned again not to mention effectively being corruption every 188 bytes. It might be better to re-apply the processes that were done on raw.ts again and produce a new transport stream. Preferably in script form this time so that if something else is found they can quickly be re-applied.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: John_L on 05/10/2014 03:46 am
Well, looks like we're not getting any new video anytime soon.  SpaceX has scrubbed the launch for tomorrow, and pushed it back to "later this month". :(
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/10/2014 04:01 am
Well, looks like we're not getting any new video anytime soon.  SpaceX has scrubbed the launch for tomorrow, and pushed it back to "later this month". :(

More motivation for us to get the video done even better. :P
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Guspaz on 05/10/2014 04:10 am
If it does come down to needing to do automated A/B testing, and humans are needed for that evaluation, I'd like to point out that there is always Amazon's Mechanical Turk. It basically allows you to farm out this sort of effort to large numbers of people fairly cheaply. I don't think you guys are at that point yet, but there's been discussions in this thread about trying to do the testing algorithmically, so I thought I'd mention it.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/10/2014 04:30 am
There's a bigger issue is that when edited.ts and try1.ts were created they had "fixed" the transport stream headers by adding the 4 byte headers to them every 188 bytes throughout the stream. If you simply shift the bytes at the moment then all the transport stream headers will become unaligned again not to mention effectively being corruption every 188 bytes. It might be better to re-apply the processes that were done on raw.ts again and produce a new transport stream. Preferably in script form this time so that if something else is found they can quickly be re-applied.
Good point - odds are that's what happened since I shifted the entire file four bits to the right. I've been heart-attack busy lately so I've resisted the temptation to learn how MPEG encoding works. I'll take a look at the resulting bitshifted file and the video coding documentation, and if you wouldn't mind doing so as well and letting me know if you're inclined. As you can see the shifting operation is straightforward in Perl so it won't be difficult to cope with the TS headers. And also perhaps set up command line arguments to shift arbitrary ranges of nybbles.

Edit: yep, there it is, right at the end: ...FF F4 71 FF F1 6F FF...  Looking at raw.ts now.

Edit: The four-bit offset is present in raw.ts as well at 0x23BD07. Coercing the TS headers in the raw.ts may have mangled things... it's extremely difficult to discern where the missing or added four bits came from, too. Hmmm... it looks like the 00-00-01-b0-03 bytes always appear exactly 28 bytes after a ts header, let's see...

In a good packet formatted to 20 columns, we see the 47 to mark the beginning of the packet, and here "D" is the sequence number (the "3" before it means "adaptation field & payload") and 28 bytes later we see the 00-00-01-B0 pattern:

00316CE4  47 43 E8 3D 07 10 00 34 5B FF 7E 00 00 00 01 E0 00 00 81 80
00316CF8  07 21 01 A3 21 DD FF FF 00 00 01 B0 03 00 00 01 B5 0D 0F 00
00316D0C  00 01 00 00 00 01 20 00 C4 8D C0 00 46 56 C0 1E C0 01 FD 30


In the bit-shifted one, counting 28 octets back from the "0-00-00-1B-0" we land in the middle of the marked 4D, meaning that we have four extra bits between the header and the sequence:

0023BCE8  FA A9 FF 4D BE 66 AF 8C EC B9 A7 CD C3 5C 74 0C 28 21 3B 00
0023BCFC  18 18 B5 00 72 10 18 D2 28 DF FF F0 00 00 1B 00 30 00 00 1B
0023BD10  50 D0 F0 00 00 10 00 00 00 12 00 0C 48 5C 00 04 65 48 01 34


Based on what appears to be a clean packet header above this region at showing "47-1F-FF-18," at 0x23bb74 in raw.ts, the sequence number at the marked packet header above should be "A", since it's about 2x188 bytes away from the clean header.

This reveals some interesting information as well - in raw.ts, 188 bytes after this clean header is "C0-D8-61-01" which was coerced in try1.ts to "47 83 E8 11" so the sequence number there is incorrect (1 instead of 9) due to a bit flip - "0001" instead of "1001."  This apparently means that the try1.ts file is stuffed full of continuity count errors - the coerced header just after this one has a sequence number of C when it should be A, for example. And if that was ignored, then no-increment-if-no-payload was probably also ignored.

Setting that aside... let's say that the 4D is supposed to be a 47 (0111 instead of 1110, and the sequence number is supposed to be A, and look what we have here...

0023BCE8  FA A9 FF 47 BE 66 AF 8C EC B9 A7 CD C3 5C 74 0C 28 21 3B 00
0023BCFC  18 18 B5 00 72 10 18 D2 28 DF FF F0 00 00 1B 00 30 00 00 1B
0023BD10  50 D0 F0 00 00 10 00 00 00 12 00 0C 48 5C 00 04 65 48 01 34


So thanks to Shanuson we know we're looking for something along the lines of: 00 00 00 01 E0 00 00 81 80 07 21 01 59 79 7D FF FF...

0023BCE8  FA A9 FF 47 BE 66 AF 8 CE CB 9A 7C DC 35 C7 40 C2 82 13 B0 01
0023BCFC  81 8B 50 07 21 01 8D 22 8D FF FF 00 00 01 B0 03 00 00 01 B5
0023BD10  0D 0F 00 00 01 00 00 00 01 20 00 C4 85 C0 00 46 54 80 13


Taking out that one "8" nybble, for example, puts the sequence back to 28 bytes after the packet header's 0x47. So basically we know that the "AF" at the end of the packet header needs to be "1A" or "3A," and that it would be reasonable to conclude that the four extra bits fall somewhere between the 0x47 and 0xFF00. I have that dangling "8" in there, but it could be something else. Someone more familiar with the encoding may be able to recognize it. Maybe it's where the 07-21-01 match breaks.

Edit: It needs to be 3A, because the "0x02" bit is the payload-unit-start indicator which should be set at the beginning of an iframe.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/10/2014 04:55 am
Ok, first try at auto-marking bad MBs based on ffmpeg log output and primitive analysis.

...

Edit: Here's the Java source if anyone else wants to play. http://pastebin.com/bKgzCRqD The mammoth inner class at the bottom is for writing the animated gif :)
Great stuff!!

One suggestion: when comparing a block with other block its best to only compare it with the one above and the one to the left.

This is because of how mpeg-4 works: the current block will directly affect the one below and the one to the right. So you are really comparing to yourself in a way. Also: if the current block is defect (or one until the one below is reached) it is very likely the block below is bit-shifted so you're not comparing the current block to what is supposed to be below it.

Keep that in mind.

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: grythumn on 05/10/2014 05:57 am
Teasing away at the top of iframe 9. 23:04 starts at 15950. Best correction I got for 19:04 was X:15522:80,X:15518:80, but there were several similar solutions, and didn't try an exhaustive 2-bit search[0], much less 3-bit.

Trying to figure out 20, 21, 22... first attempt didn't end up with the correct number of bits. :/ More tomorrow, brute force search in progress.

-Bob
[0] Ran through single bit permutations to remove the largest artifact, noted the bit locations that helped, ran a second set of permutations against for each of the first list.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mikes on 05/10/2014 08:08 am
I think the first iframe is a lost cause.

Perhaps you could turn your attention to the missing iframe "7a"?  Maybe there's a bit flipped somewhere which is causing your search for 0x000001B0 / 0x03 / 0x000001B5 not to turn it up?  (I'm assuming the existing iframe 7 is "7b" based on the timestamp image, someone correct me if i'm wrong.)

Not sure how long it would take to brute-force through the file in the area of concern (from pos 2054276 to 2940884), performing a bit-flip at each bit along the way and performing a search for 0x000001B0 / 0x03 / 0x000001B5 within the bits +/- the total message length.  If you don't see anything, flip the bit back and move onto the next bit.  When you see a new hit for 0x000001B0 / 0x03 / 0x000001B5, record the file location and then go back later to see if maybe a new iframe materializes.

Frustrating because I know I used to be able to do all I said above, but I've been away from coding for too long to do it in any expedient manner.

I think this is equivalent to flipping each of the bits of the search string, then doing a search for each of those 72 strings. I've tried that, but it didn't turn anything up.

#!/usr/bin/python
import os,sys
def binsearch(foo,ninebytes):
    want=hex(ninebytes)[2:-1].zfill(18)
    print want
    os.system("./bgrep "+want+" "+foo)
foo=sys.argv[1]
orig=0x000001b003000001b5
mask=0x800000000000000000
binsearch(foo,orig)
while mask :
    binsearch(foo,orig^mask)
    mask >>= 1


https://raw.githubusercontent.com/tmbinc/bgrep/master/bgrep.c
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/10/2014 11:25 am
Now past the 100,000 read mark for this thread. Worth noting! :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/10/2014 12:23 pm
Ok here is a new file containing a comparison of raw.ts and try1.ts

Containing:
15 I-Frames, the start position of the '47' byte and the following 94 Bytes
Only clearly sees that all those parts are I-Frames.

Most of them are quite similar:
But IF0 and IF1 have some errors

And there is IF7 which is the by 4bit shifted one.  (In the File I manually shifted if so one can compared the bytes better)

The following is how it should look like IMHO.
There are 3 blocks of XXs which are different for each frame. The first one seems to contain a counter. The rest i dont know. But it should be mentioned in that ISO*.pdf how they should look like/what they should contain.

47 43 E8 37 07 10 00 xx xx xx 7E 00 00 00 01 E0 00 00 81 80 07 21 01 xx xx xx FF FF 00 00 01 B0 03 00 00 01 B5 0D 0F 00 00 01 00 00 00 01 20 00 C4 8D C0 00 46 56 C0 1E C0 01 xx xx 9A FC 7C 2E EE 2C 08 78 28 C7 00 00 01 B2 65 6D 34 76 20 34 2E 33 2E 30 4E 31 40 C3 FF 00 00 01 B6 1x

About the problem where the shift occurred: Frames are framed by large blocks of FFs with 47 1F FF 1X in them if they are large enough and X counting up. On Both sides of that IF7 they are already align. So the shift and reshift should have occurred between those. Maybe it is just enough to add a F at the end of the block before IF7 and remove one at the start of the block after IF7 (0x23cde9).

Cheers
Shanuson
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/10/2014 12:56 pm
My I7 MBP spent the night in the freezer (literally) running brute force bit flipping (X flags), and was still chugging like a champ when I killed the process this morning. It had run ffmpeg 2,452,480 times in 14:41:50 (8 threads). Computational feasibility demonstrated.

It came up with a few decent improvements, but no where close to what SwissCheese did by hand (on try1). The best is probably

00:10:-1,00:14:-1,00:17:71044,00:24:-1,00:27:114198,08:15:61126,
19:05:20944,21:13:50333,25:14:57756,35:05:-1,36:15:-1,43:06:27582,
X:15868:3

but I haven't looked through all 1024 yet.

Reading through the latest, it seems like I need to rework the code to run through raw.ts instead. This will probably be faster anyway, since I can manipulate bits directly in the data and skip the -mbm flags argument. Just need to catch up on what has been done.

Great work Shanuson


edit: I updated my java code to include the multithreaded ffmpeg stuff http://pastebin.com/bKgzCRqD

edit2: Just a bit of warning, this will peg your CPU at 100% for a long time, make sure it has plenty of cooling and you don't have any unsaved windows open just in case ;)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/10/2014 01:12 pm
Ok, first try at auto-marking bad MBs based on ffmpeg log output and primitive analysis.

...

Edit: Here's the Java source if anyone else wants to play. http://pastebin.com/bKgzCRqD The mammoth inner class at the bottom is for writing the animated gif :)
Great stuff!!

One suggestion: when comparing a block with other block its best to only compare it with the one above and the one to the left.

This is because of how mpeg-4 works: the current block will directly affect the one below and the one to the right. So you are really comparing to yourself in a way. Also: if the current block is defect (or one until the one below is reached) it is very likely the block below is bit-shifted so you're not comparing the current block to what is supposed to be below it.

Keep that in mind.

Regards,

arnezami

Hmm ok so it is pointless to try manipulation in blocks past the first bad block? Basically we need to completely fix one bad block at a time?

Edit: By "first bad block" I'm indexing blocks index= (x + y*width) i.e. left-to-right, top-to-bottom. Or is it in the order that they are encoded (which isn't always that)?

That's actually not so bad, really makes the problem easier. Basically the chess problem. Maybe we can ask IBM to borrow Deep Blue (or Watson :D) 
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: grythumn on 05/10/2014 01:19 pm
Ok, first try at auto-marking bad MBs based on ffmpeg log output and primitive analysis.

...

Edit: Here's the Java source if anyone else wants to play. http://pastebin.com/bKgzCRqD The mammoth inner class at the bottom is for writing the animated gif :)
Great stuff!!

One suggestion: when comparing a block with other block its best to only compare it with the one above and the one to the left.

This is because of how mpeg-4 works: the current block will directly affect the one below and the one to the right. So you are really comparing to yourself in a way. Also: if the current block is defect (or one until the one below is reached) it is very likely the block below is bit-shifted so you're not comparing the current block to what is supposed to be below it.

Keep that in mind.

Regards,

arnezami

Hmm ok so it is pointless to try manipulation in blocks past the first bad block? Basically we need to completely fix one bad block at a time?

I think you're okay fixing half-block errors that occur later in the image, but don't mess with full block or greater brightness/hue changes... they come and go freely as you fix earlier blocks. At least that's what I'm noticing on Frame 9... I'm having to revert some of my earlier edits.

-Bob
(the mad bit flipper)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/10/2014 01:57 pm
Working on those headers I found that some errors are single bit flips, but there are also stripes of bytes which are simply garbage/randomly turned bits one after an other. In an image you cant repair this I think. Substitute MBs maybe.
Headers can also be fixed, but if part of an frame MB itself is messed up like this you cant really hope to fixed it by flipping random bits.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/10/2014 02:39 pm
About the problem where the shift occurred: Frames are framed by large blocks of FFs with 47 1F FF 1X in them if they are large enough and X counting up. On Both sides of that IF7 they are already align. So the shift and reshift should have occurred between those. Maybe it is just enough to add a F at the end of the block before IF7 and remove one at the start of the block after IF7 (0x23cde9).

The X is the frame sequence number, which starts at zero and counts up through "F" and then back to 0 - this Wikipedia page about the MPEG transport stream has a good explanation of those four bytes starting with 0x47. (http://en.wikipedia.org/wiki/MPEG_transport_stream) The 1F FF 1X means:

0001 1111 1111 1111 0001

000 = no transport error, no payload unit start, no high priority
1 1111 1111 1111 = null packet identifier for fixed-bandwidth padding packets
0001 = 00 - not scrambled; 0 - no adaptation field; 1 - contains payload

This detail may help suss out what the packet headers should be in the bit-shifted packet. For instance, we know that the PID field can't take certain values like 0x0004 through 0x000F, etc.

Lining up the null packets in front of the bit-shifted one in a 47-column hex editor it's easy to see bad bytes, because we know that the 184 bytes following a 471FFF header should all be FF, such as this from raw.ts:

0023BB60  FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 47 1F FF 18 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF
0023BB8F  FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF
0023BBBE  FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF
0023BBED  FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 7F FC FF FF FF FD 7F F4 FF 7C F6 FF FF FF FF FF FF FF DE 7F 7D 76 FC F4 EB 6D 6F F6 FF 1F FD BF
0023BC1C  FF FF FF FF FF FF FF FB 03 EE 76 17 E0 1B DA 08 FF EB DF 14 C0 D8 61 01 F7 5F C8 3F FE 7B FF F8 7D 53 57 70 34 EB DC F3 3F 5C 82 FF F3 FE 3E


This packet-aligned-column view also allows you to see where the good packets pick up after this mangling - at 0x23BFDC with a 0x4703E81A, followed by 0x4703E81B. This also tells us that we have the right number of bytes in the scrambled interim, because the columns still line up.

Moving up the rows, it's easy to see where there's corrupted headers:

0023BC30 C0D86101 should be 4703E815, or something with a bit set to indicate a sequence discontinuity.
0023BCEC BE66AF8C should be 4703E816
0023BDA8 4703E817 is correct
0023BE64 77034818 should be 4703E818
0023BF20 77EB35EE should be 4703E819


So it seems like it should be practical to write a tool that can walk through the raw.ts and use the rules of the packet header formatting and sequence numbers to reconstruct a more coherent transport stream. Even just getting all the errors out of the null packets by replacing everything with FF might make it easier to discern good patterns.

Edit: For example, in raw.ts there are bytes missing or added between 0x37ACC and 0x371EC - that's the first place where the columns shift in a 47-column hex display.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/10/2014 10:09 pm
iframe 6 (try1)
Left leg is not fully deployed yet, but i'm perplexed by the tip of it. At least I think it's the tip because it doesn't look like a cloud or a foam spot on the sea. It kind of resembles the tip of the right leg on iframe 5, which shows the right leg in similar stage of deployment as left leg on this frame. The thing is it doesn't align. It's off by one block (half of macroblock). Is such an alignment error even possible?

Some macroblocks are highly suspect, they may be random noise that vaguely looks like information that I forced into the picture (18:12 and 25:12 are the main concerns).

Brace for mmb
X:82804:80,X:134444:80,0:0:583,39:0:5496,43:0:-1,2:1:6129,21:1:8584,22:1:8796,6:3:18105,
7:3:18276,9:3:29527,30:3:32359,31:3:-1,43:3:33591,2:6:44738,4:6:46341,5:6:48794,35:6:52255,
38:6:52740,40:06:52938,42:06:53105,0:7:53564,2:7:-1,4:10:55390,1:11:-1,10:11:63669,27:11:-1,
2:12:67622,17:12:70590,18:12:71241,19:12:71659,21:12:71834,23:12:-1,25:12:72549,26:12:72948,
27:12:73733,29:12:75341,33:12:-1,1:13:76584,2:13:-1,05:13:77037,30:13:83673,33:13:84191,
37:13:84690,14:14:87836,17:14:-1,19:14:88523,30:14:92989,8:15:95815,12:15:96507,15:15:96729,
22:16:109491,25:18:131296,42:18:135673,05:22:-1,27:22:173820,0:24:-1,27:24:192142,4:27:-1,
25:27:215436
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: maschnitz on 05/10/2014 10:31 pm
If you zoom in on 6, it seems to me that 17:12 is sea foam, not the leg. The color and texture isn't right. (It'd be handy to have a magnifying glass on Slapbet for stuff like this.)

Then, I suspect 15:11 is actually 15:10. The bit offsets in rows 10 and 11 jump all over the place; there's a jump from 56129 to 63669 between 09:10 and 10:10, and by 00:11 we're at 68940.  The actual end of row 10 to 11 probably lives/lived early in that range; or earlier.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jakusb on 05/10/2014 10:51 pm
An improved version of the annotated iframe8 with many more common dirt spots I see in several other iframes.
They seem to be very useful for further aligning MB's in other iframes
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/10/2014 11:00 pm
If you zoom in on 6, it seems to me that 17:12 is sea foam, not the leg. The color and texture isn't right. (It'd be handy to have a magnifying glass on Slapbet for stuff like this.)
This was my first assumption too, but there is no good place I can put it other that the top of the leg. It's very bright and causes a propagating streak if placed in other places. The range it can be in is constrained and it's in a longish piece of good data so it can't be moved much without either impinging on fixed pieces or raising the question where is the tip of the leg then. The leg section on the other hand is really dark and wants a very light macroblock on top of it.

Then, I suspect 15:11 is actually 15:10.
Can't be. 4:10 to 42:10 is a solid good piece of data anchored by the smudge on 26:10
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: maschnitz on 05/10/2014 11:22 pm
Can't be. 4:10 to 42:10 is a solid good piece of data anchored by the smudge on 26:10

Good point.

Hmm. If you switch back and forth quickly between 'common_dirt2' (frame 8 ) and 6, 6's left leg is too long, not too short. And the right leg is a little short. So maybe 15:11 is more like 16:12? Then everything else shifts down and to the left one. The lower part of the leg isn't set in stone, I don't think. Or 15:12/13? Not sure, though. Wish I was more nimble with the MPEG format.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: seanpg71 on 05/10/2014 11:23 pm
So I tried to get something with iframe 4, but still a lot to do. I have to stop now for a few days so good luck to those who wish to improve it ;)

From try1.ts

5:1:4200,00:03:-1,
02:03:10968,10:03:-1,
17:06:22427,14:11:-1,
38:16:65876,19:17:-1,
00:19:78514,28:20:-1,
0:28:129509

At 17911 and 55810 there are also valid sequences, which I do not know exactly where to put, and I did not really try to improve the bottom of the frame.

Also some sequences from http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195103#msg1195103 could help if put at the accurate place.
So I tried to get something with iframe 4, but still a lot to do. I have to stop now for a few days so good luck to those who wish to improve it ;)

From try1.ts

5:1:4200,00:03:-1,
02:03:10968,10:03:-1,
17:06:22427,14:11:-1,
38:16:65876,19:17:-1,
00:19:78514,28:20:-1,
0:28:129509

At 17911 and 55810 there are also valid sequences, which I do not know exactly where to put, and I did not really try to improve the bottom of the frame.

Also some sequences from http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195103#msg1195103 could help if put at the accurate place.

I was able to add some more dubiously placed water to this.  But not getting much of anything in the parts that are actually interesting.  Not sure threre's a whole lot more I can do with 4.

5:1:-1,
11:1:5384,30:2:-1,
2:3:10968,10:3:-1,
13:3:14762,33:3:-1,
35:3:16382,
37:3:16550,34:4:-1,
6:5:17911,
30:5:19483,32:5:-1,
36:5:20286,8:6:-1,
9:6:21857,16:10:-1,
17:10:36140,13:11:-1,
17:11:39697,38:11:-1,
1:12:41855,16:12:-1,
29:12:43895,20:13:-1,
13:14:51698,38:14:-1,
14:15:51794,36:15:-1,
37:16:65876,18:17:-1,
14:18:74612,30:20:-1,
24:27:127453,6:28:-1

Note, this is from try1.ts.  It's fairly easy to port between coerced.ts and try1.ts frames though.  Just need to add/subtract an offset to account for the extra sections of data in try1.ts that aren't in coerced.ts.

Here's the diff and offset for frame 4.


Coerced      Try      Offset
0-4225        0-4225      0
              4225-5697   
4226-46912    5698-48384   1472
              48384-51328
46913-49856   51329-54272   4416
              54272-55744   
49857-66048   55745-71936   5888
              71936-73408   
66049-104323 73409-111683   7360
              111683-114627
104324-121984   114628-132288   10304
121985 - 132287
              132289-144485


Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jakusb on 05/10/2014 11:47 pm
So I tried to get something with iframe 4, but still a lot to do. I have to stop now for a few days so good luck to those who wish to improve it ;)

From try1.ts

5:1:4200,00:03:-1,
02:03:10968,10:03:-1,
17:06:22427,14:11:-1,
38:16:65876,19:17:-1,
00:19:78514,28:20:-1,
0:28:129509

At 17911 and 55810 there are also valid sequences, which I do not know exactly where to put, and I did not really try to improve the bottom of the frame.

Also some sequences from http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195103#msg1195103 could help if put at the accurate place.
So I tried to get something with iframe 4, but still a lot to do. I have to stop now for a few days so good luck to those who wish to improve it ;)

From try1.ts

5:1:4200,00:03:-1,
02:03:10968,10:03:-1,
17:06:22427,14:11:-1,
38:16:65876,19:17:-1,
00:19:78514,28:20:-1,
0:28:129509

At 17911 and 55810 there are also valid sequences, which I do not know exactly where to put, and I did not really try to improve the bottom of the frame.

Also some sequences from http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195103#msg1195103 could help if put at the accurate place.

I was able to add some more dubiously placed water to this.  But not getting much of anything in the parts that are actually interesting.  Not sure threre's a whole lot more I can do with 4.

5:1:-1,
11:1:5384,30:2:-1,
2:3:10968,10:3:-1,
13:3:14762,33:3:-1,
35:3:16382,
37:3:16550,34:4:-1,
6:5:17911,
30:5:19483,32:5:-1,
36:5:20286,8:6:-1,
9:6:21857,16:10:-1,
17:10:36140,13:11:-1,
17:11:39697,38:11:-1,
1:12:41855,16:12:-1,
29:12:43895,20:13:-1,
13:14:51698,38:14:-1,
14:15:51794,36:15:-1,
37:16:65876,18:17:-1,
14:18:74612,30:20:-1,
24:27:127453,6:28:-1

Note, this is from try1.ts.  It's fairly easy to port between coerced.ts and try1.ts frames though.  Just need to add/subtract an offset to account for the extra sections of data in try1.ts that aren't in coerced.ts.

Here's the diff and offset for frame 4.


Coerced      Try      Offset
0-4225        0-4225      0
              4225-5697   
4226-46912    5698-48384   1472
              48384-51328
46913-49856   51329-54272   4416
              54272-55744   
49857-66048   55745-71936   5888
              71936-73408   
66049-104323 73409-111683   7360
              111683-114627
104324-121984   114628-132288   10304
121985 - 132287
              132289-144485


I changed 11:1:5384 to 22:1:5384 to properly align the dark dirt in top center... I really should give them numbers ;)
And also 30:2:-1 to 3:0:-1
Now a lot of dirt blobs start to align... Now I really gonna number them.. :)

5:1:-1,22:1:5384,0:3:-1,2:3:10968,10:3:-1,13:3:14762, 33:3:-1,35:3:16382,37:3:16550,34:4:-1,6:5:17911, 30:5:19483,32:5:-1,36:5:20286,8:6:-1,9:6:21857, 16:10:-1,17:10:36140,13:11:-1,17:11:39697,38:11:-1, 1:12:41855,16:12:-1,29:12:43895,20:13:-1,13:14:51698, 38:14:-1,14:15:51794,36:15:-1,37:16:65876,18:17:-1,14:18:74612,30:20:-1,24:27:127453,6:28:-1
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jakusb on 05/10/2014 11:51 pm
A version with the dirt spots numbered for better reference in conversation

Edit: Added dirt group 14
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jakusb on 05/11/2014 12:02 am
iframe 4:

I see another dirt group (14) that needs to go 39 blocks left (or 1 row down and 5 to right)
However I could not get it to work yet
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/11/2014 12:12 am
<snip>
The following is how it should look like IMHO.
There are 3 blocks of XXs which are different for each frame. The first one seems to contain a counter. The rest i dont know. But it should be mentioned in that ISO*.pdf how they should look like/what they should contain.

47 43 E8 37 07 10 00 xx xx xx 7E 00 00 00 01 E0 00 00 81 80 07 21 01 xx xx xx FF FF 00 00 01 B0 03 00 00 01 B5 0D 0F 00 00 01 00 00 00 01 20 00 C4 8D C0 00 46 56 C0 1E C0 01 xx xx 9A FC 7C 2E EE 2C 08 78 28 C7 00 00 01 B2 65 6D 34 76 20 34 2E 33 2E 30 4E 31 40 C3 FF 00 00 01 B6 1x

<snip>

The first xx xx xx is a counter. From Iframe to Iframe it increases by 60060. (the 2 red ones are calculated with this information)

Frame   Hex    Dec
IF01   2B31E7   2830823
IF02    2c1c83    2890883
IF03    2d071f    2950943
IF04    2df1bb    3011003
IF05    2edc57    3071063
IF06    2fc6f3    3131123
IF07    30b18f    3191183
IF08    319C2B   3251243
IF09    3286c7    3311303
IF10    337163    3371363
IF11    345bff    3431423
IF12    35469b    3491483
IF13    363137    3551543
IF14    371bd3    3611603
IF15    38066f    3671663


The next block of XXs is part of the Presentation Time Stamp PTS
how to get the PTS is explained quite well here: http://dvd.sourceforge.net/dvdinfo/pes-hdr.html
PTS: 21 01 xx xx xx.
We can now decode the PTS of the good frames and define the PTS of the messed up ones.

It is late again, my bed is calling.
I will calculate the PTS tomorrow, also will work out that last xx xx.
Here are the raw 5 byte PTS of the frames:

IF00: 99 31 59 14 6C
IF01: 21 01 61 23 ED
IF02: 21 01 67 CE 5D
IF03: 21 01 6F 78 CD
IF04: 21 01 77 23 3D
IF05: 21 01 7D CD AD
IF06: 21 01 85 78 1D
IF07: 21 01 8D 22 8D
IF08: 21 01 93 CC FD
IF09:  21 01 9B 77 6D
IF10: 21 01 A3 21 DD
IF11: 21 01 A9 CC 4D
IF12: 21 01 B1 76 BD
IF13: 21 01 B9 21 2D
IF14: 21 01 BF CB 9D

Cheers
Shanuson
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/11/2014 12:20 am
Hmm. If you switch back and forth quickly between 'common_dirt2' (frame 8 ) and 6, 6's left leg is too long, not too short.

Yes while deploying the leg looks longer because of perspective and that a fully deployed leg is pointing downward.

And the right leg is a little short. So maybe 15:11 is more like 16:12? Then everything else shifts down and to the left one. The lower part of the leg isn't set in stone, I don't think. Or 15:12/13? Not sure, though. Wish I was more nimble with the MPEG format.

I can see no way that this thing can be in any other row than the one it's currently in. Leg macroblocks look in correct and unchangeable positions to me. The only place it could go is to the right. It may be possible to calculate an approximate position based on known bitpositions and average macroblock size. But try1 seems to contain large chunks of junk so that approach is questionable.

Here's what it would look like if it was shifted 1 block to the right. Pretty weird, but kind of fits. Plus this change would lighten the columns under it and darken the ones to the left. And the leg tips look pretty weird on iframe 5 as well. But again, I do not know if that such a misalignment of blocks in macroblocks is even theoretically possible.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: JBF on 05/11/2014 12:27 am
Let me just say you guys are amazing.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/11/2014 01:00 am
Here's what it would look like if it was shifted 1 block to the right. Pretty weird, but kind of fits. Plus this change would lighten the columns under it and darken the ones to the left. And the leg tips look pretty weird on iframe 5 as well. But again, I do not know if that such a misalignment of blocks in macroblocks is even theoretically possible.

I do like the fit of that image, as you can make out the curve of the tip of the leg. Perhaps the bright white is another camera window artifact, like the missing part of the left side of the left leg - maybe it just caught the white border in just the right way in the light, and you wound up with a magnified or brightened reflection. There's a similar sort of oddity at the tip of the leg in the current rendition of frame 5 as you observed - perhaps nailing that down more fully would help figure this one out.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: seanpg71 on 05/11/2014 03:18 am


I changed 11:1:5384 to 22:1:5384 to properly align the dark dirt in top center... I really should give them numbers ;)
And also 30:2:-1 to 3:0:-1
Now a lot of dirt blobs start to align... Now I really gonna number them.. :)

5:1:-1,22:1:5384,0:3:-1,2:3:10968,10:3:-1,13:3:14762, 33:3:-1,35:3:16382,37:3:16550,34:4:-1,6:5:17911, 30:5:19483,32:5:-1, 36:5:20286,8:6:-1,9:6:21857, 16:10:-1,17:10:36140,13:11:-1,17:11:39697,38:11:-1, 1:12:41855,16:12:-1,29:12:43895,20:13:-1,13:14:51698, 38:14:-1,14:15:51794,36:15:-1, 37:16:65876,18:17:-1,14:18:74612,30:20:-1,24:27:127453,6:28:-1



More uninteresting parts of the image!

4:1:-1,22:1:5384,5:2:-1,6:2:7531,0:3:-1,2:3:10968,10:3:-1, 13:3:14762,33:3:-1,35:3:16382,37:3:16550,34:4:-1,6:5:17911,30:5:19483,32:5:-1, 36:5:20286,8:6:-1,9:6:21857,16:10:-1,17:10:36140,13:11:-1, 17:11:39697,38:11:-1,2:12:42062,16:12:-1,27:12:-1,29:12:43895,20:13:-1,13:14:51698, 38:14:-1,14:15:51794,36:15:-1,37:16:65876,18:17:-1,14:18:74612, 30:20:-1,41:20:91288,32:21:-1,34:21:96927, 25:22:-1,41:22:104774,0:24:-1,17:26:122506,0:27:-1,24:27:127453,6:28:-1


Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/11/2014 04:02 am
Working on those headers I found that some errors are single bit flips, but there are also stripes of bytes which are simply garbage/randomly turned bits one after an other. In an image you cant repair this I think. Substitute MBs maybe.
Headers can also be fixed, but if part of an frame MB itself is messed up like this you cant really hope to fixed it by flipping random bits.

Yeah, been working on the raw and the corruption is much worse than I had assumed. Not only are there huge swaths of garbage, but also probable missing bits (like the 4-bit offset that was found for sure).

Now I see where the aeroquartet guy was coming from - it *should* be impossible to repair this video - but all the more reason why the work being done here is so amazing. At the current pace, we'll have silky smooth HD video before the next launch :)

I'm going to continue playing with michaelni's tsfix, but I don't see a way that this can be brute-force cleaned. Missing (and possibly added) bits expand the search space well beyond computational feasibility. We might be able to tidy up some of the intermediate frames some, but the masters will have to be done by hand. So keep up the awesome work bit flippers and block movers!

One note to the maintainer of slapbet - it would probably be very easy to integrate my ffmpeg log parsing code into the webapp to enable bad-block highlighting with a checkbox. See ~499 of http://pastebin.com/bKgzCRqD and following - just string and integer parsing in Java which is a trivial port to js. 

Edit: also, a mouseover showing the x:y would be nice :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/11/2014 04:06 am
Working on those headers I found that some errors are single bit flips, but there are also stripes of bytes which are simply garbage/randomly turned bits one after an other. In an image you cant repair this I think. Substitute MBs maybe.
Headers can also be fixed, but if part of an frame MB itself is messed up like this you cant really hope to fixed it by flipping random bits.

Yeah, been working on the raw and the corruption is much worse than I had assumed. Not only are there huge swaths of garbage, but also probable missing bits (like the 4-bit offset that was found for sure).

Now I see where the aeroquartet guy was coming from - it *should* be impossible to repair this video - but all the more reason why the work being done here is so amazing. At the current pace, we'll have silky smooth HD video before the next launch :)

I'm going to continue playing with michaelni's tsfix, but I don't see a way that this can be brute-force cleaned. Missing (and possibly added) bits expand the search space well beyond computational feasibility. We might be able to tidy up some of the intermediate frames some, but the masters will have to be done by hand. So keep up the awesome work bit flippers and block movers!

One note to the maintainer of slapbet - it would probably be very easy to integrate my ffmpeg log parsing code into the webapp to enable bad-block highlighting with a checkbox. See ~499 of http://pastebin.com/bKgzCRqD and following - just string and integer parsing in Java which is a trivial port to js. 

Edit: also, a mouseover showing the x:y would be nice :)

This is all incremental. Anything you get us thats better than we have right now is progress!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/11/2014 07:09 am
iframe 6 (try1)
Left leg is not fully deployed yet, but i'm perplexed by the tip of it. At least I think it's the tip because it doesn't look like a cloud or a foam spot on the sea. It kind of resembles the tip of the right leg on iframe 5, which shows the right leg in similar stage of deployment as left leg on this frame. The thing is it doesn't align. It's off by one block (half of macroblock). Is such an alignment error even possible?

If I understand your question correctly: yes.

It is possible the data from one block (not macroblock) is used by another block within the same macroblock. This would for example happen if somehow a bad bit was interpreted as a "stop-bit" of the data of a block. This way the next block (within the same macroblock) would use the data from its previous block. The other way around is also possible I believe.

This is all due to the variable length the VLC-codes can have. And they are the ones that have the "stop-bit". See here for more info on 8x8 block decoding (http://pastebin.com/pLZQReSP).
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/11/2014 10:44 am
It's off by one block (half of macroblock). Is such an alignment error even possible?

If I understand your question correctly: yes.

It is possible the data from one block (not macroblock) is used by another block within the same macroblock. This would for example happen if somehow a bad bit was interpreted as a "stop-bit" of the data of a block. This way the next block (within the same macroblock) would use the data from its previous block. The other way around is also possible I believe.

No, I knew that, what I meant to ask was whether blocks from one macroblock can end up inside another in a way that would result in macroblock contents being offset by half of macroblock. It probably isn't. Which would mean that the odd object is not the tip of the leg.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/11/2014 12:29 pm
No, I knew that, what I meant to ask was whether blocks from one macroblock can end up inside another in a way that would result in macroblock contents being offset by half of macroblock. It probably isn't. Which would mean that the odd object is not the tip of the leg.
Ah ok. I think that can happen too: when the first macroblock thinks it has seen the 4 blocks but those stop-bits weren't all real stop-bits the next macroblock will pick-up the blocks from the previous macroblock. But then you would also expect some color-trouble because those would also be interpreted wrong I guess. I think Michael Niedermayer knows this better though.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/11/2014 12:50 pm
@Michael Niedermayer:

I've been investigating the possibility of automatically setting all L1,L2,L3,L4,C1,C2 values for all macroblocks for a frame. Basicly inputting an mmb and letting a script detect "bad lines" in the picture. Moving from the right-bottom corner to the upper-top.

I'm starting to believe (using some extra logging and some manual experimentation) that this might actually be possible. If it does this would improve the quality of the frames quite dramatically and would save a lot of manual work!

However there is one thing that I find difficult to do myself. And I think you might be able to help us. Instead of using this kind of setting:

15:14:-1:0:-40:0:0:0:20

where I replace the current macroblock with a blank one and change the DC values.

But what I really want to do is this:

15:14:57217:0:-40:0:0:0:20

In other words: I want to keep the macroblock with its (structured) data as much as I can (at the very least the blocks in it which are set to 0 above) but change only the DC "starting" values. So not blanken it, but modifying it so these DC-values make sure propogation to other blocks works the same as now with -1 (therefore fixing the color and intensity of following blocks) but I also keep the current blocks inside the current macroblock.

Is something like that possible?

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/11/2014 02:35 pm

So it seems like it should be practical to write a tool that can walk through the raw.ts and use the rules of the packet header formatting and sequence numbers to reconstruct a more coherent transport stream. Even just getting all the errors out of the null packets by replacing everything with FF might make it easier to discern good patterns.

Edit: For example, in raw.ts there are bytes missing or added between 0x37ACC and 0x371EC - that's the first place where the columns shift in a 47-column hex display.

Working on this, in the meantime, here is a plot of the bit errors per null packet in raw.ts. This was created by finding probable null packet headers (p > 0.85) then counting bits deviating from 1 within the 188 bytes.


Edit: BTW with some quick math, assuming raw.ts is ~30s, those big spikes of 5-8 packets (30-45ms) are almost certainly lightning EMI. Must have been some storm...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/11/2014 02:49 pm
I found out why iframe 0 didn't extract - the transport stream header for the first packet of the iframe had its "transport error indicator" bit flipped, and so that packet containing the very first piece of the iframe was ignored, resulting in the entire iframe being ignored.

If you take try1.ts and change offset 0x3E3D1 from 0x43 to 0x03, clearing the TEI bit, then iframe 0 should pop out. It may have some issues since the sequence numbers are rather jumbled right there - it looks like 0x3E48F should be 0x5C rather than 0x5F, and that might contain the adaptation field info for the iframe. The sequence doesn't seem to get back on track until you get to 0x3E83B.


Edit: Coffee deficit - the 0x80 bit is the TEI, not the 0x40 bit.  ::)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/11/2014 03:10 pm
I found out why iframe 0 didn't extract - the transport stream header for the first packet of the iframe had its "transport error indicator" bit flipped, and so that packet containing the very first piece of the iframe was ignored, resulting in the entire iframe being ignored.

If you take try1.ts and change offset 0x3E3D1 from 0x43 to 0x03, clearing the TEI bit, then iframe 0 should pop out. It may have some issues since the sequence numbers are rather jumbled right there - it looks like 0x3E48F should be 0x5C rather than 0x5F, and that might contain the adaptation field info for the iframe. The sequence doesn't seem to get back on track until you get to 0x3E83B.

No its more than that. The headers are wrong so it doesn't decode and by "wrong" i mean complete garbage beyond a certain point. I don't think the TEI bit is related.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/11/2014 03:26 pm
I found out why iframe 0 didn't extract - the transport stream header for the first packet of the iframe had its "transport error indicator" bit flipped, and so that packet containing the very first piece of the iframe was ignored, resulting in the entire iframe being ignored.

If you take try1.ts and change offset 0x3E3D1 from 0x43 to 0x03, clearing the TEI bit, then iframe 0 should pop out. It may have some issues since the sequence numbers are rather jumbled right there - it looks like 0x3E48F should be 0x5C rather than 0x5F, and that might contain the adaptation field info for the iframe. The sequence doesn't seem to get back on track until you get to 0x3E83B.

No its more than that. The headers are wrong so it doesn't decode and by "wrong" i mean complete garbage beyond a certain point. I don't think the TEI bit is related.

Not complete garbage, but very close - Based on the null-packet error estimation there is something like a 15% *average* error rate during iframe 0. This is substantially worse than the rest of the data, which hangs between 0.1% and 1.5%.

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/11/2014 03:32 pm
I found out why iframe 0 didn't extract - the transport stream header for the first packet of the iframe had its "transport error indicator" bit flipped, and so that packet containing the very first piece of the iframe was ignored, resulting in the entire iframe being ignored.

If you take try1.ts and change offset 0x3E3D1 from 0x43 to 0x03, clearing the TEI bit, then iframe 0 should pop out. It may have some issues since the sequence numbers are rather jumbled right there - it looks like 0x3E48F should be 0x5C rather than 0x5F, and that might contain the adaptation field info for the iframe. The sequence doesn't seem to get back on track until you get to 0x3E83B.

No its more than that. The headers are wrong so it doesn't decode and by "wrong" i mean complete garbage beyond a certain point. I don't think the TEI bit is related.

Not complete garbage, but very close - Based on the null-packet error estimation there is something like a 15% *average* error rate during iframe 0. This is substantially worse than the rest of the data, which hangs between 0.1% and 1.5%.

It was very oddly placed, at the EXACT location that the user data header was to start, it was replaced with a different "error" header with random noise in it. All the header data before that location was perfect without a single bit flip. I could not find anything that looked remotely like the standard data (thats the same for every iframe) in it. It almost looked like it was intentional and the encoder decided to not use this iframe.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/11/2014 03:33 pm
Thanks for the clarifications!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/11/2014 03:51 pm
I found out why iframe 0 didn't extract - the transport stream header for the first packet of the iframe had its "transport error indicator" bit flipped, and so that packet containing the very first piece of the iframe was ignored, resulting in the entire iframe being ignored.

If you take try1.ts and change offset 0x3E3D1 from 0x43 to 0x03, clearing the TEI bit, then iframe 0 should pop out. It may have some issues since the sequence numbers are rather jumbled right there - it looks like 0x3E48F should be 0x5C rather than 0x5F, and that might contain the adaptation field info for the iframe. The sequence doesn't seem to get back on track until you get to 0x3E83B.

No its more than that. The headers are wrong so it doesn't decode and by "wrong" i mean complete garbage beyond a certain point. I don't think the TEI bit is related.

Not complete garbage, but very close - Based on the null-packet error estimation there is something like a 15% *average* error rate during iframe 0. This is substantially worse than the rest of the data, which hangs between 0.1% and 1.5%.

It was very oddly placed, at the EXACT location that the user data header was to start, it was replaced with a different "error" header with random noise in it. All the header data before that location was perfect without a single bit flip. I could not find anything that looked remotely like the standard data (thats the same for every iframe) in it. It almost looked like it was intentional and the encoder decided to not use this iframe.

Maybe internal buffering after the encoder was brought online? I could see this with a primitive ASIC (or FPGA (or  RTOS)) encoder with a "output valid" signal that was ignored, so that with constant bitrate the resulting junk made it to the transmitter.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/11/2014 04:12 pm
Playing more with the raw.ts error data in MATLAB, I estimate 85% of the data has error rate < 1%, and ~3.0% of the data has an error rate above 12%

Edit: BTW with some quick math, assuming raw.ts is ~30s, those big spikes of 5-8 packets (30-45ms) are almost certainly lightning EMI. Must have been some storm...

Thinking out loud: If this is the case, we can reliably match "strike regions" where P(Err) is greatest in raw.ts before frame extraction, then maybe we can have ffmpeg lock C/L values within those regions. Since we can be relatively confident, i.e. P(Err) < 1%, in the packet integrity on either side of the strike, it should be easier to resync once we are "out of the weeds".

Thoughts?


Edit: to clarify my quote above, the strikes are 5-8 null-packet-counts in duration. There are 4-5 data packets between null packets. So that's locking 25-40 total packets per strike, or about 3-5k of real data.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: gospacex on 05/11/2014 04:46 pm
Parsing raw.ts using info from http://en.wikipedia.org/wiki/MPEG_transport_stream

The file consists of 188-byte ts-packets.

1st byte is always 47.

2 next bytes are 000<packetid13bits>, and apparently
they are always 03e8 for non-null ts-packets and 1fff for null ts-packets.
High-order bits mean "Transport Error Indicator" (0),
"Payload Unit Start Indicator" (0), "Transport Priority" (0).

4th byte is always 1x in both types of packets, where x cycles from 0 to f.
High-order bits 0001 mean "no scrambling" (00), "no adaptation field" (0), "payload exists" (1).

See below the listing of first few bytes of first few dozens of ts-packets, generated with
   hexdump -e '"%07.7_ax " 188/1 "%02x" "\n"' <raw.ts
command:

0000000 4703e95f81e103e2...
00000bc 4703e8103de99073...
0000178 4703e81176eeba37...
0000234 479779692f580085
00002f0 4703e413669bd0b3
00003ac 4713e8745300b09b
0000468 4703e81541a70f05
0000524 4703e8164aa9f008
00005e0 4703e817b70130e8
000069c 4703e818795c22a1
0000758 4703e8199ac05d5c
0000814 4703439a7a535507
00008d0 4703e81b06633dc1
000098c 4703e81c264b8290
0000a48 4703e81dcc29a6ab
0000b04 4703e81e107e0382
0000bc0 4703e81ff07cd801
0000c7c 4763e810a60791d7
0000d38 4703e8114386cd32
0000df4 4703e812041d1198
0000eb0 4703e813ffb86c32
0000f6c 4703e8144008f650
0001028 4703e815909cfbd4
00010e4 4703e816589e8010
00011a0 470389071ee887cb
000125c 471ac8d8ea28a720
0001318 47be1c520419650d
00013d4 474842d0f2fde345
0001490 477883798db92230
000154c 477c538a24031797
0001608 47d08588d2899b38
00016c4 477d03c3c0c0da0c
0001780 4703d41fb89610bc
000183c 4703e8301a005fff
00018f8 4740001c0000b011
00019b4 471fff12ffffffff
0001a70 471fff13ffffffff
0001b2c 4797ef1a9fdbffff
0001be8 475fff15dbb77e4e
0001ca4 474ebfbf9fffffff
0001d60 471fff17f7ffcfff
0001e1c 471ffb1827357f43
0001ed8 471fff19ffffffff
0001f94 471fd72aff5fffef
0002050 471fff1bff3ffd7f
000210c 471b1f02bfe7ffff
00021c8 478d5fe8e7476eb9
0002284 47fc19a23faa43df
0002340 470f77d412ac2e05
00023fc 47fff10fdfff3fff
00024b8 47bfe3bffffff5fb
0002574 471fff12ffffffff
0002630 471fff13ffffffff
00026ec 4f1fcf14ffffffff
00027a8 47d64ba1bfffffff

cat rawhex | sed 's/4703e81/.START=/g' | sed 's/471fff1/.NULL==/g'

and perusing resulting text file,
it's obvious that last good ts-packet header is at 0037a10:

0037a10 4703e81e0cf8280187ea...

The next +188 byte offset is:

0037acc 4703681dc10b930c9eb5b9fc....

it looks like moderately garbled ts-packet, but after that
there are no longer any recognizable ts-packet headers at +188 offsets.

The next recognizable ts-header is at 00381ec (not a +188 offset),
and from this location 188-byte packets continue from there.

The next discontinuity is at 00d6e20:

00d6bec 4b03e81c929ef0c...
00d6ca8 45abe11d923e21a...
00d6d64 4703e81e66c3272...
00d6e20 4703e81fa2080de... <== last good hdr
00d6edc 14e09aeb3fd7402...
00d6f98 a3582165fad84dd...
00d7054 11f93b7d90446f5...
00d7110 4e6f8b9b0de6ec6...

and the re-sync happens a bit later here:

00d7250 4703e815f4f261
00d730c 4703e816209781
00d73c8 4703e817784253
00d7484 4743e998e38153
00d7540 4703e8190cbc42

Oh... that's strange... I always need to seek by exactly 132 bytes to get
ts-packet headers to realign... can't be a coincidence!

Overall, there seems to be six sections of correctly-sized ts-packets:

1st_ofs - offset_of_last_recognizable_ts_packet_header
0000000 - 0037acc
00381ec - 00d6e20
00d7250 - 0215c58
0215cdc - 0356ae0
0357378 - 03dbe8c
03dd16c - 04460f0
04461aa (eof, last packet is two bytes short)

Between each section is a block of few hundred bytes.
Its size is always N*188 + 132. ???

I can "repair" the file by padding each section so that its size is a multiple of 188,
and then repair ts-packet headers: it's easy te determine whether the packet is
a null packet or payload packet, and then fix headers to be either
4703e81x or 471fff1x.

Will this be useful in any way?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: aero on 05/11/2014 05:01 pm
Quote
Oh... that's strange... I always need to seek by exactly 132 bytes to get
ts-packet headers to realign... can't be a coincidence!

hmm
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/11/2014 05:12 pm
Overall, there seems to be six sections of correctly-sized ts-packets:

1st_ofs - offset_of_last_recognizable_ts_packet_header
0000000 - 0037acc
00381ec - 00d6e20
00d7250 - 0215c58
0215cdc - 0356ae0
0357378 - 03dbe8c
03dd16c - 04460f0
04461aa (eof, last packet is two bytes short)

Between each section is a block of few hundred bytes.
Its size is always N*188 + 132. ???

I can "repair" the file by padding each section so that its size is a multiple of 188,
and then repair ts-packet headers: it's easy te determine whether the packet is
a null packet or payload packet, and then fix headers to be either
4703e81x or 471fff1x.

Will this be useful in any way?
Yes. There were 5 times 56 bytes missing (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1191647#msg1191647) in the raw.ts. Very weird indeed.

This was fixed (among other things) in the coerced.ts. I am not sure if it was fixed in the try1.ts.

But actually it's good to have a raw_aligned.ts where only this is fixed by padding 5 times 56 bytes with FF.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: gospacex on 05/11/2014 06:13 pm
The script to cut up raw.ts into sections with 188-byte ts-packets.
The last two commands prove that concatenation of sections is the same as raw.ts (both have 042433dfc39c70a51203fae581a5461f md5sum).

#!/bin/sh
mk_sect()
{
        local start=$1
        local end=$2
        dd bs=1 skip=$(($start)) count=$(($end - $start)) <raw.ts >$3.ts
        cat section?.ts >z.$$
        hexdump -s $(($start)) -v -e '"%07.7_ax " 188/1 "%02x" "\n"' <z.$$ >$3.hex
        rm z.$$
}
# 1st_ofs - offset_of_last_recognizable_ts_packet_header
# 0000000 - 0037acc
# 00381ec - 00d6e20
# 00d7250 - 0215c58
# 0215cdc - 0356ae0
# 0357378 - 03dbe8c
# 03dd16c - 04460f0
# 04461aa (eof, last packet is two bytes short)
#
# It is not obvious where exactly realignment happens.
# What can be seen is where the last packet with "old" alignment is,
# and where is the first recognizable packet with "new" alignment is.
#
rm section?.ts
mk_sect 0x0000000 0x00381ec section1
mk_sect 0x00381ec 0x00d7250 section2
mk_sect 0x00d7250 0x0215cdc section3
mk_sect 0x0215cdc 0x0357378 section4
mk_sect 0x0357378 0x03dd16c section5
mk_sect 0x03dd16c 0xfffffff section6
cat section?.ts | md5sum
cat raw.ts | md5sum
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/11/2014 06:21 pm
The script to cut up raw.ts into sections with 188-byte ts-packets.
The last two commands prove that concatenation of sections is the same as raw.ts (both have 042433dfc39c70a51203fae581a5461f md5sum).

#!/bin/sh
mk_sect()
{
        local start=$1
        local end=$2
        dd bs=1 skip=$(($start)) count=$(($end - $start)) <raw.ts >$3.ts
        cat section?.ts >z.$$
        hexdump -s $(($start)) -v -e '"%07.7_ax " 188/1 "%02x" "\n"' <z.$$ >$3.hex
        rm z.$$
}
# 1st_ofs - offset_of_last_recognizable_ts_packet_header
# 0000000 - 0037acc
# 00381ec - 00d6e20
# 00d7250 - 0215c58
# 0215cdc - 0356ae0
# 0357378 - 03dbe8c
# 03dd16c - 04460f0
# 04461aa (eof, last packet is two bytes short)
#
# It is not obvious where exactly realignment happens.
# What can be seen is where the last packet with "old" alignment is,
# and where is the first recognizable packet with "new" alignment is.
#
rm section?.ts
mk_sect 0x0000000 0x00381ec section1
mk_sect 0x00381ec 0x00d7250 section2
mk_sect 0x00d7250 0x0215cdc section3
mk_sect 0x0215cdc 0x0357378 section4
mk_sect 0x0357378 0x03dd16c section5
mk_sect 0x03dd16c 0xfffffff section6
cat section?.ts | md5sum
cat raw.ts | md5sum

Just a note. Dump any code-like text into
[code]
[/code]
blocks
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: gospacex on 05/11/2014 06:35 pm
And after this, it's instructive to check what kind of packets do we have there?

grep . section?.hex | sort -t' ' -k2

There are lots of data and null packets, of course, but also there are:

474000xx packets:

section3.hex:00e7da0 474000100000b0110000c100000000e0100001e020d36af0acfffffffff...
section3.hex:01c2c98 474000100000b0110000c100000000e0100001e020d36af0acfffffffff...
section4.hex:028cae4 474000100000b0110000c100000000e0100001e020d36af0acfffffffff...
section4.hex:0353f8c 474000100000b0110000c100000000e0100001e020d36af0acfffffffff...
section6.hex:04065a0 474000100000b0110000c100000000e0100001e020d36af0acfffffffff...

474020xx packets:

section3.hex:00eab28 474020100002b01f0001c10000e3e8f00010e3e8f0031b01f580e3e9f00081e3f3f0003f64f115ffff...
section3.hex:01c6468 474020100002b01f0001c10000e3e8f00010e3e8f0031b01f580e3e9f00081e3f3f0003f64f115ffff...
section4.hex:029d868 474020100002b01f0001c10000e3e8f00010e3e8f0031b01f580e3e9f00081e3f3f0003f64f115ffff...
section6.hex:0408768 474020100002b01f0001c10000e3e8f00010e3e8f0031b01f580e3e9f00081e3f3f0003f64f115ffff...

0x40 bit is Payload Unit Start Indicator. I have no idea what's that,
but it has quite easily recognizable bit pattern in the payload.
Meaning: it's easy to detect such packets and fix their headers even
if it's severely corrupted.

And the 474000 + 4703e8 combination also exists:

section2.hex:004817c 4743e8300710002b495d7e00400181e00000818007410019d755ffff000001b6597...
section2.hex:00ae4f0 4743e8300710002c9d8c7e00000001e000008180072101652811ffff000001b658c...
section3.hex:0117d0c 4743e8300710002df1bb7e00000001c000c081800721016f78cdffff000001b0030...
section3.hex:011bdac 4743e8300710002dfd767e00000001e0000081800721016fa7b9ffff000001b658c...

The 30 byte is "Adaptation field exist" bit 0x20 set in addition to the usual "Payload present" 0x10 bit.

Looking for less obvious "unusual" packets, "Adaptation field exist" bit seems to be present
in some payload packets too. They are visible because they have a bunch of FF bytes
in the beginning:

section3.hex:017cbf0 4703e8300b00ffffffffffffffffffff843695a94aad38fbed264a01a2503272de7134078280741e03f93613ff2abc4e935...
section1.hex:000183c 4703e8301a005ffffffffffd7fffffffffffc7ff6ffffffffffffffffffffffffffffffff5ffc3ffffffffffffffff18211...

Note 4703e83x header instead of more typical 4703e81x.
It's not a one-bit corruption because there are a number of these,
they all have ...ffffffffff... at the start of their payload.
"Usual" data packets don't.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/11/2014 06:41 pm
I can "repair" the file by padding each section so that its size is a multiple of 188, and then repair ts-packet headers: it's easy te determine whether the packet is a null packet or payload packet, and then fix headers to be either 4703e81x or 471fff1x.

Will this be useful in any way?

I think you don't want to force all headers to either of those two values - I think we would just want to clear the TEI and the scrambling bits only, and make a 0x471FFFX into a 0x471FFF1. There appear to be instances where the adaptation field and payload unit start bits are significant.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: gospacex on 05/11/2014 06:51 pm
Trivial script to concatenate sections with padding, so that every ts-packet is 188-byte aligned:
{
cat section1.ts; dd bs=56 count=1 </dev/zero
cat section2.ts; dd bs=56 count=1 </dev/zero
cat section3.ts; dd bs=56 count=1 </dev/zero
cat section4.ts; dd bs=56 count=1 </dev/zero
cat section5.ts; dd bs=56 count=1 </dev/zero
cat section6.ts
} >aligned188.ts
hexdump -v -e '"%07.7_ax " 188/1 "%02x" "\n"' <aligned188.ts >aligned188.hex
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/11/2014 08:08 pm
An improved version of the annotated iframe8 with many more common dirt spots I see in several other iframes.
They seem to be very useful for further aligning MB's in other iframes
I truly believe that in the long run this is gonna help us enormously.

Thank you Jakusb. Incredibly good!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: gospacex on 05/11/2014 09:38 pm
First cut at ts-packet fixing tool.
How to use:

(0) compile: gcc -Os -Wall fix_tspackets.c -o fix_tspackets
(1) run cut_188byte_sections.sh.txt (cuts raw.ts into sections)
(2) fix a section:

$ ./fix_tspackets -o 0x03dd16c section5.ts section5-1.ts
Loaded 2917 packets
03ea430: 4740001x packet (h:47400011 diff:0.000000)
03f0b00: 4740001x packet (h:47400012 diff:0.000000)
03fd08c: 4740001x packet (h:47800013 diff:0.000000)
03fd08c: fixing header from 47800013 to 47400013
03fd08c: fixing payload
0408d48: 4740001x packet (h:47400014 diff:0.000000)
0410094: 4740001x packet (h:47400015 diff:0.000000)
0424a50: 4740001x packet (h:47400017 diff:0.000000)
042947c: 4740001x packet (h:46000018 diff:0.083333)
042947c: fixing header from 46000018 to 47400018
042947c: fixing payload
0437188: 4740001x packet (h:47400019 diff:0.000000)
0445764: 4740001x packet (h:4740001a diff:0.000000)
044e230: 4740001x packet (h:4740001b diff:0.000000)
045e9d4: 4740001x packet (h:4740001c diff:0.000000)
045e9d4: fixing payload
Saved to 'section5-1.ts'

(-o OFFSET is cosmetic: it adjusts packet offsets so that they can match raw.ts)

(gosh... I need to add .txt to attachments for them to be posted. strip it...)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/12/2014 12:20 am
<snip>
The next block of XXs is part of the Presentation Time Stamp PTS
how to get the PTS is explained quite well here: http://dvd.sourceforge.net/dvdinfo/pes-hdr.html
PTS: 21 01 xx xx xx.
We can now decode the PTS of the good frames and define the PTS of the messed up ones.

It is late again, my bed is calling.
I will calculate the PTS tomorrow, also will work out that last xx xx.
Here are the raw 5 byte PTS of the frames:

IF00: 99 31 59 14 6C
IF01: 21 01 61 23 ED
IF02: 21 01 67 CE 5D
IF03: 21 01 6F 78 CD
IF04: 21 01 77 23 3D
IF05: 21 01 7D CD AD
IF06: 21 01 85 78 1D
IF07: 21 01 8D 22 8D
IF08: 21 01 93 CC FD
IF09:  21 01 9B 77 6D
IF10: 21 01 A3 21 DD
IF11: 21 01 A9 CC 4D
IF12: 21 01 B1 76 BD
IF13: 21 01 B9 21 2D
IF14: 21 01 BF CB 9D

Cheers
Shanuson

Here is the next part:
PTS values are separated by 120120:
All except the first one are OK:
The first should be:
0x210159797D

Also I missed one byte that changes from IF to IF, namely the 4th in each sequence: 47 43 E8 3x ......
This X is a continuity counter. But it is different from the counter of the 47 1F FF  1Y packets.
In the better part of the file one find that X count continue for packets with 47 03 E8 1X,  47 03 E8 3X and 47 43 E8 3X. so it seems it is a counter for each packet with the same PID (13 bit long), namely 0x13E8 or 0x1FFF
So to find the right value of X for each corrupt frame one has to check the surrounding packets with the same PID, which i did:
Here are the 15 correct continuity counter X values for all IF's: (red ones contained errors in raw.ts
B, D, 7, 0, D, D, 6, 6, F, 4, D, 4, 8, 7, 5

Finally back to that 4bit shifted part:
Thanks to gospaceX work I know now what to look for:

Here the part of the file (4x188 bytes copied out of raw.ts, starting from 0x23bb74):
Black part is the start of  an I-Frame header (What you find in raw-headers.txt)
First red block: Aligned right; some errors at the end where all should be ff.
Last red block: Also aligned right and all following blocks too.
Blue Block: Errors at the start, so no 47 start byte; should look like the one before, all FFs
Green Block, Start of the I-Frame, but shifted by 4 bits to the LEFT as can be seen by comparing that one to a correct one (first 4 bits are the D at the end of the blue block)
So, the Blue block could also be shifted but it does not matter since it should contain only FFs.
Finally to fix this, I think, one only has to move that black-green block by 4 bits to the right (including that last D of the blue block and remove the last green B).


47 1F FF 18 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 7F FC FF FF FF FD 7F F4 FF 7C F6 FF FF FF FF FF FF FF DE 7F 7D 76 FC F4 EB 6D 6F F6 FF 1F FD BF FF FF FF FF FF FF FF FB 03 EE 76 17 E0 1B DA 08 FF EB DF 14

C0 D8 61 01 F7 5F C8 3F FE 7B FF F8 7D 53 57 70 34 EB DC F3 3F 5C 82 FF F3 FE 3E BB 78 7F FF FF FF FF FF BB FE 67 FF FF FF CF FF 5F FF FE FF F9 FF FF E7 FF AE FF F9 FF FF FF FF FF FF FF FF FF DF F3 3F 9F F5 FF FF F3 DF D7 67 FA 37 EF AF CF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FA 7F E7 FF FF FF FD 38 E9 69 AF E7 EC F0 8F FF 2B 30 E5 53 5F FD BF FF FF B7 78 34 FA EE FF FF FF FD FF F3 FF FF FF FF 5F FC 3F EF FD 9F 63 FC 9C 0F F7 40 FC BA 86 2D 65 44 70 BF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FC 12 BA 06 FA A9 FF 4D

BE 66 AF 8C EC B9 A7 CD C3 5C 74 0C 28 21 3B 00 18 18 B5 00 72 10 18 D2 28 DF FF F0 00 00 1B 00 30 00 00 1B 50 D0 F0 00 00 10 00 00 00 12 00 0C 48 5C 00 04 65 48 01 34 00 9E FD E9 AF E7 C2 2E E2 C0 87 86 8C 2B C1 04 98 06 C8 D3 47 62 03 42 E3 32 E3 02 E3 00 0C 3F F0 00 00 1B 61 0C F0 C0 D6 08 65 8D 9F BD C7 A4 B8 18 F1 19 57 32 42 08 44 41 E0 08 B5 27 E4 46 BA 7A ED 28 0A AA 89 BD 3F 60 BA F5 9F 96 07 05 53 CA E7 AE 0B 8A 3D 2E FA 25 D2 79 3E 7F 88 72 DE 8E B1 EC FF C1 3C 88 13 31 E7 79 26 19 C7 E9 AF 33 38 CE 46 3A 56 4F 34 34 A8 9E 73 54 76 C4 D1 76 82 EB

47 03 E8 17 3D DB B8 2A AE 02 77 05 E6 E1 48 38 8F CB 54 AB 3F CD D5 EA 29 75 70 60 A4 B3 5C A8 41 C4 26 13 A4 F3 7F 6B FB 9D AD 22 45 A1 9F B4 5D 5D 5E 39 58 FC C9 F0 77 47 99 FD CF 6E 6B 9E 4C 6C 4D 48 C9 FB 7C FC 9F EA 0F F0 15 21 27 13 F3 FA B8 79 2F D7 4F 73 D2 58 28 A7 B4 F2 90 5D 4B B8 39 00 46 E1 40 4D BA C8 9C 33 25 A0 BA F6 B9 45 E9 E5 05 1D D7 71 D5 E6 73 B9 BD B9 3D 15 73 BD FD E1 A6 75 70 FC 82 47 D2 DE ED 7F 3D E8 F4 A0 0A 94 86 F9 17 D1 17 23 EA 4D BF D5 84 AF A7 EB 6B 1E 1B DB D2 BE 14 D0 60 90 B7 94 B8 AA 86 B0 20 47 18 0E 0A 23 E5 C9 8A 0B


Tomorrow, I will work out that last XX XX of the headers. I will then change the raw.ts file to include the right headers and that shift and upload it.

Cheers
Shanuson
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: gospacex on 05/12/2014 12:21 am
How to use:

(-1) download these files into an empty dir, drop raw.ts there too
(0) run cut_188byte_sections.sh (cuts raw.ts into sections)
(1) compile: gcc -Os -Wall fix_tspackets.c -o fix_tspackets
(2) run fix_tspackets_in_sections.sh
    - fixes each section
    - copies fixed sections into fixed_sections/ subdir
    - glues them into "new" fixed_sections/raw.ts
    - runs cut_188byte_sections.sh on it
    - produces raw.hex.diff for you to peruse

With this source, resulting "new" raw.ts plays no worse than original
(maybe even better, it's hard to tell).

There is a catch, though. The most interesting part of this exercise
is to decode more data from packets with damaged headers.
The part of code which fixes packets is here:

        fix_4740001x_packets(pkt);
        fix_4740201x_packets(pkt);
        fix_4743e83x_packets(pkt);
        fix_null_packets(pkt);

and the part which tries to decide which packets are data, follows:

        // Does a bit too much damage by "fixing":
        //fix_data_packets(pkt, 0.3);
        // Fares better:
        //fix_data_packets(pkt, 0.25);

As you see, it is commented out. There are some heuristics in fix_data_packets() which avoid a few cases where damage is done, but apparently not enough.

I encourage interested people to uncomment last line, run the program,
look at raw.hex.diff, and suggest improvements to fix_data_packets() logic.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: michaelni on 05/12/2014 12:31 am
Ah ok. I think that can happen too: when the first macroblock thinks it has seen the 4 blocks but those stop-bits weren't all real stop-bits the next macroblock will pick-up the blocks from the previous macroblock. But then you would also expect some color-trouble because those would also be interpreted wrong I guess. I think Michael Niedermayer knows this better though.

yes, with enough (bad) luck a block could end up in the wrong position, but the next MBs MCBPC & cbpy would then not be where they are expected very likely
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: michaelni on 05/12/2014 12:41 am
Parsing raw.ts using info from http://en.wikipedia.org/wiki/MPEG_transport_stream

The file consists of 188-byte ts-packets.

1st byte is always 47.

2 next bytes are 000<packetid13bits>, and apparently
they are always 03e8 for non-null ts-packets and 1fff for null ts-packets.

there are more than 2 PID values in the ts packets/ file
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: michaelni on 05/12/2014 12:43 am

Yes. There were 5 times 56 bytes missing (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1191647#msg1191647) in the raw.ts. Very weird indeed.

This was fixed (among other things) in the coerced.ts. I am not sure if it was fixed in the try1.ts.

IIRC edited, coerced and try1 all had the 188byte packets aligned (except the 4bit thing)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: gospacex on 05/12/2014 12:53 am

Yes. There were 5 times 56 bytes missing (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1191647#msg1191647) in the raw.ts. Very weird indeed.

This was fixed (among other things) in the coerced.ts. I am not sure if it was fixed in the try1.ts.

IIRC edited, coerced and try1 all had the 188byte packets aligned (except the 4bit thing)

Aligning is easy, and possibly not that useful.
For one, it changes file offsets of ts-packets, making it harder for us to refer to the packets in speech.
E.g. "packet at offset XYZ needs its header fixed this way" - offset XYZ in *which .ts file*?

The hard part is repairing damaged headers, and subsequently, data.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/12/2014 02:38 am
Operating on the ts packet would be easier if you set up the struct to map it all out:

typedef struct {
    unsigned char fourseven;
    unsigned tei:1;
    unsigned pus:1;
    unsigned pri:1;
    unsigned pid:13;
    unsigned scramble:2
    unsigned hasadapt:1;
    unsigned haspayload:1;
    unsigned count:4;
    unsigned char payload[184];
} tspkt;

See http://www.cs.cf.ac.uk/Dave/C/node13.html for more info.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/12/2014 04:12 am

Yes. There were 5 times 56 bytes missing (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1191647#msg1191647) in the raw.ts. Very weird indeed.

This was fixed (among other things) in the coerced.ts. I am not sure if it was fixed in the try1.ts.

IIRC edited, coerced and try1 all had the 188byte packets aligned (except the 4bit thing)

Aligning is easy, and possibly not that useful.
For one, it changes file offsets of ts-packets, making it harder for us to refer to the packets in speech.
E.g. "packet at offset XYZ needs its header fixed this way" - offset XYZ in *which .ts file*?

The hard part is repairing damaged headers, and subsequently, data.

Aligning is easy, but also critical. There is no way to decode misaligned data. Luckily for us, michaelni has already written code to do this automatically (tsalign.c).

Referring to packet header locations is not that useful, as they are easily automatically repaired by the other program michaelni has written (tsfix.c).

It's as simple as

cat raw.ts | tsalign | tsfix > fixed.ts


The only hard part is repairing the data.

P.S. I saw you read all about mpeg on wikipedia, but were I you I'd listen to what he says:

Wooow! I didn't realize this at first. But we have Michael Niedermayer (http://www.ffmpeg.org/consulting.html) in the house!  8)  One of the creators of ffmpeg itself.

Just saying  ;)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Axiom on 05/12/2014 06:10 am
Hi! After following this forum for several years as a silent consumer, I felt compelled today to try and add something to this superb effort (first post, so go easy :)

About the Iframes, i was thinking that manual moving the macroblocks around will be difficult if you don't have a lot of reference (i.e. if there aren't many features on the block). So what about getting some help from statistics. So, would it be helpful to statistically compare all valid macroblocks for the closest matching edges, then automatically reshuffle the macroblocks. Some results may be false, yet if you create several iterations of the image for visual confirmation, those could rapidly be checked. You could even pin correctly placed macroblocks in place, excluding them from the automatic reshuffle. Would that be of help? 

Furthermore, and this may not be feasable but wanted to post the idea anyway, would it be possible to use the last frame before the next iframe, as a reference point. So if you have a very good Iframe upstream, say iframe8, let the video run until just one frame before iframe9, and use that as a reference for reconstructing Iframe9. Just my 50c.

Cheers!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Adaptation on 05/12/2014 06:11 am
Alpha 3 of my transport stream search and repair tool is finished.

Seems like michaelni has been doing a good job at making me redundant.   ;)
Really some top notch work here, I'm impressed. 

Hopefully my program can still help with manual searches and bit manipulation needs... for people with windows. 
 

Made the search capable of finding results that are not byte aligned.  (This feature includes a wildcard filter and can return results with a specified tolerated number of bit errors.) 
Enabled bit level insertion and deletion. (will bit shift everything after that point in the file)
Bit deletion at arbitrary point in the byte, bit insertion is after the byte selected, (first bit of next byte) to insert in the middle of a byte you can bit shift the whole file so your desired insertion point is between two bytes or select the byte ahead, insert and twiddle the bits back to how you want them. 

Switched most of the operations out of managed managed memory and most of the math is now 64 bit.

Fixed quite a few bugs but several remain, it will still crash if you operate most of the controls before opening a ts file so make sure to open one first.

Still plan on adding a bit level find/replace feature, adding an autosave/undo and colorizing the background between packet boundaries and search results in the hex editor. 

<edit> removed transport stream program </edit>
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/12/2014 06:43 am
Hi guys,

I have a bit of a mystery going on and it's holding me back from doing great stuff. ;) Hopefully somebody else has an idea what is going on.

This is essentially about how luminance/chrominance (errors) propagate. From what I understood so far each (macro)block propagates to its right (macro)block and to its bottom (macro)block. Luminance does it on a block (8x8pixel) basis while chrominance does it on a macroblock basis. So if you set macroblock 0,0 to "more red" the entire picture becomes that much "more red". Right?

Now here is my mystery example.

Take iframe 12 (try1) and enter 0:0:-1,13:24:72730 as mmb. It shows the bottom part of the picture and it looks quite nice.

Now enter 0:0:-1,12:24:72687 and suddenly you see a luminosity error appearing far from the block we just changed. How is this possible??

I suspect this has to do with the fact that the lower-right block of 24:26 is getting different input from above or left, but I don't understand why it could be so different from before. Maybe some kind of dc-clipping? (but there is no error log of that I think) Is this block reading the bits differently because it has different input? Because I always thought that the reading/meaning of the bits was not determined by the input.

I'm confused by this.

Regards,

arnezami

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/12/2014 07:16 am
Hi guys,

I have a bit of a mystery going on and it's holding me back from doing great stuff. ;) Hopefully somebody else has an idea what is going on.

This is essentially about how luminance/chrominance (errors) propagate. From what I understood so far each (macro)block propagates to its right (macro)block and to its bottom (macro)block. Luminance does it on a block (8x8pixel) basis while chrominance does it on a macroblock basis. So if you set macroblock 0,0 to "more red" the entire picture because that much "more red". Right?

Now here is my mystery example.

Take iframe 12 (try1) and enter 0:0:-1,13:24:72730 as mmb. It shows the bottom part of the picture and it looks quite nice.

Now enter 0:0:-1,12:24:72687 and suddenly you see a luminosity error appearing far from the block we just changed. How is this possible??

I suspect this has to do with the fact that the lower-right block of 24:26 is getting different input from above or left, but I don't understand why I could be so different from before. Maybe some kind of dc-clipping? (but there is no error log of that I think) Is this block reading the bits differently because it has different input? Because I always thought that the reading/meaning of the bits was not determined by the input.

I'm confused by this.

Regards,

arnezami



I've seen this too. Either the offending block is messed up in some way that propagates the error, or MPEG behaves badly when values are clipped. The problem seems to be worse on frames where the top row is missing. I attached an image that shows the path from the bad block to where it gets noticeably bad.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: gospacex on 05/12/2014 07:33 am

Yes. There were 5 times 56 bytes missing (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1191647#msg1191647) in the raw.ts. Very weird indeed.

This was fixed (among other things) in the coerced.ts. I am not sure if it was fixed in the try1.ts.

IIRC edited, coerced and try1 all had the 188byte packets aligned (except the 4bit thing)

Aligning is easy, and possibly not that useful.
For one, it changes file offsets of ts-packets, making it harder for us to refer to the packets in speech.
E.g. "packet at offset XYZ needs its header fixed this way" - offset XYZ in *which .ts file*?

The hard part is repairing damaged headers, and subsequently, data.

Aligning is easy, but also critical. There is no way to decode misaligned data. Luckily for us, michaelni has already written code to do this automatically (tsalign.c).

Referring to packet header locations is not that useful, as they are easily automatically repaired by the other program michaelni has written (tsfix.c).

It's as simple as

cat raw.ts | tsalign | tsfix > fixed.ts


The only hard part is repairing the data.

Unfortunately, it is not that easy to automatically
fix headers without damaging data even more.

I think it's better to have the corrections explained (so that they can be cross-checked and tuned).
For example, take edited.ts and coerced.ts. edited.ts has this sequence of packets:

00011a0 4703e8171ee887cbdfca43de18b9c2ea1a00c602180987dea67319134
000125c 4703e818ea28a720cc02d4aab0f256ba257cbc142a249ece63371f217
0001318 47be1c520419650d5d53bbcdb599034d0a9671ca97fbe0f475e8a40e0
00013d4 474842d0f2fde345f6dfbfd879228b38560c9b12c310942450f11ef62
0001490 477883798db922308145ac41ec97875bbc361389625238f2a0c8ec2ae
000154c 477c538a240317970fb67007230fca81c10d7055a74983e73efec042c
0001608 47d08588d2899b38c980f24655dd453b0f1fef99c87541c3458f9a9e4
00016c4 477d03c3c0c0da0c5ea8033e10419583170d016759e418bc1003fc9a0
0001780 4703e81fb89610bc12a1b164f81e27fe900e06540e126806979704207
000183c 4703e8301a005ffffffffffd7fffffffffffc7ff6ffffffffffffffff
00018f8 4740001c0000b0110000c100000000e0100001e020d36af0acfffffff
00019b4 471fff12ffffffffffffffffffffffffffffffffffffffff8480e4ffe
0001a70 471fff13fffffffffffffffffffffffffffffffffffffffffffffffff
0001b2c 4703e8149fdbfffffffffffffffffffffffffffffdfff3ffffaffe1fe
0001be8 4703e815dbb77e4e2fffefff9fffcfff5fffe17fb8cfe799def8fe67f

and in coerced.ts, the corresponding packets are

0001318 4703e8171ee887cbdfca43de18b9c2ea1a00c602180987dea67319134
00013d4 4703e818ea28a720cc02d4aab0f256ba257cbc142a249ece63371f217
0001490 471fff12fffffffffffffffffffffffffffffffffffffffffffffffff
000154c 471fff13fffffffffffffffffffffffffffffffffffffffffffffffff
0001608 471fff14fffffffffffffffffffffffffffffffffffffffffffffffff
00016c4 471fff15fffffffffffffffffffffffffffffffffffffffffffffffff
0001780 471fff16fffffffffffffffffffffffffffffffffffffffffffffffff
000183c 471fff17fffffffffffffffffffffffffffffffffffffffffffffffff
00018f8 4703e819b89610bc12a1b164f81e27fe900e06540e126806979704207
00019b4 4703e83a1a005ffffffffffd7fffffffffffc7ff6ffffffffffffffff
0001a70 474000110000b0110000c100000000e0100001e020d36af0acfffffff
0001b2c 471fff18fffffffffffffffffffffffffffffffffffffffffffffffff
0001be8 471fff19fffffffffffffffffffffffffffffffffffffffffffffffff
0001ca4 4703e81b9fdbfffffffffffffffffffffffffffffdfff3ffffaffe1fe
0001d60 4703e81cdbb77e4e2fffefff9fffcfff5fffe17fb8cfe799def8fe67f

As you see, packets 2..8 were marked as "null" and filled with FF's.
Granted, their headers look severely corrupted, but it's wrong
to turn them to "nulls". Check 8'th nibble. First two packets
have 7, 8 there. Continue incrementing it through the corrupted packets.
9,a,b,c,d,e... When you reach F, you get a match again, with the
4703e81fb89610bc12a1b... packet (see? it's awful when packets
have no fixed offsets - I need to refer to them by *contents*...)
Next is 0 - matches next packet.
Next is 1 - corrupted.
Next is 2 - match. 3 - match. 4 - match. 5 - match!

Those six packets with headers has been auto-fixed to "nulls", subsequent packets have their seq fields renumbered. This destroyed some data which was there.

We need to be more careful than that.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jakusb on 05/12/2014 07:49 am

Hi guys,

I have a bit of a mystery going on and it's holding me back from doing great stuff. ;) Hopefully somebody else has an idea what is going on.

This is essentially about how luminance/chrominance (errors) propagate. From what I understood so far each (macro)block propagates to its right (macro)block and to its bottom (macro)block. Luminance does it on a block (8x8pixel) basis while chrominance does it on a macroblock basis. So if you set macroblock 0,0 to "more red" the entire picture because that much "more red". Right?

Now here is my mystery example.

Take iframe 12 (try1) and enter 0:0:-1,13:24:72730 as mmb. It shows the bottom part of the picture and it looks quite nice.

Now enter 0:0:-1,12:24:72687 and suddenly you see a luminosity error appearing far from the block we just changed. How is this possible??

I suspect this has to do with the fact that the lower-right block of 24:26 is getting different input from above or left, but I don't understand why I could be so different from before. Maybe some kind of dc-clipping? (but there is no error log of that I think) Is this block reading the bits differently because it has different input? Because I always thought that the reading/meaning of the bits was not determined by the input.

I'm confused by this.

Regards,

arnezami

I had a similar experience a few days back. I cannot remember which frame. I think 4 or 5.
I had to -1 the very last mb to have it mess up the clock. It is very strange to have a block down stream mess up the image up stream. Or isn't it?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/12/2014 08:52 am
I've seen this too. Either the offending block is messed up in some way that propagates the error, or MPEG behaves badly when values are clipped. The problem seems to be worse on frames where the top row is missing. I attached an image that shows the path from the bad block to where it gets noticeably bad.
Yeah. I'm starting to realize that it is even harder than I thought it would be.

I wanted to create a "clean enviroment" for checking bad block propagation by starting at the end and working backward therefore avoiding the issue of having propagation from above and left.

But this won't work if block propagation below and to the right is different when adding more blocks to the left and above.

Hmmm. A rethink is required here.

Thanks for that diff!

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: hutchel on 05/12/2014 11:38 am
I've seen this too. Either the offending block is messed up in some way that propagates the error, or MPEG behaves badly when values are clipped. The problem seems to be worse on frames where the top row is missing. I attached an image that shows the path from the bad block to where it gets noticeably bad.
Yeah. I'm starting to realize that it is even harder than I thought it would be.

I wanted to create a "clean enviroment" for checking bad block propagation by starting at the end and working backward therefore avoiding the issue of having propagation from above and left.

But this won't work if block propagation below and to the right is different when adding more blocks to the left and above.

Hmmm. A rethink is required here.

Thanks for that diff!

Regards,

arnezami


Quick thought from a quiet computer Geek in the audience.... Can Space X provide you with a good 30 sec chunk of video from the same setup so you can see what good video is supposed to look like?  That would get you ground truth as to formatting etc.  It would also give you the ability to selectively damage the bit stream to better understand propagation related errors.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/12/2014 02:33 pm

Unfortunately, it is not that easy to automatically
fix headers without damaging data even more.

I think it's better to have the corrections explained (so that they can be cross-checked and tuned).
For example, take edited.ts and coerced.ts. edited.ts has this sequence of packets:



Those six packets with headers has been auto-fixed to "nulls", subsequent packets have their seq fields renumbered. This destroyed some data which was there.

We need to be more careful than that.

If you look at tsfix.c, it defaults to assume a packet is video rather than null. And it does not fill null packets with 0xFF, it just fixes the header so they are skipped by ffmpeg. Also, I have been working on statistical analysis of packet contents (rather than just the header) and context (previous and next packets) to increase accuracy.

The problem with your approach is that it assumes the error rate in the header is low enough for a human to be able to recognize the different parts. It isn't, at least in a lot of the data, the error rate is 16-20% everywhere, including the header, not just the data. Humans can't help finding patterns in things, even if they are not there. Especially when you start taking packets out of order/context, it is hard to get the whole picture.

For example, as I posted earlier, there are spikes in noise level where error rate spikes above 30% for a few thousand bytes. If this occurred in a null packet it might look to you like a mildly corrupt data packet, but in reality it is junk.

The goal of cleaning the raw stream is to give the folks here the very best mpg4-img files to tweak visually. There are 23838 packets in raw.ts - IMO It is better to tune the algorithm rather than numbering the data and tuning by hand. I don't understand your concern, for any given code run on any given input, the output will always be the same. It's not like my output will be different than yours, cross referencing is always possible by offset.

Edit: gospacex - look at http://pastebin.com/HmDAzu1q to see my version of null packet identification. Looking again, it seems to be similar to yours. My next iteration will count runs of diffs instead of just the number of diffs.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/12/2014 03:06 pm
Ok, I am done with my work on the headers

attached you find a raw_edit1.ts (edit: file contained a bug with frame169(old Iframe8), you find the corrected version  here (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1197348#msg1197348)) where only the 15 I-Frame headers are changed (plus a few '47' bytes maybe), Also the one 188 byte block is reshifted by 4 bits to align it.

The last XX XX was part of vbv_occupancy (1 marker bit and the frist 7 of the latter half of vbv_occupancy)
According to michaelni, the actually value is not needed to extrace I-frames.

Cheers
Shanuson
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/12/2014 04:34 pm
I encourage interested people to uncomment last line, run the program,
look at raw.hex.diff, and suggest improvements to fix_data_packets() logic.

After wading through this code, I think we have roughly the same idea. Just took me a while to get used to your coding style before I could see what you were doing.

I've started to add your extra header patterns to the paste bin code, and tweaked what michaelni had done for fixing packets. Overall, I think of the three the newest style is the most concise and fastest performing.

The things that still need to be done:
(1) check the rest of your packet types, and adjust the masks as needed.

(2) integrate min(score) over packet number, and use this to allow the probabilities to relax in high-error regions. When p(error) exceeds a certain value, it might be best to "drop" packets all together rather than feed them to the decoder.

(3) rather than use a simple bit-count as the diff, we should really be checking the number of continuous matching bits. In other words, we should compute Poisson rather than binomial probability.

(4) in addition to expected and mask, there should be a third row of bit probability. when the expected is not known (thus, nothing to diff). This would solve the issue of null body matching, since the p(1) should be close to .5 for real data and close to 1 for null data. Note this only matters when (3) has been implemented, otherwise expected = ..0x55 0x55 0x55.. has the same effect for the binomial probability.

Edit (5): evaluate all possibilities, rather than the first best match, then choose the highest.

Edit (6): fix sequence numbers where possible 
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/12/2014 04:57 pm
I think we can also assume that the transport error and the scrambling control bits should never be set in any header, right? That may help constrain the problem a bit. Turning off the transport priority bit in everything may also be helpful, since we're dealing with a file rather than an over-the-air stream and wouldn't expect an overworked decoder to need to choose which packet to deal with first.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: cscott on 05/12/2014 05:19 pm
About the Iframes, i was thinking that manual moving the macroblocks around will be difficult if you don't have a lot of reference (i.e. if there aren't many features on the block).

As it turns out, it appears that the dirty splash on liftoff has been a great help, as there are lots of small dirty smudges in each frame which provide reference features.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jakusb on 05/12/2014 06:21 pm
About the Iframes, i was thinking that manual moving the macroblocks around will be difficult if you don't have a lot of reference (i.e. if there aren't many features on the block).

As it turns out, it appears that the dirty splash on liftoff has been a great help, as there are lots of small dirty smudges in each frame which provide reference features.

Yes, see my earlier post with an image annotated with 14 dirt spots that clear are on the lens.


A version with the dirt spots numbered for better reference in conversation

Edit: Added dirt group 14

(http://img.tapatalk.com/d/14/05/13/7eje5ete.jpg)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/12/2014 06:35 pm
About the Iframes, i was thinking that manual moving the macroblocks around will be difficult if you don't have a lot of reference (i.e. if there aren't many features on the block).

As it turns out, it appears that the dirty splash on liftoff has been a great help, as there are lots of small dirty smudges in each frame which provide reference features.

Yes, see my earlier post with an image annotated with 14 dirt spots that clear are on the lens.


A version with the dirt spots numbered for better reference in conversation

Edit: Added dirt group 14

(http://img.tapatalk.com/d/14/05/13/7eje5ete.jpg)

Dont forget the one that is overlapping with part of the right leg. (wiki IF 5 shows a complete right leg)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/12/2014 06:40 pm
Hi guys,

I have a bit of a mystery going on and it's holding me back from doing great stuff. ;) Hopefully somebody else has an idea what is going on.

This is essentially about how luminance/chrominance (errors) propagate. From what I understood so far each (macro)block propagates to its right (macro)block and to its bottom (macro)block. Luminance does it on a block (8x8pixel) basis while chrominance does it on a macroblock basis. So if you set macroblock 0,0 to "more red" the entire picture becomes that much "more red". Right?

Now here is my mystery example.

Take iframe 12 (try1) and enter 0:0:-1,13:24:72730 as mmb. It shows the bottom part of the picture and it looks quite nice.

Now enter 0:0:-1,12:24:72687 and suddenly you see a luminosity error appearing far from the block we just changed. How is this possible??

I suspect this has to do with the fact that the lower-right block of 24:26 is getting different input from above or left, but I don't understand why it could be so different from before. Maybe some kind of dc-clipping? (but there is no error log of that I think) Is this block reading the bits differently because it has different input? Because I always thought that the reading/meaning of the bits was not determined by the input.

I'm confused by this.

Regards,

arnezami

So this confused me for a while too. First off a block does not propagate to the left AND right. It is better to say that a given block propagates from EITHER the left OR the top block. Further, this is not defined in the file specifically. It is defined by comparing the block above, the block to the left, AND the block to the upper left. Based on the DC (brightness/color) of the blocks in these directions it PICKs which block to propagate from. Because of this when you adjust a value you can cause a given block to swap which block it inherits from.

This is even worse because this prediction direction even determines the scan order through the DCT table. So causing a block to swap which block it inherits from can cause the DCT scan order to become screwed up resulting in the pixels in the block to be transposed (mirrored and flipped vertically).
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Lars_J on 05/12/2014 06:42 pm
A version with the dirt spots numbered for better reference in conversation

Edit: Added dirt group 14

Also, don't forget the large smudge under dirt #3 (just below to the left), which obscures/blurs part of the leg in every frame. (that has a leg)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/12/2014 06:42 pm
Hi guys,

I just noticed that there are no B-frames in the file. Only I-frames and P-frames.

Quote
./ffprobe try1.ts -show_frames > frames.txt

Not sure yet what the implications are yet. Hopefully it will get easier that way.

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/12/2014 06:45 pm
So this confused me for a while too. First off a block does not propagate to the left AND right. It is better to say that a given block propagates from EITHER the left OR the top block. Further, this is not defined in the file specifically. It is defined by comparing the block above, the block to the left, AND the block to the upper left. Based on the DC (brightness/color) of the blocks in these directions it PICKs which block to propagate from. Because of this when you adjust a value you can cause a given block to swap which block it inherits from.
Aha! Interesting!

Thank you for that. That way it does indeed make more sense.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/12/2014 06:56 pm
I've seen this too. Either the offending block is messed up in some way that propagates the error, or MPEG behaves badly when values are clipped. The problem seems to be worse on frames where the top row is missing. I attached an image that shows the path from the bad block to where it gets noticeably bad.
Yeah. I'm starting to realize that it is even harder than I thought it would be.

I wanted to create a "clean enviroment" for checking bad block propagation by starting at the end and working backward therefore avoiding the issue of having propagation from above and left.

But this won't work if block propagation below and to the right is different when adding more blocks to the left and above.

Hmmm. A rethink is required here.

Thanks for that diff!

Regards,

arnezami


Quick thought from a quiet computer Geek in the audience.... Can Space X provide you with a good 30 sec chunk of video from the same setup so you can see what good video is supposed to look like?  That would get you ground truth as to formatting etc.  It would also give you the ability to selectively damage the bit stream to better understand propagation related errors.

Yes we asked about that. But nothing has come yet. Supposedly something is in the works.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/12/2014 07:06 pm
Parsing raw.ts using info from http://en.wikipedia.org/wiki/MPEG_transport_stream

The file consists of 188-byte ts-packets.

1st byte is always 47.

2 next bytes are 000<packetid13bits>, and apparently
they are always 03e8 for non-null ts-packets and 1fff for null ts-packets.

there are more than 2 PID values in the ts packets/ file


To find out how many different TS-PIDs exists, I counted all different "0x47xxxx" appearances in raw_edit1.ts:
This are all which occured 20 times or more:
20 471be8
20 4733e8
21 470fe8
25 4703e9
63 474703
74 474020
81 474000
269 4743e8
4968 471fff
15016 4703e8


I think the first 4 are permutations of the last ones by bitflips.
4743e8 and 4703e8 have the same PID,
When one checks where 474703 is found, one sees that the next byte is E8. So the first 47 is the last byte of a previous packet, and therefore 474703 is not part of a TS-header.
So we end up with only 4 PIDs (13 bit long):
0x0000, 0x0020, 0x03E8,0x1FFF
Each PID has its own continuity counter, which should help deciding which packet is which.
 
With some different flagbits we end with the following TS-headers:
0x4740001W
0x4740201X
0x4703E81Y, 0x4703E83Y and 0x4743E83Y
0x471FFF1Z
with the 4 different counters W,X,Y,Z
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/12/2014 07:46 pm
Offtopic: An updated version of the sequence from video I-Frames that I posed some pages ago, now with the legs clearly moving!


Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jakusb on 05/12/2014 08:35 pm
Offtopic: An updated version of the sequence from video I-Frames that I posed some pages ago, now with the legs clearly moving!
Nice! Clearly great progress already.
If only we could somehow get the legs in Iframe 4 resolved...
I assume they should be just opened and slightly away from the stage. Or not

Ps: very much on topic if you ask me. ;)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/12/2014 09:11 pm
Ok, I am done with my work on the headers

attached you find a raw_edit1.ts where only the 15 I-Frame headers are changed (plus a few '47' bytes maybe), Also the one 188 byte block is reshifted by 4 bits to align it.

The last XX XX was part of vbv_occupancy (1 marker bit and the frist 7 of the latter half of vbv_occupancy)
According to michaelni, the actually value is not needed to extrace I-frames.

Cheers
Shanuson

I tried it, some iframes appear to have extra data and are clearer, others are worse. iframe8 is missing, and I'm not sure if there are new iframes or not. Several of the frames have apparently significantly less errors though.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IRobot on 05/12/2014 09:19 pm
As soon as the work finishes on recovering the information, I guess some post processing can be done, namely noise filtering/smothering and some morphing to create intermediate frames!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jakusb on 05/12/2014 09:20 pm

Ok, I am done with my work on the headers

attached you find a raw_edit1.ts where only the 15 I-Frame headers are changed (plus a few '47' bytes maybe), Also the one 188 byte block is reshifted by 4 bits to align it.

The last XX XX was part of vbv_occupancy (1 marker bit and the frist 7 of the latter half of vbv_occupancy)
According to michaelni, the actually value is not needed to extrace I-frames.

Cheers
Shanuson

I tried it, some iframes appear to have extra data and are clearer, others are worse. iframe8 is missing, and I'm not sure if there are new iframes or not. Several of the frames have apparently significantly less errors though.

Any idea yet how to proceed?
Is it useful to add them to the online editor v2?
I am getting pretty curious to see this potential new or improved data in actual images...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: AncientU on 05/12/2014 09:33 pm
About the Iframes, i was thinking that manual moving the macroblocks around will be difficult if you don't have a lot of reference (i.e. if there aren't many features on the block).

As it turns out, it appears that the dirty splash on liftoff has been a great help, as there are lots of small dirty smudges in each frame which provide reference features.

Yes, see my earlier post with an image annotated with 14 dirt spots that clear are on the lens.


A version with the dirt spots numbered for better reference in conversation

Edit: Added dirt group 14

(http://img.tapatalk.com/d/14/05/13/7eje5ete.jpg)

Dont forget the one that is overlapping with part of the right leg. (wiki IF 5 shows a complete right leg)
Also, don't forget the rocket body and it's dirty water 'paint job' which should remain constant except for lighting effects.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/12/2014 09:37 pm

Ok, I am done with my work on the headers

attached you find a raw_edit1.ts where only the 15 I-Frame headers are changed (plus a few '47' bytes maybe), Also the one 188 byte block is reshifted by 4 bits to align it.

The last XX XX was part of vbv_occupancy (1 marker bit and the frist 7 of the latter half of vbv_occupancy)
According to michaelni, the actually value is not needed to extrace I-frames.

Cheers
Shanuson

I tried it, some iframes appear to have extra data and are clearer, others are worse. iframe8 is missing, and I'm not sure if there are new iframes or not. Several of the frames have apparently significantly less errors though.

Any idea yet how to proceed?
Is it useful to add them to the online editor v2?
I am getting pretty curious to see this potential new or improved data in actual images...

First there is a small python script which helps to get the iframes.
After ffmpeg -vstats -i file.ts-vcodec copy -an -f image2 frame%d.mpg4-img produced all frames
use the following python script

import os
import sys

ld=os.listdir('./')

for ff in ld:
    if "frame" in ff:
        with open(ff, "rb") as f:
            b1 = f.read(4).encode("hex")
            i=0       
            #print b1,ff
            if b1=='000001b0':
                print ff
                os.system('cp '+ff+' i'+ff)


It will make copys of those frames that are I-frames.

attached you find a zipfile containing all 15. But some still need some work, 1 is only 78 byte large, will another is of 30Kb.
But this is progress. Maybe to early to add them to the wiki, i dont know.

Edit: maybe should give a maping from the new ones to the old ones:
New -> Old:
iframe1.mpg4-img -> IF0
iframe14.mpg4-img ->
iframe32.mpg4-img ->  IF1
iframe52.mpg4-img -> IF2
iframe72.mpg4-img -> IF4
iframe92.mpg4-img -> IF5
iframe112.mpg4-img  ->  IF6
iframe131.mpg4-img  ->
iframe150.mpg4-img  ->  IF7
iframe169.mpg4-img  ->  IF8 
iframe190.mpg4-img  -> IF9
iframe210.mpg4-img  -> IF10
iframe230.mpg4-img  -> IF11
iframe249.mpg4-img  -> IF12
iframe269.mpg4-img  -> IF13
 

Edit2:Fixed the 78byte IF, it is now 20472 Bytes large
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/12/2014 09:56 pm
As soon as the work finishes on recovering the information, I guess some post processing can be done, namely noise filtering/smothering and some morphing to create intermediate frames!

Already working on it. Want to help?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/12/2014 10:00 pm
As soon as the work finishes on recovering the information, I guess some post processing can be done, namely noise filtering/smothering and some morphing to create intermediate frames!

Already working on it. Want to help?

A lot of work still needs to be done on iframes before that can be done.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/12/2014 10:16 pm
To find out how many different TS-PIDs exists, I counted all different "0x47xxxx" appearances in

Back when I was messing with the raw.ts I looked at the PMTs (because I thought corrupted PMTs may mess up decoding I made a consensus PAT and PMT and attached those to the start of the file and removed all others, don't think it made much difference). I remember there were 3 PIDs defined. 3e8 - video, 3e9 and 3f0. I have no idea what the last two are, but I think they're unneeded. Those packets may still be present though, but I don't remember actually seeing one. Even if the header said 3e9 the counter suggested it was actually still 3e8.

This is even worse because this prediction direction even determines the scan order through the DCT table. So causing a block to swap which block it inherits from can cause the DCT scan order to become screwed up resulting in the pixels in the block to be transposed (mirrored and flipped vertically).

That's just disgusting.

If only we could somehow get the legs in Iframe 4 resolved...
I assume they should be just opened and slightly away from the stage. Or not

I've spent the last two evenings on this and it's been awful. I'm inching closer, but the data is in dreadful condition. I've seen no leg at all. They appear out of thin air in iframe 5 for all I can tell. I'm calling it a night for today. My best effort so far is pretty abysmal, but still shows a bit more than the previous efforts (I hope it's more or less correct)

mmb: 5:1:4200,0:3:10968,8:3:14762,28:3:16550,24:4:20286,40:4:-1,9:6:21857,13:11:39488,37:11:41549,
10:12:42693,29:12:43895,21:13:50986,10:14:-1,13:14:-1,21:15:-1:60:60::::,22:15:55249,2:16:-1,
8:16:58952,13:16:59649,27:16:63538,18:17:69075,27:17:-1,13:18:74459
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/12/2014 10:17 pm
As soon as the work finishes on recovering the information, I guess some post processing can be done, namely noise filtering/smothering and some morphing to create intermediate frames!

Already working on it. Want to help?

A lot of work still needs to be done on iframes before that can be done.

Everything that I am working on involves transformations on the i- and p- frames. I can fold decoding improvements back into the project as they happen.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/12/2014 10:27 pm
Here are my I-Frames.
Yes for various reasons am i sure that these are the I-Frames, and that they are the correct ones.
But it can be that some data is missing from them or that there is data in them from the next frame.

As for the different Iframe mlindner and i got, i think that is because he is using raw_edit1.ts directly while i put it through tsalign and tsfix first. those i got 269 frames.
here are the pngs from those:
Also as can be seen by the numbering: most of them are 20 frames apart, some are less, indicating where something is still not ok.

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/12/2014 10:33 pm
Those look very very nice!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/12/2014 10:56 pm
Here is the fixed.ts where those and the img-files in that zipfolder a few posts back came from:

Edit: And the raw_edit1.zip file with the header of frame169 fixed. I will delete the wrong one.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: bilbo on 05/12/2014 10:59 pm
Has any work been done on the section where the stage tips over?
so far, I have only seen work on the landing, and initial splashdown.
Keep up the good work!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: gospacex on 05/12/2014 11:06 pm
For example, as I posted earlier, there are spikes in noise level where error rate spikes above 30% for a few thousand bytes. If this occurred in a null packet it might look to you like a mildly corrupt data packet, but in reality it is junk.

I was thinking about it and here sequence numbers are a godsend. They are often corrupted too, yes, but it would take a lot of corruption to make long run of them completely unrecognizable!

And as long as you can reasonably guess and restore them, most of the time it conclusively answers the question "is this packet data or null?" - they have separately counting sequences.

Quote
The goal of cleaning the raw stream is to give the folks here the very best mpg4-img files to tweak visually.

Exactly. My worry is that by dropping just one data ts-packet, even corrupted one, we can easily make it orders of magnitude harder to restore the frame.

ts-packet fixing operates on 188-byte blocks, macroblock fixing works on BITS! Spending a minute more on fixing a packet can save hours!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/12/2014 11:08 pm
Here are my I-Frames.

Wow. That iframe 6 is almost perfect.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/12/2014 11:11 pm
Going to port my iframe8 changes to this new iframe8.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/12/2014 11:28 pm
Going to port my iframe8 changes to this new iframe8.
How do you do that?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/12/2014 11:48 pm
Alpha 3 of my transport stream search and repair tool is finished.

Were you able to find any other sub-byte shifts aside from the one we already know about?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/13/2014 12:01 am
I have tsfix.c matching the patterns from gospacex correctly now, and fixing sequence numbers: http://pastebin.com/dirZVxfS

It seems like there are a ton of sequence number errors, mostly for PID=1000 (which is forced more often than not as all PIDs not 0, 8191, 32, or 64 are forced to 1000).

This does make the sequence number errors in ffmpeg go away, but brings on many new errors, such as:

DTS 4301262034 < 8597009660 out of order
timestamp discontinuity -47730595911, new offset= -47770843290
etc.

The output video looks different, but I would not say better (just playing in VLC).
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/13/2014 12:10 am
The time stamp discontinuities should be able to be fixed without too much heartache since we know this is a monotonic stream. There should be a valid range of values and I suspect the flipped bits would become apparent upon inspection of the prior and subsequent stamps, much like the last four bits of the header.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/13/2014 12:26 am
i fixed manually (by hand) the TS-Headers including the continuity counter throughout that 2nd IFrame (frame13 in raw_edit1.ts or frame14 in fixed.ts) in both those files.
before it had ~9KB in fixed.ts.
After fixing the sequence in fixed.ts It got a bit larger ~10Kb. But there was one 188 KB block of mostly FFs that somehow got included into fixed.ts by tsfix (or less likely tsalign) that could not be accounted for with a continuous continuity counter.
So i also handfixed raw_edit1.ts the same way, which results in the attached png and img-file :D. It is now 16kb large.

Also i am not sure some script can fix all those TS-headers correctly. Sometimes the bytes are completely wrong.

Edit: this is not frame14 at 0x85bf8, but the next one, old IF1, sorry for messing up the frames
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/13/2014 12:58 am

To find out how many different TS-PIDs exists, I counted all different "0x47xxxx" appearances in raw_edit1.ts:
This are all which occured 20 times or more:
20 471be8
20 4733e8
21 470fe8
25 4703e9
63 474703
74 474020
81 474000
269 4743e8
4968 471fff
15016 4703e8


I think the first 4 are permutations of the last ones by bitflips.
4743e8 and 4703e8 have the same PID,
When one checks where 474703 is found, one sees that the next byte is E8. So the first 47 is the last byte of a previous packet, and therefore 474703 is not part of a TS-header.
So we end up with only 4 PIDs (13 bit long):
0x0000, 0x0020, 0x03E8,0x1FFF
Each PID has its own continuity counter, which should help deciding which packet is which.
 
With some different flagbits we end with the following TS-headers:
0x4740001W
0x4740201X
0x4703E81Y, 0x4703E83Y and 0x4743E83Y
0x471FFF1Z
with the 4 different counters W,X,Y,Z

For completeness, I counted PIDs by absolute bit position in the raw.ts, here are the top 50:

PID       HEX      COUNT 
1000    x03E8      15592   -- Video
8191    x1FFF      5070     -- Null packet
   0    x0000      84
  32    x0020      75
5096    x13E8      49       -- 1 bit off from 1000
1001    x03E9      48        -- 1 bit off from 1000
 904    x0388      40                -- 2 bit off from 1000
7144    x1BE8      30       -- 2 bit off from 1000
 984    x03D8      29                -- 2 bit off from 1000
1003    x03EB      29       -- 1 bit off from 1000
 616    x0268      28
3048    x0BE8      28
1512    x05E8      27
2000    x07D0      27
 996    x03E4      26
1008    x03F0      26
1002    x03EA      25
4095    x0FFF      24
 500    x01F4      23
1006    x03EE      23
4072    x0FE8      23
8190    x1FFE      23
 808    x0328      19
1004    x03EC      19
 992    x03E0      17
 232    x00E8      16
 488    x01E8      16
 968    x03C8      16
1016    x03F8      16
2024    x07E8      16
6120    x17E8      15
 872    x0368      14
 936    x03A8      14
4000    x0FA0      14
 680    x02A8      13
 744    x02E8      13
1005    x03ED      13
2047    x07FF      13
2536    x09E8      12
8167    x1FE7      11
8189    x1FFD      11
7807    x1E7F      10
 840    x0348      9
5119    x13FF      9
7423    x1CFF      9
8095    x1F9F      9
8127    x1FBF      9
8185    x1FF9      9
 125    x007D      8
 994    x03E2      8


All 1650 are in the attached csv.

I think your 4-PID theory is correct.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/13/2014 01:24 am
Going to port my iframe8 changes to this new iframe8.
How do you do that?

Apparently the new and the old iframe8 are byte identical. Was hoping some of the issues would be fixed, but apparently not.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/13/2014 01:52 am
I think your 4-PID theory is correct.

And it makes sense, too - it's not like it's a very complicated stream with a bunch of different elementary streams in it. One camera and you're done. The 0x0000 is the program association table which "contains a directory listing of all Program Map Tables," (Wikipedia) then 0x0020 is the first available number in the range of assignable data tables, and finally 1000 is a nice round number in decimal.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Adaptation on 05/13/2014 02:01 am
Hi guys,

I just noticed that there are no B-frames in the file. Only I-frames and P-frames.

Quote
./ffprobe try1.ts -show_frames > frames.txt

Not sure yet what the implications are yet. Hopefully it will get easier that way.

Regards,

arnezami

That is very fortunate happenstance.  P frames will only have compound errors from any damage from the frames before them in the sequence.  B frames would have compound errors from both before and after them in the sequence. 
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Swatch on 05/13/2014 03:36 am

To find out how many different TS-PIDs exists, I counted all different "0x47xxxx" appearances in raw_edit1.ts:
...

For completeness, I counted PIDs by absolute bit position in the raw.ts, here are the top 50:
...

I think your 4-PID theory is correct.


Out of curiosity, did anyone do the counting (based on absolute bit position) on the streams that were re-aligned to 188 bits (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1196838#msg1196838)?  I was under the impression that raw.ts was not, so this might throw off your count if you're doing absolute bit position.

How many differenct PID permutations might that produce?

Then it might help for Adaptation's tool (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1197014#msg1197014) to be able to display/color based on the PID permutations.  That might also help identify what type of PID a corrupted packet is supposed to be.

Just a suggestion after digesting all of your work the last few pages.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/13/2014 04:46 am

To find out how many different TS-PIDs exists, I counted all different "0x47xxxx" appearances in raw_edit1.ts:
...

For completeness, I counted PIDs by absolute bit position in the raw.ts, here are the top 50:
...

I think your 4-PID theory is correct.


Out of curiosity, did anyone do the counting (based on absolute bit position) on the streams that were re-aligned to 188 bits (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1196838#msg1196838)?  I was under the impression that raw.ts was not, so this might throw off your count if you're doing absolute bit position.

How many differenct PID permutations might that produce?

Then it might help for Adaptation's tool (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1197014#msg1197014) to be able to display/color based on the PID permutations.  That might also help identify what type of PID a corrupted packet is supposed to be.

Just a suggestion after digesting all of your work the last few pages.

Sorry, to be clear I always run raw.ts | tsalign before doing anything on it. So the packets are lined up correctly in the stats above.

Edit: the reason I like to start with raw.ts is that it would be nice to have the final solution be generated completely automatically. Maybe then Michael will add a -spacex flag to ffmpeg that will allow raw.ts to play flawlessly ;)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Digitalchromakey on 05/13/2014 05:01 am
Here are my I-Frames.

Wow. That iframe 6 is almost perfect.
I think iframe112-mpg4.img.png is decoded from the ES video data set that was previously used to decode iframe_6 (coerced).

The time stamp in the new  iframe131.mpg4-img.png  frame is 19:35:10:23, whereas the time stamp you can see in iframe_6 (coerced) is 19:35:08:23.

As iframe_8 has a time stamp of 19:35:12:97 - a difference of nearly 5 seconds, it looks like maybe a new I frame has been teased out of the transport stream between the old iframe_6 and iframe_8.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/13/2014 06:46 am
Alpha 3 of my transport stream search and repair tool is finished.

Were you able to find any other sub-byte shifts aside from the one we already know about?

I recall there are at least two ts packets in raw.ts in iframe 8 that are missing one bit and the next packet has an extra bit causing the following ones to be aligned again. I expect there are more in other frames.

Going to port my iframe8 changes to this new iframe8.
How do you do that?

Apparently the new and the old iframe8 are byte identical. Was hoping some of the issues would be fixed, but apparently not.

That made for an easy port.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: JamesH on 05/13/2014 08:45 am
Hi guys,

I just noticed that there are no B-frames in the file. Only I-frames and P-frames.

Quote
./ffprobe try1.ts -show_frames > frames.txt

Not sure yet what the implications are yet. Hopefully it will get easier that way.

Regards,

arnezami

That is very fortunate happenstance.  P frames will only have compound errors from any damage from the frames before them in the sequence.  B frames would have compound errors from both before and after them in the sequence.

Not really that 'fortunate' - there are very few if any real time encoders of the sort that would be used here that use B-frames. They need to encode in near real time, which means basing frames on frames that haven't yet appeared doesn't really work.  B-frames tends to work best when doing offline encoding. It does give a better compression ratio, but is more computationally expensive.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: seanpg71 on 05/13/2014 08:52 am
...

Going to port my iframe8 changes to this new iframe8.
How do you do that?

Apparently the new and the old iframe8 are byte identical. Was hoping some of the issues would be fixed, but apparently not.

Comparing try1 to the frames from the edit1 zip above:
iframes 1, 4, 5, 7, 8, 9 and 13 match the frames from try1 exactly.
iframe 11 and 12 match up except for a couple bitflips
iframes 2, 6, and 10 have extra bits in try1 that aren't in the edit1 (dunno if they're important bits).

So we're probably not going to get any better versions of the current frames using the new iframes in Shanuson's edit1-fixed-iframe.zip.

The new frames are exciting though.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Kaputnik on 05/13/2014 09:38 am
A long thread but worth it. I feel like I have just been on a crash course in digital video processing!
Keep up the amazing work- the world is watching :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/13/2014 11:07 am
So I quickly tried the new frame 131 (between 6 and 7)! I threw away some valid blocks so it should be possible to improve it further.

0:0:-1,
1:3:7967,0:7:-1,
12:8:23866,26:14:-1,
34:16:65834,25:20:-1,
14:21:92080,24:25:-1,
14:27:120887,10:29:-1
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: gospacex on 05/13/2014 12:07 pm

To find out how many different TS-PIDs exists, I counted all different "0x47xxxx" appearances in raw_edit1.ts:
This are all which occured 20 times or more:
20 471be8
20 4733e8
21 470fe8
25 4703e9
63 474703
74 474020
81 474000
269 4743e8
4968 471fff
15016 4703e8


I think the first 4 are permutations of the last ones by bitflips.
4743e8 and 4703e8 have the same PID,
When one checks where 474703 is found, one sees that the next byte is E8. So the first 47 is the last byte of a previous packet, and therefore 474703 is not part of a TS-header.
So we end up with only 4 PIDs (13 bit long):
0x0000, 0x0020, 0x03E8,0x1FFF
Each PID has its own continuity counter, which should help deciding which packet is which.
 
With some different flagbits we end with the following TS-headers:
0x4740001W
0x4740201X
0x4703E81Y, 0x4703E83Y and 0x4743E83Y
0x471FFF1Z
with the 4 different counters W,X,Y,Z

For completeness, I counted PIDs by absolute bit position in the raw.ts, here are the top 50:

PID       HEX      COUNT 
1000    x03E8      15592   -- Video
8191    x1FFF      5070     -- Null packet
   0    x0000      84
  32    x0020      75
5096    x13E8      49       -- 1 bit off from 1000
1001    x03E9      48        -- 1 bit off from 1000
 904    x0388      40                -- 2 bit off from 1000
7144    x1BE8      30       -- 2 bit off from 1000
 984    x03D8      29                -- 2 bit off from 1000
1003    x03EB      29       -- 1 bit off from 1000
 616    x0268      28
3048    x0BE8      28
1512    x05E8      27
2000    x07D0      27
 996    x03E4      26
1008    x03F0      26
1002    x03EA      25
4095    x0FFF      24
 500    x01F4      23
1006    x03EE      23
4072    x0FE8      23
8190    x1FFE      23
 808    x0328      19
1004    x03EC      19
 992    x03E0      17
 232    x00E8      16
 488    x01E8      16
 968    x03C8      16
1016    x03F8      16
2024    x07E8      16
6120    x17E8      15
 872    x0368      14
 936    x03A8      14
4000    x0FA0      14
 680    x02A8      13
 744    x02E8      13
1005    x03ED      13
2047    x07FF      13
2536    x09E8      12
8167    x1FE7      11
8189    x1FFD      11
7807    x1E7F      10
 840    x0348      9
5119    x13FF      9
7423    x1CFF      9
8095    x1F9F      9
8127    x1FBF      9
8185    x1FF9      9
 125    x007D      8
 994    x03E2      8


All 1650 are in the attached csv.

I think your 4-PID theory is correct.

I think "ultimate fix" for ts-packet stream is to identify the nature and sequence numbers of nearly every packet. It is in fact possible despite instances of heavy local corruption.

Example how it can be done (looking at raw.hex):

00d9ce8 471fff1bffffffffffffffffffffffffffffffffffffffff...
00d9da4 071e7f1cfffffffffffce7ffaffffffffffbffe7ffffffff...
00d9e60 5e9eff1bfffffc3ff77fffffffffffff53ffffffffffffff...
00d9f1c 465eff389fcebbfd67f0ffffffbffe79ffebffaffe1fdf7f
00d9fd8 26bebae7fbffdfe73fafff646f669f7ecd5c63a4e271d4a5

above packets are null packets. last two are heavily corrupted.
Now data packets follow:

00da094 4743e8380710002d2a507e00000001e00000818007210169
00da150 4703e819cdc0466941eb5c9df5f0320710bc3a330e091ba7
00da20c c703e81a059c70c40701a982190958cd31e2f06e6605a0b3
00da2c8 470fe41bb1d1204529aa6b20c9e3161765247f8f27400625
00da384 4703e81c2b371e0c9e2220df2056a413bc0c4f06c727fd05
00da440 4703e81ddfeeb416075502c8320cc4680dab3f6dafe923dc
00da4fc 4703e81e8d3801135e80608ecd4ee8a6b831473a5c0704c6
00da5b8 4703e81f19c3e015e44317a641ab463daf6c9474f5404178
00da674 5303e810845492731ce7088df908d5c69c9d8650f83de0c9
00da730 c0aa09d1aff4edb39423532036d6414be3bff3774fec9ff7
00da7ec 3870d4b906f3dc1101ac21f8154e663f444e7a684e908e5f
00da8a8 98167426424d9a350043e7e988b7648e76139028fef4b339
00da964 5060f71f0180f1b65c02ad0414880a6a0594d1211547cc01
00daa20 8e07902a9908c0d17d52b1e4cd2a16ed7895f5209a251ca7

last four packets above are heavily corrupted - header is completely
unrecognizable, but they *must be* data packets, otherwise seq numbers
don't match: there is 8,9,a,b,c,d,e,f,0,1 sequence before them,
and below the same sequence continues: 6,7,8,9,...
So these four packets above *have to be* 2,3,4,5,
and since seq numbers are counting independently for different PIDs,
they must be data packets.

00daadc 4703e816f0cd370cc3300b7a55347680436640e0c8456c32
00dab98 4703e81752d05c5b6234200ae027f951e53499e182195038
00dac54 4703e81893671ee0c8058f24c320c9ce0b250c9cf878496e
00dad10 4703e81973877ea096f1181004e2301e774163c04f72c038
00dadcc 4703e81a40b2a83cdc8e085b9d3413de15028243200af9e7
00dae88 4703e81bb9ec065fc3da74f299fd281e0b551ffcd3aab013
00daf44 4703e81c5938420482ded1e067066230400bdd77cf833832
00db000 4f03d81d380ff2904f9ae02286676e2bd40704b73e1a17e9
00db0bc 4703e8127c77ab0958cc54ac742ce54c00c3a0b2fd029871

Corrupted seq no above, should be 'e'.

00db178 471fff10ffffffffffffffffffffffffffffffffffffffff
00db234 4703e81f82ba45e0090670958f067b80dac02da97f1904f8

As you see, every packet can be identified in this example, despite
some headers being shot to hell.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jakusb on 05/13/2014 12:21 pm

Offtopic: An updated version of the sequence from video I-Frames that I posed some pages ago, now with the legs clearly moving!

Could someone maybe make a small addition where in top right corner per frame an estimation from how stage would look from the side?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/13/2014 01:12 pm

last four packets above are heavily corrupted - header is completely
unrecognizable, but they *must be* data packets, otherwise seq numbers
don't match: there is 8,9,a,b,c,d,e,f,0,1 sequence before them,
and below the same sequence continues: 6,7,8,9,...
So these four packets above *have to be* 2,3,4,5,
and since seq numbers are counting independently for different PIDs,
they must be data packets.


Careful with this assumption - the continuity counter/sequence number is just 4 bits. It is not special nor immune to noise than any other four bits in the header or data. Yes, including it in the match will increase the chances of getting the right PID, but it's not a magic bullet in itself.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/13/2014 01:18 pm
Careful with this assumption - the continuity counter/sequence number is just 4 bits. It is not special nor immune to noise than any other four bits in the header or data. Yes, including it in the match will increase the chances of getting the right PID, but it's not a magic bullet in itself.

The info is not in the 4 bits of an individual counter, but the sequence of thousands of them. The question is whether including completely garbled packets does more good or harm.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/13/2014 01:34 pm
Careful with this assumption - the continuity counter/sequence number is just 4 bits. It is not special nor immune to noise than any other four bits in the header or data. Yes, including it in the match will increase the chances of getting the right PID, but it's not a magic bullet in itself.
I don't think he's trying to say that, but rather merely pointing out that there's a second dimension from which the correct header can be derived, kind of like a macroblock referring to the one above or to the left. There's only four PIDs in the stream, and only two which occur frequently (0x03e8 and 0x1fff), so even with a completely trashed header like 0x5060F71, pulling these known quantities together you can figure out with a very high level of confidence that it's supposed to be 0x4703E814.

Of course, it might be 0x34, not 0x14, if it's an adaptation packet, but there's also context available to figure that out too - such as if the next byte after the header is greater than 184 (0xB8) since that would mean the adaptation field is longer than the rest of the packet and so either that byte is wrong or the 0x34 should be 0x14.

Now whether the likely completely trashed payload that goes with that completely trashed header will be of any use is another question, but at least we have the payload.

Of course this doesn't help much if you go too far beyond 16 packets in either direction (3008 bytes), but I don't think we've seen that much contiguous unrecognizable data anywhere in the transport stream.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/13/2014 01:39 pm
I updated frame 131 (and can finally make .png as output):

0:0:-1,
7:2:5895,4:7:-1,
11:7:20490,26:14:-1,
18:15:56376,13:16:-1,
14:16:62505,20:16:-1,
21:16:64068,27:16:-2,28:16:-1,
32:16:65707,25:20:-1,
14:21:92080,24:25:-1,
14:26:117098,41:26:-1,
14:27:120887,10:29:-1

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/13/2014 02:35 pm
Here are the best results on the first 2 I-frames IF0 and IF 0.5 and the ts-file where they came from:

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Okie_Steve on 05/13/2014 02:37 pm
As you see, every packet can be identified in this example, despite
some headers being shot to hell.
Interesting approach. Do you see more 0->1 or 1->0 bit flips are they about equal? If non-random then the people doing genetic bit flips might want to bias their attempts.

Also, might be interesting to see an "XOR" file the same size as the input that could be applied to generate the "repaired" headers if someone wants to look for patterns and apply them to the data too.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/13/2014 04:50 pm
@Michael Niedermayer:

I've been investigating the possibility of automatically setting all L1,L2,L3,L4,C1,C2 values for all macroblocks for a frame. Basicly inputting an mmb and letting a script detect "bad lines" in the picture. Moving from the right-bottom corner to the upper-top.

I'm starting to believe (using some extra logging and some manual experimentation) that this might actually be possible. If it does this would improve the quality of the frames quite dramatically and would save a lot of manual work!

However there is one thing that I find difficult to do myself. And I think you might be able to help us. Instead of using this kind of setting:

15:14:-1:0:-40:0:0:0:20

where I replace the current macroblock with a blank one and change the DC values.

But what I really want to do is this:

15:14:57217:0:-40:0:0:0:20

In other words: I want to keep the macroblock with its (structured) data as much as I can (at the very least the blocks in it which are set to 0 above) but change only the DC "starting" values. So not blanken it, but modifying it so these DC-values make sure propogation to other blocks works the same as now with -1 (therefore fixing the color and intensity of following blocks) but I also keep the current blocks inside the current macroblock.

Is something like that possible?

Regards,

arnezami
Hi guys,

I've created a new feature that implements the above.

Simply put you can now do something like this:

15:14:57217:0:-40:0:0:0:20

This will keep the macroblock (so not nullify it) but it will change the DC values. With this you can really fix the lum/chrom issues in the current I-frames. Should have a big effect. I hope.

[edit] Forgot: it now also logs the 6 DC values for each macroblock. Which can come in handy in combination with this new feature.

Also you can do this now:

15:14:57217:0:0:0:0:0:0:0
15:14:57217:0:0:0:0:0:0:63

What this does is change direction from where a DC prediction are coming/inherited from: left or top. For each 1-bit in this number it tells it to get the DC-prediction from the top for a specific DC (4 x lum, 2 x chrom). A 0-bit means left. The number consists of 6 bit (hence max is 63). So in effect a 0 is all left, a 63 is all top. I don't know yet whether this feature is going to be very useful but it can't hurt.

I've attached the 3 source files that were changed. If somebody can check if it doesn't conflict with the existing branch (and maybe test it) and then push it to github, that would be great. Its these files btw:

libavcodec/mpegvideo.h
libavcodec/mpeg4video.h
libavcodec/h263dec.c

[edit] Fixed a bug: when using -2 it would get stuck in that mode. zip re-uploaded.

I need some sleep now ;)

Regards,

arnezami

PS. I was also trying to make a reverse-macroblock finder of sorts. So instead of telling its starting bitposition you would tell it its ending bitposition (which would usually be a starting bitposition of a mb you already have). But I ran into a lot of issues. Even when you try all possible DC-directions it still won't find the right starting position. Even when I fake the right DC values left to the block. I'm not sure why yet. Probably something I don't understand yet. This part of the code is in there but its commented out. It was also extremely error-log-heavy...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/13/2014 05:28 pm
Hi guys,

I just noticed that there are no B-frames in the file. Only I-frames and P-frames.

Quote
./ffprobe try1.ts -show_frames > frames.txt

Not sure yet what the implications are yet. Hopefully it will get easier that way.

Regards,

arnezami

That is very fortunate happenstance.  P frames will only have compound errors from any damage from the frames before them in the sequence.  B frames would have compound errors from both before and after them in the sequence.

Not really that 'fortunate' - there are very few if any real time encoders of the sort that would be used here that use B-frames. They need to encode in near real time, which means basing frames on frames that haven't yet appeared doesn't really work.  B-frames tends to work best when doing offline encoding. It does give a better compression ratio, but is more computationally expensive.

I am trying to recreate the full movie using the I-frames, a few "nice-looking" P-frames and fill the rest with empty P-frames. How can I convert a png into a P-frame? Or should I create the empty P-frames differently?

For the I-frames, I use for example:
ffmpeg -i frame_131.png -s 704x480 -aspect 22:15 -f image2 -r 44999/3003 frame131.mpg4-img
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/13/2014 06:47 pm
Careful with this assumption - the continuity counter/sequence number is just 4 bits. It is not special nor immune to noise than any other four bits in the header or data. Yes, including it in the match will increase the chances of getting the right PID, but it's not a magic bullet in itself.
I don't think he's trying to say that, but rather merely pointing out that there's a second dimension from which the correct header can be derived, kind of like a macroblock referring to the one above or to the left. There's only four PIDs in the stream, and only two which occur frequently (0x03e8 and 0x1fff), so even with a completely trashed header like 0x5060F71, pulling these known quantities together you can figure out with a very high level of confidence that it's supposed to be 0x4703E814.

Of course, it might be 0x34, not 0x14, if it's an adaptation packet, but there's also context available to figure that out too - such as if the next byte after the header is greater than 184 (0xB8) since that would mean the adaptation field is longer than the rest of the packet and so either that byte is wrong or the 0x34 should be 0x14.

Now whether the likely completely trashed payload that goes with that completely trashed header will be of any use is another question, but at least we have the payload.

Of course this doesn't help much if you go too far beyond 16 packets in either direction (3008 bytes), but I don't think we've seen that much contiguous unrecognizable data anywhere in the transport stream.

Yes, and if null packets occurred one at a time (or even N at a time, N in known set) every ~M data packets, this would be useful, and I agree that looking at any particular segment you might believe you have recovered a header based on the counter, but it adds much less value than you might hope. The problem is that null packets do not occur in a predictable way. I ran some stats to illustrate..

The first graph shows the frequency of such runs vs their length. There is obviously some outliers resulting from problematic data-packet matching, but the variance between 1 and 32 looks real enough to me. Strings of 1 occur often, but percentage-wise (third graph) they only constitute 2.58% of null packets.

The second graph shows the distance between null packets. Excluding the case where d=0 (i.e. a continuous string per above, 5103 counts) and presuming that d=1..3 (120 counts) might be d=0 with over-opomistic data-packet identification, there is still a huge variance in the potential number of data packets before the next null packet.

The third graph is the most important, since it answers the question "suppose I know for sure I have N null packets, what is the chance that the next packet is also a null packet". The chance is pretty even to about N=38, meaning that the 16 values offered by the continuity counter could alias twice with equal probability.

That said, I did implement the counter into the packet-matcher data (see attached). As I expected, it does not seem to have made much of a difference.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/13/2014 07:32 pm
I am trying to recreate the full movie using the I-frames, a few "nice-looking" P-frames and fill the rest with empty P-frames. How can I convert a png into a P-frame? Or should I create the empty P-frames differently?

For the I-frames, I use for example:
ffmpeg -i frame_131.png -s 704x480 -aspect 22:15 -f image2 -r 44999/3003 frame131.mpg4-img

When you convert the p-frames to PNGs, some information about what to do with the blocks is lost. I don't think it is possible to reassemble the movie that way.

I think probably mlindner's idea to add a frame specifier to the mmb parameter is going to work best for contiguous sequences of i- and p-frames. In cases where we are missing p-frames, there is going to need to be some manual interpolation and reassembly.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/13/2014 07:39 pm
Regarding the update of the ffmpeg:

I just fixed a bug: "when using -2 it would get stuck in that mode. zip re-uploaded".

Please re-download the new one above.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/13/2014 08:08 pm
Could pframes be used to help fine tune iframes? If you start with a high quality iframe like 6, and then look for spots where the pframes indicate no change, could you simply carry the block from the good iframe forward (or backward) or does the compression in the iframe make that impractical?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/13/2014 08:20 pm
Hi guys,

I just noticed that there are no B-frames in the file. Only I-frames and P-frames.

Quote
./ffprobe try1.ts -show_frames > frames.txt

Not sure yet what the implications are yet. Hopefully it will get easier that way.

Regards,

arnezami

That is very fortunate happenstance.  P frames will only have compound errors from any damage from the frames before them in the sequence.  B frames would have compound errors from both before and after them in the sequence.

Not really that 'fortunate' - there are very few if any real time encoders of the sort that would be used here that use B-frames. They need to encode in near real time, which means basing frames on frames that haven't yet appeared doesn't really work.  B-frames tends to work best when doing offline encoding. It does give a better compression ratio, but is more computationally expensive.

I am trying to recreate the full movie using the I-frames, a few "nice-looking" P-frames and fill the rest with empty P-frames. How can I convert a png into a P-frame? Or should I create the empty P-frames differently?

For the I-frames, I use for example:
ffmpeg -i frame_131.png -s 704x480 -aspect 22:15 -f image2 -r 44999/3003 frame131.mpg4-img

How does it look with just the iframes? care to join us in the IRC channel?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/13/2014 08:49 pm
Hi guys,

I just noticed that there are no B-frames in the file. Only I-frames and P-frames.

Quote
./ffprobe try1.ts -show_frames > frames.txt

Not sure yet what the implications are yet. Hopefully it will get easier that way.

Regards,

arnezami

That is very fortunate happenstance.  P frames will only have compound errors from any damage from the frames before them in the sequence.  B frames would have compound errors from both before and after them in the sequence.

Not really that 'fortunate' - there are very few if any real time encoders of the sort that would be used here that use B-frames. They need to encode in near real time, which means basing frames on frames that haven't yet appeared doesn't really work.  B-frames tends to work best when doing offline encoding. It does give a better compression ratio, but is more computationally expensive.

I am trying to recreate the full movie using the I-frames, a few "nice-looking" P-frames and fill the rest with empty P-frames. How can I convert a png into a P-frame? Or should I create the empty P-frames differently?

For the I-frames, I use for example:
ffmpeg -i frame_131.png -s 704x480 -aspect 22:15 -f image2 -r 44999/3003 frame131.mpg4-img

How does it look with just the iframes? care to join us in the IRC channel?

So I created empty P-frames with an hex editor (just taking a valid P-frame and replacing everything except the 4 first bytes with FF, somehow it seems it worked ;))

I used then the I-frames, the empty P-frames and the following nearly intact P-frames (fixed.ts): 135, 172, 188, 199, 225, 233, 251

And finally created the movie with: ffmpeg -framerate 44999/3003 -i frame%d.mpg4-img -vcodec copy fixed_ts.avi

Not really better than the gif, but it's a beginning ;)

Does someone work on a way to edit the P-frames?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/13/2014 09:06 pm
Could pframes be used to help fine tune iframes? If you start with a high quality iframe like 6, and then look for spots where the pframes indicate no change, could you simply carry the block from the good iframe forward (or backward) or does the compression in the iframe make that impractical?

You could in theory carry blocks forward from one i-frame to the next i-frame. However you would need a good decode for that block in every one of the ~19 p-frames in between. I think we are unlikely to see that in practice.

The p-frames also contain block updates, so you could carry those forward to update the following i-frame. That doesn't help much either, because the blocks that are most likely to get updates are the most dynamic. There are lots of updates of the water which is either spinning or full of smoke, spray, or waves. Not so many updates of the rocket body which is most in need of "filling in". Up to iframe 11, at least. This method might be useful for reconstructing some of the water after splashdown.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/13/2014 09:32 pm
Been talking to SpaceX today about things. Got another awesome from them on this effort, so that's for you guys.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/13/2014 09:42 pm
My thought was to deal with what I presume is low-hanging fruit: blocks with no or virtually no change from one iframe to the next. If there's no updates, does that reduce the pframe coding length enough that it'd be more likely to have dodged corruption?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/13/2014 09:55 pm
My thought was to deal with what I presume is low-hanging fruit: blocks with no or virtually no change from one iframe to the next.

Do P-frame blocks within a given frame propagate like I-frame blocks? Or is it strictly a diff from the same MB?

If we can create fake P-frames with no change, then fill in the data changes one at a time, we might make faster progress. But if they are interdependent you'd need to guess C/L data for each which could get annoying fast (and give people creative license to make the video into whatever)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/13/2014 09:58 pm
My thought was to deal with what I presume is low-hanging fruit: blocks with no or virtually no change from one iframe to the next. If there's no updates, does that reduce the pframe coding length enough that it'd be more likely to have dodged corruption?

Heh. It's a rocket landing. With a hover-slam. There are virtually no blocks with no change from one frame to the next.

The pframes seem to be an all-or-nothing deal as far as corruption goes, as you might expect. Some of them are quite good, but a lot of them are really bad.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/13/2014 10:01 pm
Do P-frame blocks within a given frame propagate like I-frame blocks? Or is it strictly a diff from the same MB?

If we can create fake P-frames with no change, then fill in the data changes one at a time, we might make faster progress. But if they are interdependent you'd need to guess C/L data for each which could get annoying fast (and give people creative license to make the video into whatever)

No, they do not seem to suffer from the same error propagation problems as the i-frames.

Edit: Actually, some of them have groups of block updates which can propagate errors. It will be much easier to fix, though, because there are fewer opportunities for errors to get compounded.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/14/2014 10:19 am
Actually we can edit the P-frames the same as the I-frames, the generated PNG image then looks good, but I cannot manage to write them back as P-frames, also when using something like this:

ffmpeg -s:0 704:480 -i frameXXX.mpg4-img -f image2 frameXXX_modified.mpg4-img

I hoped it would just copy the file, but it automatically converts the data into an I-frame it seems.

I also tried to use -f mp4 or to add -vf '[out]select=eq(pict_type\,P)', with no more success.

Does someone know how to force ffmpeg to write a P-frame?

Edit:

I finally managed to do it with:
ffmpeg -s:0 704:480 -i frameXXX.mpg4-img -vcodec copy -an -f image2 frameXXX_mod.mpg4-img

Edit(2):

It copies the P-frame but does not handle the -mmb ...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/14/2014 01:32 pm
Hi All,

Just a little update on the Wiki.

The current table with iFrames is a bit of a mess, as it has frames extracted from three different sources (coerced.ts from arnezami, try1.ts from mlindner and -until minutes ago- edit5.ts from Shanuson).

The most recent version (edit5.ts) is going to have the complete set of  15 workable IFrames. In order to organize things a bit, I created a separate page on the wiki to focus on these, hoping  these will be the definitive iframes. You can access from the main page, or directly here http://spacexlanding.wikispaces.com/Progress+on+edit5.ts . The previous table still has all the edits on frames from coerced.ts  and try1.

As of today, there is only one iframe in the new page (the one of frame #14 in the video) as its the only one that is completely fixed and ready to be edited. If anyone has some time available, please work on that one.  As other iframes become available (Shanuson is working on some), they will start appearing there. The online editor 2 will also have an updated version of those iframes for those that are using that tool.

And important note regarding backward compatibility. The new set of iframes includes previously unseen frames, but also some that are  equivalent to those we have already fixed. However, due to changes in iframe sizes, positions of MBs will change. This means that for existing iframes, previous -mmb codes will be only be partially applicable.  Keep this in mind as some .mmb codes will have to get reworked.

Thanks everyone for the good work!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/14/2014 04:18 pm
The online editor 2 will also have an updated version of those iframes for those that are using that tool.

Just to add on this: Big thanks to Iain for implementing some pretty cool new features in http://spacex.slapbet.org

- MB Errors (from the log) are now shown next to the MB X:Y display above the image when an MB is clicked. This includes DC clipping, dquant, ibpc, ac-tex, and stuffing (though I'm not even sure what they all mean ;) ).

- Total number of MB with errors are show on the right. It would be expected that a good -mbm would reduce the total number of MB with errors by more than one. This also allows a quick check to see which of two similar MBM strings is "better"

- Checkbox to enable highlighting MB with errors. This provides invaluable visual assistance in identifying where problems start. It would have been almost impossible to get this clear of a picture just looking at the log. It is interesting to watch how blocks with problems don't always look like it, and how visually defective blocks don't have any errors. This might drastically simplify debugging mbms.

So if you are using Online Editor V2 and haven't refreshed the page in a while, give it a F5. Great work Iain!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/14/2014 07:28 pm
So I was building the latest FFMpeg for windows from here https://github.com/michaelni/FFmpeg/tree/spacexdebug1

I managed to actually get it working (yay for me) but when I was testing it I found that the output for the same image with no -mmb applied was different to the previous version of ffmpeg I was using.

OLD: http://i.imgur.com/CeHOIrB.png
NEW: http://i.imgur.com/sMInZkY.png

The output seems a bit cleaner but I don't know why it's different, if anyone knows why this is, or more importantly if it's actually expected and OK, then I'll deploy it to the Online Editor to include the latest functionality.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/14/2014 08:09 pm
So I was building the latest FFMpeg for windows from here https://github.com/michaelni/FFmpeg/tree/spacexdebug1

I managed to actually get it working (yay for me) but when I was testing it I found that the output for the same image with no -mmb applied was different to the previous version of ffmpeg I was using.

OLD: http://i.imgur.com/CeHOIrB.png
NEW: http://i.imgur.com/sMInZkY.png

The output seems a bit cleaner but I don't know why it's different, if anyone knows why this is, or more importantly if it's actually expected and OK, then I'll deploy it to the Online Editor to include the latest functionality.
I'm not sure what version you were on but I also noticed some positive changes a few days ago when I updated. I think somebody did some bugfixes/improvements. Very similar to yours.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/14/2014 09:22 pm
Cool, I put the latest ffmpeg on the image handler the editor uses. If someone can summarise what the changes to the editor itself should be (I think it's an additional parameter fr the macroblock operation but I don't know what it's for) then I can add it :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/14/2014 09:36 pm
The output seems a bit cleaner but I don't know why it's different, if anyone knows why this is, or more importantly if it's actually expected and OK, then I'll deploy it to the Online Editor to include the latest functionality.

Iain, the new version of that frame you're seeing was extracted from a cleaned-up version of the transport stream, where errors in the transport stream packet identifiers, flag bits, and sequence numbers were corrected in order to allow more 184-byte pieces of the MPEG data to be incorporated. If you review from about page 38, that's where Shanuson posts the first pass at the extracted images from the error-corrected transport stream, and prior pages are where we were discussing how to get from point A to pont B (or point 0x03E8) with the protocol.

So yes, it's expected and OK, and you can go ahead and deploy.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/14/2014 09:36 pm
Only I ran the new ffmpeg on the old images not the new ones :/

Anyway it's deployed, if it's wrong (I can't tell) I can revert easily enough.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Joffan on 05/14/2014 10:15 pm
Only I ran the new ffmpeg on the old images not the new ones :/

Anyway it's deployed, if it's wrong (I can't tell) I can revert easily enough.

Hi Iain
Interface request for http://spacexlanding.wikispaces.com/Online+Editor : can you set it so that when the user clicks on a block in the image, the relevant part of the log is shown? and maybe fill in the existing pos value in the edit window (if it's a previously unspecified block).
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/14/2014 10:20 pm
Which bits of the log do you need? There's already quite a bit there above the image.

As for auto adding pos, currently, because of the way it's built, that would end up adding a bunch of effectively useless parts to the mmb (i.e. if you just click around and don't actually do anything) There might be a way round this, I'll have a think tomorrow when I get back from work =)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/14/2014 10:20 pm
@Michael Niedermayer:

I've been investigating the possibility of automatically setting all L1,L2,L3,L4,C1,C2 values for all macroblocks for a frame. Basicly inputting an mmb and letting a script detect "bad lines" in the picture. Moving from the right-bottom corner to the upper-top.

I'm starting to believe (using some extra logging and some manual experimentation) that this might actually be possible. If it does this would improve the quality of the frames quite dramatically and would save a lot of manual work!

However there is one thing that I find difficult to do myself. And I think you might be able to help us. Instead of using this kind of setting:

15:14:-1:0:-40:0:0:0:20

where I replace the current macroblock with a blank one and change the DC values.

But what I really want to do is this:

15:14:57217:0:-40:0:0:0:20

In other words: I want to keep the macroblock with its (structured) data as much as I can (at the very least the blocks in it which are set to 0 above) but change only the DC "starting" values. So not blanken it, but modifying it so these DC-values make sure propogation to other blocks works the same as now with -1 (therefore fixing the color and intensity of following blocks) but I also keep the current blocks inside the current macroblock.

Is something like that possible?

Regards,

arnezami
Hi guys,

I've created a new feature that implements the above.

Simply put you can now do something like this:

15:14:57217:0:-40:0:0:0:20

This will keep the macroblock (so not nullify it) bit it will change the DC values. With this you can really fix the lum/chrom issues in the current I-frames. Should have a big effect. I hope.

[edit] Forgot: it now also logs the 6 DC values for each macroblock. Which can come in handy in combination with this new feature.

Also you can do this now:

15:14:57217:0:0:0:0:0:0:0
15:14:57217:0:0:0:0:0:0:63

What this does is change direction from where a DC prediction are coming/inherited from: left or top. For each 1-bit in this number it tells it to get the DC-prediction from the top for a specific DC (4 x lum, 2 x chrom). A 0-bit means left. The number consists of 6 bit (hence max is 63). So in effect a 0 is all left, a 63 is all top. I don't know yet whether this feature is going to be very useful but it can't hurt.

I've attached the 3 source files that were changed. If somebody can check if it doesn't conflict with the existing branch (and maybe test it) and then push it to github, that would be great. Its these files btw:

libavcodec/mpegvideo.h
libavcodec/mpeg4video.h
libavcodec/h263dec.c

[edit] Fixed a bug: when using -2 it would get stuck in that mode. zip re-uploaded.

I need some sleep now ;)

Regards,

arnezami

PS. I was also trying to make a reverse-macroblock finder of sorts. So instead of telling its starting bitposition you would tell it its ending bitposition (which would usually be a starting bitposition of a mb you already have). But I ran into a lot of issues. Even when you try all possible DC-directions it still won't find the right starting position. Even when I fake the right DC values left to the block. I'm not sure why yet. Probably something I don't understand yet. This part of the code is in there but its commented out. It was also extremely error-log-heavy...

Iain, is it possible to add this in the online editor interface? It is quite useful :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/14/2014 10:25 pm
Yup, I'll look at that tomorrow too.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Joffan on 05/14/2014 10:28 pm
Which bits of the log do you need? There's already quite a bit there above the image.

As for auto adding pos, currently, because of the way it's built, that would end up adding a bunch of effectively useless parts to the mmb (i.e. if you just click around and don't actually do anything) There might be a way round this, I'll have a think tomorrow when I get back from work =)

I meant the scrolling image interpretation log below the image - can you position that to show the relevant block as maybe the third line in the window? And actually now you point out the line above the image (which I had been looking for and never found for some reason), that has the current position of the block anyway so it's fairly easy to cut and paste across.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/15/2014 03:36 am
Which bits of the log do you need? There's already quite a bit there above the image.

As for auto adding pos, currently, because of the way it's built, that would end up adding a bunch of effectively useless parts to the mmb (i.e. if you just click around and don't actually do anything) There might be a way round this, I'll have a think tomorrow when I get back from work =)

I meant the scrolling image interpretation log below the image - can you position that to show the relevant block as maybe the third line in the window? And actually now you point out the line above the image (which I had been looking for and never found for some reason), that has the current position of the block anyway so it's fairly easy to cut and paste across.

I suspect you hadn't seen it because it was added yesterday :) I think this change (the extra info at the top) is better than scrolling the log, right? Or do you still see utility in that? This might actually be relevant with the new dc value logging (see edit2).


IainCole: Since the MB already auto-populates the mmb field, I think he's referring to the byte offset of the MB into the "Invert Bits" box. In your code, that's var pos = parseInt(match[4]); near line 714. It could just fill in a new invert bits with a mask of 0.

Edit: I don't know about utility, but it would also be fun to have 8 checkboxes rather than typing in the hex. It would make it easier for non-software people to visualize. Maybe a shift left/right/invert button. I can code it if you're busy

Edit2: I didn't think that it was actually possible to scroll a textarea in js but apparently it is http://features.sheep.art.pl/AutoscrolledTextarea , though due to interop issues it's non-trivial. The trick would be to append one line at a time, recording the length of the string when you hit a mb_pos_info and storing it with the rest of the MB data. 
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/15/2014 04:10 am
The online editor 2 will also have an updated version of those iframes for those that are using that tool.

Just to add on this: Big thanks to Iain for implementing some pretty cool new features in http://spacex.slapbet.org

- MB Errors (from the log) are now shown next to the MB X:Y display above the image when an MB is clicked. This includes DC clipping, dquant, ibpc, ac-tex, and stuffing (though I'm not even sure what they all mean ;) ).

...

Just to follow up with this, michaelni gave a nice description of each error on the IRC:

Quote from: michaelni
dquant means that a MB that changes the QP has occured, thats no error but possibly is not used by the encoder which in that case could be used to detect errors

ac-tex damaged means that something went wrong during decoding of the ac coefficients, for example the sum of runs being larger than the space in a 8x8 block or a invalid vlc

I cbpy damaged means that where the cbpy is stored there was a bit sequence that maps to no valid intra cbpy

marker bit missing in X. esc means that a bit that is required to be 1 in the ac coefficient escape coding was not 1

stuffing is a method used to increase the bitrate, aka to stuff (useless) bits in the stream, its allowed in mpeg4 but i suspect the encoder doesnt use it so it could be used to identify errors. That is, one MB type is for stuffing and when that occurs the decoder throws it away and reads the mb type again until it finds a non stuffing one, that way the bitstream can be filled up to reach some bitrate. So like the null TS packets, but at a MB level, probably meaning that the MB type is corrupt
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: michaelni on 05/15/2014 04:50 am
Hi guys,

I've created a new feature that implements the above.

Simply put you can now do something like this:

15:14:57217:0:-40:0:0:0:20

This will keep the macroblock (so not nullify it) bit it will change the DC values. With this you can really fix the lum/chrom issues in the current I-frames. Should have a big effect. I hope.

[edit] Forgot: it now also logs the 6 DC values for each macroblock. Which can come in handy in combination with this new feature.

Also you can do this now:

15:14:57217:0:0:0:0:0:0:0
15:14:57217:0:0:0:0:0:0:63

What this does is change direction from where a DC prediction are coming/inherited from: left or top. For each 1-bit in this number it tells it to get the DC-prediction from the top for a specific DC (4 x lum, 2 x chrom). A 0-bit means left. The number consists of 6 bit (hence max is 63). So in effect a 0 is all left, a 63 is all top. I don't know yet whether this feature is going to be very useful but it can't hurt.

I've attached the 3 source files that were changed. If somebody can check if it doesn't conflict with the existing branch (and maybe test it) and then push it to github, that would be great. Its these files btw:

libavcodec/mpegvideo.h
libavcodec/mpeg4video.h
libavcodec/h263dec.c

pushed changes  into github  (except what was commented out)

thanks
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: michaelni on 05/15/2014 05:10 am
My thought was to deal with what I presume is low-hanging fruit: blocks with no or virtually no change from one iframe to the next.

Do P-frame blocks within a given frame propagate like I-frame blocks? Or is it strictly a diff from the same MB?

If we can create fake P-frames with no change, then fill in the data changes one at a time, we might make faster progress. But if they are interdependent you'd need to guess C/L data for each which could get annoying fast (and give people creative license to make the video into whatever)

P- frames can contain both intra MBs (like I frames) and inter MBs, each inter MB has 1(common) or 4(not so common) motion vector(s) that points to some location in the previous frame from where data is copied, such MB then stores the difference to that previous area. MVs are predicted somewhat similar to DC values of intra MBs, allthough MVs use the left, top and top right MBs for prediction. You can visualize the motion vectors with -vismv 7
MVs are in half pixel units
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/15/2014 08:53 am
PS. I was also trying to make a reverse-macroblock finder of sorts. So instead of telling its starting bitposition you would tell it its ending bitposition (which would usually be a starting bitposition of a mb you already have). But I ran into a lot of issues. Even when you try all possible DC-directions it still won't find the right starting position. Even when I fake the right DC values left to the block. I'm not sure why yet. Probably something I don't understand yet. This part of the code is in there but its commented out. It was also extremely error-log-heavy...
Good news: I finally got the reverse searching of macroblocks to work!  8)

It's still quite hacky right now but hopefully I can release some working code within a day or so.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/15/2014 10:14 am
Hi Guys,

Here is my by-hand-aligned raw.ts, and all iframes after piping it through tsfix (tsalign still complains about one region but that's an misreport. Compare that part to raw.ts and you will see that all 47's are there and at the right position in that part. so don't use tsalign my that ts.)
I will update  http://spacexlanding.wikispaces.com/Progress+on+edit5.ts  (http://spacexlanding.wikispaces.com/Progress+on+edit5.ts) after this post.
But i have to go back to do my original work, So this will probably be my last contribution. 
I used the continuity bit and my best judgement to edit raw.ts to get that outcome.
There was on 56byte missing part where i am not 100% sure I add the 56 FF at the right position, I was a gap of 5 188 blocks with one missing 56 bytes. I added the 56 byte to the last one and marked it by 12 34 56 78 for the first 4 of those 56 Bytes.

Have fun with it :D

Cheers Shanuson
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/15/2014 10:18 am
And here a Zip with all img-files and the raw_edit8.ts
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/15/2014 10:37 am
An here is the rest of the frames.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/15/2014 12:41 pm
Amazing job! The wiki page with these net iframes is up and working :)
Looking forward to see your edits
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mhenderson on 05/15/2014 02:39 pm
Progress on old iFrame 10 / frame 209 in edit5.ts.

X:14630:01,X:14708:01,X:15737:01,X:15817:1,X:15843:1,X:17180:1,43:2:8818,2:3:
8916,5:5:-2:16:25:25:16,6:5:14708,18:5:-1:-1:1:1:
-1:0:0,21:5:-1:-4:1:-5:-5,23:5:-1:-2:3:-4:2:0:0,25:5:16467,31:5:-1:-5:0:0:-5:
0:0,32:5:16740,37:5:-2:0:0:-5:-5:0:0,0:6:-1,0:7:-1,21:7:21000,0:15:-2,7:17:55313

This is basically back to where it was before the switch to edit5.ts. Some improvements in row 5.

I use the online v2 editor. Thanks so much Iain.  :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/15/2014 02:59 pm
Remember to break up your code lines or the width of the thread goes haywire.

And a welcome to mhenderson! :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/15/2014 03:21 pm
And here a Zip with all img-files and the raw_edit8.ts

Thank you for your work!

Two questions:

How can I get the "fixed_edit8.ts" version? I read about the "tsfix.c" and "tsalign.c" and the command line "cat raw.ts | tsfix | tsalign > fixed.ts", do I have to compile the files and how? For now the command line does not work. I found the files in the "FFmpeg-spacexdebug1/tools" folder.

When directly extracting the frames from raw_edit8.ts, I get only 241 frames, is it normal?

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/15/2014 03:32 pm
And here a Zip with all img-files and the raw_edit8.ts

Thank you for your work!

Two questions:

How can I get the "fixed_edit8.ts" version? I read about the "tsfix.c" and "tsalign.c" and the command line "cat raw.ts | tsfix | tsalign > fixed.ts", do I have to compile the files and how? For now the command line does not work. I found the files in the "FFmpeg-spacexdebug1/tools" folder.

When directly extracting the frames from raw_edit8.ts, I get only 241 frames, is it normal?

yes raw_edit8.ts only gives 241 frames or so. that why you need to use tsfix too.

simply do gcc -o tsfix tsfix.c and likewise for tsalign. After that only do cat raw_edit8.ts | tsfix > fixed_edit8.ts
and dont use tsalign on raw_edit8.ts

Cheers
Shanuson
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/15/2014 04:00 pm
And here a Zip with all img-files and the raw_edit8.ts

Thank you for your work!

Two questions:

How can I get the "fixed_edit8.ts" version? I read about the "tsfix.c" and "tsalign.c" and the command line "cat raw.ts | tsfix | tsalign > fixed.ts", do I have to compile the files and how? For now the command line does not work. I found the files in the "FFmpeg-spacexdebug1/tools" folder.

When directly extracting the frames from raw_edit8.ts, I get only 241 frames, is it normal?

yes raw_edit8.ts only gives 241 frames or so. that why you need to use tsfix too.

simply do gcc -o tsfix tsfix.c and likewise for tsalign. After that only do cat raw_edit8.ts | tsfix > fixed_edit8.ts
and dont use tsalign on raw_edit8.ts

Cheers
Shanuson

Now I can run it, but it only creates a 32 bytes file... I made a print screen of the command prompt output. Is it possible to post your fixed_edit8.ts file?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/15/2014 04:15 pm
Now I can run it, but it only creates a 32 bytes file... I made a print screen of the command prompt output. Is it possible to post your fixed_edit8.ts file?

Are you on windows? The 'cat' command is *nix, IIRC the equivalent on windows is 'type'. So type raw_edit8.ts | tsfix > fixed_edit8.ts

Also, tsfix < raw_edit8.ts > fixed_edit8.ts might work better.

 (disclaimer: I don't have Windows)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/15/2014 04:23 pm

yes raw_edit8.ts only gives 241 frames or so. that why you need to use tsfix too.

simply do gcc -o tsfix tsfix.c and likewise for tsalign. After that only do cat raw_edit8.ts | tsfix > fixed_edit8.ts
and dont use tsalign on raw_edit8.ts

Cheers
Shanuson

Now I can run it, but it only creates a 32 bytes file... I made a print screen of the command prompt output. Is it possible to post your fixed_edit8.ts file?

I can do when i am back at home.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/15/2014 04:28 pm
Yes I am on windows (using MinGW).

So I tried again your propositions and got the same as before, I compiled tsfix.c with Visual Studio and also got the same, and finally logged to a Unix machine and managed to do it there.

So I finally got all the frames, thank you for the help! :)

edit: I put the file in attachment
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Untribium on 05/15/2014 04:42 pm
Some quick work on frame 52/old iframe 2. The bit flips are pretty much random...
X:25254:1,X:25299:1,X:53423:1,40:3:-1,6:4:9364,4:5:-1,6:5:11443,7:5:11487,0:7:-1,28:11:25250
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/15/2014 04:43 pm
So I adapted iframe32 using the new options with:

X:57182:4,
40:0:-1,
17:1:5444,13:2:-1,
5:4:14914,43:4:-1,
7:5:19298,14:06:24261:0:0:0:0:0:0:0,23:11:45354:0:0:0:0:0:0:0,26:12:-1,
7:13:52167,7:14:-1,
17:15:62855,28:15:-1,
27:16:68962,21:17:72509:0:0:0:0:0:0:0,21:18:76689:0:0:0:0:0:0:0,24:18:76921:0:0:0:0:0:0:0,7:21:-1,
10:21:-1:-5:10:0,
11:21:-1,
20:21:90326:10:0:0:0,22:27:-1,
27:27:112406,0:28:-1,
3:28:115143,9:29:-1,
33:29:135545
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/15/2014 04:52 pm
So back to the P-frames...

So this is an example of a P-frame (frame192). I ploted the motion vectors to see if they are also correctly modified. I used the following command lines for both images:

ffmpeg -debug mb_pos_size -err_detect ignore_err -vismv 7 -s:0 704:480 -i frame192.mpg4-img -f image2 frame_192_mv.png
ffmpeg -mmb 2:0:-1,19:0:1092 -debug mb_pos_size -err_detect ignore_err -vismv 7 -s:0 704:480 -i frame192.mpg4-img -f image2 frame_192_mv_mod.png

Now instead of creating a png, is it possible to directly store the modified one as a P-frame? If yes, how? Or does something have to be implemented?

When using:
ffmpeg -mmb 2:0:-1,19:0:1092 -debug mb_pos_size -err_detect ignore_err -s:0 704:480 -i frame192.mpg4-img -f image2 frame192_mod.mpg4-img
it stores an I-frame (we can check the frame type with option -vf 'showinfo')

When using:
ffmpeg -mmb 2:0:-1,19:0:1092 -debug mb_pos_size -err_detect ignore_err -s:0 704:480 -i frame192.mpg4-img -vodec copy -an -f image2 frame192_mod.mpg4-img
it just copies the P-frame without modifying it.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/15/2014 06:17 pm
So I adapted iframe32 using the new options with:

X:57182:4,
40:0:-1,
17:1:5444,13:2:-1,
5:4:14914,43:4:-1,
7:5:19298,14:06:24261:0:0:0:0:0:0:0,23:11:45354:0:0:0:0:0:0:0,26:12:-1,
7:13:52167,7:14:-1,
17:15:62855,28:15:-1,
27:16:68962,21:17:72509:0:0:0:0:0:0:0,21:18:76689:0:0:0:0:0:0:0,24:18:76921:0:0:0:0:0:0:0,7:21:-1,
10:21:-1:-5:10:0,
11:21:-1,
20:21:90326:10:0:0:0,22:27:-1,
27:27:112406,0:28:-1,
3:28:115143,9:29:-1,
33:29:135545
Just to be sure: the extra six 0's (0:0:0:0:0:0) won't do anything here I believe. The 6 numbers are only needed if you fill in at least one non-zero number. They change the lum/chrom by the number. They don't set it to 0.

Just in case I wasn't very clear about that.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/15/2014 06:23 pm
So I adapted iframe32 using the new options with:

X:57182:4,
40:0:-1,
17:1:5444,13:2:-1,
5:4:14914,43:4:-1,
7:5:19298,14:06:24261:0:0:0:0:0:0:0,23:11:45354:0:0:0:0:0:0:0,26:12:-1,
7:13:52167,7:14:-1,
17:15:62855,28:15:-1,
27:16:68962,21:17:72509:0:0:0:0:0:0:0,21:18:76689:0:0:0:0:0:0:0,24:18:76921:0:0:0:0:0:0:0,7:21:-1,
10:21:-1:-5:10:0,
11:21:-1,
20:21:90326:10:0:0:0,22:27:-1,
27:27:112406,0:28:-1,
3:28:115143,9:29:-1,
33:29:135545
Just to be sure: the extra six 0's (0:0:0:0:0:0) won't do anything here I believe. The 6 numbers are only needed if you fill in at least one non-zero number. They change the lum/chrom by the number. They don't set it to 0.

Just in case I wasn't very clear about that.

I know, it's the 7th "0" which is useful there
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/15/2014 06:36 pm
At the risk of introducing additional confusion and coding difficulty, may I suggest IPV6-style notation for zeros,

:: --> 0:0:0:0:0:0

23:11:45354:0:0:0:0:0:0:0
23:11:45354::0

The '::' can only occur once and only fills zeros.

if the 7th 0 is different than having only 6 zeros, maybe change it to requiring a '-0' rather than '0' to distinguish the cases.

Edit: I have this implemented in a fork, waiting on michaelni to merge
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/15/2014 06:36 pm
So I adapted iframe32 using the new options with:

X:57182:4,
40:0:-1,
17:1:5444,13:2:-1,
5:4:14914,43:4:-1,
7:5:19298,14:06:24261:0:0:0:0:0:0:0,23:11:45354:0:0:0:0:0:0:0,26:12:-1,
7:13:52167,7:14:-1,
17:15:62855,28:15:-1,
27:16:68962,21:17:72509:0:0:0:0:0:0:0,21:18:76689:0:0:0:0:0:0:0,24:18:76921:0:0:0:0:0:0:0,7:21:-1,
10:21:-1:-5:10:0,
11:21:-1,
20:21:90326:10:0:0:0,22:27:-1,
27:27:112406,0:28:-1,
3:28:115143,9:29:-1,
33:29:135545

I have not been able to replicate this using the online editor. Can you confirm that this is edit5?  ;)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/15/2014 06:38 pm
Some quick work on frame 52/old iframe 2. The bit flips are pretty much random...
X:25254:1,X:25299:1,X:53423:1,40:3:-1,6:4:9364,4:5:-1,6:5:11443,7:5:11487,0:7:-1,28:11:25250

Same here! I´m afraind you guys were using an old version of edit5  :P
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/15/2014 06:40 pm
That's assuming I haven't done something wrong ;)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/15/2014 06:53 pm
hehehe indeed. Can someone using the FFmpeg utility directly confirm to us if the output of the editor is consistent?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/15/2014 06:54 pm
I know, it's the 7th "0" which is useful there
Oooh. Sorry. I didn't count ;)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Untribium on 05/15/2014 07:15 pm
Some quick work on frame 52/old iframe 2. The bit flips are pretty much random...
X:25254:1,X:25299:1,X:53423:1,40:3:-1,6:4:9364,4:5:-1,6:5:11443,7:5:11487,0:7:-1,28:11:25250

Same here! I´m afraind you guys were using an old version of edit5  :P

Yeah I just realized that...oh well, here we go again :D

EDIT: The new version is a lot cleaner! Great work Shanuson!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/15/2014 07:40 pm
Hi Guys,

Here is my by-hand-aligned raw.ts, and all iframes after piping it through tsfix (tsalign still complains about one region but that's an misreport. Compare that part to raw.ts and you will see that all 47's are there and at the right position in that part. so don't use tsalign my that ts.)
I will update  http://spacexlanding.wikispaces.com/Progress+on+edit5.ts  (http://spacexlanding.wikispaces.com/Progress+on+edit5.ts) after this post.
But i have to go back to do my original work, So this will probably be my last contribution. 
I used the continuity bit and my best judgement to edit raw.ts to get that outcome.
There was on 56byte missing part where i am not 100% sure I add the 56 FF at the right position, I was a gap of 5 188 blocks with one missing 56 bytes. I added the 56 byte to the last one and marked it by 12 34 56 78 for the first 4 of those 56 Bytes.

Have fun with it :D

Cheers Shanuson

I just want to give you a very big thank you for this.  :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: michaelni on 05/15/2014 07:48 pm
So back to the P-frames...

So this is an example of a P-frame (frame192). I ploted the motion vectors to see if they are also correctly modified. I used the following command lines for both images:

ffmpeg -debug mb_pos_size -err_detect ignore_err -vismv 7 -s:0 704:480 -i frame192.mpg4-img -f image2 frame_192_mv.png
ffmpeg -mmb 2:0:-1,19:0:1092 -debug mb_pos_size -err_detect ignore_err -vismv 7 -s:0 704:480 -i frame192.mpg4-img -f image2 frame_192_mv_mod.png

Now instead of creating a png, is it possible to directly store the modified one as a P-frame? If yes, how? Or does something have to be implemented?

You can flip bits or move bits around in the frame and or the ts file using a hex editor or some other tool, keeping the data otherwise exactly intact. Or all the changes which are possible through mmb could be applied while decoding, this would require to make mmb work with multiple frames.
Some changes now possible with mmb like changing dc prediction direction can not be represented within the bitstream so they could not be added into frames.  Unless the frames are simply reencoded to high quality I-Frames (which in some sense is cheating but still may be a usefull tool)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Untribium on 05/15/2014 07:59 pm
Frame 52 (now using the most recent ts :)) )
40:3:-1,6:4:9364,4:5:-1,6:5:11443,7:5:11487,5:6:13496:0:0:0:0:0:-2,0:7:-1,38:11:26080:0:-4:-1:2:0:0,
31:12:28067:3:1:0:0,34:12:28226:-1,40:14:34617:-3:0:2:0,40:16:42426:-1:0:-4:-5,33:17:44961:0:-7:0:0:-2:0
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: FutureSpaceTourist on 05/15/2014 08:11 pm
Amazing thread; it's fantastic to see what you are all achieving.

Am I the only one thinking some of the frames currently look very Turneresque?!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/15/2014 08:21 pm
Frame 52 (now using the most recent ts :)) )
40:3:-1,6:4:9364,4:5:-1,6:5:11443,7:5:11487,5:6:13496:0:0:0:0:0:-2,0:7:-1,38:11:26080:0:-4:-1:2:0:0,
31:12:28067:3:1:0:0,34:12:28226:-1,40:14:34617:-3:0:2:0,40:16:42426:-1:0:-4:-5,33:17:44961:0:-7:0:0:-2:0

Perfect! Great work!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/15/2014 09:02 pm
Haven't had time to work on this for a while, but here's my latest attempt at the cursed frame 72 (ex iframe4), incorporating work by SwissCheese and seanpg71. I've been trying to get to the badly corrupted business end of the rocket but I fear this is little more than bad art and figments of my imagination. My current techiques don't allow much better, may need to develop new ones for this.

mmb attached as .txt file:
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: theshadow27 on 05/15/2014 09:18 pm
Ugg dumb question guys.

I assumed (as does the logic in the online editor) for X:pos:mask, mask was a hexadecimal value ("00" to "FF").

When I was adding the "::" to h263dec.c, I noticed the sscanf is using %d, which means it's decimal.

Which way is right/intended? Edit: so that I can update the wiki with the correct info…


Edit2: Ok, I was looking at the wrong line. The X:pos:mask IS a hex mask

but

The new DC inherit direction is a decimal mask. This should probs be changed to hex, if not too many people are using it.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: lewing on 05/15/2014 09:25 pm
Here is my best attempt at frame 150 from edit8.ts

X:18494:1,20:5:-1,33:6:23509,14:12:-1,18:12:46628,15:13:-1:24,16:13:50797:20,19:14:57637,
18:15:-1:0:0:4:2::14,25:15:66746,24:16:72453,32:18:-1:1:10:-3:6,35:18:87166,39:19:94065,
43:19:-1,0:20:94827,12:20:-1,43:20:101568,31:21:106253,33:22:112514,22:23:-1,
23:23:119825,26:23:-2,0:26:128548,5:27:-1,9:27:133159,0:29:142356,2:29:147166
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/15/2014 09:34 pm
Haven't had time to work on this for a while, but here's my latest attempt at the cursed frame 72 (ex iframe4), incorporating work by SwissCheese and seanpg71. I've been trying to get to the badly corrupted business end of the rocket but I fear this is little more than bad art and figments of my imagination. My current techiques don't allow much better, may need to develop new ones for this.

Perhaps someone from SpaceX can tell us whether or not we should expect to see some indication of leg deployment 1.3 seconds before what's depicted in the next iframe, shown here:

(http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=582517;image)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/15/2014 10:07 pm
Haven't had time to work on this for a while, but here's my latest attempt at the cursed frame 72 (ex iframe4), incorporating work by SwissCheese and seanpg71. I've been trying to get to the badly corrupted business end of the rocket but I fear this is little more than bad art and figments of my imagination. My current techiques don't allow much better, may need to develop new ones for this.

mmb: 5:1:4200,0:3:10968,8:3:14762,28:3:16550,24:4:20286,40:4:-1,9:6:21857,13:11:39488,37:11:41549,
10:12:-1,11:12:42693,30:12:-1,32:12:43895,24:13:-1,18:14:50986,8:15:-1,19:15:54479,20:15:54791,
2:16:-1,8:16:58952,13:16:-1,15:16:59896,20:16:61472,24:16:-1,27:16:63538,18:17:69075,27:17:-1,
13:18:74459,30:20:-1,41:20:91288,32:21:-1,34:21:96927,25:22:-1,41:22:104774,0:24:-1,
17:26:122506,0:27:-1,24:27:127453,6:28:-1

Looks amazing. But what .ts are you using? I´m getting something slightly differnt on edit8
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/15/2014 10:19 pm

Looks amazing. But what .ts are you using? I´m getting something slightly differnt on edit8

Edit5. The last one in the online editor. It's a useful tool. It was the same in edit5 and try1 so I assumed it would be the same in edit8 too. I'll check edit8.

Edit: I got the same thing from edit8. The one posted here: http://forum.nasaspaceflight.com/index.php?topic=34597.msg1198706#msg1198706

Edit2: I have somehow managed to post a bad mmb, I apologize.

Still bad, something's wrong with my pasting skills.
5:1:4200,0:3:10968,8:3:14762,28:3:16550,24:4:20286,40:4:-1,9:6:21857,13:11:39488,37:11:41549,
10:12:-1,11:12:42693,30:12:-1,32:12:43895,24:13:-1,18:14:50986,8:15:-1,19:15:54479,
20:15:54791,2:16:-1,8:16:58952,13:16:-1,15:16:59896,20:16:61472,24:16:-1,27:16:63538,
18:17:69075,27:17:-1,13:18:74459,30:20:-1,41:20:91288,32:21:-1,34:21:96927,25:22:-1,
41:22:104774,0:24:-1,17:26:122506,0:27:-1,24:27:127453,6:28:-1
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/15/2014 10:26 pm
Here is also the fixed_edit8.ts for those of you using windows :D
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/15/2014 10:36 pm

Looks amazing. But what .ts are you using? I´m getting something slightly differnt on edit8

Edit5. The last one in the online editor. It's a useful tool. It was the same in edit5 and try1 so I assumed it would be the same in edit8 too. I'll check edit8.

Edit: I got the same thing from edit8. The one posted here: http://forum.nasaspaceflight.com/index.php?topic=34597.msg1198706#msg1198706

I'm getting a bulish version on the online editor... no idea why  :o

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Oersted on 05/15/2014 10:44 pm
Am I the only one thinking some of the frames currently look very Turneresque?!

Turneresque indeed, with a bit of Mondrian thrown in, unfortunately!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/15/2014 10:45 pm
I'm getting a bulish version on the online editor... no idea why  :o

I'm attaching the mmb as text. For some reason it's not pasting correctly.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/15/2014 10:51 pm
Sweet! That was it!  ;)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/15/2014 11:08 pm
A few updates to the UI of the editor today, so refresh if you haven't recently :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/16/2014 06:00 am
Hi guys,

Here is the source for reverse searching of macroblocks.  8)

Only this file was changed:

libavcodec/h263dec.c

Please test first and then push to github.

How it works:

Lets say you know that a correct macroblock starts here:

0:0-1,19:06:22832

and you want to find 18:06 but you don't know its bitposition. So you do this:

0,0:0:-1,18:06:--22832:0,19:06:22832

With this you give the instruction: when reaching mb 18:06 start with the position 22832 (which is the starting bitposition of 19:06) and try to find a possible starting bitposition before 22832. What it will do is first set bitposition of 18:06 to 22831, then decode from there and if the decoding of the macroblock does not end at 22832 it will start from 22830 etc. If it does at some point end at 22832 it will keep that as the solution. You can see the final position in the log.

If you don't think it's the real solution you can tell it to skip the one or more candidates:

0,0:0:-1,18:06:--22832:1,19:06:22832

This will skip the first hit and will take the second hit as the solution.

Now after experimenting with this I saw there can be quite a few false positives: for one example I found 11 possible solutions. So you will still have to determine yourselves which one is the most likely starting position (probably the one with the lowest bitposition number). In practice the real starting position is the one where the macroblock before that (in this case 17:06) ends again. So I think you can see where this is going ;)

There is a max of 2048 tries at the moment btw. If it doesn't find a solution it will set the starting position at the postion you told it to start searching from (22832 in this case).

Sometimes it crashes after finding no solution. I'm not sure why yet. So handle with care.

Important note: do not post a reverse searching command in your solution online! If you find a bitposition using this technique then replace the search-command with a normal command (eg 18:06:22712). Otherwise you will create a LOT of unnecessary log to others.

Regards,

arnezami

PS. This technique will cause quite a bit of error-log because you are trying to start from a lot of wrong bitpositions. Never do two searches at the same time!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/16/2014 06:38 pm
Still trying to get something out of the p-frames...

We can visualize quite well the macroblock types with -debug vis_mb_type and the motion vectors with -vismv 7.  It's nice but does not help so much as it is only a visual information. Would it be possible to write the macroblock type and motion vector values besides the macroblock dc values? I think it would help quite a lot.

edit: found that we can write out the MB types with -debug mb_type

edit2: wrote some code lines to do it :), will post tomorrow
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/17/2014 11:34 am
The updated version displaying the MB type and the motion vectors is in attachment, it was based on the latest version of Arnezami. I hope it does not introduce any bug ;)

I also tried to rebuild frames +/- "as the encoder does" with some basic matlab code, the problem is that many P-frames are highly corrupted... but it worked not too badly for frames 169 (I-frame), 171 and 172.

I don't know if it's a good idea to go this way, but I was stuck trying to repair P-frames: as soon as I touched them (tried to correct them with matlab), they would not be used during the video encoding (even when simply reading and rewriting a frame, without changing anything).

Btw some P-frames like 171 or 176 have issues in their first data bytes or at the end of the header (bytes 4/5/6), I don't really know, and are not converted to PNG, a copy/paste of these bytes from a valid frame solves the issue.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/17/2014 11:41 am
Hi guys,

I noticed a problem in the online editor. It is removing zero's in the DC sections. This goes wrong if you want to do something like this: 16:05:65003:0:0:-5:0:0:0. Now it makes 16:05:65003:::-5,0 out of that. Which is wrong.

Only if all 6 DC values (L1:L2:L3:L4:C1:C2) are zero should it remove the 6 zero's (L1:L2:L3:L4:C1:C2). Otherwise there should always be 6 numbers there. The direction-number after that is optional.

Also I think it is not updated to the latest commit (https://github.com/michaelni/FFmpeg/commit/77d24d6a036e18064bcd25281f8d458e92de9b7e) where the short version 16:05:65003::15 of changing the directions was introduced.

Regards,

arenzami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/17/2014 12:01 pm
The updated version displaying the MB type and the motion vectors is in attachment, it was based on the latest version of Arnezami. I hope it does not introduce any bug ;)

I also tried to rebuild frames +/- "as the encoder does" with some basic matlab code, the problem is that many P-frames are highly corrupted... but it worked not too badly for frames 169 (I-frame), 171 and 172.

I don't know if it's a good idea to go this way, but I was stuck trying to repair P-frames: as soon as I touched them (tried to correct them with matlab), they would not be used during the video encoding (even when simply reading and rewriting a frame, without changing anything).

Btw some P-frames like 171 or 176 have issues in their first data bytes or at the end of the header (bytes 4/5/6), I don't really know, and are not converted to PNG, a copy/paste of these bytes from a valid frame solves the issue.
Cool! And interesting. The engine is just igniting! :)

Regarding the P-frames: I think it would be a good idea to extend the -mmb syntax with a framenumber.

So you would have something like this:

-mmb FRAME168:16:04:37259,23:02:73229=FRAME169:01:01:2342,34:02:-1  etc.

Something like that.

That way you can give it either the complete .ts or a .ts that starts with an I-frame until the P-frame you are actually working on. It would run through all the frames changing their macroblocks. And the P-frame you're interested in will inherit all the fixes (and errors of course) that have preceded it.

The only big problem is that the mmb will get quite big. We would need some kind of script that assembles all the mmb's of the frames and calls ffmpeg with that long mmb.

Good idea?

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Avron on 05/17/2014 12:13 pm
...I was stuck trying to repair P-frames:...



Amazing, quite amazing to see P- frames. I would hope Elon send you all an invite to tour the factory for the work effort he triggered, I am so sure he will do something great to say thanks.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/17/2014 12:42 pm
The only big problem is that the mmb will get quite big. We would need some kind of script that assembles all the mmb's of the frames and calls ffmpeg with that long mmb.

In condor_submit, you just specify the "-append" argument multiple times to add more parameters. Seems like the same idea would work here, and allow a bit more flexibility in the context of this idea.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/17/2014 01:40 pm
The updated version displaying the MB type and the motion vectors is in attachment, it was based on the latest version of Arnezami. I hope it does not introduce any bug ;)

I also tried to rebuild frames +/- "as the encoder does" with some basic matlab code, the problem is that many P-frames are highly corrupted... but it worked not too badly for frames 169 (I-frame), 171 and 172.

I don't know if it's a good idea to go this way, but I was stuck trying to repair P-frames: as soon as I touched them (tried to correct them with matlab), they would not be used during the video encoding (even when simply reading and rewriting a frame, without changing anything).

Btw some P-frames like 171 or 176 have issues in their first data bytes or at the end of the header (bytes 4/5/6), I don't really know, and are not converted to PNG, a copy/paste of these bytes from a valid frame solves the issue.
Cool! And interesting. The engine is just igniting! :)

Regarding the P-frames: I think it would be a good idea to extend the -mmb syntax with a framenumber.

So you would have something like this:

-mmb FRAME168:16:04:37259,23:02:73229=FRAME169:01:01:2342,34:02:-1  etc.

Something like that.

That way you can give it either the complete .ts or a .ts that starts with an I-frame until the P-frame you are actually working on. It would run through all the frames changing their macroblocks. And the P-frame you're interested in will inherit all the fixes (and errors of course) that have preceded it.

The only big problem is that the mmb will get quite big. We would need some kind of script that assembles all the mmb's of the frames and calls ffmpeg with that long mmb.

Good idea?

Regards,

arnezami
Hi guys,

I've extended the mmb-syntax so it can now handle multiple frames at the time. See the above quote.

The syntax is:

Quote
-mmb FRAME0:16:04:37259,23:02:73229=FRAME1:01:01:2342,34:02:-1

Note the '=' sign!

The old syntax still works on files with one frame in it.

I've also incorporated SwissCheese' logging (and changed back the && into & ;))

I haven't actually tested it with a file with multiple frames in it. But it *should* work. Let me know!

[edit] Important: the frame numbers should be the frame numbers in your file. If you use a file with say 3 frames in it their frame numbers should be 0, 1 and 2. So not 168, 169, 170 in that case. Should I add the frame number in the log too? Not sure.

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/17/2014 01:51 pm
Coming up on the 150,000 view mark for this thread/effort. Noting it as there's a small active team of people really working hard on this, but also a HUGE amount of people watching that work. Always nice to have a large appreciative audience, both inside and outside the space industry.

You'd be astonished how many views are from IP addresses in a place called Hawthorne, California! ;D
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/17/2014 02:02 pm
The updated version displaying the MB type and the motion vectors is in attachment, it was based on the latest version of Arnezami. I hope it does not introduce any bug ;)

I also tried to rebuild frames +/- "as the encoder does" with some basic matlab code, the problem is that many P-frames are highly corrupted... but it worked not too badly for frames 169 (I-frame), 171 and 172.

I don't know if it's a good idea to go this way, but I was stuck trying to repair P-frames: as soon as I touched them (tried to correct them with matlab), they would not be used during the video encoding (even when simply reading and rewriting a frame, without changing anything).

Btw some P-frames like 171 or 176 have issues in their first data bytes or at the end of the header (bytes 4/5/6), I don't really know, and are not converted to PNG, a copy/paste of these bytes from a valid frame solves the issue.
Cool! And interesting. The engine is just igniting! :)

Inspiring work SwissCheese! this is Amazing! Yet another piece of the puzzle (engine ignition) has been uncovered.

The idea of using matlab sounds good to me. I think we should accept that some of the frames of the video are beyond recovery.  Considering this, a good approach could be to deconstruct the video frame by frame recovering as many usable frames as possible, and then reconstruct it back again keeping only the good frames, to then interpolate (like the guys on the curiosity landing did) the missing ones in order to obtain a decent dynamic video.

My guess is the p-frames immediately after a fixed iframe will be the best looking, as only some macro blocks should have changed..... but as the sequence continues moving forward, the subsequent frames will start to accumulate errors and they will look worse and worse. That is, naturally, until the video gets to the next iframe that has been manually fixed.

For how many frames did you apply the process?  Could you the same technique for the frames after of after 72 or 209 (which could show leg deployment and touchdown)? do you thing my hyphotesis of constant deterioration will hold?

Edit: On other news, I tried to apply your very long mmb code for frame 92 (the frame formally known as iframe # 5) to the newest version (the one obtained from edit5.ts) and unfortunately is not working anymore. AS you where the main author, maybe you will be able to figure out what has changed. I'm guessing a small adjustment should be sufficient to get that frame visible again! It could also be a problem on how the code was pasted in the forum: I believe this happened to Saliva_sweet
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: billh on 05/17/2014 02:16 pm
The updated version displaying the MB type and the motion vectors is in attachment, it was based on the latest version of Arnezami. I hope it does not introduce any bug ;)

I also tried to rebuild frames +/- "as the encoder does" with some basic matlab code, the problem is that many P-frames are highly corrupted... but it worked not too badly for frames 169 (I-frame), 171 and 172.

I don't know if it's a good idea to go this way, but I was stuck trying to repair P-frames: as soon as I touched them (tried to correct them with matlab), they would not be used during the video encoding (even when simply reading and rewriting a frame, without changing anything).

Btw some P-frames like 171 or 176 have issues in their first data bytes or at the end of the header (bytes 4/5/6), I don't really know, and are not converted to PNG, a copy/paste of these bytes from a valid frame solves the issue.
Cool! And interesting. The engine is just igniting! :)

I don't think that's the engine igniting. It looks more like the plume is just beginning to interact with the ocean surface.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: edfishel on 05/17/2014 02:25 pm
For the small group of people who are working really, really hard on this...thank you!  It's beyond my comprehension.  But I thoroughly appreciate the work you are doing.  Chris Bergin is so right.  You are doing wonderful work on behalf of all of us.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/17/2014 02:29 pm
The updated version displaying the MB type and the motion vectors is in attachment, it was based on the latest version of Arnezami. I hope it does not introduce any bug ;)

I also tried to rebuild frames +/- "as the encoder does" with some basic matlab code, the problem is that many P-frames are highly corrupted... but it worked not too badly for frames 169 (I-frame), 171 and 172.

I don't know if it's a good idea to go this way, but I was stuck trying to repair P-frames: as soon as I touched them (tried to correct them with matlab), they would not be used during the video encoding (even when simply reading and rewriting a frame, without changing anything).

Btw some P-frames like 171 or 176 have issues in their first data bytes or at the end of the header (bytes 4/5/6), I don't really know, and are not converted to PNG, a copy/paste of these bytes from a valid frame solves the issue.
Cool! And interesting. The engine is just igniting! :)

Regarding the P-frames: I think it would be a good idea to extend the -mmb syntax with a framenumber.

So you would have something like this:

-mmb FRAME168:16:04:37259,23:02:73229=FRAME169:01:01:2342,34:02:-1  etc.

Something like that.

That way you can give it either the complete .ts or a .ts that starts with an I-frame until the P-frame you are actually working on. It would run through all the frames changing their macroblocks. And the P-frame you're interested in will inherit all the fixes (and errors of course) that have preceded it.

The only big problem is that the mmb will get quite big. We would need some kind of script that assembles all the mmb's of the frames and calls ffmpeg with that long mmb.

Good idea?

Regards,

arnezami
Hi guys,

I've extended the mmb-syntax so it can now handle multiple frames at the time. See the above quote.

The syntax is:

Quote
-mmb FRAME0:16:04:37259,23:02:73229=FRAME1:01:01:2342,34:02:-1

Note the '=' sign!

The old syntax still works on files with one frame in it.

I've also incorporated SwissCheese' logging (and changed back the && into & ;))

I haven't actually tested it with a file with multiple frames in it. But it *should* work. Let me know!

[edit] Important: the frame numbers should be the frame numbers in your file. If you use a file with say 3 frames in it their frame numbers should be 0, 1 and 2. So not 168, 169, 170 in that case. Should I add the frame number in the log too? Not sure.

Regards,

arnezami

I don't exactly understand how to use it, could you make a small .zip package with an example? (text file with command lines and used .ts or .mp4-img files)

(sorry for the &  ::)  :P)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/17/2014 02:32 pm
The updated version displaying the MB type and the motion vectors is in attachment, it was based on the latest version of Arnezami. I hope it does not introduce any bug ;)

I also tried to rebuild frames +/- "as the encoder does" with some basic matlab code, the problem is that many P-frames are highly corrupted... but it worked not too badly for frames 169 (I-frame), 171 and 172.

I don't know if it's a good idea to go this way, but I was stuck trying to repair P-frames: as soon as I touched them (tried to correct them with matlab), they would not be used during the video encoding (even when simply reading and rewriting a frame, without changing anything).

Btw some P-frames like 171 or 176 have issues in their first data bytes or at the end of the header (bytes 4/5/6), I don't really know, and are not converted to PNG, a copy/paste of these bytes from a valid frame solves the issue.
Cool! And interesting. The engine is just igniting! :)

Inspiring work SwissCheese! this is Amazing! Yet another piece of the puzzle (engine ignition) has been uncovered.

The idea of using matlab sounds good to me. I think we should accept that some of the frames of the video are beyond recovery.  Considering this, a good approach could be to be to deconstruct the video frame by frame recovering as many usable frames as possible, and then reconstruct it back again keeping only the good frames, to then interpolate (like the guys on the curiosity landing did) in order to obtain a decent dynamic video.

My guess is the p-frames immediately after a fixed iframe will be the best looking, as only some macro blocks should have changed..... but as the sequence continues moving forward, the subsequent frames will start to accumulate errors and they will look worse and worse. That is, naturally, until the video gets to the next iframe that has been manually fixed.

For how many frames did you apply the process?  Could you the same technique for the frames after of after 72 or 209 (which could show leg deployment and touchdown)? do you thing my hyphotesis of constant deterioration will hold?

Edit: On other news, I tried to apply your very long mmb code for frame 92 (the frame formally known as iframe # 5) to the newest version (the one obtained from edit5.ts) and unfortunately is not working anymore. AS you where the main author, maybe you will be able to figure out what has changed. I'm guessing a small adjustment should be sufficient to get that frame visible again! It could also be a problem on how the code was pasted in the forum: I believe this happened to Saliva_sweet

Yes it is exactly the problem, it works just after the I-frame and as long as we have good quality P-frames, but if a serie of P-frames is missing/corrupted it does not give great results anymore... Also seen in frame 176: some MB have somehow a wrong type (coded as stand-alone MB, when they are clearly difference MB...) and other have really low values (very dark) for no reason that I understand.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/17/2014 04:15 pm
Interpolated descent between frames 150 and 168.

Many thanks to SwissCheese for demonstrating how to apply p-frames.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/17/2014 04:20 pm
Video time  8)

I've used the new multiframe command on fixed_edit8.ts using all known mmb's for the I-frames that work on edit8 (from the wiki). I then reencoded it back into a video.

I used this command:

Quote
./ffmpeg -r 44999/3003 -mmb `cat mmb.txt` -debug mb_pos_size -err_detect ignore_err  -i fixed_edit8.ts -c:v mpeg4 -q:v 3 -tag:v xvid spacex_langing_nsf_20140517.avi

The mmb.txt file is attached. As you can see it's simply all the mmb's concatenated with '=' between them and 'FRAMExxx:' in front of each.

[edit] Almost forgot: for some reason the framenumbers are offset by 8. I think there are some frames before the first I-frame that cause this.

If you just want all the pictures you can do this:

Quote
./ffmpeg -r 44999/3003 -mmb `cat mmb.txt` -debug mb_pos_size -err_detect ignore_err  -i fixed_edit8.ts -f image2 img-%03d.png

The video (also attached) already looks very cool :) At some point you can actually see it flying!

https://www.youtube.com/watch?v=PgY2NZwRxBY

Of course the P-frames disturb the video heavily. So there is still a lot of work to do.

Enjoy!

arnezami

@SwissCheese: I've tried to add the P-frame mmb's you provided. But somehow this doesn't work: the changed P-frames and the P-frames after that stopped working. We'll have to figure out why. I thought that should have worked.

[edit] Added youtube video.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/17/2014 04:25 pm
Video time  8)

I've used the new multiframe command on fixed_edit8.ts using all known mmb's for the I-frames that work on edit8 (from the wiki). I then reencoded it back into a video.

I used this command:

Quote
./spx/ffmpeg-spacex-spacexdebug1/ffmpeg -r 44999/3003 -mmb `cat mmb.txt` -debug mb_pos_size -err_detect ignore_err  -i fixed_edit8.ts -c:v mpeg4 -q:v 3 -tag:v xvid spacex_langing_nsf_20140517.avi

The mmb.txt file is attached. As you can see it's simply all the mmb's concatenated with =FRAMExxx: between them and just FRAMExxx: at the beginning.

[edit] Almost forgot: for some reason the framenumbers are offset by 8. I think there are some frames before the first I-frame that cause this.

If you just want all the pictures you can do this:

Quote
./spx/ffmpeg-spacex-spacexdebug1/ffmpeg -r 44999/3003 -mmb `cat mmb.txt` -debug mb_pos_size -err_detect ignore_err  -i fixed_edit8.ts -f image2 img-%03d.png

The video (also attached) already looks very cool :) At some point you can actually see it flying!

Of course the P-frames disturb the video heavily. So there is still a lot of work to do.

Enjoy!

arnezami

@SwissCheese: I've tried to add the P-frame mmb's you provided. But somehow this doesn't work: the changed P-frames and the P-frames after that stopped working. We'll have to figure out why. I thought that should have worked.

That things looks amazing!!!!!!!!!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/17/2014 04:29 pm
I don't know if it's a good idea to go this way, but I was stuck trying to repair P-frames: as soon as I touched them (tried to correct them with matlab), they would not be used during the video encoding (even when simply reading and rewriting a frame, without changing anything).

Are there some headers and metadata around the pframes that might be able to be repaired in a similar way as the transport stream and iframe headers were? I can't seem to find a good explanation so far as to how pframes are encoded, though. If there's some reference points like timestamp values and what have you, that might help resuscitate some of them.

Edit: NICE jobs on that video and animation, you two! I think that explains why the orange color in iframe 209 wants to get pulled down farther than you'd expect - it's reflecting off the rocket body.

Edit: By the way, at the very beginning of the video are we perhaps seeing some evidence that the first couple of iframes were almost completely fogged in? Until we get up to iframe 32 which SwissCheese combed out, there's a huge amount of grey MBs in there, and then just a hint of orange as the engine glow passes through the fog. What if we hardcoded the first iframe or two to just a solid grey foggy expanse and see what happens as the pframes come through?

Would the triangles at the lower corners be visible in dense fog, also? Seems like they would be, but that would depend on how the housing is designed.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/17/2014 04:38 pm
Hi guys,

I've added a youtube video link to my previous post (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1199895#msg1199895).

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Sohl on 05/17/2014 04:46 pm
Video time  8)

That is fantastic!

But I was wondering, is there a relatively simple way to coerce the hue/color values in the p-frames to something in the blue-to-gray/white range before re-encoding the video?  Well, and maybe some yellow/orange near the frame center when the plume is visible.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Asmegin on 05/17/2014 04:52 pm
Amazing video! Nice to see all the hard work starting to come together.

Will the work on the P-Frames be more automatic or manual similar to the I-Frames? I'm sure we have enough people here with a bit of knowledge to help work on them if lots of manual labour is required. Otherwise it's all up to the few smart guys here :D

The next daytime landing won't be for many months, so there is lots of time to produce a good product. The upcoming Orbcomm launch will likely be in the early morning darkness, and the following two Asiasat launches won't have legs. Which means we have to wait until CRS-4, which I believe is scheduled for early August.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: adrianwyard on 05/17/2014 05:23 pm
Apologies if that has come up before (this is a big thread!) but has any consideration been given to post-processing the final 'best' rendering of the actual data?

It's a good bet that others have tried to recover highly corrupted MPEG streams before so there may be filters and techniques already developed out there beyond NSF. But some ideas:

+ Restrict the color palette to those we know are real. For those crazy colored larger macroblocks, substitute a color by averaging that of the neighbors. A really simple version of this would be to just forgo color and go black-and-white.

+ Detect hard 90 degree edges (of macroblocks) and blur those away; we know those edges didn't exist in the camera source image.

+ At some point you need to determine if there's any useful engineering data in this project. If so, then you'd be cautious about adding blur and generating colors, if if not, then post-processing choices would be geared to producing the most pleasing perceived video. You may find that adding generated 'snow' produces subjectively 'better' video because we're more used to seeing signals degraded this way than bright-green macroblocks.

Just my $0.02. Thanks everyone for your hard work; it may be just a footnote, but this landing is going in the history books.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: meadows.st on 05/17/2014 05:24 pm
Amazing video! Nice to see all the hard work starting to come together.

Will the work on the P-Frames be more automatic or manual similar to the I-Frames? I'm sure we have enough people here with a bit of knowledge to help work on them if lots of manual labour is required. Otherwise it's all up to the few smart guys here :D

The next daytime landing won't be for many months, so there is lots of time to produce a good product. The upcoming Orbcomm launch will likely be in the early morning darkness, and the following two Asiasat launches won't have legs. Which means we have to wait until CRS-4, which I believe is scheduled for early August.

First, let me say that I echo the comment on the Amazing video and simulation and offer any brute-force labor that may be required for P-Frames.

re: "The next daytime landing", I first started to do a calculation for what the light might be like at between ~1000m and 2000m ASL where the landing burn should start on 27 May, 2014 and then I found SunCalc.net that seems to indicate that the landing area should be between "civil twilight and sunrise" between 03:00 and 03:30 EDT on 27 May, 2014.  If the launch actually happens, this light should make for some reasonable (but perhaps not great) footage.  I have attached a screenshot of the resulting map presented by the SunCalc simulation.

EDIT: Actually, I suspect the launch images will be spectacular in the early light if the weather is clear and the further East the landing is located, the more light there will be for the landing (so no pressure) ;).

@Comga gives more detailed (and contradictory) info here:
http://forum.nasaspaceflight.com/index.php?topic=33089.msg1199888#msg1199888

EDIT2: Sadly I think that SunCalc is giving the time in my local timezone (PDT) so unfortunately I (now) agree with @Comga that the flight will still be in darkness. :(  In my own defence, I am used to higher latitudes (49N) and it starts to get light at 4AM around here as we approach the summer solstice. (added updated image with time in EDT)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Llian Rhydderch on 05/17/2014 07:57 pm
With the "NSF community-edited video" getting further along, it is probably not too early to ask this question:

What will be the license attached to any video that comes of this community editing effort by NSFers?

It would be great if the finished product ends up with one of the acceptable Creative Commons licenses (https://en.wikipedia.org/wiki/Wikipedia:Copyrights) on it that would make the video okay for Wikimedia and all the Wikipedia Foundation projects.  Two of the acceptable licenses for use there include the Creative Commons Attribution-Sharealike 3.0 Unported License (CC-BY-SA) and the GNU Free Documentation License (GFDL). 

Given that this started out as a SpaceX raw file, with what license I don't know, what will the edited/cleaned final version license be?  ... or should it be?


Edited:  fixed a single-letter typo
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/17/2014 09:24 pm
Edit: On other news, I tried to apply your very long mmb code for frame 92 (the frame formally known as iframe # 5) to the newest version (the one obtained from edit5.ts) and unfortunately is not working anymore. AS you where the main author, maybe you will be able to figure out what has changed. I'm guessing a small adjustment should be sufficient to get that frame visible again! It could also be a problem on how the code was pasted in the forum: I believe this happened to Saliva_sweet
This is my best attempt so far:

0:0:550,14:0:-1,
25:10:17410,
17:11:33650,
17:12:37648,
25:12:39009,29:12:-1,32:12:-1,10:14:-1,
17:14:52765,18:14:-1,24:14:-1,21:15:-1,22:15:-1,
34:15:56061,24:16:-1,
27:16:61466,42:19:-1,
43:19:83153,43:20:-1,
1:21:90585,11:22:-1,
12:22:99201,26:22:-1,
17:23:105394,22:24:-1,
23:24:111581,0:27:-1,
16:27:124815,24:27:-1,
25:27:125621,28:28:-1,
3:29:140154

Hope that helps others to make it better again. I've tried to use as much as possible from the last (much better) attempt (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195949#msg1195949). This though is using edit8.

But somehow I feel (while more data is certainly added to this frame) something got lost too. Not sure. Particularly around the legs. But I'm not so good at this. Hope I'm wrong.

Please help.

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Asmegin on 05/17/2014 10:28 pm

This is my best attempt so far:

0:0:550,14:0:-1,
25:10:17410,
17:11:33650,
17:12:37648,
25:12:39009,29:12:-1,32:12:-1,10:14:-1,
17:14:52765,18:14:-1,24:14:-1,21:15:-1,22:15:-1,
34:15:56061,24:16:-1,
27:16:61466,42:19:-1,
43:19:83153,43:20:-1,
1:21:90585,11:22:-1,
12:22:99201,26:22:-1,
17:23:105394,22:24:-1,
23:24:111581,0:27:-1,
16:27:124815,24:27:-1,
25:27:125621,28:28:-1,
3:29:140154


Bit more of legs: 19:13:42998, 18:14:47626
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/17/2014 11:03 pm
Interpolated descent between frames 150 and 168.

Many thanks to SwissCheese for demonstrating how to apply p-frames.

It looks good! How did you do the interpolation? Did you use information from the P-frames? I was thinking about interpolating the moving vector fields for the non-usable P-frames, or somehow better using them, but I'm stuck...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/17/2014 11:12 pm

This is my best attempt so far:
Bit more of legs: 19:13:42998, 18:14:47626

This looks very close to the first attempt! If someone can tease out more of the ocean, and perhaps find dirt spot #3 again, we'd be in good shape.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Asmegin on 05/17/2014 11:15 pm

This is my best attempt so far:
Bit more of legs: 19:13:42998, 18:14:47626

This looks very close to the first attempt! If someone can tease out more of the ocean, and perhaps find dirt spot #3 again, we'd be in good shape.

Dirt spot: 26:10:31150. Kinda messes up the rest though.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/17/2014 11:19 pm

This is my best attempt so far:
Bit more of legs: 19:13:42998, 18:14:47626

This looks very close to the first attempt! If someone can tease out more of the ocean, and perhaps find dirt spot #3 again, we'd be in good shape.

Dirt spot: 26:10:31150

Love that dirt spot! They should put them in the camera for the next flight too.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/17/2014 11:25 pm
Better if they turn off interlacing and enable CRC instead. :D
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: somepitch on 05/17/2014 11:58 pm
Coming up on the 150,000 view mark for this thread/effort. Noting it as there's a small active team of people really working hard on this, but also a HUGE amount of people watching that work. Always nice to have a large appreciative audience, both inside and outside the space industry.

You'd be astonished how many views are from IP addresses in a place called Hawthorne, California! ;D

How many from Centennial, CO or Evry-Courcouronnes, France...?  ;D
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Asmegin on 05/18/2014 12:24 am

This is my best attempt so far:

0:0:550,14:0:-1,
25:10:17410,
17:11:33650,
17:12:37648,
25:12:39009,29:12:-1,32:12:-1,10:14:-1,
17:14:52765,18:14:-1,24:14:-1,21:15:-1,22:15:-1,
34:15:56061,24:16:-1,
27:16:61466,42:19:-1,
43:19:83153,43:20:-1,
1:21:90585,11:22:-1,
12:22:99201,26:22:-1,
17:23:105394,22:24:-1,
23:24:111581,0:27:-1,
16:27:124815,24:27:-1,
25:27:125621,28:28:-1,
3:29:140154


Bit more of legs: 19:13:42998, 18:14:47626
Dirt: 26:10:31150

I found what appears to be more of the left leg @ 17:13:42121 and 19:13:42516, but they only kindaaa fit and really mess up everything else. Maybe someone else can find the correct ones??
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jakusb on 05/18/2014 12:30 am
A repost for convenience. :)
For those new:
The image has annotated most of the common dirt groups that reoccur in all frames. It helps align the macroblocks. :)(http://img.tapatalk.com/d/14/05/18/emugerav.jpg)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/18/2014 02:19 am
Here's the collected mmb's for frame 92 from the above posts:

0:0:550,14:0:-1,25:10:17410,26:10:31150,
17:11:33650,17:12:37648,25:12:39009,
29:12:-1,32:12:-1,17:13:42121,18:13:42526,
10:14:-1,17:14:52765,18:14:-1,24:14:-1,
21:15:-1,22:15:-1,34:15:56061,24:16:-1,
27:16:61466,42:19:-1,43:19:83153,43:20:-1,
1:21:90585,11:22:-1,12:22:99201,26:22:-1,
17:23:105394,22:24:-1,23:24:111581,0:27:-1,
16:27:124815,24:27:-1,25:27:125621,28:28:-1,
3:29:140154
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Untribium on 05/18/2014 03:00 am
Some progress on the top half of the frame and the left leg. Also removed some duplicate -1s
0:0:550,14:0:-2:0:0:0:0:12:-6,7:5:14749,12:5:15312,39:7:-1,18:9:27558,
25:10:17410,26:10:31150,17:11:33650,17:12:37648,25:12:39009,29:12:-1,
32:12:-1,17:13:42121,18:13:42526,10:14:-1,18:14:47626,26:14:-1,34:15:56061,
24:16:-1,27:16:61466,42:19:-1,43:19:83153,43:20:-1,1:21:90585,11:22:-1,
12:22:99201,26:22:-1,17:23:105394,22:24:-1,23:24:111581,0:27:-1,
16:27:124815,24:27:-1,25:27:125621,28:28:-1,3:29:140154
I think 23:14 needs some bits flipped...the part of the right leg is one block (half a macro block) off, which is pretty bad...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Asmegin on 05/18/2014 03:27 am
Some progress on the top half of the frame and the left leg. Also removed some duplicate -1s
0:0:550,14:0:-2:0:0:0:0:12:-6,7:5:14749,12:5:15312,39:7:-1,18:9:27558,
25:10:17410,26:10:31150,17:11:33650,17:12:37648,25:12:39009,29:12:-1,
32:12:-1,17:13:42121,18:13:42526,10:14:-1,18:14:47626,26:14:-1,34:15:56061,
24:16:-1,27:16:61466,42:19:-1,43:19:83153,43:20:-1,1:21:90585,11:22:-1,
12:22:99201,26:22:-1,17:23:105394,22:24:-1,23:24:111581,0:27:-1,
16:27:124815,24:27:-1,25:27:125621,28:28:-1,3:29:140154
I think 23:14 needs some bits flipped...the part of the right leg is one block (half a macro block) off, which is pretty bad...

Nice job on the ocean! Before I had to step out I got a bit at the top but I didn't save it! Ahh, will keep trying. Also gonna try and work on the base of the legs. Dunno about that block on the right leg, I'm not knowledgeable enough to figure out what's happening there  ;D
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Untribium on 05/18/2014 03:42 am
Some progress on the top half of the frame and the left leg. Also removed some duplicate -1s
-snip-
I think 23:14 needs some bits flipped...the part of the right leg is one block (half a macro block) off, which is pretty bad...

Nice job on the ocean! Before I had to step out I got a bit at the top but I didn't save it! Ahh, will keep trying. Also gonna try and work on the base of the legs. Dunno about that block on the right leg, I'm not knowledgeable enough to figure out what's happening there  ;D

Happened to me with frame 52, closed the tab by accident :'( :) The ocean seems to be pretty much intact, it's just really time consuming to find the address of the next block after a bad one...brute force ftw! :D
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Asmegin on 05/18/2014 04:04 am
Frame 92:

23:14:49203 gives you a piece of the circle between the legs. Only the left blocks are correct though... Doesn't help the right side of the right leg either :p

Also there is something at 54419. I thought it was the edge of the rocket base but it doesn't fit anywhere. Might be a wave...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/18/2014 04:35 am
Interpolated descent between frames 150 and 168.

Many thanks to SwissCheese for demonstrating how to apply p-frames.

It looks good! How did you do the interpolation? Did you use information from the P-frames? I was thinking about interpolating the moving vector fields for the non-usable P-frames, or somehow better using them, but I'm stuck...

I used the ocean blobs from the i-frames at each end and p-frames 153, 159, and 166 as control points, then I interpolated each transformation variable with a spline. I scripted Hugin to do the transformations. P-frame 159 was applied directly, except for a bit of shifting and masking to get the timestamp in the right place. P-frames 151 and 154 were also applied but only half of the ocean for each was useable. Basically I took the previous frame, pretended it was an i-frame, created a movie with just that and one p-frame, and then broke it up into images again.

Bits and pieces of other p-frames were pasted in, but I did not attempt to apply the mpg4-img in those cases.

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Asmegin on 05/18/2014 04:52 am
Frame 229

0:0:-1,4:13:63400,20:15:74510,2:16:77453
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/18/2014 04:53 am
Hi!

You guys are amazing! I really thought the legs were a goner on that frame. But oooh no! :)

Here is my version of 229 with edit8. It was still missing from the video.

0:0:-1,
9:12:59273:25:1:1:1:1:1,
0:21:99181,0:25:-1,
1:25:113405,7:29:-1

It needs some love though. ;)

I am going to make a new video again with these new frames.

Regards,

arnezami

PS. The 1:1:1:1:1 is in these because the online editor removes the 0:0:0:0:0. Maybe somebody can take a look at that (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1199765#msg1199765)...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Asmegin on 05/18/2014 04:58 am
Looks like we had the same idea. Yours looks a bit cleaner though :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/18/2014 06:01 am
Hmmm. I've got some issues. I don't understand whats causing them. Looks like ffmpeg is not cooperating...

For some reason when I use this command frames 189, 229 and 268 are missing. (these are frames 197, 237 and 276 according to ffmpeg btw).

Quote
./ffmpeg -r 44999/3003 -mmb `cat mmb.txt` -debug mb_pos_size -err_detect ignore_err  -i fixed_edit8.ts -f image2 img-%03d.png 2> debug.log

See attached mmb.txt (you will have to remove the newlines before using it).

If you look at the pictures it generated (careful the frame number in the filename is 1 higher then the frame number according to ffmpeg) you see all the i-frames except the above mentioned.

When I run the same command with the three i-frame commands removed then 189 appears again (in its original form) while 229 and 268 are still missing.

Why is ffmpeg not adding these frames to the output?? I can't see anything in the log either. Can we force it somehow? But I'm no expert at using ffmpeg. Hopefully somebody else can help here.

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/18/2014 09:46 am
Hmmm. I've got some issues. I don't understand whats causing them. Looks like ffmpeg is not cooperating...

For some reason when I use this command frames 189, 229 and 268 are missing. (these are frames 197, 237 and 276 according to ffmpeg btw).

Quote
./ffmpeg -r 44999/3003 -mmb `cat mmb.txt` -debug mb_pos_size -err_detect ignore_err  -i fixed_edit8.ts -f image2 img-%03d.png 2> debug.log

See attached mmb.txt (you will have to remove the newlines before using it).

If you look at the pictures it generated (careful the frame number in the filename is 1 higher then the frame number according to ffmpeg) you see all the i-frames except the above mentioned.

When I run the same command with the three i-frame commands removed then 189 appears again (in its original form) while 229 and 268 are still missing.

Why is ffmpeg not adding these frames to the output?? I can't see anything in the log either. Can we force it somehow? But I'm no expert at using ffmpeg. Hopefully somebody else can help here.

Regards,

arnezami

For me the mmb for multiple frames does only work for the first one... (FRAME0) after that it does not work at all, somehow it uses some of the mmb of the first frame for the next one. No idea why.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/18/2014 10:34 am
Hi!

You guys are amazing! I really thought the legs were a goner on that frame. But oooh no! :)

Here is my version of 229 with edit8. It was still missing from the video.

0:0:-1,
9:12:59273:25:1:1:1:1:1,
0:21:99181,0:25:-1,
1:25:113405,7:29:-1

It needs some love though. ;)

I am going to make a new video again with these new frames.

Regards,

arnezami

PS. The 1:1:1:1:1 is in these because the online editor removes the 0:0:0:0:0. Maybe somebody can take a look at that (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1199765#msg1199765)...

This is fixed.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/18/2014 10:58 am
Hi guys,

Ok. I think it's time to really start going into the P-frames.  :)

Individual P-frames:

Attached is a zip with all the .mpg4-img files of all the 270 frames. These we will need to do check the P-frames individually. P-frames are "delta" frames so they will look a little weird. For now the most important thing to do with these P-frames is aligning the MBs to the bitpositions. Luminosity and chrominance (the DC-values) will come later. Also: focus first on the P-frames right after the I-frames, since all other P-frames after that will look better if they get better.

Here are the most common things to do (taken from here (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1198858#msg1198858)).

1. To check individual P-frames (original):

Quote
./ffmpeg -debug mb_pos_size -err_detect ignore_err -s:0 704:480 -i frame192.mpg4-img -f image2 frame_192.png

2. To check individual P-frames (original, with arrows indicating movement):

Quote
./ffmpeg -debug mb_pos_size -err_detect ignore_err -vismv 7 -s:0 704:480 -i frame192.mpg4-img -f image2 frame_192_mv.png

3. To check individual P-frames (modified by mmb):

Quote
./ffmpeg -mmb 2:0:-1,19:0:1092 -debug mb_pos_size -err_detect ignore_err -s:0 704:480 -i frame192.mpg4-img -f image2 frame_192_mod.png

4. To check individual P-frames (modified by mmb, and with arrows indicating movement):

Quote
./ffmpeg -mmb 2:0:-1,19:0:1092 -debug mb_pos_size -err_detect ignore_err -vismv 7 -s:0 704:480 -i frame192.mpg4-img -f image2 frame_192_mv_mod.png

We will still have to learn how to modify the P-frames effectively. But you can already experiment with them. This way we may get some basic mmb's for certain P-frames.

Using multiple frames mmb's: making pngs and video

In order for all this to be somewhat doable/managable I've split the fixed_edit8.ts into 15 parts, each starting with an I-frame. These 15 .ts files are attached. They have the i-frame number in their file name for good reference. I propose that we make a wiki page for each of these video parts (each containing about 20 frames: 1 I-frame and the rest P-frames). And on the wiki page where all I-frames (of edit8) are shown we put a link to each of these wiki pages.

We can use multiple mmb's together now, thereby creating a "fixed" part-video or series of pngs. So instead of decoding the whole video we decode only a part.

Each part starts with an I-frame.

Here are the steps to take:

1. Create mmb file for a part. For example:

Quote
vi mmb_starting_from_169.txt

In it you put the mmb's you have made yourself or have gotten from somebody else. Make sure it always starts with FRAME0 (even though it is a higher frame number in the original fixed_edit8.ts). Frame 0 is always the I-frame.

Like so:

Quote
FRAME0:0:0:-1,9:12:59273:25:1:1:1:1:1,0:21:99181=FRAME1:0:25:-1,1:25:113405,7:29:-1

So you make this by adding FRAMExxx: to each mmb you want to add (where xxx is the frame number in this part-file (so if the part starts with frame 169 which would be FRAME0 then frame 171 would be FRAME2). After that you concatenate/put them together with '=' in between.

2. To make pngs:

Quote
./ffmpeg -r 44999/3003 -mmb `cat mmb_starting_from_169.txt` -debug mb_pos_size -err_detect ignore_err  -i fixed_edit8_part_169.ts -f image2 fixed_edit8_part_169_%03d.png

3. To make video:

Quote
./ffmpeg -r 44999/3003 -mmb `cat mmb_starting_from_169.txt` -debug mb_pos_size -err_detect ignore_err  -i fixed_edit8_part_169.ts -c:v mpeg4 -q:v 3 -tag:v xvid fixed_edit8_part_169.avi

Here is an example of the video part 169:

https://www.youtube.com/watch?v=x5yGSwAudi0

I used the current mmb for I-frame 169 and the mmb for P-frame 171 from SwissCheese.

** currently I am still looking into an issue with using multiple-mmbs, but I hope to figure out soon why this doesn't always work as expected. It *should* work, right ;)


Regards,

arnezami

PS. In the attached splitting_into_ts_parts.zip I explain how I split the video into 15 parts. The make_ts_part.c is also in that zip.
PPs. I've also attached a slightly altered h263dec.c which now logs the frame number too.


For me the mmb for multiple frames does only work for the first one... (FRAME0) after that it does not work at all, somehow it uses some of the mmb of the first frame for the next one. No idea why.

Ok. Thanks. I will look into this. It maybe caused by the mmb-reading/interpretation.

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/18/2014 01:14 pm
For me the mmb for multiple frames does only work for the first one... (FRAME0) after that it does not work at all, somehow it uses some of the mmb of the first frame for the next one. No idea why.
Hi SwissCheese,

I've been trying to find the cause of the multi-mmb's not always working on the P-frames. I'm pretty convinced the mmb-interpretation code is ok and no positions of mb's are changed that shouldn't be changed (I logged a lot to see if all is ok).

I've disabled large parts of the added features (such as X, -1, -2, dc, direction) and just kept the "move mb to bitpos".

In this simplified mode: when I use your mmb from frame 172 in run_sequence.txt and add it to that of 169 it seems to work. Also when I combine the mmb's from 169 and 171. No problem. Only when I combine all three then things go wrong: 172 seems to "freeze" (that is: it doesn't change anything). Like the P-frame is empty. Ok there is a little motion and softening of sharp lines but that's it. Almost if 172 has halted all P-frame activity. Because everything from there stays basicly the same.

But if I set the mmb from 172 to 173 (which is a bit silly of course) everything works just fine. So it seems that the mmb from 171 and 172 are somehow conflicting (or better: the effect of those mmb's).

Do you have simpler example where only a combination of 2 doesn't work? Because I think it's related to what the mmb's are doing to the frames.

I think though that it is something in ffmpeg: some kind of reaction without logging about it. I could sure use some help in this department.

Regards,

arnezami

[edit] I noticed that setting frame 170 to 0:0:-1 (basicly turning it off) it works again! Maybe it's wise to disable unfixed P-frames prior to the one that you are trying to fix?

Basicly you get this: (without the newlines)

FRAME0:08:14:-1:15:-15:1:14:-2:1,
09:14:56683,
05:21:-1:0:-20:-5:2,
06:21:-1:-20:-20:6:5,
07:21:-1:0:0:3:5,
08:21:-1:0:0:0:0,
09:21:-1:10:15:0:0,
10:21:-1:15:15:0:0,
11:21:-1:5:10,
12:21:-1:5:3:0:0:-2,
13:21:-1:4:4,
14:21:-1,
28:21:108878,
X:126932:80,
04:27:-1:0:-4:13:7,
05:27:135412,
X:143273:80,
X:143386:80,
06:28:144012
=FRAME1:0:0:-1
=FRAME2:0:0:-1,31:0:4850,13:2:-1,14:2:12422,2:3:-1,3:3:13866,32:8:50739
=FRAME3:6:20:-1,7:20:84967

I use this command:

Quote
./ffmpeg -r 44999/3003 -mmb `cat mmb_starting_from_169.txt` -debug mb_pos_size -err_detect ignore_err  -i fixed_edit8_part_169.ts -f image2 fixed_edit8_part_169_%03d.png
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Untribium on 05/18/2014 01:19 pm
Nice work on the P-frames guys! The bits (literally :) ) and pieces are finally coming together!
Can someone more knowledgeable than me and has looked into the bitstream have a look at this and tell me whether it's accurate?
http://www.cmlab.csie.ntu.edu.tw/cml/dsp/training/coding/h263/format_p.html (http://www.cmlab.csie.ntu.edu.tw/cml/dsp/training/coding/h263/format_p.html)
Could be really helpful for bit flipping :)

edit: Also, is it possible to add the bitstream to the online editor?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/18/2014 02:14 pm
Great work guys. I am happy that my work help many.

The following is important for all of us that are working on the raw video stream file. I will attach 15 parts of raw_edit8.ts each containing a I-Frame and the following P-frames too a  follow on post (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1200471#msg1200471).
 
Some things that should still be done to raw_edit8.ts that again may(!) change the size of the I-Frames, and the numbering for sure.
When you look at the current numbering of the frames you will see that most I-Frames are 20 Frames apart. So there are 19 P-Frames between each I-Frame. There was made no effort at all to find all those in the code and repair the headers.
There should be 14*20+3+(how many P-Frames there are before the first I-Frame) Frames in raw.ts. The 3 is because the last I-frame is followed only by 2 P-frames.
A few days ago I had a discussion with  theshadow27 about that. I think i should summarize it here for everyone:

Like i said before, There are 4 PIDs used, with one 3E8 marking all real data,
With PID 3E8 are 3 kind of headers marked:
4743E83X:
This marks the start of a Frame, Both I-Frame and P-Frame, More to this later.

4703E83X: This should mark the END of a Frame. It is a fill packet, containing the last part if data and the rest of the Packet consists more or less of only FF. I have only checked a few but all those looked like: 4703E8X YY 00 FF FF FF ... Dont know if the YY contains any important information. YY was only one byte, but AFAIUI YY could also be many bytes.

4703E81X: All the rest of the data packets are headed by these 4 bytes.

X incrementing throughout the video stream over all those 3 Headers. It tells how many data packets there should be modulo 16. This helps to identify in messed up parts if there are some data packets missing or if some null packets are interpreted as data.

There are 3 more continuity counters used in the Video, each one for the other 3 PIDs used (000 020 and 1FF, the latter one being a bit tricked, see the end of this post)

I fixed the header of the I-Frames, both the first counter (Green below), and the PTS (time stamp, blue below). The first counter is increasing by 60060(dez) and the PTS is increasing by 120120(dez) between each I-Frame.
The header of P-frame also has both of those in it (the first part is similar to I-Frames):
Here is one P-Frame Header:
47 43 E8 3X 07 10 00 33 59 ED 7E 00 00 00 01 E0 00 00 81 80 07 21 01 9B 19 95 FF FF 00 00 01 B6 5*

So all P-frames should start like that. 00 00 01 B6 5 marking the following data as a P-Frame. (I-Frames have more header data before the 00 00 01 B1 1*)


The other most common PID is 1FFF marking the packet as a null packet, full of FFs and not containing any data.
The CC for those packets is also continuous throughout the file, also from one block of multiple null packets to the next. Yet there is one exception. single null packets entered during the streaming of one frame have a CC of 0. 

So the order of Headers is:

47 43 E8 3X I-Frame header packet
47 03 E8 1X Data packets
47 03 E8 3X Last Data packet partly filled with FFs
maybe one packet with 000 or 020 as PID
47 1F FF 1X Null packets

Followed by 19 times:

47 43 E8 3X P-Frame header packet
47 03 E8 1X Data packets
47 03 E8 3X Last Data packet partly filled with FFs
maybe one packet with 000 or 020 as PID
47 1F FF 1X Null packets

It can be that the Null packets are skipped between frames.
Also single null packets can be added throughout the 47 03 E8 1X data part. Check the CC to see if there could be one.

Lastly, there are still large part of the raw_edit8.ts part where there are no 47 marks to indicate the start of a TS-packet. the largest one is 20*188 byte large.

Cheers Shanuson

(edit: added the link the to follow on post)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/18/2014 02:31 pm
Nice work on the P-frames guys! The bits (literally :) ) and pieces are finally coming together!
Can someone more knowledgeable than me and has looked into the bitstream have a look at this and tell me whether it's accurate?
http://www.cmlab.csie.ntu.edu.tw/cml/dsp/training/coding/h263/format_p.html (http://www.cmlab.csie.ntu.edu.tw/cml/dsp/training/coding/h263/format_p.html)
Could be really helpful for bit flipping :)

edit: Also, is it possible to add the bitstream to the online editor?

It seems that that is describing the "6.2.5.2  Video Plane with Short Header" from ISO-IEC-14496-2_2001_MPEG4_Visual.pdf (http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=582064). Also see: "6.3.5.2  Video Plane with Short Header". Not entirely sure.

I'm actually not sure our video uses the "short header". I have mostly been focused on the macroblock/variable stuff. But it would be good to confirm, because there could definitely be problems in these headers.

This is also a little interesting:

Quote
short_video_header:  The short_video_header is an internal flag which is set to 1 when an abbreviated header
format is used for video content. This indicates video data which begins with a short_video_start_marker rather
than a longer start code such as visual_object_ start_code.  The short header format is included herein to provide
forward compatibility with video codecs designed using the earlier video coding specification ITU-T
Recommendation H.263.  All decoders which support video objects shall support both header formats
(short_video_header equal to 0 or 1) for the subset of video tools that is expressible in either form.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/18/2014 03:11 pm
This is fixed.
Great! Works well now :)

I've been thinking about the P-frames for the online editor. I think what would be useful and fairly easy to add is a third dropdown. What you get is first the option of the ts file (raw_edit8 for example). Then you select the "base I-frame" in the second dropdown. And then you can choose the actual frame (which would default to the first frame (=I-frame)) in the third dropdown.

In the case of edit8 the third drowdown not only contains the I-frame but also the P-frames within the chosen base I-frame. And you keep using the framexxx.mpg4-img files as your input to ffmpeg. That way more ppl can start working on the P-frames. In the case of try1 and coerced you only get one option in the third dropdown, which is always chosen.

Of course this will only show the P-frame individually (not fed by I-frame data). And we should also make sure we can run it though the multi-mmb call to ffmpeg, but thats for later I guess.

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/18/2014 05:28 pm
[edit] I noticed that setting frame 170 to 0:0:-1 (basicly turning it off) it works again! Maybe it's wise to disable unfixed P-frames prior to the one that you are trying to fix?

Basicly you get this: (without the newlines)

FRAME0:08:14:-1:15:-15:1:14:-2:1,
09:14:56683,
05:21:-1:0:-20:-5:2,
06:21:-1:-20:-20:6:5,
07:21:-1:0:0:3:5,
08:21:-1:0:0:0:0,
09:21:-1:10:15:0:0,
10:21:-1:15:15:0:0,
11:21:-1:5:10,
12:21:-1:5:3:0:0:-2,
13:21:-1:4:4,
14:21:-1,
28:21:108878,
X:126932:80,
04:27:-1:0:-4:13:7,
05:27:135412,
X:143273:80,
X:143386:80,
06:28:144012
=FRAME1:0:0:-1
=FRAME2:0:0:-1,31:0:4850,13:2:-1,14:2:12422,2:3:-1,3:3:13866,32:8:50739
=FRAME3:6:20:-1,7:20:84967

I use this command:

Quote
./ffmpeg -r 44999/3003 -mmb `cat mmb_starting_from_169.txt` -debug mb_pos_size -err_detect ignore_err  -i fixed_edit8_part_169.ts -f image2 fixed_edit8_part_169_%03d.png

Do you get "nice images"? For me it corrects the first image but does nothing to the next ones...

I checked the code to understand a bit better, and put a printf there:

// if framenumber is in the mmb, check if it is the current one
if ((frame_number_mmb < 0) || (frame_number_mmb == s->avctx->frame_number)) {
   if (frame_number_mmb == s->avctx->frame_number)
      printf("\nframe number ffmpeg: %d\nframe number mmb: %d\ntoken_frame = %s\n\n",s->avctx->frame_number,frame_number_mmb,token_frame);
   while (bitpos == INT_MIN && token_frame && *token_frame) {
      char *token = av_get_token(&token_frame, ",");
      char *token_orig;
      token_orig = token;
      if (*token_frame)
         token_frame++;
      bitpos = get_bitpos_from_mmb_part(s, gb, gb_blank, mb_x, mb_y, token, do_reverse, nr_of_hits_to_ignore);
      av_free(token_orig);
   }
}
av_free(token_frame_orig);

it gives me for example this:
frame number ffmpeg: 3
frame number mmb: 3
token_frame = 6:20:-1,7:20:84967

it reads correctly all mmb sequences, and works as expected for the frame 0, but after that the frame numbers jump between frames 1,2 and 3 quite randomly. Is it the normal behavior? At this point I'm really lost... Maybe it's something related to windows?

Having this in the web app would for sure help a lot, we would have a reference program and only one set of data ;)

I also have mmbs for several P-frames, but can only use them on stand-alone *.mpg4-img frames
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Untribium on 05/18/2014 06:32 pm
More progress on #52. Found 3 more rows worth of actual data (including dirt spot #3!), which brings us down to 3 rows of -1 blocks :)
lum/chroma correction is a nightmare though, particularly row 10 (dirt spot)...the luminance error doesn't seem to be a propagation problem, i.e. it occurs in several blocks (which is odd...)
X:76768:80,X:22120:80,40:3:-1,42:3:9061,0:7:-2:-10:-10:-10:-10:8:-5,29:9:21074,34:9:-1,39:9:21626,0:29:81738

edit: also found a bit of the left leg on #150, pretty messed up though...17:12:46027
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/18/2014 06:40 pm
lum/chroma correction is a nightmare though, particularly row 10 (dirt spot)...the luminance error doesn't seem to be a propagation problem, i.e. it occurs in several blocks (which is odd...)

What if it's a rainbow effect from the cloud we're passing through, or reflected light from the engine flare? Peering at the reconstructed Youtube video you can get a little sense of that in some of the intervening pframes.

Edit: https://www.youtube.com/watch?v=PgY2NZwRxBY

See, for example, pale orange MB's in seconds 2 and 3, and nice orange flame around second 9 at the left side.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/18/2014 07:38 pm
Added some data to I-frame 248, also difficult with dc values, it can for sure be improved a lot:

0:0:-1,
37:0:3175,
27:1:-1,
29:1:5452,
35:2:-1,
42:4:15183,
5:9:-1,
12:9:29306,

edit: argh made a mistake at the top and all dc correction are awful now... so giving the mmb without corrections... see the wiki for the full mmb used in the image
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/18/2014 07:43 pm
A gif animation (reconstruction with matlab as described in a previous post) and all my mmb for the P-frames.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/18/2014 07:50 pm
A gif animation (reconstruction with matlab as described in a previous post) and all my mmb for the P-frames.

That looks super!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Xspace_engineerX on 05/18/2014 08:09 pm
A gif animation (reconstruction with matlab as described in a previous post) and all my mmb for the P-frames.

Wow. The quality in that is much, much better than I thought could be achieved. You guys are incredible.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/18/2014 08:11 pm
PROGRESS REPORT: WHAT WE KNOW SO FAR

For almost three weeks, we have been working on a video that was (for the most part) indecipherable. After many hours of work from a bunch of highly committed experts, now we have enough images to actually interpret what exactly happened one month ago (April 18), with Space X's CRS-3 first stage booster. And let me tell you: the recovered images tell an amazing story!

For this analysis, I considered all the i-frames that are on the wiki and a few p-frames that have been posted here. To put things on a time line, I used the same clock that appears on the video itself.

Phase 1: Controlled Descent (Frames 1-52)
The first 4 seconds of the video (19:35:01-19:35:04), reveal the controlled descent of the booster. This part of the video is composed of 4 i-frames and around 48 p-frames. All i-frames from this phase where successfully extracted, but only two have been repaired so far. To this day, the p-frames also remain untouched. As most of the work has been oriented into other (more interesting) sections of the video, the effort to recover images from this phase has been limited.

However, with the small number of images we have obtained so far, we can confirm that the booster went in a controlled descent over the ocean, falling vertically in the intended position for landing.  As we do not have any reference points, at this time it is impossible to asses the degree of rotation or the speed of the rocket. However, we can confirm that the booster was in one piece and looking good (expect for some splashes of dirty water, that actually have played an important role in the reconstruction of the frames)

Phase 2: Leg Deployment (Frames 72-150)
The next seven seconds of the video (19:35:5-19:35:12), with 5 iframes and about 80 p-frames, show the entire process of leg deployment. All five iframes have been fixed, and p-frames are slowly starting to emerge. From these, we can confirm that the leg deployment was successful. The latest version of frame 72 show no legs yet but leg deployment is clearly visible on frame 92. We can assume that the process started somewhere in between. 

From the images we have so far, it seems that there was a slight difference between how right and left legs deployed. It seems either the left one (from the perspective of the camara) deployed some milliseconds earlier or the legs actually deployed at different speeds. Legs appear to reach a 90 degree angle with respect of rocket around frame 131 (in this frame they appear to be longer for this reason). That frame has also the best view of the legs, and with that we can confirm that they were in prime condition (there was some speculation here at some point of a possible hole in the right leg: that was just an image artifact that has been used by some here to actually get the macro blocks in the right place). Deployment continues and legs get to be close or at landing position at iframe 150. The precise end of leg deployment is somewhere in the p-frames that precede or follow iframe 150.

Phase 3: Touch down (Frames 169-229)
The next five seconds of the video (19:35:12-19:35:17), show a successful landing. We currently have 4 very clear iframes and several fixed p-frames from this phase. The rocket continued to fall vertically, this time with the legs (at least the two visible ones) completely deployed. The supersonic exhaust stream from the rocket becomes visible on iframe 189, as it hits the water and dissipates horizontally. Considering the tests done with the grasshopper and the 9FR, we can assume that the engine has been running all this time, but the flame was not visible before as it was directly under the rocket (I'm guessing only the center engine is running, while the ones outside are only used during lunch).  From the most recent p-frames (the sequence was just posted by SwissCheese) in this stage we can confirm that there is almost no rotation in the rocket body (we have enough detail on the ocean to see how things move from one frame to the next). Flame becomes smaller as the rocket gets even closer to the water (Frame 209). Actual touchdown is somewhere in the p-frames between 209-229. On iframe 229, legs seem to be submerged in water following a sucesful touchdown. 8)

Phase 4: Tip Off (Frames 229+)
The last two seconds (19:35:18-19:35:20) show what happens to the booster after it hits the water.  We currently have 2 reconstructed images of this phase. Legs appear to partially resurface after the landing. Instants later, the booster starts to tip off. From the original video it seems that it fells on its right side (we will be able to confirm this, once we analyze the pframes from this section). Camera finally dies, only two p-frames after iframe 268.

Many times, when things develop slowly, changes tend to pass unnoticed. So let's take a moment to step back, and actually reflect on what has been achieved.  Many of us came to this forum for the first time, looking for more details on this landing. And well... we have been able to obtain them!

Thanks to everybody that has contributed in some way or another! But in particular to the guys that have been working directly with the frames, and developing the different systems that we all have been using! You are all amazing!

Lets continue with the great work! there are still many frames to be fixed, and many more details to be uncovered!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/18/2014 08:15 pm
Do you get "nice images"? For me it corrects the first image but does nothing to the next ones...

I checked the code to understand a bit better, and put a printf there:

[snip]

it gives me for example this:
frame number ffmpeg: 3
frame number mmb: 3
token_frame = 6:20:-1,7:20:84967

it reads correctly all mmb sequences, and works as expected for the frame 0, but after that the frame numbers jump between frames 1,2 and 3 quite randomly. Is it the normal behavior? At this point I'm really lost... Maybe it's something related to windows?

Having this in the web app would for sure help a lot, we would have a reference program and only one set of data ;)

I also have mmbs for several P-frames, but can only use them on stand-alone *.mpg4-img frames
I will take a look into it, but my feeling is starting to be that P-frames act different individually than they do in combination with other frames. Even without mmb-changes. For example frame 170 looks very different individually compared to what it seems to change to frame 169 if its combined with it. Not sure yet.

Here is one test you can do. Add this green line of code:

Quote
static int get_bitpos_from_mmb_part (MpegEncContext *s, GetBitContext *gb, GetBitContext *gb_blank, int mb_x, int mb_y, const char *mmb_part, int *do_reverse, int *nr_of_hits_to_ignore) {

[snip]

        if (mb_x == mmb_x && mb_y == mmb_y && mb_y >= 0 && mb_x >= 0) {
            int i;
            if (mmb_pos < -2 || mmb_pos > gb->size_in_bits - 1) {
                av_log(NULL, AV_LOG_ERROR, "mmb bit pos invalid\n");
                return INT_MIN;
            }

            av_log(NULL, AV_LOG_ERROR, "new bitpos for %d:%d: from %d to %d\n", mmb_x, mmb_y, bitpos, mmb_pos);
            bitpos = mmb_pos;
That way you can see for which blocks it is actually changing its position. It will be colored red since its logged as error.

I will also try to figure this out myself. Since we trying something new here (P-frames in combination with others) we can expect some unexpected results. We'll just have to figure out what is going on.

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/18/2014 08:33 pm
A gif animation (reconstruction with matlab as described in a previous post) and all my mmb for the P-frames.
Wooow! Super!! :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Untribium on 05/18/2014 09:06 pm
lum/chroma correction is a nightmare though, particularly row 10 (dirt spot)...the luminance error doesn't seem to be a propagation problem, i.e. it occurs in several blocks (which is odd...)

What if it's a rainbow effect from the cloud we're passing through, or reflected light from the engine flare? Peering at the reconstructed Youtube video you can get a little sense of that in some of the intervening pframes.

Edit: -video-

See, for example, pale orange MB's in seconds 2 and 3, and nice orange flame around second 9 at the left side.

interesting point! I was also thinking that since the encoding is interlaced (right?), a very sudden change in brightness could actually result in different luminance values within the same macroblock. Apparently (according to the spec sheet posted earlier 6.1.3.6), the brightness values are not recorded simultaneously for all 4 subblocks...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: billh on 05/18/2014 09:28 pm
It's been said many times but, really, you guys are amazing!!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Untribium on 05/18/2014 09:35 pm
Frame 92:

23:14:49203 gives you a piece of the circle between the legs. Only the left blocks are correct though... Doesn't help the right side of the right leg either :p

Also there is something at 54419. I thought it was the edge of the rocket base but it doesn't fit anywhere. Might be a wave...

I think it *is* part of the rocket base! Put it at 24:15, perfect match. You can also remove what's at 34:15 to check, it works :) Nice find!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/18/2014 10:50 pm
PROGRESS REPORT

For one month now, we have been working on a video that was (for the most part) indecipherable. After many hours of work from a bunch of highly committed experts, now we have enough images to actually interpret what exactly happened with Space X's CRS-3 first stage booster, that evening of April 18. And let me tell you: these images tell an amazing story!

For this analysis, I considered all the recovered frames that are on the wiki and a few p-frames that have been posted here. To put things on a time line, I used the same clock that appears on the video itself.

Phase 1: Controlled Descent (Frames 1-52)
The first 4 seconds of the video (19:35:01-19:35:04), reveal the Controlled Descent of the booster. This part of the video isof 4 iframes and around 48 p-frames. All iframes from this phase where successfully extracted, but only two have been repaired. The p-frames also remain untouched The effort to recover images from this phase has been limited, as most of the work has been oriented into other sections of the video.

However, with the small number of images we have obtained so far, we can confirm that the booster went in a controlled descent over the ocean, falling vertically in the intended position for landing.  As we do not have any reference points, at this time it is impossible to asses the degree of rotation or the speed of the rocket. However, we can confirm that the booster was in one piece and looking good (expect for some splashes of dirty water, that actually have played an important role in the reconstruction of the frames)

Phase 2: Leg Deployment (Frames 72-150)
The next seven seconds of the video (19:35:5-19:35:12), with 5 iframes and about 80 p-frames, show the entire process of leg deployment. All five iframes have been fixed, and p-frames are slowly starting to emerge. From these, we can confirm that the leg deployment was successful. The latest version of frame 72 show no legs yet but leg deployment is clearly visible on frame 92. We can assume that the process started somewhere in between. 

From the images we have so far, it seems that there was a slight difference between how right and left legs deployed. It seems either the left one (from the perspective of the camara) deployed some milliseconds earlier or the legs actually deployed at different speeds. Legs appear to reach a 90 degree angle with respect of rocket around frame 131 (in this frame they appear to be longer for this reason). That frame has also the best view of the legs, and with that we can confirm that they were in prime condition (there was some speculation here at some point of a possible hole in the right leg: that was just an image artifact that has been used by some here to actually get the macro blocks in the right place). Deployment continues and legs get to be close or at landing position at iframe 150. The precise end of leg deployment is somewhere in the p-frames that precede or follow iframe 150.

Phase 3: Touch down (Frames 169-229)
The next five seconds of the video (19:35:12-19:35:17), show a successful landing. We currently have 4 very clear iframes and several fixed p-frames from this phase. The rocket continued to fall vertically, this time with the legs (at least the two visible ones) completely deployed. Flame from the rocket becomes visible on iframe 189, as it hits the water and dissipates horizontally. Considering the tests done with the grasshopper and the 9FR, we can assume that the engine has been running all this time, but the flame was not visible before as it was directly under the rocket (I'm guessing only the center engine is running, while the ones outside are only used during lunch).  From the most recent p-frames (the sequence was just posted by SwissCheese) in this stage we can confirm that there is almost no rotation in the rocket body (we have enough detail on the ocean to see how things move from one frame to the next). Flame becomes smaller as the rocket gets even closer to the water (Frame 209). Actual touchdown is somewhere in the p-frames between 209-229. On iframe 229, legs seem to be submerged in water following a sucesful touchdown. 8)

Phase 4: Tip Off (Frames 229+)
The last two seconds (19:35:18-19:35:20) show what happened to the booster after it hit water.  We currently have 2 reconstructed images of this phase. Legs appear to have partially resurfaced after landing, and afterwards the booster started to tip off. From the original video it seems that it fell on its right side. Camera finally died, only two p-frames after iframe 268.

Many times, when things develop slowly, changes tend to pass unnoticed. So let's take a moment to step back, and actually reflect on what has been achieved.  Many of us came to this forum for the first time, looking for more details on this landing. And well... know we have been able to obtain them!

Thanks to everybody that has contributed in some way or another! But in particular to the guys that have been working directly with the frames, and developing the different systems that we have been using! You are all amazing!

Lets continue with the great work! there are still many frames to be fixed, and many more details to be uncovered!
Love this post.  :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/18/2014 11:49 pm

The following is important for all of us that are working on the raw video stream file. I will attach 15 parts of raw_edit8.ts each containing a I-Frame and the following P-frames too a follow on post.

<snip>

Ok, now had time to make those parts: There is overlap between those parts. except for the last and the first it is always one 188 byte TS Packet before and one Packet after a block of I+P frames. This can give you some reference information on CC or so. Maybe you have to look also into raw.ts or raw_edit8.ts for further info if that block is messed up.
rawsplit_part_0 contains everything before the first I-Frame. Don't know if that is usefull at all.
rawsplit_part_15 contains only 2 P-frames AFAIK.
part_1 to part_14 should contain 1 I-Frame and 19 P-Frames each.
My idea is that people can pick on of the 15 and start fixing the TS-headers and more importantly the  P-Frame headers. I wrote how those should look like in the post quoted. First try to find the position of the P-Frame headers, they should be more or less equally distributed through the file. See if the counter bytes are messed up, calculate what they are, compare what they should be and change the header bytes if necessary.
If you have questions you can write me or ask here. I will check when I have more time.
When all parts are fixed we can put them all pack together and have us a new improved ts file from which we can hopefully extract all Frames.

Maybe someone can add a wiki page where people can write down which part they are fixing.

Cheers
Shanuson

But even with the work done already the latest video looks really impressive considering with what we started :D
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Sohl on 05/18/2014 11:50 pm
Yes, very nice summary post, moralec!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/19/2014 01:01 am
My idea is that people can pick on of the 15 and start fixing the TS-headers and more importantly the  P-Frame headers. I wrote how those should look like in the post quoted. First try to find the position of the P-Frame headers, they should be more or less equally distributed through the file. See if the counter bytes are messed up, calculate what they are, compare what they should be and change the header bytes if necessary.

The transport stream headers are mostly correct after the work to suss out the correct program IDs and sequence numbers, right? Are there any details we need to know about that? Also, if you have any other links regarding bitmaps for the packetized elementary stream headers and what have you, that'd be helpful. Thanks for all your work!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Untribium on 05/19/2014 02:00 am
Improved frame #92 and cleaned up the mmb...this is tough...
X:54419:00,X:53655:00,0:0:550,14:0:-2:0:0:0:0:12:-6,2:5:14749,39:7:-1,18:9:27558,25:12:39009,29:12:-1,
17:13:42121,18:13:42526,10:14:-1,18:14:47626,26:14:-2:0:0:0:0:8:-6,30:14:50837,1:15:-1,5:15:51900,9:15:-1,
21:15:53655,27:16:61466,42:19:-1,43:19:83153,43:20:-1,1:21:90585,11:22:-1,17:23:105394,22:24:-1,
23:24:111581,0:27:-1,25:27:125621,28:28:-1,3:29:140154
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/19/2014 02:46 am

The following is important for all of us that are working on the raw video stream file. I will attach 15 parts of raw_edit8.ts each containing a I-Frame and the following P-frames too a follow on post.

<snip>

Maybe someone can add a wiki page where people can write down which part they are fixing.

Cheers
Shanuson


I will take care of this tomorrow. I'm currently debating over different options... the current structure is not ideal. I hope I'll be able to come with a better design for the wiki, in order to keep things more organized. Any suggestions will be greatly appreciated.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/19/2014 06:06 am
I updated frame 131 (and can finally make .png as output):

Frame 131 from edit8:

X:614:84,X:638:1,X:822:a1,X:1166:4,X:1246:8,X:1430:4,12:0:-1,
7:2:5895,4:7:-1,11:7:20490,26:14:-1,18:15:56376,13:16:-1,
14:16:62505,20:16:-1,21:16:64068,27:16:-2,28:16:-1,32:16:65707,
25:20:-1,14:21:92080,24:25:-1,14:26:117098,41:26:-1,
14:27:120887,10:29:-1

Edit: Linked SwissCheese's previous version.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/19/2014 08:43 am

I will also try to figure this out myself. Since we trying something new here (P-frames in combination with others) we can expect some unexpected results. We'll just have to figure out what is going on.

Regards,

arnezami
Hi guys,

I think I have figured it out! :)

We were doing it wrong. By extracting the P-frames individually and then making png's of the mpg4-img files we were interpreting the P-frames as if they were I-frames! As you can imagine the mmb for such an instance would not work when the same frame was interpreted as a P-frame.

Attached are two interpretations of frame 170. One is simply the extraction of the png using the frame170.mpg4-img file. The other is an extraction of the png using fixed_edit8_part_169.ts with "FRAME0:0:0:-1". This mmb-command completely erases frame 169. Because of that you can see 170 in the next frame interpreted as a P-frame. Looks much better!

So when making mmb's for P-frames set all frames before it to "FRAMExx:0:0:-1". Then you will see the frame as interpreted as a P-frame.

Regards,

arnezami

PS. This is still an hypotheses, but I think this is actually correct or close to it.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/19/2014 09:11 am
My idea is that people can pick on of the 15 and start fixing the TS-headers and more importantly the  P-Frame headers. I wrote how those should look like in the post quoted. First try to find the position of the P-Frame headers, they should be more or less equally distributed through the file. See if the counter bytes are messed up, calculate what they are, compare what they should be and change the header bytes if necessary.

The transport stream headers are mostly correct after the work to suss out the correct program IDs and sequence numbers, right? Are there any details we need to know about that? Also, if you have any other links regarding bitmaps for the packetized elementary stream headers and what have you, that'd be helpful. Thanks for all your work!


All 15 I-Frame headers are ok. with all the right values for the variable parts.
For the 4 Byte TS-Header, there are good parts and bad parts in the ts-file. But at least they are all aligned now i think. so every 188th byte starts a new TS-packet like it should be. That does not mean there is a "47"-byte at every 188th position. Sometimes there are huge gaps spanning 10-20 ts-packets with no correct ts-header.

See the picture below. It shows the first few ts-packets of rawsplit_part_8.ts. You can see where Ts-Packets should be, marked by the black line. Blue circles show the correct CC bytes, in Ts-Headers in green are not correct. there are some bytes wrong, due to biterrors. Red shows the I-Frame Header. Cyan marks the two variable parts of the header, which are increasing.

Here they are in dez:
0x329c2B -> 3251243
0x8d228d -> 9249421

The following P-header is: 47 43 E8 3E 07 10 00 31 A7 E6 7E 00 00 00 01 E0 00 00 81 80 07 21 01 8D 51 79 FF FF 00 00 01 B6 51
coloured are both variable parts: there value in dez is:
31A7E6 -> 3254246 -> 3003 more than before, this is correct, because 20*3003=60060, the difference between I-Frames

The second is more complicated I just found out: There is a no equal distance between all PTS values of the I-Frames, but there is a pattern: So i would suggest to decode all PTS values and try to find a pattern. Apply it to all those where the pattern brakes.

       PTS Hex   PTS Dez   Difference to the last value
IF1    59797D     5863805   
IF2    6123ED     6366189   502384
IF3    67CE5D     6803037   436848
IF4    6F78CD     7305421   502384
IF5    77233D     7807805   502384
IF6    7DCDAD     8244653   436848
IF7    85781D     8747037   502384
IF8    8D228D     9249421   502384
IF9    93CCFD     9686269   436848
IF10   9B776D    10188653   502384
IF11   A321DD    10691037   502384
IF12   A9CC4D    11127885   436848
IF13   B176BD    11630269   502384
IF14   B9212D    12132653   502384
IF15   BFCB9D    12569501   436848


This was wrong, PTS has to be decode since it is 33 Bits spread over 5 bytes. See here (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1201981#msg1201981)

I hope this helps some.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/19/2014 10:35 am
I think I have figured it out! :)

Way to go!!  ;D That pframe looks absolutely delicious!

And look at all this nasty  interlacing - talk about different luminance values in the same block! Sheesh!  :o
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Untribium on 05/19/2014 11:33 am
I think I have figured it out! :)

Way to go!!  ;D That pframe looks absolutely delicious!

And look at all this nasty  interlacing - talk about different luminance values in the same block! Sheesh!  :o

ouch...and I got ticked off by this one row in 52... :o Hopefully it doesn't show in the video :)
I love how the legs are "cut out" in 170 :D
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: JamesH on 05/19/2014 02:51 pm
Please forgive me if this has already been covered somewhere in the previous 50 pages....trying to get my understanding correct.

As I understand it I-frames are complete standalone frames - they need no other data to represent aa full frame. P frames are difference frames (as are B but I don't think they are used here). A p-frame contains all the information required to go from the previous P or I frame to the current frame. The result of applying a P-frame to the previous frame should be the current frame. A p-frame is a bunch of 64x64 macroblocks, plus offset information (where the 64x64 base data came from). These contain the difference information from wherever the MB came from and where is it now. So the current frame is basically the previous frame (fully rendered) with the current MB set applied over the top. The result should be a perfect representation of the frame, minus any high frequency compression differences (a bit like JPEG)

So, if you have a broken/completely wrecked) I-frame you can generate a new one by working out what the previous frame looks like, and using that. Simply inject a new i-frame in to the bitstream. Or, you could, if you have a partial I-frame, generate one from a combination of what you do have that looks OK, plus any data from the prewious frame.

And of course in H264 every frame can be an I-frame, so you could process through, creating a new bitstream from the old one, replacing all P-frames with I-frames.

Is that what people are attempting?

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/19/2014 03:31 pm
So, if you have a broken/completely wrecked) I-frame you can generate a new one by working out what the previous frame looks like, and using that. Simply inject a new i-frame in to the bitstream. Or, you could, if you have a partial I-frame, generate one from a combination of what you do have that looks OK, plus any data from the prewious frame.

Pretty good understanding so far. However, the trouble is that every last one of the iframes in the stream that was just barely received from the first stage were more or less wrecked, so there wasn't a previous frame to use.

If you check out the progress site (http://spacexlanding.wikispaces.com/Progress+on+edit5.ts), you can see how bad things were at the beginning:

(http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=583287)

The focus over the past few weeks has been to tease out reasonable-looking iframes from the jumble of bits, and since we're in reasonably good shape there, the focus has started to shift to teasing out reasonable-looking pframes to go with them, as you've observed in the last several posts.

Here's what a seven-year veteran of video repair, Benoît Joossen, said (http://aeroquartet.com/wordpress/2014/05/08/spacex-falcon-first-stage-landing-part-ii/) after SpaceX had engaged him to scrape out the two partial frames that Elon posted on Twitter:

Quote
Why the video stream can’t be recovered
...
I’ve been repairing videos full time for the last 7 years, so I immediately knew that the bitstream was corrupt and that at best we could hope to partially fix some frames. Maybe a 50% chance of recovering one or two frames.

Based on previous “edge-of-impossible” repairs, I know that the error rate must be close to zero. A 20kb frame, for example, contains 160,000 bits. If more than 2 or 3 of them are wrong, it becomes impossible to repair it.

This was absolutely true when he wrote it, but the amazing fact is that over the past 50 pages and few weeks worth of work, the participants here have greatly advanced the state of the art of video recovery, through the repair of the transport stream headers, the presentation timestamps, correction of individual macroblocks, and so on. And of course, the indispensible aid from Michael Niedermayer himself, the curator of the "ffmpeg" tool since 2004, was a key part of that advance.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/19/2014 03:32 pm
Please forgive me if this has already been covered somewhere in the previous 50 pages....trying to get my understanding correct.

As I understand it I-frames are complete standalone frames - they need no other data to represent aa full frame. P frames are difference frames (as are B but I don't think they are used here). A p-frame contains all the information required to go from the previous P or I frame to the current frame. The result of applying a P-frame to the previous frame should be the current frame. A p-frame is a bunch of 64x64 macroblocks, plus offset information (where the 64x64 base data came from). These contain the difference information from wherever the MB came from and where is it now. So the current frame is basically the previous frame (fully rendered) with the current MB set applied over the top. The result should be a perfect representation of the frame, minus any high frequency compression differences (a bit like JPEG)

So, if you have a broken/completely wrecked) I-frame you can generate a new one by working out what the previous frame looks like, and using that. Simply inject a new i-frame in to the bitstream. Or, you could, if you have a partial I-frame, generate one from a combination of what you do have that looks OK, plus any data from the prewious frame.

And of course in H264 every frame can be an I-frame, so you could process through, creating a new bitstream from the old one, replacing all P-frames with I-frames.

Is that what people are attempting?
Hi JamesH,

Thanks for the idea. I believe it has not been suggested, but I don't think it will work in practice. It might work in situation where there is a I-frame broken due to some kind of localized media defect. But with random noise like we have its unlikely to be feasable.

The reason for this is that it is very unlikely that the cumulative errors from 19 P-frames are going to less than the errors in the I-frame following it. In other words: all errors propagate so the last P-frame before an I-frame is going to be the worst of all frames.

Besides that: we are not even close to decoding the 19th P-frame. More like first or second.

That is unless we get lucky and there are some intra-macroblocks in this last P-frame, then it might work a little.

But I like your thinking. Maybe you can come up with other ideas. It's a tough problem though when you go into the details. ;)

Regards,

arnezami

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: JamesH on 05/19/2014 03:40 pm
If I get a chance, I'll have a chat with our H264 codec team (I work for a company that amongst other things designs and build H264 encoder/decoder chips). It's not my field, though I do have to deal with it sometimes, and sadly the guy who sat opposite me until he left to cycle round NZ a few months ago was quite an expert.

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/19/2014 03:44 pm
If I get a chance, I'll have a chat with our H264 codec team (I work for a company that amongst other things designs and build H264 encoder/decoder chips). It's not my field, though I do have to deal with it sometimes, and sadly the guy who sat opposite me until he left to cycle round NZ a few months ago was quite an expert.
Just point them to this thread (or the wiki). Any help is welcome. :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: JamesH on 05/19/2014 03:45 pm
Actually, thinking more about repairing I-frames, although you may not be able to reconstruct an entire I-frame from the previous P-frame set due to bad cumulative error, the P-frame set MAY be able to fill in gaps in the I-frame if valid data does exist. Might recover a few blocks here and there.

What is the I-frame rate in the bitstream btw? (Intra-refresh?)  If I were SpaceX I'd increase that at the expense of higher bandwidth - gives much better frame recovery in the event of transmission issues.


Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/19/2014 03:59 pm

I will also try to figure this out myself. Since we trying something new here (P-frames in combination with others) we can expect some unexpected results. We'll just have to figure out what is going on.

Regards,

arnezami
Hi guys,

I think I have figured it out! :)

We were doing it wrong. By extracting the P-frames individually and then making png's of the mpg4-img files we were interpreting the P-frames as if they were I-frames! As you can imagine the mmb for such an instance would not work when the same frame was interpreted as a P-frame.

Attached are two interpretations of frame 170. One is simply the extraction of the png using the frame170.mpg4-img file. The other is an extraction of the png using fixed_edit8_part_169.ts with "FRAME0:0:0:-1". This mmb-command completely erases frame 169. Because of that you can see 170 in the next frame interpreted as a P-frame. Looks much better!

So when making mmb's for P-frames set all frames before it to "FRAMExx:0:0:-1". Then you will see the frame as interpreted as a P-frame.

Regards,

arnezami

PS. This is still an hypotheses, but I think this is actually correct or close to it.

That looks good! But when I try doing the same, I don't get this, I get +/- empty frames until the 6th P-frame... Could you generate the *.mpg4-img files for everyone? or at least a test sample between 169 and 189?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/19/2014 04:00 pm
Actually, thinking more about repairing I-frames, although you may not be able to reconstruct an entire I-frame from the previous P-frame set due to bad cumulative error, the P-frame set MAY be able to fill in gaps in the I-frame if valid data does exist. Might recover a few blocks here and there.

What is the I-frame rate in the bitstream btw? (Intra-refresh?)  If I were SpaceX I'd increase that at the expense of higher bandwidth - gives much better frame recovery in the event of transmission issues.

There are plenty of opportunities to pull blocks forward to make up for missing data. At full speed it noticeably degrades video quality, so the less we have to do this the better.

We get an i-frame once every 1.3 seconds or so. I-frame plus 19 p-frames at 15 fps.

There are a number of things SpaceX could do to improve video transmission on future flights. Most importantly adding some sort of data integrity check and turning off interlacing as mvpel mentioned a few pages back. I think they are probably using an off-the-shelf video system, so any improvements would have to come from the vendor.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/19/2014 04:30 pm
That looks good! But when I try doing the same, I don't get this, I get +/- empty frames until the 6th P-frame... Could you generate the *.mpg4-img files for everyone? or at least a test sample between 169 and 189?
Hi,

I posted all .mpg4-img files here (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1200144#msg1200144) in a zip called fixed_edit8_all_frames_mpg4-img.zip (http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=584047)

What happens if you download fixed_edit8_part_169.zip (http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=584057). Unzip it. Then edit mmb_starting_from_169.txt to contain "FRAME0=0:0:-1=FRAME1=0:0:-1=FRAME2=0:0:-1". And run this command:

Quote
./ffmpeg -r 44999/3003 -mmb `cat mmb_starting_from_169.txt` -debug mb_pos_size -err_detect ignore_err  -i fixed_edit8_part_169.ts -f image2 fixed_edit8_part_169_%03d.png

You should get three gray frames followed by the 3rd P-frame. And you should be able to do this for every n-th frame. I will test this myself later when I have some more time but I expect this to work.

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/19/2014 04:49 pm
Hi,

I just tested it. It works fine for me.

See attachments  8)

Regards,

arnezami

PS. I used these mmmb's (multiple-mmb's)

For frame 170:
Quote
FRAME0:0:0:-1

For frame 171:
Quote
FRAME0:0:0:-1=FRAME1:0:0:-1

For frame 172:
Quote
FRAME0:0:0:-1=FRAME1:0:0:-1=FRAME2:0:0:-1

For frame 173:
Quote
FRAME0:0:0:-1=FRAME1:0:0:-1=FRAME2:0:0:-1=FRAME3:0:0:-1

And you can rinse and repeat this of course. Of course ignore the gray frames you've just disabled. They obviously won't contain data. The first non-gray picture is the one you're after. Which is non-disturbed by the gray frames before it.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/19/2014 05:06 pm
That's really great stuff, Arnezami!

You can really see how each pframe relates back to the previous pframe, like in 171-172-173 where you see the expanding front of the water being blasted outward with the saturated white froth blocks remaining unchanged. It's an excellent way to visualize how the MPEG encoding actually works.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/19/2014 05:07 pm
That looks good! But when I try doing the same, I don't get this, I get +/- empty frames until the 6th P-frame... Could you generate the *.mpg4-img files for everyone? or at least a test sample between 169 and 189?
Hi,

I posted all .mpg4-img files here (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1200144#msg1200144) in a zip called fixed_edit8_all_frames_mpg4-img.zip (http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=584047)

What happens if you download fixed_edit8_part_169.zip (http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=584057). Unzip it. Then edit mmb_starting_from_169.txt to contain "FRAME0=0:0:-1=FRAME1=0:0:-1=FRAME2=0:0:-1". And run this command:

Quote
./ffmpeg -r 44999/3003 -mmb `cat mmb_starting_from_169.txt` -debug mb_pos_size -err_detect ignore_err  -i fixed_edit8_part_169.ts -f image2 fixed_edit8_part_169_%03d.png

You should get three gray frames followed by the 3rd P-frame. And you should be able to do this for every n-th frame. I will test this myself later when I have some more time but I expect this to work.

Regards,

arnezami

I know, but I cannot manage to make the multiple mmb work correctly, still the same issue... Maybe I should try to install everything on a unix/linux machine?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/19/2014 06:20 pm
I know, but I cannot manage to make the multiple mmb work correctly, still the same issue... Maybe I should try to install everything on a unix/linux machine?
Are you using the mmb's your earlier created for the mpg4-img files? Because they are indeed incompatible with multiple frame decoding. In other words: they are useless.

You have to create new mmb's by using the multiple mmb format and with the fixed_edit8_part_169.ts file. Or is that also not working for you?

I will try it myself soon.

Regards,

arnezami

[edit] To make it absolutely clear: you should not use the mpg4-img files for P-frames!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/19/2014 06:23 pm

The following is important for all of us that are working on the raw video stream file. I will attach 15 parts of raw_edit8.ts each containing a I-Frame and the following P-frames too a follow on post.

<snip>

Maybe someone can add a wiki page where people can write down which part they are fixing.

Cheers
Shanuson


I will take care of this tomorrow. I'm currently debating over different options... the current structure is not ideal. I hope I'll be able to come with a better design for the wiki, in order to keep things more organized. Any suggestions will be greatly appreciated.

Hello everyone. I have now finished updating the wiki. The main page now contains:

- A brief introduction
- Links to the most important technical posts from this thread (it was nice to read the entire thread again)
- Some external resources, particularly useful for those that are just starting to contribute,
- Links to the official tools we are using (FFmpeg utility, Online editors)
- A Progress Reports and Media section, for those that are mostly interested in the results.
- A Work in Progress with links to the pages where we are pasting the latest reconstructed iframes AND a new page for the work that is being done on the pframes
- Links to the places where the discussion is being held (Here and in IRC)
- Additional External Links

Enjoy
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Swatch on 05/19/2014 06:25 pm
And as a general reminder/note to newcomers...

Wiki:  http://spacexlanding.wikispaces.com/
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/19/2014 06:37 pm
This is fixed.
Great! Works well now :)

I've been thinking about the P-frames for the online editor. I think what would be useful and fairly easy to add is a third dropdown. What you get is first the option of the ts file (raw_edit8 for example). Then you select the "base I-frame" in the second dropdown. And then you can choose the actual frame (which would default to the first frame (=I-frame)) in the third dropdown.

In the case of edit8 the third drowdown not only contains the I-frame but also the P-frames within the chosen base I-frame. And you keep using the framexxx.mpg4-img files as your input to ffmpeg. That way more ppl can start working on the P-frames. In the case of try1 and coerced you only get one option in the third dropdown, which is always chosen.

Of course this will only show the P-frame individually (not fed by I-frame data). And we should also make sure we can run it though the multi-mmb call to ffmpeg, but thats for later I guess.

Regards,

arnezami
Hi IainCole,

Since I last posted this we have come to a better understanding of how to decode the P-frames. Most importantly it looks like we should not use the framexxx.mpg4-img files for P-frames!

Instead what can be done is the following:

* Let somebody choose a P-frame (as described in the above quote with a third dropdown)
* Instead of using ffmpeg with the mpg4.img file you should use this command:
Quote
./ffmpeg -r 44999/3003 -mmb <the mmb commands> -debug mb_pos_size -err_detect ignore_err  -i fixed_edit8_part_169.ts -f image2 fixed_edit8_part_169_%03d.png
Note that this command is only valid for the P-frames 170 - 188. So the 169 should be replaced by the I-frame number this P-frame "belongs to" (the P-frame is in a part that starts with an I-frame, so you need to tell ffmpeg to use that .ts). The mmb-commands are the mmb commands the user submit. But these commands are prefixed with a series of null-commands like this (in the case this is the fourth P-frame counted from the I-frame which would be 173):
Quote
FRAME0:0:0:-1=FRAME1:0:0:-1=FRAME2:0:0:-1=FRAME3:0:0:-1=FRAME4=
* Since this creates around 20 files it is wise to remove all the pngs expect the one from the fourth P-frame Keep in mind ffmpeg starts counting with 1 for the files while it starts counting with 0 for the mmbs. So it would be file number 5 in this case.
* [edit] As for the log: you probably don't want the entire log in the browser. Only the part that concerns the frame in question. You would somehow have to detect when it starts with that frame.

That way we should be able to create valid mmb's for P-frames online :).

Regards,

arnezami

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/19/2014 06:45 pm
OK, I'll try and think of a decent way to make this work
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/19/2014 06:53 pm
How am I supposed to interpret the log? before I was just looking as MB positions and parsing that to give you info n the editor. How does this work for p frames?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/19/2014 07:08 pm
How am I supposed to interpret the log? before I was just looking as MB positions and parsing that to give you info n the editor. How does this work for p frames?

You could count the number of occurrences of this:

Quote
[force CFR for input from stream 0:0 @ 0x1b354a0] N:0 PTS:0 T:0.000000 POS:0 INTERLACED:0 -> PTS:0 T:0.000000
[AVIOContext @ 0x1b410e0] Statistics: 0 seeks, 1 writeouts

This is repeated for each frame. This is the next:

Quote
[force CFR for input from stream 0:0 @ 0x1b354a0] N:1 PTS:6006 T:400.809307 POS:21432 INTERLACED:0 -> PTS:1 T:0.066735
[AVIOContext @ 0x1ba7cc0] Statistics: 0 seeks, 1 writeouts

Also, the log was extended with the frame number:

Quote
[mpeg4 @ 0x1b49d00] MB pos/size: 0 002:03:01:3521 128 dc: 140 159 28 32 - 128 129, MB_type: 513, MV: 0 0

But I dont think thats a good way of filtering all log lines since error lines will not contain the frame number. So I would go for "[AVIOContext @ 0x1ba7cc0] Statistics: 0 seeks, 1 writeouts" and count the number of times you say that. Keep in mind that 0x1ba7cc0 could be different. Also the number of seeks and writeouts.

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/19/2014 07:12 pm
It the latest version of the ffmpeg code on the git repo? I'm thinking of doing a completely new editor in a different way which should lend itself to doing this a bit better.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/19/2014 07:23 pm
It the latest version of the ffmpeg code on the git repo? I'm thinking of doing a completely new editor in a different way which should lend itself to doing this a bit better.
I believe not.

My latest update is in this post: http://forum.nasaspaceflight.com/index.php?topic=34597.msg1200144#msg1200144

I believe if you take the latest git and overwrite the h263dec.c from my latest update you should be ok.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/19/2014 07:29 pm
Hi guys,

Proof of concept time 8)

Here are the first 4 frames of the fixed_edit8_part_169.ts file.

I did a bit of a quick and dirty hack job but you can see multiple mmb commands are working nicely :).

Here is the mmmb I used:

FRAME0:08:14:-1:15:-15:1:14:-2:1,
09:14:56683,
05:21:-1:0:-20:-5:2,
06:21:-1:-20:-20:6:5,
07:21:-1:0:0:3:5,
08:21:-1:0:0:0:0,
09:21:-1:10:15:0:0,
10:21:-1:15:15:0:0,
11:21:-1:5:10,
12:21:-1:5:3:0:0:-2,
13:21:-1:4:4,
14:21:-1,
28:21:108878,
X:126932:80,
04:27:-1:0:-4:13:7,
05:27:135412,
X:143273:80,
X:143386:80,
06:28:144012
=FRAME1:2:8:21884
=FRAME2:0:0:16348

You can see the P-frames blend in now with the frames before them. Also added an animated GIF just for kicks.  8)

Regards,

arnezami

[edit] I used this command:

Quote
./ffmpeg -r 44999/3003 -mmb `cat mmb_starting_from_169.txt` -debug mb_pos_size -err_detect ignore_err  -i fixed_edit8_part_169.ts -f image2 fixed_edit8_part_169_%03d.png
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/19/2014 07:34 pm
As far as the editor functionality is concerned, do p frames only contain some macroblocks per frame? should the user only be able to add mmb operations in certain x/y positions? Or should they be able to do what they want?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/19/2014 07:39 pm
As far as the editor functionality is concerned, do p frames only contain some macroblocks per frame? should the user only be able to add mmb operations in certain x/y positions? Or should they be able to do what they want?
No. They contain all macro blocks. Some macro blocks contain very tiny bits of data (like: don't change this block) but it's still data. In fact it's possible P-frames might need some additional features: we might sometime be able to change the block type for example.

In other words: let them change whatever they want.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/19/2014 07:45 pm
OK cool, am I right in thinking there's a whole bunch of extra stuff coming out of the log now? Also some things don't seem to be coming out in "order" as far as frame numbers go.

Quote
[mpeg4 @ 0x60008d3c0] MB pos/size: 0 001:29:08:0 1 dc: 128 128 128 128 - 128 128MB pos/size: 0 006:29:03:0 1 dc: 0 0 0 0 - 0 0, MB_type: 14344, MV: 0 0
[mpeg4 @ 0x60009a2a0] , MB_type: 14344, MV: 0 0
[mpeg4 @ 0x6000a36a0] MB pos/size: 0 009:31:00:0 1 dc: 0 0 0 0 - 0 0, MB_type: 14344, MV: 0 0
[mpeg4 @ 0x60009d440] MB pos/size: 0 007:30:02:0 1 dc: 0 0 0 0 - 0 0, MB_type: 14344, MV: 0 0
[mpeg4 @ 0x60008dde0] MB pos/size: 0 002:29:07:0 1 dc: 0 0 0 0 - 0 0, MB_type: 14344, MV: 0 0
[mpeg4 @ 0x600090f80] MB pos/size: 0 003:30:06:0 1 dc: 0 0 0 0 - 0 0, MB_type: 14344, MV: 0 0
[mpeg4 @ 0x6000a0500] MB pos/size: 0 008:31:01:0 1 dc: 0 0 0 0 - 0 0, MB_type: 14344, MV: 0 0
[mpeg4 @ 0x600094040] MB pos/size: 0 004:28:05:0 1 dc: 0 0 0 0 - 0 0, MB_type: 14344, MV: 0 0
[mpeg4 @ 0x6000971e0] MB pos/size: 0 005:29:04:0 1 dc: 0 0 0 0 - 0 0, MB_type: 14344, MV: 0 0
[mpeg4 @ 0x60008d3c0] MB pos/size: 0 001:30:08:0 1 dc: 128 128 128 128 - 128 128, MB_type: 14344, MV: 0 0

Is that normal?

I also have a few lines that look like this:

Quote
[mpeg4 @ 0x600090f80] MB pos/size: 0 000:09:20:0 1 dc: 0 0 0 0 - 0 0MB pos/size: 0 000:10:19:0 1 dc: 0 0 0 0 - 0 0, MB_type: 14344, MV: 0 0
[mpeg4 @ 0x6000971e0] MB pos/size: 0 000:11:18:0 1 dc: 0 0 0 0 - 0 0, MB_type: 14344, MV: 0 0
[mpeg4 @ 0x6000971e0] , MB_type: 14344, MV: 0 0

There's 2 MB pos/size things on 1 line and a little further down some stuff is missing.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/19/2014 07:49 pm
OK cool, am I right in thinking there's a whole bunch of extra stuff coming out of the log now? Also some things don't seem to be coming out in "order" as far as frame numbers go.

Quote
[mpeg4 @ 0x60008d3c0] MB pos/size: 0 001:29:08:0 1 dc: 128 128 128 128 - 128 128MB pos/size: 0 006:29:03:0 1 dc: 0 0 0 0 - 0 0, MB_type: 14344, MV: 0 0
[mpeg4 @ 0x60009a2a0] , MB_type: 14344, MV: 0 0
[mpeg4 @ 0x6000a36a0] MB pos/size: 0 009:31:00:0 1 dc: 0 0 0 0 - 0 0, MB_type: 14344, MV: 0 0
[mpeg4 @ 0x60009d440] MB pos/size: 0 007:30:02:0 1 dc: 0 0 0 0 - 0 0, MB_type: 14344, MV: 0 0
[mpeg4 @ 0x60008dde0] MB pos/size: 0 002:29:07:0 1 dc: 0 0 0 0 - 0 0, MB_type: 14344, MV: 0 0
[mpeg4 @ 0x600090f80] MB pos/size: 0 003:30:06:0 1 dc: 0 0 0 0 - 0 0, MB_type: 14344, MV: 0 0
[mpeg4 @ 0x6000a0500] MB pos/size: 0 008:31:01:0 1 dc: 0 0 0 0 - 0 0, MB_type: 14344, MV: 0 0
[mpeg4 @ 0x600094040] MB pos/size: 0 004:28:05:0 1 dc: 0 0 0 0 - 0 0, MB_type: 14344, MV: 0 0
[mpeg4 @ 0x6000971e0] MB pos/size: 0 005:29:04:0 1 dc: 0 0 0 0 - 0 0, MB_type: 14344, MV: 0 0
[mpeg4 @ 0x60008d3c0] MB pos/size: 0 001:30:08:0 1 dc: 128 128 128 128 - 128 128, MB_type: 14344, MV: 0 0

Is that normal?
Nope. Never seen that before. Which command did you use?
Quote
I also have a few lines that look like this:

Quote
[mpeg4 @ 0x600090f80] MB pos/size: 0 000:09:20:0 1 dc: 0 0 0 0 - 0 0MB pos/size: 0 000:10:19:0 1 dc: 0 0 0 0 - 0 0, MB_type: 14344, MV: 0 0
[mpeg4 @ 0x6000971e0] MB pos/size: 0 000:11:18:0 1 dc: 0 0 0 0 - 0 0, MB_type: 14344, MV: 0 0
[mpeg4 @ 0x6000971e0] , MB_type: 14344, MV: 0 0

There's 2 MB pos/size things on 1 line and a little further down some stuff is missing.
Same question.

You use this in linux right?

Did you clone the latest version from git and added my h263dec.c?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/19/2014 07:50 pm
I'm compiling for windows under cygwin, so could be that, using the latest git repo with that file changed.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/19/2014 08:02 pm
I'm compiling for windows under cygwin, so could be that, using the latest git repo with that file changed.
'
Which command you use?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Llian Rhydderch on 05/19/2014 08:03 pm
Hi guys,

Proof of concept time 8)

Here are the first 4 frames of the fixed_edit8_part_169.ts file.

I did a bit of a quick and dirty hack job but you can see multiple mmb commands are working nicely :).

Here is the mmmb I used:

FRAME0:08:14:-1:15:-15:1:14:-2:1,
---snip--
You can see the P-frames blend in now with the frames before them. Also added an animated GIF just for kicks.  8)

Regards,

arnezami

[edit] I used this command:

Quote
./ffmpeg -r 44999/3003 -mmb `cat mmb_starting_from_169.txt` -debug mb_pos_size -err_detect ignore_err  -i fixed_edit8_part_169.ts -f image2 fixed_edit8_part_169_%03d.png

Thanks Arnezami!  It's great to see those particular four frames as they show the supersonic exhaust stream impacting the water over time.

This is really awesome work you are all doing! 

There are a great many of us watching the results you are getting in the OpenSource video recovery effort!  We are very appreciative of what all of you are building, a first-rate video documenting an important first in history of human technology, right before our eyes.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/19/2014 08:05 pm
I'm compiling for windows under cygwin, so could be that, using the latest git repo with that file changed.
'
Which command you use?

This one

Quote
ffmpeg.exe -r 44999/3003 -mmb `cat mmb.txt` -debug mb_pos_size -err_detect ignore_err -i fixed_edit8_part_169.ts -f image2 fixed_edit8_part_169_%03d.png

Edit: this is mmb.txt "FRAME0:0:0:-1=FRAME1:0:0:-1=FRAME2:0:0:-1=FRAME3:0:0:-1"
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/19/2014 08:34 pm
You can see the P-frames blend in now with the frames before them. Also added an animated GIF just for kicks.  8)

(http://moodyeyeview.files.wordpress.com/2013/04/we.png)

Ultra-sick excellent, my friend. Well done!! Even the reflection off the body of the rocket - niiiice!  ;D That's one heck of a proof of concept!

Edit: Elon should Tweet that landing GIF!! Someone call him!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Asmegin on 05/19/2014 08:37 pm
Amazin work, arnezami! Can't wait to get started on the pframes :D

"It always seems impossible until it's done"
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/19/2014 08:42 pm
Proof of concept time 8)


That gif looks awsome! (I'm adding it on the wiki).


On other news the table in the wiki is ready, for those that are fixing the headers on the pframes :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/19/2014 08:50 pm
I know, but I cannot manage to make the multiple mmb work correctly, still the same issue... Maybe I should try to install everything on a unix/linux machine?
Are you using the mmb's your earlier created for the mpg4-img files? Because they are indeed incompatible with multiple frame decoding. In other words: they are useless.

You have to create new mmb's by using the multiple mmb format and with the fixed_edit8_part_169.ts file. Or is that also not working for you?

I will try it myself soon.

Regards,

arnezami

[edit] To make it absolutely clear: you should not use the mpg4-img files for P-frames!

That's what I'm doing, just copy paste the command lines you gave, and it doesn't work... It only works for the first frame. Already spent one day trying to understand why, but nothing... It reads correctly the mmb for all frames, just does not apply them correctly.

I'm trying to install ffmpeg on a unix machine, but since I have no superuser rights, I have issues, can't get ffmpeg using yasm, so I cannot compile... and I'm really no linux/unix specialist :P
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/19/2014 08:57 pm
Proof of concept time 8)


That gif looks awsome! (I'm adding it on the wiki).


On other news the table in the wiki is ready, for those that are fixing the headers on the pframes :)
I have difficulty finding stuff in the wiki now. Shouldn't the stuff we are working on be right at the top? Also this pframe pages, how do I find them? I'm really confused atm.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: catdlr on 05/19/2014 09:01 pm
Proof of concept time 8)


That gif looks awsome! (I'm adding it on the wiki).


On other news the table in the wiki is ready, for those that are fixing the headers on the pframes :)
I have difficulty finding stuff in the wiki now. Shouldn't the stuff we are working on be right at the top? Also this pframe pages, how do I find them? I'm really confused atm.

Click on any of the three links in the "Work In Progress"
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/19/2014 09:03 pm
I'm trying to install ffmpeg on a unix machine, but since I have no superuser rights, I have issues, can't get ffmpeg using yasm, so I cannot compile... and I'm really no linux/unix specialist :P

I am, so let me give you a hand. I set up a virtual CentOS 6.5 box on my Windows 7 laptop just yesterday, in fact. Download and install Oracle Virtualbox, and then find the image at http://virtualboxes.org/images/centos/ - open that up as a virtual disk, and then you'll have your own full-fledged Linux system complete with yum repositories already configured for installing software like gcc and yasm.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/19/2014 09:09 pm
Proof of concept time 8)


That gif looks awsome! (I'm adding it on the wiki).


On other news the table in the wiki is ready, for those that are fixing the headers on the pframes :)
I have difficulty finding stuff in the wiki now. Shouldn't the stuff we are working on be right at the top? Also this pframe pages, how do I find them? I'm really confused atm.

Thanks for the feedback. I'm moving that section up now.

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/19/2014 09:10 pm
It works! I found the problem!

 :D ;D :)

I simply have to disable the multi-threading with option -threads 1

So yes, it was related to windows...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/19/2014 09:12 pm
It works! I found the problem!

 :D ;D :)

I simply have to disable the multi-threading with option -threads 1

So yes, it was related to windows...
I love you for that!  :) :) :)

I didn't have a clue what was wrong.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/19/2014 09:20 pm
Was working on iframe 52 today. All I got was a couple of macroblocks (some questionable) and a deep appreciation of the work done by Untribium. But I think it's a small improvement anyway.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/19/2014 09:29 pm
Proof of concept time 8)


That gif looks awsome! (I'm adding it on the wiki).


On other news the table in the wiki is ready, for those that are fixing the headers on the pframes :)
I have difficulty finding stuff in the wiki now. Shouldn't the stuff we are working on be right at the top? Also this pframe pages, how do I find them? I'm really confused atm.

Thanks for the feedback. I'm moving that section up now.

Hey guys,
I just moved the Work in Progress section in the wiki to the beginning of the page (see picture).

There you will find links to both active and archived  pages. For example, as of now we have:
1) A page for the reconstruction of i and p frames using edit8
2) The fixing of p frames headers
3) The reconstruction of i-frames using old ts files - which no one should be using now.

:)

Hope this works. Whenever you have a second, add your progress there.

If you prefer to focus on your work, just post things here and let me know. I can take care of it.

http://spacexlanding.wikispaces.com/home
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/19/2014 11:59 pm
So, it seems all the puzzle pieces are finally getting together, a big congratulation to Arnezami for finding how to handle these P-frames :)

Here the best I could quickly get for the same frames:

% I-Frame 169
08:14:-1:15:-15:1:14:-2:1,09:14:56683,05:21:-1:0:-20:-5:2,06:21:-1:-20:-20:6:5,
07:21:-1:0:0:3:5,08:21:-1:0:0:0:0,09:21:-1:10:15:0:0,10:21:-1:15:15:0:0,11:21:-1:5:10,
12:21:-1:5:3:0:0:-2,13:21:-1:4:4,14:21:-1,28:21:108878,X:126932:80,04:27:-1:0:-4:13:7,
05:27:135412,X:143273:80,X:143386:80,5:28:-2:-30:-30:-10:-10,06:28:144012,2:29:-1

% Frame 170
% 6:1:-1,23:3:9697,40:7:-1,37:8:22779 (strange interlace effect)
6:1:-1,23:3:9697,40:7:-1,42:7:19526,3:8:-1,4:8:19780,13:8:-1,15:8:20531,28:8:-1,30:8:21401,37:8:22779 (effect less visible)

% Frame 171
16:0:-1,31:0:4850,4:2:13866,35:7:-1,42:7:46457,22:8:46996,32:8:50739

% Frame 172
6:20:-1,7:20:84967

% Frame 173
0:7:-1,3:8:21583,17:11:-1,20:11:37003,8:12:-1,32:13:43824,0:19:-1 (too tired to do better :P)

And also a gif animation!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: S.Paulissen on 05/20/2014 12:15 am
I've been following this thread and website for some time and have been looking for a way to have a productive first post.  This, however, will not be that productive.  Fantastic work everyone who's working on this project.  I'll keep tooling around with frames but ... holy smokes you guys are good at this.  I don't have a lot to bring to the table.

On frame 52 of raw_edit8.ts I got a few extra blocks in the bottom left with some highly suspect changes.

40:3:-1,6:4:9364,4:5:-1,6:5:11443,7:5:11487,5:6:13496:0:0:0:0:0:-2,0:7:-1,38:11:26080:0:-4:-1:2:0:0,
31:12:28067:3:1:0:0:0:0,34:12:28226:-1:0:0:0:0:0,40:14:34617:-3:0:2:0:0:0,40:16:42426:-1:0:-4:-5:0:0,
33:17:44961:0:-7:0:0:-2:0,7:28:18
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/20/2014 12:31 am
OK cool, am I right in thinking there's a whole bunch of extra stuff coming out of the log now? Also some things don't seem to be coming out in "order" as far as frame numbers go.

Quote
[mpeg4 @ 0x60008d3c0] MB pos/size: 0 001:29:08:0 1 dc: 128 128 128 128 - 128 128MB pos/size: 0 006:29:03:0 1 dc: 0 0 0 0 - 0 0, MB_type: 14344, MV: 0 0
[mpeg4 @ 0x60009a2a0] , MB_type: 14344, MV: 0 0
[mpeg4 @ 0x6000a36a0] MB pos/size: 0 009:31:00:0 1 dc: 0 0 0 0 - 0 0, MB_type: 14344, MV: 0 0
[mpeg4 @ 0x60009d440] MB pos/size: 0 007:30:02:0 1 dc: 0 0 0 0 - 0 0, MB_type: 14344, MV: 0 0
[mpeg4 @ 0x60008dde0] MB pos/size: 0 002:29:07:0 1 dc: 0 0 0 0 - 0 0, MB_type: 14344, MV: 0 0
[mpeg4 @ 0x600090f80] MB pos/size: 0 003:30:06:0 1 dc: 0 0 0 0 - 0 0, MB_type: 14344, MV: 0 0
[mpeg4 @ 0x6000a0500] MB pos/size: 0 008:31:01:0 1 dc: 0 0 0 0 - 0 0, MB_type: 14344, MV: 0 0
[mpeg4 @ 0x600094040] MB pos/size: 0 004:28:05:0 1 dc: 0 0 0 0 - 0 0, MB_type: 14344, MV: 0 0
[mpeg4 @ 0x6000971e0] MB pos/size: 0 005:29:04:0 1 dc: 0 0 0 0 - 0 0, MB_type: 14344, MV: 0 0
[mpeg4 @ 0x60008d3c0] MB pos/size: 0 001:30:08:0 1 dc: 128 128 128 128 - 128 128, MB_type: 14344, MV: 0 0

Is that normal?

I also have a few lines that look like this:

Quote
[mpeg4 @ 0x600090f80] MB pos/size: 0 000:09:20:0 1 dc: 0 0 0 0 - 0 0MB pos/size: 0 000:10:19:0 1 dc: 0 0 0 0 - 0 0, MB_type: 14344, MV: 0 0
[mpeg4 @ 0x6000971e0] MB pos/size: 0 000:11:18:0 1 dc: 0 0 0 0 - 0 0, MB_type: 14344, MV: 0 0
[mpeg4 @ 0x6000971e0] , MB_type: 14344, MV: 0 0

There's 2 MB pos/size things on 1 line and a little further down some stuff is missing.

I was getting the same, you have to disable multithreading with option "-threads 1"
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/20/2014 02:11 am
I've been following this thread and website for some time and have been looking for a way to have a productive first post.  This, however, will not be that productive.  Fantastic work everyone who's working on this project.  I'll keep tooling around with frames but ... holy smokes you guys are good at this.  I don't have a lot to bring to the table.

Don't sell yourself short, my friend.  Passion, perseverance, willingness to learn, and a sharp mind is the essence of what some other folks who've been working on this brought to the table as well, and that was plenty. You fixed stuff! That's great! You may also very well have fixed an error that would have carried forward and spread out into a mangled mess in later frames. Jump on the IRC channel, and keep up the good work!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/20/2014 04:17 am
Hi guys,

I think to get started we should begin with the P-frames 169 - 188. That way we can get used to the process and figure out how to deal with certain problems/practicalities. Of course you can play around with any P-frame you want: there is lots of goodies out there now!. ;)

To kick this off I created all the original frames for 169 - 188. We can put these in a separate wiki page and try to make them better individually. From time to time we can combine them to make a video/animated gif.

Good luck!

arnezami

[edit]

Just to be clear: you create an original P-frame by adding "FRAMExxx:0:0:-1" to the mmb for every frame before it. You edit it's mmb by simply adding "FRAMExxx:" to it and of course your mmb commands for that frame.

So for frame 173 (which is FRAME4 in the file) you should do this to get the original:

Quote
FRAME0:0:0:-1=FRAME1:0:0:-1=FRAME2:0:0:-1=FRAME3:0:0:-1


And if you want to fix 173 you should do this:

Quote
FRAME0:0:0:-1=FRAME1:0:0:-1=FRAME2:0:0:-1=FRAME3:0:0:-1=FRAME4:<your mmb commands>
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: michaelni on 05/20/2014 05:19 am
Nice work on the P-frames guys! The bits (literally :) ) and pieces are finally coming together!
Can someone more knowledgeable than me and has looked into the bitstream have a look at this and tell me whether it's accurate?
http://www.cmlab.csie.ntu.edu.tw/cml/dsp/training/coding/h263/format_p.html (http://www.cmlab.csie.ntu.edu.tw/cml/dsp/training/coding/h263/format_p.html)
Could be really helpful for bit flipping :)

edit: Also, is it possible to add the bitstream to the online editor?

It seems that that is describing the "6.2.5.2  Video Plane with Short Header" from ISO-IEC-14496-2_2001_MPEG4_Visual.pdf (http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=582064). Also see: "6.3.5.2  Video Plane with Short Header". Not entirely sure.

I'm actually not sure our video uses the "short header". I have mostly been focused on the macroblock/variable stuff. But it would be good to confirm, because there could definitely be problems in these headers.

This is also a little interesting:

Quote
short_video_header:  The short_video_header is an internal flag which is set to 1 when an abbreviated header
format is used for video content. This indicates video data which begins with a short_video_start_marker rather
than a longer start code such as visual_object_ start_code.  The short header format is included herein to provide
forward compatibility with video codecs designed using the earlier video coding specification ITU-T
Recommendation H.263.  All decoders which support video objects shall support both header formats
(short_video_header equal to 0 or 1) for the subset of video tools that is expressible in either form.

short video headers are not used in the file we have
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: michaelni on 05/20/2014 05:23 am
PPs. I've also attached a slightly altered h263dec.c which now logs the frame number too.

merged most of this in github (not the reverse search stuff as i didngt had time to really look at it)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: meadows.st on 05/20/2014 05:25 am
So, it seems all the puzzle pieces are finally getting together, a big congratulation to Arnezami for finding how to handle these P-frames :)

Here the best I could quickly get for the same frames:

% I-Frame 169
<snip>
% Frame 170
% 6:1:-1,23:3:9697,40:7:-1,37:8:22779 (strange interlace effect)
6:1:-1,23:3:9697,40:7:-1,42:7:19526,3:8:-1,4:8:19780,13:8:-1,15:8:20531,28:8:-1,30:8:21401,37:8:22779 (effect less visible)
<snip>
And also a gif animation!


This is great.  In watching the gif, I saw something interesting related to the "dirt spot" that obscures part of the right leg.  I am referring to the un-numbered spot below and to the left of spot 3 see:


A version with the dirt spots numbered for better reference in conversation

Edit: Added dirt group 14

Due to the flare of the flame in (I think) frame 170, the spot (that I am currently numbering dirt group 15) becomes clearly outlined.  (see my two attached pics).  I do not have the expertise (yet) to see this difference in the hex values of the macroblocks but I suspect it will be there in some way.  Looking at the other dirt spots (bless the dirty bird) I see some of them also flaring out when the flame flares (as it impinges) significantly after frame 170.  I don't know if this will help but it seems that there may be some additional info that could assist in the reconstruction of some of the mb's of the P-Frames.

EDIT: added - I grabbed the second picture below via screen grab from SwissCheese's gif.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Swatch on 05/20/2014 06:34 am
...

Don't sell yourself short!  I took what you had and tweaked it a bit more.

40:3:-1,6:4:9364,4:5:-1,6:5:11443,7:5:11487,5:6:13496:0:0:0:0:0:-2,0:7:-1, 38:11:26080:0:-4:-1:2:0:0,31:12:28067:3:1:0:0:0:0,34:12:28226:-1:0:0:0:0:0, 40:14:34617:-3:0:2:0:0:0,40:16:42426:-1:0:-4:-5:0:0,4:28:76082,5:28:-2, 0:29:81738,1:29:81815,2:29:82246,3:29:83389,4:29:84447,5:29:85968,6:29:86644
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/20/2014 06:44 am
Those that messaged me about the bitmask bug on the editor, this is now fixed.

Thanks :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Marams on 05/20/2014 09:25 am
I'm trying to find flipped bits.

The problem I'm having is, it's almost almost impossible to tell if I've found a flipped bit. Just because the next few blocks or rows improve that doesn't mean you found a flipped one. It's very well possible you just found by luck a way to get the alignment of the next macroblocks right, but there's still a subtle propagating error.

The advantage of finding a flipped bit is, you preserve the most information possible in the frame. But it's only possible in regions where the error rate is less then perhaps 1 flipped bit per 1000.

Here the ones with a fairly high confidence (against raw_edit8.ts):

Frame 72: X:4231:01
Frame 112: X:560:01,X:5373:01,X:5986:01,X:8742:01
Frame 209: X:8799:01,X:8802:01

I'm also attaching the growing list of candidates
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/20/2014 10:57 am
Frame 72: X:4231:01

This is very good. I was about to xor through that macroblock myself. Added it to full mmb and fixed some sea alignment.

PS. The online editor is not showing log data for me.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Untribium on 05/20/2014 11:10 am
Frame 72: X:4231:01

This is very good. I was about to xor through that macroblock myself. Added it to full mmb and fixed some sea alignment.

PS. The online editor is not showing log data for me.

I agree, great stuff! I'm currently trying to understand the macroblock bitstream representation but it's quite challenging, as pretty much all the values are variable size...However, if you start at the end of a block and start going down the address, you can see changes happening in the chroma2 followed by chroma1, then some bits that I'll have to investigate, then the brightness values from 4 to 1 (which is consistent with the encoding described in the spec pdf posted earlier). I'll keep you updated if I find anything interesting :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: PaulNY on 05/20/2014 12:46 pm
I know we are trying to replace bad mmb with good mmb. So would it be useful to document the XY and POS of good mmb's? I am thinking if we can map out what we know to be good, figuring out what to replace the bad with may be easier.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/20/2014 02:02 pm
Hi guys,

To kick this off I created all the original frames for 169 - 188. We can put these in a separate wiki page and try to make them better individually. From time to time we can combine them to make a video/animated gif.

Hi guys.
I'm on the process of creating 15 new wiki pages, following the sections of the video arnezami posted above. Each page will contain the i-frame and 19 p-frames of that section. Everything will be organized in a table just like the one we where using to record our progress on the iframes. The table will include a field to to add a preview and to paste the mmb code used.  The  page for frames 169 - 188 is ready (http://spacexlanding.wikispaces.com/Frames+169-188). I'll add the rest as we progress other parts of the video.

I have also created a central page (http://spacexlanding.wikispaces.com/Progress+using+multi+frame+MMBs) with the description of the tasks and links to the each of the 15 new pages.

The old page that recorded the progress on iframes it still there. If you are continue working on single iframes, you can continue to post the results there. However, as we move towards fixing p-frames, eventually all modifications will have to be recorded in the new pages.


Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/20/2014 03:17 pm
Hi guys,

To kick this off I created all the original frames for 169 - 188. We can put these in a separate wiki page and try to make them better individually. From time to time we can combine them to make a video/animated gif.

Hi guys.
I'm on the process of creating 15 new wiki pages, following the sections of the video arnezami posted above. Each page will contain the i-frame and 19 p-frames of that section. Everything will be organized in a table just like the one we where using to record our progress on the iframes. The table will include a field to to add a preview and to paste the mmb code used.  The  page for frames 169 - 188 is ready (http://spacexlanding.wikispaces.com/Frames+169-188). I'll add the rest as we progress other parts of the video.

I have also created a central page (http://spacexlanding.wikispaces.com/Progress+using+multi+frame+MMBs) with the description of the tasks and links to the each of the 15 new pages.

The old page that recorded the progress on iframes it still there. If you are continue working on single iframes, you can continue to post the results there. However, as we move towards fixing p-frames, eventually all modifications will have to be recorded in the new pages.

Maybe we should rather put the "stand-alone" p-frame, since the final image depends on all previous frames. Or maybe put both? But for the people editing them, having the final result does not really help.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/20/2014 03:18 pm
Hi guys,

To kick this off I created all the original frames for 169 - 188. We can put these in a separate wiki page and try to make them better individually. From time to time we can combine them to make a video/animated gif.

Hi guys.
I'm on the process of creating 15 new wiki pages, following the sections of the video arnezami posted above. Each page will contain the i-frame and 19 p-frames of that section. Everything will be organized in a table just like the one we where using to record our progress on the iframes. The table will include a field to to add a preview and to paste the mmb code used.  The  page for frames 169 - 188 is ready (http://spacexlanding.wikispaces.com/Frames+169-188). I'll add the rest as we progress other parts of the video.

I have also created a central page (http://spacexlanding.wikispaces.com/Progress+using+multi+frame+MMBs) with the description of the tasks and links to the each of the 15 new pages.

The old page that recorded the progress on iframes it still there. If you are continue working on single iframes, you can continue to post the results there. However, as we move towards fixing p-frames, eventually all modifications will have to be recorded in the new pages.
Hi,

Great!

I've been thinking about the small pictures from the frames: maybe it's better to put the individual P-frame in the "Most Recent Image". Not the one merged with previous frames. Otherwise a good/fixed P-frame might look corrupt even though the previous frame caused that corruption. So it's aiming at getting the P-frames themselves fixed not the whole video. I posted the original individual P-frames here (http://).

Beside that we could of course still put a second picture next to it showing it in combination with the other frames.

Just a thought.

Regards,

Jeffrey

[edit] I see SwissCheese had the same idea. The word "stand-alone" is a better term than the term "individual" actually.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/20/2014 03:25 pm
Hi,

I've also been thinking about the frame numbers. If we are going to find more P-frame (we almost surely are) then all these frame numbers we use now are going to be wrong.

I suggest doing the following to prevent this: take the byte-position of the frame in the .ts file (you can for example use ffprobe for that) and remove the last 4 digits of the postion (eg 3552521 becomes 355). That way we will not be bothered by new frames. And these numbers will at most be 3 digits long. Not too bad.

Good idea?

Regards,

arnezami

PS. While it is theoretically possible to get collisions this way this is currently not the case: I checked it. And if if that happens once or twice we can come up with a simple solution like 355.2 and 355.8.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/20/2014 04:09 pm
I updated http://spacexlanding.wikispaces.com/Frames+169-188 :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/20/2014 05:39 pm
Excellent edits! and I agree with both having the stand-alone frame instead of the combined one, and including the byte-position (I just added a new column in the table for that).

I'm going to make the rest of the 14 pages using this template.

And SwissCheese, that latest gif looks terrific! Great work.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/20/2014 06:15 pm
I updated http://spacexlanding.wikispaces.com/Frames+169-188 :)
Impressive! :)

You just more than doubled the amount of decoded frames!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/20/2014 07:08 pm
I have two questions for the rocket experts!  8)

When looking at the latest result from SwissCheese:

(http://spacexlanding.wikispaces.com/file/view/frames_169_188.gif/510110544/420x288/frames_169_188.gif)

1. Does the widening of the "bright circle" mean the rocket is going downwards and therefore the flame widens because the flame hits the water earlier and earlier and therefore it spends more time over the water (instead of going down towards the water) to die out going to the sides? Or does it mean the rocket is almost hovering and the fire/flames are simply accumulating at one spot and has no more room so it widens?

2. Can you deduce something from the movement from the waves in the water (see for example the waves on the left) about either the height or the speed the rocket is going down? Of maybe deduce a change in velocity? Keep in mind this should actually be run at 15 frames per second.

Regards,

arnezami

[edit] Maybe it is wise that we are going to have a separate thread on the interpretation of the video besides this thread about the repair of the video.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/20/2014 08:41 pm
I know we are trying to replace bad mmb with good mmb. So would it be useful to document the XY and POS of good mmb's? I am thinking if we can map out what we know to be good, figuring out what to replace the bad with may be easier.
Welcome to the forum PaulNY!

From what I understand your idea is to catalog good macro blocks in all the frames. Good as in decided by human beings not by some algorithm I assume.

This may be of some use. I'm not sure. We sort of implicitly rely on our vision to detect good vs bad blocks. We also have a nice feature in the online tool that highlights all error-macroblocks in a frame. There are still plenty of non-error-blocks left that look "bad", but we don't know if they are easily repairable or are really bad. We usually judge their "badness" by the destruction they cause to other blocks.

So depending on what you think 1. what the criteria of "bad" is, or even multiple classes of bad and 2. how you think we can deal with those kinds of "badness", your idea might have some value.

Maybe you can try to "fertilize" your ideas a little bit so it becomes more clear if we can use it. :)

Regards,

arnezami

PS. We refer to mmb as "modify macro block". I believe when you say mmb you actually ment to say "macro block".
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: michaelni on 05/20/2014 08:50 pm

However there is one thing that I find difficult to do myself. And I think you might be able to help us. Instead of using this kind of setting:

15:14:-1:0:-40:0:0:0:20

where I replace the current macroblock with a blank one and change the DC values.

But what I really want to do is this:

15:14:57217:0:-40:0:0:0:20

In other words: I want to keep the macroblock with its (structured) data as much as I can (at the very least the blocks in it which are set to 0 above) but change only the DC "starting" values. So not blanken it, but modifying it so these DC-values make sure propogation to other blocks works the same as now with -1 (therefore fixing the color and intensity of following blocks) but I also keep the current blocks inside the current macroblock.

Is something like that possible?

yes but i am not sure i understand what you are trying to correct.
That is if the "current" MB is ending up with a bad dc value, that would either be caused by one of its 6 dc coefficients being damaged in the bitstream/decoded incorrectly due to surrounding damage or one of the surrounding 11 (left/topleft/top) dc values being wrong. Whatever is wrong, should be corrected ideally.
OTOH, if some dc value is overridden on top of the decoded MB then for example the bitstream might store a dc delta in 5 bits while the overridden value could need 6. If it actually was 6 in the undamaged file, ideally the damaged bit(s) should be flipped to make things match or if instead some previous block had a wrong dc value then it should be corrected so the 5bit vlc works out to the correct result.
What iam trying to say is, iam a bit concerned that by editing dc values after decoding MBs or by editing the dc prediction direction in a way that contradicts the surrounding DC values, errors are more covered up than corrected. But maybe thats the best that can be done, i dont know. but Ill look into implementing this as it should give more options/flexibility
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/20/2014 08:59 pm

2. Can you deduce something from the movement from the waves in the water (see for example the waves on the left) about either the height or the speed the rocket is going down? Of maybe deduce a change in velocity? Keep in mind this should actually be run at 15 frames per second.


Considering this sequence of frames corresponds to less than one second in the video, the waves seem to me moving fast. This could be however explained by different factors. The rocket may be  still coming down at a fast speed. Also, the thruster may be generating a huge movement in the surface water (if even a small boat generates disturbances, think of the supersonic exhaust stream from the rocket.). We also need to consider that the sea was very agitated that day.....

Questions, questions questions....
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/20/2014 09:01 pm
yes but i am not sure i understand what you are trying to correct.
That is if the "current" MB is ending up with a bad dc value, that would either be caused by one of its 6 dc coefficients being damaged in the bitstream/decoded incorrectly due to surrounding damage or one of the surrounding 11 (left/topleft/top) dc values being wrong. Whatever is wrong, should be corrected ideally.
OTOH, if some dc value is overridden on top of the decoded MB then for example the bitstream might store a dc delta in 5 bits while the overridden value could need 6. If it actually was 6 in the undamaged file, ideally the damaged bit(s) should be flipped to make things match or if instead some previous block had a wrong dc value then it should be corrected so the 5bit vlc works out to the correct result.
What iam trying to say is, iam a bit concerned that by editing dc values after decoding MBs or by editing the dc prediction direction in a way that contradicts the surrounding DC values, errors are more covered up than corrected. But maybe thats the best that can be done, i dont know. but Ill look into implementing this as it should give more options/flexibility

I have already implemented this since then. And it works.

Maybe you want to look at what I did in the code (I think you already commited that). Maybe I am doing something that would be unwise. Essentially I use the forced DC code you already implemented earlier but do it in a relative way (instead of absolute).

From what I understand if you change a dc value to effectively its original value (which you hope you guess right of course) then blocks after that will correctly choose from which block (left or top) they should take their prediction. In other words: changing these values *should* have a good impact on correct propagation.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/20/2014 09:02 pm
I found the missing p-frame between i-frames 150 and 169, it is just following the i-frame. Here the debug log at the end of the i-frame, the next frame is a p-frame but keeps the number 000 when it should have 001. Should we try to find the position in the ts file and hand-edit the "ghost" p-frame header?

[mpeg4 @ 03d9f020] MB pos/size: 0 000:38:29:0 22 dc: 83 84 83 84 - 129 108
[mpeg4 @ 03d9f020] MB pos/size: 0 000:39:29:0 22 dc: 81 78 81 78 - 129 108
[mpeg4 @ 03d9f020] MB pos/size: 0 000:40:29:0 22 dc: 68 65 68 65 - 129 109
[mpeg4 @ 03d9f020] MB pos/size: 0 000:41:29:0 22 dc: 54 55 54 55 - 130 107
[mpeg4 @ 03d9f020] MB pos/size: 0 000:42:29:0 22 dc: 58 73 58 73 - 131 104
[mpeg4 @ 03d9f020] MB pos/size: 0 000:43:29:0 22 dc: 79 72 79 72 - 130 104
[mpegts @ 0178f6a0] Continuity check failed for pid 1000 expected 10 got 9
[mpegts @ 0178f6a0] Continuity check failed for pid 1000 expected 10 got 11
[mpegts @ 0178f6a0] Continuity check failed for pid 1000 expected 6 got 12
(20 such lines in total)
[mpegts @ 0178f6a0] Continuity check failed for pid 1000 expected 5 got 6
[mpeg4 @ 03d9f020] MB pos/size: 0 000:00:00:66 96 dc: -15 0 0 0 - -9 -9, MB_type: 12296, MV: 1 -1
[mpeg4 @ 03d9f020] MB pos/size: 0 000:01:00:162 151 dc: 0 39 0 0 - 0 0, MB_type: 12296, MV: 0 0
[mpeg4 @ 03d9f020] MB pos/size: 0 000:02:00:313 109 dc: 27 0 -9 0 - 0 0, MB_type: 12296, MV: 16 5
[mpeg4 @ 03d9f020] MB pos/size: 0 000:03:00:422 159 dc: 0 33 -9 -21 - 0 0, MB_type: 12296, MV: -6 2
[mpeg4 @ 03d9f020] MB pos/size: 0 000:04:00:581 239 dc: 123 116 111 112 - 144 122, MB_type: 1, MV: 0 0
[mpeg4 @ 03d9f020] MB pos/size: 0 000:05:00:820 47 dc: 0 0 0 0 - 0 0, MB_type: 12296, MV: 22 9
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/20/2014 09:04 pm

2. Can you deduce something from the movement from the waves in the water (see for example the waves on the left) about either the height or the speed the rocket is going down? Of maybe deduce a change in velocity? Keep in mind this should actually be run at 15 frames per second.


Considering this sequence of frames corresponds to less than one second in the video, the waves seem to me moving fast. This could be however explained by different factors. The rocket may be  still coming down at a fast speed. Also, the thruster may be generating a huge movement in the surface water (if even a small boat generates disturbances, think of the supersonic exhaust stream from the rocket.). We also need to consider that the sea was very agitated that day.....

Questions, questions questions....

I don't think they are waves, but vapour. There are still 4 unused (and more difficult to correct, especially one) p-frames which could fill that in white.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: meadows.st on 05/20/2014 09:16 pm
I updated http://spacexlanding.wikispaces.com/Frames+169-188 :)

Looks fantastic!

I also noted that the impinged plume in the final frame looks to be about the same size as the plume from the one picture of the Cassiope water "landing". (and after the servers came back up, I saw that Arnezami is thinking along the same lines...)

I have two questions for the rocket experts!  8)

When looking at the latest result from SwissCheese:

(http://spacexlanding.wikispaces.com/file/view/frames_169_188.gif/510110544/420x288/frames_169_188.gif)

1. Does the widening of the "bright circle" mean the rocket is going downwards and therefore the flame widens because the flame hits the water earlier and earlier and therefore it spends more time over the water (instead of going down towards the water) to die out going to the sides? Or does it mean the rocket is almost hovering and the fire/flames are simply accumulating at one spot and has no more room so it widens?

2. Can you deduce something from the movement from the waves in the water (see for example the waves on the left) about either the height or the speed the rocket is going down? Of maybe deduce a change in velocity? Keep in mind this should actually be run at 15 frames per second.

Regards,

arnezami

[edit] Maybe it is wise that we are going to have a separate thread on the interpretation of the video besides this thread about the repair of the video.

The first picture is a marked up version of the only picture from the "alightment" of the F9S1 for the Cassiope mission.  We know that this attempt resulted in the engines failing at some point so I do not yet have any concrete understanding of how significantly the plume/water surface interaction differs from the CRS-3 return.

The second picture is (I believe) frame 188 from SwissCheese's latest Gif.  As Arnezami indicates above, the plume appears to expand from ~5 to 10m diameter at frame 169 to almost the entire width of the image at frame 188 (about 1.3s) but I don't want to hazzard a guess about the relationship to the Cassiope plume until I simulate the height of the craft at the time, the rate of descent and the throttle position.

I am sure there are many, many more people on this site that can run the fluid flow sims faster than I can (and much faster than I will have time to even attempt it)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/20/2014 09:17 pm
Ok let me ask the simpler question.

Does anybody know (and can explain why) what these white things are?

Because in this picture we should be quite close to the water right?

I am just curious  :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: AncientU on 05/20/2014 09:18 pm

2. Can you deduce something from the movement from the waves in the water (see for example the waves on the left) about either the height or the speed the rocket is going down? Of maybe deduce a change in velocity? Keep in mind this should actually be run at 15 frames per second.


Considering this sequence of frames corresponds to less than one second in the video, the waves seem to me moving fast. This could be however explained by different factors. The rocket may be  still coming down at a fast speed. Also, the thruster may be generating a huge movement in the surface water (if even a small boat generates disturbances, think of the supersonic exhaust stream from the rocket.). We also need to consider that the sea was very agitated that day.....

Questions, questions questions....
Moving downward quickly and the surface of the water viewed by camera is 'shrinking' -- waves/white caps in upper right of the frame are moving radially outward as are the ones on the left.  The spreading flames are same as in videos of Grasshopper of F9R Dev 1 as it approaches the pad.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: AncientU on 05/20/2014 09:19 pm
Ok let me ask the simpler question.

Does anybody know (and can explain why) what these white things are?

Because in this picture we should be quite close to the water right?

I am just curious  :)
whitecaps
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/20/2014 09:19 pm
Ok let me ask the simpler question.

Does anybody know (and can explain why) what these white things are?

Because in this picture we should be quite close to the water right?

I am just curious  :)

oo I did not think about those  ::)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/20/2014 09:20 pm
A couple more mbs from the top of frame 72. This is a rather painstaking process, but I intend to keep at it because I want this frame.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/20/2014 09:23 pm
whitecaps
Exactly. Thats what I thought. We must be very close to the water here already.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/20/2014 10:03 pm
I updated http://spacexlanding.wikispaces.com/Frames+169-188 :)

Looks fantastic!

I also noted that the impinged plume in the final frame looks to be about the same size as the plume from the one picture of the Cassiope water "landing". (and after the servers came back up, I saw that Arnezami is thinking along the same lines...)

I have two questions for the rocket experts!  8)

When looking at the latest result from SwissCheese:

(http://spacexlanding.wikispaces.com/file/view/frames_169_188.gif/510110544/420x288/frames_169_188.gif)

1. Does the widening of the "bright circle" mean the rocket is going downwards and therefore the flame widens because the flame hits the water earlier and earlier and therefore it spends more time over the water (instead of going down towards the water) to die out going to the sides? Or does it mean the rocket is almost hovering and the fire/flames are simply accumulating at one spot and has no more room so it widens?

2. Can you deduce something from the movement from the waves in the water (see for example the waves on the left) about either the height or the speed the rocket is going down? Of maybe deduce a change in velocity? Keep in mind this should actually be run at 15 frames per second.

Regards,

arnezami

[edit] Maybe it is wise that we are going to have a separate thread on the interpretation of the video besides this thread about the repair of the video.

The first picture is a marked up version of the only picture from the "alightment" of the F9S1 for the Cassiope mission.  We know that this attempt resulted in the engines failing at some point so I do not yet have any concrete understanding of how significantly the plume/water surface interaction differs from the CRS-3 return.

The second picture is (I believe) frame 188 from SwissCheese's latest Gif.  As Arnezami indicates above, the plume appears to expand from ~5 to 10m diameter at frame 169 to almost the entire width of the image at frame 188 (about 1.3s) but I don't want to hazzard a guess about the relationship to the Cassiope plume until I simulate the height of the craft at the time, the rate of descent and the throttle position.

I am sure there are many, many more people on this site that can run the fluid flow sims faster than I can (and much faster than I will have time to even attempt it)
Love your post. And your encouragement through PM's ;). That really makes it all worth it. Because all this is truly hard to do.

I think we think along the same lines here then. We'll have to see what the final verdict is after more analysis. But it's soo cool we have this video now!

Thank you SwissCheese for that!! :)

Have a nice day,

arnezami

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/20/2014 10:11 pm
I found the missing p-frame between i-frames 150 and 169, it is just following the i-frame. Here the debug log at the end of the i-frame, the next frame is a p-frame but keeps the number 000 when it should have 001. Should we try to find the position in the ts file and hand-edit the "ghost" p-frame header?

[mpeg4 @ 03d9f020] MB pos/size: 0 000:38:29:0 22 dc: 83 84 83 84 - 129 108
[mpeg4 @ 03d9f020] MB pos/size: 0 000:39:29:0 22 dc: 81 78 81 78 - 129 108
[mpeg4 @ 03d9f020] MB pos/size: 0 000:40:29:0 22 dc: 68 65 68 65 - 129 109
[mpeg4 @ 03d9f020] MB pos/size: 0 000:41:29:0 22 dc: 54 55 54 55 - 130 107
[mpeg4 @ 03d9f020] MB pos/size: 0 000:42:29:0 22 dc: 58 73 58 73 - 131 104
[mpeg4 @ 03d9f020] MB pos/size: 0 000:43:29:0 22 dc: 79 72 79 72 - 130 104
[mpegts @ 0178f6a0] Continuity check failed for pid 1000 expected 10 got 9
[mpegts @ 0178f6a0] Continuity check failed for pid 1000 expected 10 got 11
[mpegts @ 0178f6a0] Continuity check failed for pid 1000 expected 6 got 12
(20 such lines in total)
[mpegts @ 0178f6a0] Continuity check failed for pid 1000 expected 5 got 6
[mpeg4 @ 03d9f020] MB pos/size: 0 000:00:00:66 96 dc: -15 0 0 0 - -9 -9, MB_type: 12296, MV: 1 -1
[mpeg4 @ 03d9f020] MB pos/size: 0 000:01:00:162 151 dc: 0 39 0 0 - 0 0, MB_type: 12296, MV: 0 0
[mpeg4 @ 03d9f020] MB pos/size: 0 000:02:00:313 109 dc: 27 0 -9 0 - 0 0, MB_type: 12296, MV: 16 5
[mpeg4 @ 03d9f020] MB pos/size: 0 000:03:00:422 159 dc: 0 33 -9 -21 - 0 0, MB_type: 12296, MV: -6 2
[mpeg4 @ 03d9f020] MB pos/size: 0 000:04:00:581 239 dc: 123 116 111 112 - 144 122, MB_type: 1, MV: 0 0
[mpeg4 @ 03d9f020] MB pos/size: 0 000:05:00:820 47 dc: 0 0 0 0 - 0 0, MB_type: 12296, MV: 22 9
Very interesting.

Maybe somebody can take a look at this. And/or maybe fix it.

Since it is the first P-frame after the I-frame it's quite important ;)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/20/2014 10:37 pm
There's a new Online editor up for doing p-frame stuff here: http://spacex2.slapbet.org/

It should be fairly self explanatory for those doing p-frame work, it's also very new and is missing some stuff (like log / error info) which I will get to when I can.

Have a play.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Asmegin on 05/20/2014 10:58 pm
New editor looks nice - love the slider feature!

Doesn't have any bit info when you click on a block yet, so can't really use it right now.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Maciej Olesinski on 05/20/2014 10:59 pm
You guyz are doing amazing job!

From above gif I can tell that stage was landing with wild angle (tipped to right from viewer perspective) and doing very nice hoover slam.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: meadows.st on 05/20/2014 11:15 pm
You guyz are doing amazing job!

From above gif I can tell that stage was landing with wild angle (tipped to right from viewer perspective) and doing very nice hoover slam.

I think we need the "video interpretation/analysis" thread sooner rather than later given the stellar work that the BFE "bit flippers extraordinaire" group is doing.


So, if the mods want to move this response, please do.

re: "wild angle" hypothesis.  I do not agree since the frame 188 image I posted above appears to have a virtually round plume (the yellow arcs were drawn as a circle and positioned to bound the outer limits of the visible plume). 
I would expect this plume to be significantly elliptical (I understand it is already distorted by the fish-eye camera) if the stage were descending at a "wild angle" (unless our use of the term is very different).

EDIT: After re-reading your comment it occurred to me that you might have meant to write "mild angle"? I can accept the hypothesis of a mild angle especially given the reported weather conditions.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Lars_J on 05/20/2014 11:25 pm
Yes, the video interpretation should be in it own thread. I'm interested in reading about the data recovery efforts here - and not having it mixed in with interpretations (mostly wrong) based on what people think they are seeing.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/21/2014 12:20 am
Its journey started with a huge plume of water, and ended exactly the same way.  ;D
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: ajmartin on 05/21/2014 12:36 am
I'm trying to find flipped bits.

The problem I'm having is, it's almost almost impossible to tell if I've found a flipped bit. Just because the next few blocks or rows improve that doesn't mean you found a flipped one. It's very well possible you just found by luck a way to get the alignment of the next macroblocks right, but there's still a subtle propagating error.

The advantage of finding a flipped bit is, you preserve the most information possible in the frame. But it's only possible in regions where the error rate is less then perhaps 1 flipped bit per 1000.

Looking at the null packets, I noticed some patterns in the bit errors that might be useful for this kind of effort.

Specifically, many (but not all) of the bit-flips come in groups of 3, with one bit flipped, then 13 unaffected, then the next two also flipped.  This is most obvious in a hex-dump of the null packets, which are supposed to be filled with 0xff bytes: patterns such as "7f fc" or "fb ff e7" are common.  But the same bit-flip patterns apply where errors overlay the TS headers, and presumably also when they occur in image data.

Combinations (with XOR) of these 3-bit errors are also seen; mathematically, these error patterns are multiples of the polynomial x^15 + x + 1 over the binary field.  This could be the result of FEC coding applied to a signal with too many errors for successful correction.  I haven't figured it out completely but there are indications that the data may have been coded in 28-byte or 56-byte blocks (not respecting the TS packet boundaries); in particular, errors that don't fit this pattern seem to be found near offsets that are multiples of 28 bytes.

There is still the problem of figuring out if the change helped or not, but this should help to reduce the search space.  In low-error regions, there's a good chance that some of the flaws can be fixed with a single instance of this pattern.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/21/2014 01:39 am
EDIT: I concur that this discussion should go in a parallel thread.

Let start the discussion! http://forum.nasaspaceflight.com/index.php?topic=34794.0
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/21/2014 01:48 am
I found the missing p-frame between i-frames 150 and 169, it is just following the i-frame. Here the debug log at the end of the i-frame, the next frame is a p-frame but keeps the number 000 when it should have 001. Should we try to find the position in the ts file and hand-edit the "ghost" p-frame header?

I: 47 43 E8 3F 07 10 00 32 86 C7 7E 00 00 00 01 E0 00 00 81 80 07 21 01 93 CC FD FF FF 00 00 01 B0 03 00 00 01 B5 0D 0F 00 00 01 00 00 00 01 20

This is the first bit of the transport stream for the iframe out of rawsplit_part9.ts - the 000001B0 is a visual sequence start marker.

The next start sequence we see is this,. at 0x115:

0x0115 : 00 00 01 B6 14

The 000001B6 code means that it's a video object plane start marker, and the next two bits tell us whether it's a "VOP-I" or a "VOP-P" - the four bits after the B6 are 0001, and since the first two are 00, we know it's an iframe that follows.

This page has a good explanation of these pieces: http://stackoverflow.com/questions/4082080/parsing-mpeg4-frames-from-rtp-packets

At the 0x5184 offset in rawsplit_part9.ts:

P: 47 43 E8 3C 07 10 00 32 92 82 7E 00 00 00 01 E0 00 00 81 80 07 21 01 93 FB E9 FF FF 00 00 01 B6 55 38 0C 06 42 9A 7F 7E 2A 00 93 E3 C5 78 29

This is a video object plane indicated by 000001B6, and the first two bits of the highlighted "5"  (0101) are "01", indicating that it's a pframe. You can see this continue on down the line for subsequent pframes:

P: 47 43 E8 37 07 10 00 32 9E 3D 7E 00 00 00 01 E0 00 00 81 80 07 21 01 95 2A D5 FF FF 00 00 01 B6 55 F3 BE 04 42 99 62 9D A3 70 61 5F B7 94 1F

P: 47 43 E8 3B 07 10 00 32 A9 F8 7E 00 00 00 01 E0 00 00 81 80 07 21 01 95 59 C1 FF FF 00 00 01 B6 56 AF 6C 06 53 57 38 76 4A E1 99 55 02 D2 39

P: 47 43 E8 31 07 10 00 32 B5 B3 7E 00 00 00 01 E0 00 00 81 80 07 21 01 95 88 AD FF FF 00 00 01 B6 57 6B 1E 06 53 02 00 82 7D B0 60 48 08 0A 03


Now, I see that the bits after the VOP type indicator, is a counter incrementing on each pframe, but I don't seem to have the right documentation on hand to tell me how many bits it's supposed to be. It looks like it skipped an increment, but I'm not sure if the counter is limited to that one nybble or not. I seem to recall Shanuson figured that out in a post a while back, I'll check more and see what I can find, or if we're lucky Michael Niedermyer will stop in. :)

(Edit: checked rawsplit-6, and it is a much longer than four bit counter. )

AJMartin - that's one heck of a first post!! Nice find! There were some other areas which had 56-byte multiples, which didn't make any sense in the context of a 188-byte transport stream packet. It looks like you may have solved a mystery here. We haven't gotten further information from SpaceX about the characteristics of the radio transmission from the booster, but it sounds like that would fit together with what you've observed.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: meekGee on 05/21/2014 02:05 am

Looking at the null packets, I noticed some patterns in the bit errors that might be useful for this kind of effort.

Specifically, many (but not all) of the bit-flips come in groups of 3, with one bit flipped, then 13 unaffected, then the next two also flipped.  This is most obvious in a hex-dump of the null packets, which are supposed to be filled with 0xff bytes: patterns such as "7f fc" or "fb ff e7" are common.  But the same bit-flip patterns apply where errors overlay the TS headers, and presumably also when they occur in image data.

Combinations (with XOR) of these 3-bit errors are also seen; mathematically, these error patterns are multiples of the polynomial x^15 + x + 1 over the binary field.  This could be the result of FEC coding applied to a signal with too many errors for successful correction.  I haven't figured it out completely but there are indications that the data may have been coded in 28-byte or 56-byte blocks (not respecting the TS packet boundaries); in particular, errors that don't fit this pattern seem to be found near offsets that are multiples of 28 bytes.

There is still the problem of figuring out if the change helped or not, but this should help to reduce the search space.  In low-error regions, there's a good chance that some of the flaws can be fixed with a single instance of this pattern.

If there was a bigger "Like" button, I'd use it.  Consider this a super-like.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/21/2014 03:20 am
A 300% like-to-post ratio - is that record? I wonder how high it'll get when the Europeans sign on? :)

(Edit: now it's 1,200% like-to-post.  :D )

I went through rawsplit-9 with Wireshark and bvi, and found a few dozen instances where Shanuson's fixing tools were too conservative to pound a particularly mangled transport stream header into conformance, as he had mentioned. I did this by hand in the attached file, and it now shows no protocol warnings in Wireshark.

If someone more conversant with ffmpeg could take a look at it and compare it to rawsplit-9 to see what comes out the other end of it, I'd appreciate it. I'm curious if that might help the ghost pframe reported earlier, or if it will make any noticeable difference in the frames that come out or change any offsets. In any case, a clean transport stream is better than dirty, one would assume.

If it's worth the effort, we can come up with a more aggressive transport stream header solver.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: meadows.st on 05/21/2014 03:52 am
Coming up on the 150,000 view mark for this thread/effort. Noting it as there's a small active team of people really working hard on this, but also a HUGE amount of people watching that work. Always nice to have a large appreciative audience, both inside and outside the space industry.

You'd be astonished how many views are from IP addresses in a place called Hawthorne, California! ;D

I was curious how this thread ranked overall in the history of SpaceX threads (currently as of this writing, reads are >180,000) so I sorted the other SpaceX dedicated threads and only the 100,000+ and Missions sections have threads that exceed 180,000. Currently this thread is the 27th most read SpaceX thread and I think the NSF server (sorry Chris  ;D) will explode when you all get the video restored "even" to the level of SwissCheese's latest gif (which is awesome BTW). see:
I updated http://spacexlanding.wikispaces.com/Frames+169-188 :)

Graphic representation of the historical significance of the work that the video restoration group is doing (see pic).

Added: Updated context with consolidated "conversations" (multiple threads on same topic now rolled-up into a single "conversation").
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/21/2014 04:13 am
I'm trying to find flipped bits.

The problem I'm having is, it's almost almost impossible to tell if I've found a flipped bit. Just because the next few blocks or rows improve that doesn't mean you found a flipped one. It's very well possible you just found by luck a way to get the alignment of the next macroblocks right, but there's still a subtle propagating error.

The advantage of finding a flipped bit is, you preserve the most information possible in the frame. But it's only possible in regions where the error rate is less then perhaps 1 flipped bit per 1000.

Looking at the null packets, I noticed some patterns in the bit errors that might be useful for this kind of effort.

Specifically, many (but not all) of the bit-flips come in groups of 3, with one bit flipped, then 13 unaffected, then the next two also flipped.  This is most obvious in a hex-dump of the null packets, which are supposed to be filled with 0xff bytes: patterns such as "7f fc" or "fb ff e7" are common.  But the same bit-flip patterns apply where errors overlay the TS headers, and presumably also when they occur in image data.

Combinations (with XOR) of these 3-bit errors are also seen; mathematically, these error patterns are multiples of the polynomial x^15 + x + 1 over the binary field.  This could be the result of FEC coding applied to a signal with too many errors for successful correction.  I haven't figured it out completely but there are indications that the data may have been coded in 28-byte or 56-byte blocks (not respecting the TS packet boundaries); in particular, errors that don't fit this pattern seem to be found near offsets that are multiples of 28 bytes.

There is still the problem of figuring out if the change helped or not, but this should help to reduce the search space.  In low-error regions, there's a good chance that some of the flaws can be fixed with a single instance of this pattern.
Incredibly great find!!!

I hope we can use it somehow. But finding patterns in the errors is potentially HUGE!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/21/2014 04:16 am
There's a new Online editor up for doing p-frame stuff here: http://spacex2.slapbet.org/

It should be fairly self explanatory for those doing p-frame work, it's also very new and is missing some stuff (like log / error info) which I will get to when I can.

Have a play.

Sir. You are a magician! I love the slider :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/21/2014 05:53 am
Looking at the null packets, I noticed some patterns in the bit errors that might be useful for this kind of effort.

Specifically, many (but not all) of the bit-flips come in groups of 3, with one bit flipped, then 13 unaffected, then the next two also flipped.  This is most obvious in a hex-dump of the null packets, which are supposed to be filled with 0xff bytes: patterns such as "7f fc" or "fb ff e7" are common.  But the same bit-flip patterns apply where errors overlay the TS headers, and presumably also when they occur in image data.

Combinations (with XOR) of these 3-bit errors are also seen; mathematically, these error patterns are multiples of the polynomial x^15 + x + 1 over the binary field.  This could be the result of FEC coding applied to a signal with too many errors for successful correction.  I haven't figured it out completely but there are indications that the data may have been coded in 28-byte or 56-byte blocks (not respecting the TS packet boundaries); in particular, errors that don't fit this pattern seem to be found near offsets that are multiples of 28 bytes.

There is still the problem of figuring out if the change helped or not, but this should help to reduce the search space.  In low-error regions, there's a good chance that some of the flaws can be fixed with a single instance of this pattern.

That is brilliant. And something that never occurred to me. It could be incredibly important. Just yesterday (posted here: http://forum.nasaspaceflight.com/index.php?topic=34597.msg1201317#msg1201317) through a lot of trial and error I ended up doing this to frame72: X:11807:80,X:11822:80. We could really use more statistics though. Need to know what is the probability of single flips vs patterned flips. Perhaps a graph showing probability of flipped bit in positions following another flipped bit going up to 28 bytes.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/21/2014 06:49 am
Just in case someone else runs into this, I misplaced the -threads option and it was causing ffmpeg to ignore it. This is on OS X, so apparently the threads thing is not just Windows. Here is the command line I used successfully:

./ffmpeg -threads 1 -r 44999/3003 -mmb `cat mmb_starting_from_209.txt` -debug mb_pos_size -err_detect ignore_err  -i fixed_edit8_part_209.ts -f image2 fixed_edit8_part_209_%03d.png
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/21/2014 06:52 am
A 300% like-to-post ratio - is that record? I wonder how high it'll get when the Europeans sign on? :)

I went through rawsplit-9 with Wireshark and bvi, and found a few dozen instances where Shanuson's fixing tools were too conservative to pound a particularly mangled transport stream header into conformance, as he had mentioned. I did this by hand in the attached file, and it now shows no protocol warnings in Wireshark.

If someone more conversant with ffmpeg could take a look at it and compare it to rawsplit-9 to see what comes out the other end of it, I'd appreciate it. I'm curious if that might help the ghost pframe reported earlier, or if it will make any noticeable difference in the frames that come out or change any offsets. In any case, a clean transport stream is better than dirty, one would assume.

If it's worth the effort, we can come up with a more aggressive transport stream header solver.


My tools are more or less my hands and a few python scripts. :D and yes, i was conservative or not concerned about that problem yet. Did you only fix some 4byte TS-headers? or more? And only for some part or the whole stream? If only for one part. could you apply those changes to the Part_X.ts file so we can more easily merge your and my changes together?

@ajmartin
nice find. That could explain why there were 56 byte missing in raw.ts at 5 points. Never 28 though.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/21/2014 10:25 am
My tools are more or less my hands and a few python scripts. :D and yes, i was conservative or not concerned about that problem yet. Did you only fix some 4byte TS-headers? or more? And only for some part or the whole stream? If only for one part. could you apply those changes to the Part_X.ts file so we can more easily merge your and my changes together?

In the part9 I uploaded, it was strictly the transport stream headers, since that's what was easily discernable with Wireshark, and I only dealt with that one segment of the stream. Working through the procedure by hand, though, gave me a sense of what the algorithm would need to be to implement it, and extend it to the PES and MPEG headers. I'm currently thinking that breaking out the entire stream into an array of nested C structs will make it easier to manipulate and parse, and apply the known anchor points to the bits.

I recall you mentioning a section where the 188-byte alignment goes out the window at one point - something about AJMartin's 28- or 56-byte frames, perhaps? Do you have the offset where that occurs?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/21/2014 10:58 am
Hello everyone, latest update of the gif  :) (frames 169-173,175-177,180-181,183-199)

It looks really cool!

Note that many p-frames only need very little correction, some even don't need any.

One issue seems to be corrupt motion vectors (see the legs), would it be possible to implement something to change the motion vectors values? or at least set them to 0?

(Click the image to make the gif animate).
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: CuddlyRocket on 05/21/2014 01:38 pm
^ Amazing, awesome, astonishing - yes, I'm impressed with the achievement here (and more to come!).

Hopefully, there'll be some appropriate thank you for all this work coming the workers' way. A guided tour or an invite to a launch, perhaps?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Oersted on 05/21/2014 01:39 pm
Thanks Swisscheese, it looks great.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/21/2014 01:46 pm
One issue seems to be corrupt motion vectors (see the legs), would it be possible to implement something to change the motion vectors values? or at least set them to 0?

I think we'd also want the ability to specify whole swaths of motion vectors to zero out, considering that we know with certainty that almost nothing in the lower half of the frame should be moving in any way at all.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/21/2014 02:15 pm
My tools are more or less my hands and a few python scripts. :D and yes, i was conservative or not concerned about that problem yet. Did you only fix some 4byte TS-headers? or more? And only for some part or the whole stream? If only for one part. could you apply those changes to the Part_X.ts file so we can more easily merge your and my changes together?

In the part9 I uploaded, it was strictly the transport stream headers, since that's what was easily discernable with Wireshark, and I only dealt with that one segment of the stream. Working through the procedure by hand, though, gave me a sense of what the algorithm would need to be to implement it, and extend it to the PES and MPEG headers. I'm currently thinking that breaking out the entire stream into an array of nested C structs will make it easier to manipulate and parse, and apply the known anchor points to the bits.

I recall you mentioning a section where the 188-byte alignment goes out the window at one point - something about AJMartin's 28- or 56-byte frames, perhaps? Do you have the offset where that occurs?

I don't have the position anymore, the only missing 56 byte where i could not decide exactly were it occured are marked by 12 34 56 78 sequence followed by a lot (52) of FFs. But i think it was much earlier in the file, not in part9.
If you want to know, use tsalign on the original raw.ts. Ignore the 2 500000 pos parts mentioned, they look fine in a hex editor. all the other (3 or 4 I think) are parts with 56 byte missing .

Anyways, the splitup rawfile was corrected, so there should not be any 56 byte missing parts, those I filled with FFs

Cheers,
Shanuson
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/21/2014 02:46 pm
Hopefully, there'll be some appropriate thank you for all this work coming the workers' way. A guided tour or an invite to a launch, perhaps?

That'd be icing on the delicious cake of a salvaged video, I suspect.  ;D

I'll be in McGregor with my nose pressed to their fence on June 13...  ;)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: meadows.st on 05/21/2014 02:47 pm
^ Amazing, awesome, astonishing - yes, I'm impressed with the achievement here (and more to come!).

Hopefully, there'll be some appropriate thank you for all this work coming the workers' way. A guided tour or an invite to a launch, perhaps?

Given the context, I think attendance at the first land-landing would befit the effort. :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/21/2014 04:08 pm
Hello everyone, latest update of the gif  :) (frames 169-173,175-177,180-181,183-199)

It looks really cool!

Note that many p-frames only need very little correction, some even don't need any.

One issue seems to be corrupt motion vectors (see the legs), would it be possible to implement something to change the motion vectors values? or at least set them to 0?


Ok, that's crazy good!! Wow!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Untribium on 05/21/2014 04:45 pm
Does it make sense to start working on fixing p-frames before their counter bytes and stuff was checked (wiki -> extraction of p-frames)? I was going to start on frames 52 and up, since the iframe looks very promising, so I'm wondering if I should try to fix the headers first...

Also, this gif looks terrific!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/21/2014 04:50 pm
The new online editor isn't showing bitpositions. Could we get the old one back until it's fixed?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/21/2014 04:55 pm
Does it make sense to start working on fixing p-frames before their counter bytes and stuff was checked (wiki -> extraction of p-frames)? I was going to start on frames 52 and up, since the iframe looks very promising, so I'm wondering if I should try to fix the headers first...

Also, this gif looks terrific!

You can start with this one, the problematic ones are those where not all of the 19 p-frames are detected.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Untribium on 05/21/2014 04:59 pm
Does it make sense to start working on fixing p-frames before their counter bytes and stuff was checked (wiki -> extraction of p-frames)? I was going to start on frames 52 and up, since the iframe looks very promising, so I'm wondering if I should try to fix the headers first...

Also, this gif looks terrific!

You can start with this one, the problematic ones are those where not all of the 19 p-frames are detected.

Oh I see, thanks. Just noticed that some aren't actually on the editor, so I'm guessing these are the ones that need to be fixed first :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Asmegin on 05/21/2014 05:24 pm
The new online editor isn't showing bitpositions. Could we get the old one back until it's fixed?

Yeah, I started working on Frameset 92 without the bit positions...let's just say it made things very difficult :D Especially since the P-Frames jump all over the place for this set. They settle down a bit at the end (albeit in the wrong position)...but it's definitely a challenge.

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: rocketguy101 on 05/21/2014 05:42 pm
Coming up on the 150,000 view mark for this thread/effort. Noting it as there's a small active team of people really working hard on this, but also a HUGE amount of people watching that work. Always nice to have a large appreciative audience, both inside and outside the space industry.

You'd be astonished how many views are from IP addresses in a place called Hawthorne, California! ;D

I was curious how this thread ranked overall in the history of SpaceX threads (currently as of this writing, reads are >180,000) so I sorted the other SpaceX dedicated threads and only the 100,000+ and Missions sections have threads that exceed 180,000. Currently this thread is the 27th most read SpaceX thread and I think the NSF server (sorry Chris  ;D) will explode when you all get the video restored "even" to the level of SwissCheese's latest gif (which is awesome BTW). see:
I updated http://spacexlanding.wikispaces.com/Frames+169-188 :)

Graphic representation of the historical significance of the work that the video restoration group is doing (see pic).

I have just been watching this thread with total amazement, as I have nothing to add to it -- learning the background of digital video is interesting too!!  I think Chris will have to add a "cheerleading" thread for this effort :)   Keep up the fantastic work folks!!!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/21/2014 05:43 pm
Standby folks. Attempting to tweet out the gif, with a link to that post from SwissCheese.

Here it is:
https://twitter.com/NASASpaceflight/statuses/469171624072470528

And get ready for a surprise!!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/21/2014 06:44 pm
Do we have a list of all people who worked on this?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/21/2014 06:58 pm
Do we have a list of all people who worked on this?

In different pages of the wiki there are references to those that have contributed with MMB codes. However, in order to make the list comprehensive, we would need to check this thread. Other kind of contributors (online tools, wiki, etc.) would need to be added to the list too.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/21/2014 07:43 pm
The new online editor isn't showing bitpositions. Could we get the old one back until it's fixed?

This bug was introduced due to the new format of the log, this is now fixed in the i-frame only editor which is still available here http://spacex.slapbet.org/
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/21/2014 07:45 pm
Elon asks who's leading the effort?

Eight Tips for Collaborative Leadership (http://www.forbes.com/sites/carolkinseygoman/2014/02/13/8-tips-for-collaborative-leadership/)

Quote
Today’s corporation exists in an increasingly complex and ever-shifting ocean of change. As a result, leaders need to rely more than ever on the intelligence and resourcefulness of their staff. Collaboration is not a “nice to have” organizational philosophy. It is an essential ingredient for organizational survival and success.

That's what's been interesting about the effort - each individual has brought their individual talents, experience, knowledge, and skills to bear on different pieces of the problem to the extent that they are able to contribute, and everyone has been learning from everyone else.

Outsized kudos are due, of course, to the people who have set up the framework that allows this collaboration to occur in the first place - Michael Niedermyer for his ffmpeg patches to simplify the work on the frames, Iain Cole for his magnificent online editor, Moralec for his work in maintaining the Wiki page, etc. But are they "leaders" in the traditional sense of the word? They're fellow collaborators.

I think the effort and the fruit that it has borne so far is just a really excellent example of the power of "crowdsourcing" (I think someone will eventually have to add this video repair effort to the Wikipedia page for that term. :D), and a testament to how many people around the world are cheering for Elon and his entire team at SpaceX, and the vision of a multiplanetary human society.

I remember fondly my first Mars Society convention years ago in Boulder, and my work with the California Mars Society around 2000, and it's just magnificent to see how much closer we are to that goal today thanks to the efforts of not only Elon but also countless other people who were inspired - by Ray Bradbury, or Robert Zubrin, or Kim Stanley Robinson, or in any number of other ways - to each work in their own way and in their own capacity towards achieving it.

In a way, this video repair mission is just another example of that same phenomenon. It's great to see it unfold right before our eyes.

Edit: (You can see why I never got into Twitter.  ;) )
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Req on 05/21/2014 07:53 pm
Elon asks who's leading the effort?

Eight Tips for Collaborative Leadership (http://www.forbes.com/sites/carolkinseygoman/2014/02/13/8-tips-for-collaborative-leadership/)

Quote
Today’s corporation exists in an increasingly complex and ever-shifting ocean of change. As a result, leaders need to rely more than ever on the intelligence and resourcefulness of their staff. Collaboration is not a “nice to have” organizational philosophy. It is an essential ingredient for organizational survival and success.

That's what's been interesting about the effort - each individual has brought their individual talents, experience, knowledge, and skills to bear on different pieces of the problem to the extent that they are able to contribute, and everyone has been learning from everyone else.

Outsized kudos are due, of course, to the people who have set up the framework that allows this collaboration to occur in the first place - Michael Niedermyer for his ffmpeg patches to simplify the work on the frames, Iain Cole for his magnificent online editor, Moralec for his work in maintaining the Wiki page, etc. But are they "leaders" in the traditional sense of the word? They're fellow collaborators.

I think the effort and the fruit that it has borne so far is just a really excellent example of the power of "crowdsourcing" (I think someone will eventually have to add this video repair effort to the Wikipedia page for that term. :D), and a testament to how many people around the world are cheering for Elon and his entire team at SpaceX, and the vision of a multiplanetary human society.

I remember fondly my first Mars Society convention years ago in Boulder, and my work with the California Mars Society around 2000, and it's just magnificent to see how much closer we are to that goal today thanks to the efforts of not only Elon but also countless other people who were inspired - by Ray Bradbury, or Robert Zubrin, or Kim Stanley Robinson, or in any number of other ways - to each work in their own way and in their own capacity towards achieving it.

In a way, this video repair mission is just another example of that same phenomenon. It's great to see it unfold right before our eyes.

He only gets the space of a tweet, why not just have him link to this thread or the wiki page?

Also, having kept up with the thread since the beginning, I think that Arnezami would be on the short list as well.  In fact, if anybody is playing a leadership role, it's probably him.  At least in my opinion.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: morningdew76 on 05/21/2014 07:54 pm
Looking at the null packets, I noticed some patterns in the bit errors that might be useful for this kind of effort.

Specifically, many (but not all) of the bit-flips come in groups of 3, with one bit flipped, then 13 unaffected, then the next two also flipped.  This is most obvious in a hex-dump of the null packets, which are supposed to be filled with 0xff bytes: patterns such as "7f fc" or "fb ff e7" are common.  But the same bit-flip patterns apply where errors overlay the TS headers, and presumably also when they occur in image data.

Combinations (with XOR) of these 3-bit errors are also seen; mathematically, these error patterns are multiples of the polynomial x^15 + x + 1 over the binary field.  This could be the result of FEC coding applied to a signal with too many errors for successful correction.  I haven't figured it out completely but there are indications that the data may have been coded in 28-byte or 56-byte blocks (not respecting the TS packet boundaries); in particular, errors that don't fit this pattern seem to be found near offsets that are multiples of 28 bytes.

There is still the problem of figuring out if the change helped or not, but this should help to reduce the search space.  In low-error regions, there's a good chance that some of the flaws can be fixed with a single instance of this pattern.

That is brilliant. And something that never occurred to me. It could be incredibly important. Just yesterday (posted here: http://forum.nasaspaceflight.com/index.php?topic=34597.msg1201317#msg1201317) through a lot of trial and error I ended up doing this to frame72: X:11807:80,X:11822:80. We could really use more statistics though. Need to know what is the probability of single flips vs patterned flips. Perhaps a graph showing probability of flipped bit in positions following another flipped bit going up to 28 bytes.

Short time reader, first time poster.

I've done some further work on the bit-flips for frame 72, using the "8003" pattern, with results for two potential upgrades.  This is the basic form that I tried to fit: X:offset:80,X:offset+8:03

X:11807:80,X:11822:80 -> X:11807:80,11815:03
X:11505:80 -> X:11493:80,X:11501:03

In both cases the previously found bits were preserved.. the 8003 pattern was just fitted around them via trial and error.

The other bit inversion for this frame (X:4238:80) did not fit into an 8003 pattern.  Edit: but it did fit the 040018 pattern!  X:4238:80 -> X:4233:04,X:4249:18 Edit2: jumped the gun.

I have not yet tried looking for an inversion of "FB FF E7", but it would take the form of X:offset:04,X:offset+16:18 .  We lucked out with the middle FF block, otherwise this would be three 8-bit inversions.

Now for a few suggestions for the online editor-

First would be a way to increment through 16- or 24-bit inversions with a single click.  This could be handled directly with modifications to the editor, but perhaps there's potential for a new type of inversion syntax- something like X:1234:P1 for 8003, X:1234:P2 for 040018, etc.

Second, and this is probably harder, would be to have the editor calculate the following frame or two on a keyboard arrow hit.  For example- say you are incrementing the bit position for X:1234:80.  When I hit the up arrow in the bit position box, it loads X:1235:80, doing a server round trip for the image for position 1235.  It would be helpful if the editor also cached up the same edit for bit position 1236.

I am but a man, standing on the shoulders of giants.  The work that you collectively have been doing here is nothing short of amazing!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: KEdward5 on 05/21/2014 08:03 pm

He only gets the space of a tweet, why not just have him link to this thread or the wiki page?

If you go back one short page you can see it's clearly being dealt with.

http://forum.nasaspaceflight.com/index.php?topic=34597.msg1201779#msg1201779

And yes, this thread is much better, as it's a journey. Really is impressive.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Req on 05/21/2014 08:05 pm

He only gets the space of a tweet, why not just have him link to this thread or the wiki page?

If you go back one short page you can see it's clearly being dealt with.

http://forum.nasaspaceflight.com/index.php?topic=34597.msg1201779#msg1201779

And yes, this thread is much better, as it's a journey. Really is impressive.

In fact you missed several posts which are now gone that made it clear that there was still confusion, at least among several posters here.  But who cares, OT.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/21/2014 08:15 pm
X:11807:80,X:11822:80 -> X:11807:80,11815:03
X:11505:80 -> X:11493:80,X:11501:03
Cool. I have already done the same for the first of the two flips. I'll investigate the second one. I have since added more flips, some using ajmartins triple flip pattern. I'm using a bit different convention than you, but it's equivalent. Here's my latest flip string for this frame:
X:4238:80,X:10711:80,X:11505:80,X:11807:80,X:11821:C0,X:11947:80,X:11961:C0,X:12158:80

edit: The second one looks like a tiny improvement too. Thanks.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/21/2014 08:16 pm
Elon asks who's leading the effort?

He only gets the space of a tweet, why not just have him link to this thread or the wiki page?

Also, having kept up with the thread since the beginning, I think that Arnezami would be on the short list as well.  In fact, if anybody is playing a leadership role, it's probably him.  At least in my opinion.

If Elon really is asking that question I think Chris (as an observer) can make the best assessment of that.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/21/2014 08:28 pm
Elon asks who's leading the effort?

He only gets the space of a tweet, why not just have him link to this thread or the wiki page?

Also, having kept up with the thread since the beginning, I think that Arnezami would be on the short list as well.  In fact, if anybody is playing a leadership role, it's probably him.  At least in my opinion.

If Elon really is asking that question I think Chris (as an observer) can make the best assessment of that.

It was already in work by the time I noted I was tweeting it out. :) That was supposed to be the surprise for the people doing the amazing work here....and still will be if what I was told will happen :)

Oh and a big welcome to morningdew76 and Req!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: laika_fr on 05/21/2014 08:46 pm
I'am totally useless at this but this hexa editing stuff, reminds me good ol' cheats for DOS games, keep up the good work guys, George Lucas would be proud.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/21/2014 09:03 pm
p-frame editor has various bugs fixed and now has log parsing / error showing info http://spacex2.slapbet.org/
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: rsnellenberger on 05/21/2014 09:05 pm
Speaking as yet another member of the admiring bleacher team, this is like watching a group of grandmaster-level jigsaw puzzlists work.  Starting with the i-frames as the edges, and now filling in the field with the p-frames...

It couples nicely with Tapatalk's tendency to display so many of the mmb edit strings as smileys X:X:X:

At some point, it would be nice to show the repaired video side-by-side with the raw.ts original...

Thanks!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/21/2014 09:22 pm
Latest mmb for frame 72. Just some more sea macroblocks.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/21/2014 09:56 pm
p-frame editor has various bugs fixed and now has log parsing / error showing info http://spacex2.slapbet.org/

Just to be clear, for people who want to help cleaning up the p-frames, you have to put 0:0:-1 to all previous frames to edit the p-frames. The field "MMB for Entire Segment" will look like this for example for frame 172: FRAME0:0:0:-1=FRAME1:0:0:-1=FRAME2:0:0:-1=FRAME3:6:20:-1,7:20:84967
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/21/2014 10:03 pm
p-frame editor has various bugs fixed and now has log parsing / error showing info http://spacex2.slapbet.org/

Just to be clear, for people who want to help cleaning up the p-frames, you have to put 0:0:-1 to all previous frames to edit the p-frames. The field "MMB for Entire Segment" will look like this for example for frame 172: FRAME0:0:0:-1=FRAME1:0:0:-1=FRAME2:0:0:-1=FRAME3:6:20:-1,7:20:84967

So that's how you work on a p-frame without the previous frames affecting it?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/21/2014 10:16 pm
p-frame editor has various bugs fixed and now has log parsing / error showing info http://spacex2.slapbet.org/

Just to be clear, for people who want to help cleaning up the p-frames, you have to put 0:0:-1 to all previous frames to edit the p-frames. The field "MMB for Entire Segment" will look like this for example for frame 172: FRAME0:0:0:-1=FRAME1:0:0:-1=FRAME2:0:0:-1=FRAME3:6:20:-1,7:20:84967

So that's how you work on a p-frame without the previous frames affecting it?
Yes. It would be awesome if we had the option to temporarily "turn off" all previous frames this way in the online editor. When sending the command to the server/ffmpeg your would simply discard/ignore all mmb's from frames previous to the current one and put this series of FRAME0:0:-1=FRAME1:0:0:-1 etc before it.

You would still keep all the mmb's in the online editor, you just won't send it to the server during this "single P-frame mode". ;) And when somebody switches back you send all mmmb's again.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: morningdew76 on 05/21/2014 10:19 pm
Latest mmb for frame 72. Just some more sea macroblocks.

Found another expansion- X:10711:80 -> X:10695:80,X:10709:C0 which appears to remove the weird coloring in 43:2 and 43:3

It also looks like 37:3:13414 can be safely backed up to 35:03:13276, for two more blocks of sea.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/21/2014 10:43 pm
p-frame editor has various bugs fixed and now has log parsing / error showing info http://spacex2.slapbet.org/

Just to be clear, for people who want to help cleaning up the p-frames, you have to put 0:0:-1 to all previous frames to edit the p-frames. The field "MMB for Entire Segment" will look like this for example for frame 172: FRAME0:0:0:-1=FRAME1:0:0:-1=FRAME2:0:0:-1=FRAME3:6:20:-1,7:20:84967

So that's how you work on a p-frame without the previous frames affecting it?
Yes. It would be awesome if we had the option to temporarily "turn off" all previous frames this way in the online editor. When sending the command to the server/ffmpeg your would simply discard/ignore all mmb's from frames previous to the current one and put this series of FRAME0:0:-1=FRAME1:0:0:-1 etc before it.

You would still keep all the mmb's in the online editor, you just won't send it to the server during this "single P-frame mode". ;) And when somebody switches back you send all mmmb's again.

This is now implemented as "isolate frame" near "show errors"
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/21/2014 10:54 pm
FYI sometimes generating the log for the p-frame editor errors and doesn't return anything, not sure what it is yet but be aware.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/21/2014 10:55 pm
This is now implemented as "isolate frame" near "show errors"

You keep amazing me. This is exactly what we wanted.

Now we can see the P-frames as is. The way they are shown on the wiki pages like here  (http://spacexlanding.wikispaces.com/Frames+169-188).

Amazing job!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/21/2014 11:22 pm
My idea is that people can pick on of the 15 and start fixing the TS-headers and more importantly the  P-Frame headers. I wrote how those should look like in the post quoted. First try to find the position of the P-Frame headers, they should be more or less equally distributed through the file. See if the counter bytes are messed up, calculate what they are, compare what they should be and change the header bytes if necessary.

The transport stream headers are mostly correct after the work to suss out the correct program IDs and sequence numbers, right? Are there any details we need to know about that? Also, if you have any other links regarding bitmaps for the packetized elementary stream headers and what have you, that'd be helpful. Thanks for all your work!


All 15 I-Frame headers are ok. with all the right values for the variable parts.
For the 4 Byte TS-Header, there are good parts and bad parts in the ts-file. But at least they are all aligned now i think. so every 188th byte starts a new TS-packet like it should be. That does not mean there is a "47"-byte at every 188th position. Sometimes there are huge gaps spanning 10-20 ts-packets with no correct ts-header.

See the picture below. It shows the first few ts-packets of rawsplit_part_8.ts. You can see where Ts-Packets should be, marked by the black line. Blue circles show the correct CC bytes, in Ts-Headers in green are not correct. there are some bytes wrong, due to biterrors. Red shows the I-Frame Header. Cyan marks the two variable parts of the header, which are increasing.

Here they are in dez:
0x329c2B -> 3251243
0x8d228d -> 9249421

The following P-header is: 47 43 E8 3E 07 10 00 31 A7 E6 7E 00 00 00 01 E0 00 00 81 80 07 21 01 8D 51 79 FF FF 00 00 01 B6 51
coloured are both variable parts: there value in dez is:
31A7E6 -> 3254246 -> 3003 more than before, this is correct, because 20*3003=60060, the difference between I-Frames

The second is more complicated I just found out: There is a no equal distance between all PTS values of the I-Frames, but there is a pattern: So i would suggest to decode all PTS values and try to find a pattern. Apply it to all those where the pattern brakes.

       PTS Hex   PTS Dez   Difference to the last value
IF1    59797D     5863805   
IF2    6123ED     6366189   502384
IF3    67CE5D     6803037   436848
IF4    6F78CD     7305421   502384
IF5    77233D     7807805   502384
IF6    7DCDAD     8244653   436848
IF7    85781D     8747037   502384
IF8    8D228D     9249421   502384
IF9    93CCFD     9686269   436848
IF10   9B776D    10188653   502384
IF11   A321DD    10691037   502384
IF12   A9CC4D    11127885   436848
IF13   B176BD    11630269   502384
IF14   B9212D    12132653   502384
IF15   BFCB9D    12569501   436848


I hope this helps some.

What I have written above about the PTS is wrong, I forgot that the PTS is 33bits spread over 5 bytes with some marker 1s in between.
Please ignore that, the PTS itself is increasing by 120120 between I-frames and 6006 between frames.

Attached you find a python-script I use the find P-Frame headers, there are some routines that de(en)code the PTS. Maybe that is usefull for someone else too.

Cheers,
Shanuson
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/21/2014 11:46 pm
Finished part_1.ts so far that it is now producing 19 P-Frames.

Next i will start work in part_2

Cheers
Shanuson

Edit: Updated 1 File in that zip which I forgot to save before creating the zip.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Req on 05/22/2014 12:40 am
Elon asks who's leading the effort?

Eight Tips for Collaborative Leadership (http://www.forbes.com/sites/carolkinseygoman/2014/02/13/8-tips-for-collaborative-leadership/)

Quote
Today’s corporation exists in an increasingly complex and ever-shifting ocean of change. As a result, leaders need to rely more than ever on the intelligence and resourcefulness of their staff. Collaboration is not a “nice to have” organizational philosophy. It is an essential ingredient for organizational survival and success.

That's what's been interesting about the effort - each individual has brought their individual talents, experience, knowledge, and skills to bear on different pieces of the problem to the extent that they are able to contribute, and everyone has been learning from everyone else.

Outsized kudos are due, of course, to the people who have set up the framework that allows this collaboration to occur in the first place - Michael Niedermyer for his ffmpeg patches to simplify the work on the frames, Iain Cole for his magnificent online editor, Moralec for his work in maintaining the Wiki page, etc. But are they "leaders" in the traditional sense of the word? They're fellow collaborators.

I think the effort and the fruit that it has borne so far is just a really excellent example of the power of "crowdsourcing" (I think someone will eventually have to add this video repair effort to the Wikipedia page for that term. :D), and a testament to how many people around the world are cheering for Elon and his entire team at SpaceX, and the vision of a multiplanetary human society.

I remember fondly my first Mars Society convention years ago in Boulder, and my work with the California Mars Society around 2000, and it's just magnificent to see how much closer we are to that goal today thanks to the efforts of not only Elon but also countless other people who were inspired - by Ray Bradbury, or Robert Zubrin, or Kim Stanley Robinson, or in any number of other ways - to each work in their own way and in their own capacity towards achieving it.

In a way, this video repair mission is just another example of that same phenomenon. It's great to see it unfold right before our eyes.

He only gets the space of a tweet, why not just have him link to this thread or the wiki page?

Also, having kept up with the thread since the beginning, I think that Arnezami would be on the short list as well.  In fact, if anybody is playing a leadership role, it's probably him.  At least in my opinion.

I read this over again, and I just want to point out that Shanuson, saliva_sweet, Untribium, ummmm, those are the ones that come to mind.  I made this post mainly because I don't want to leave anybody out.  There's probably more.  So, on to my main point:

I don't even like this approach.  I just wanted to point out that if I were to pick a leader, it would be Arnezami.  He's the one that got the people over here from reddit, he provided momentum in the most critical early stages, he's been refining, defining, and requesting paradigms for collaborative work, etc.  My original statement was that Elon should just link to the thread or the wiki page.  Let the masses who care enough read everybodies contribution and come to their own conclusions.

However, if Elon needs the name of a leader, it should obviously be Arnezami, in my opinion.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Untribium on 05/22/2014 01:04 am
Okay, so I spent a not so little while on the p-frames following 52 and I got 1 (one!) frame fixed...and I don't even know if I did this right :))
It's really tedious, because there's almost no difference between frames 52 and 53, so there's almost no information for the lower part of the p-frame (which makes sense, but it makes it a lot more difficult to find markers...). In the lower part of the frame, you can just barely make out the shape of the rocket, but it took a while until I realized that :) I really wish the legs were extended at this point ;)
22:15:36982,30:15:-1,0:16:38358,25:17:-1,41:18:46765

Off-topic: Good luck to people who used to make a living out of repairing video files...I guess customers might no longer accept "a 50% chance of recovering one or two frames" as an answer after seeing the latest gifs from this video :))
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Asmegin on 05/22/2014 01:06 am
Took a quick try at Frameset 92 - holy smokes. This section is brutal  ;D I just tried to align the legs for now, to see them finish unfolding and the engine firing up.

Having a problem though... It seems after corrections to certain p-frames, the editor stops updating. For example, using the following:

Quote
FRAME0:X:54419:00,X:53655:00,0:0:550,14:0:-2:0:0:0:0:12:-6,2:5:14749,
39:7:-1,18:9:27558,25:12:39009,29:12:-1,17:13:42121,18:13:42526,10:14:-1,
18:14:47626,26:14:-2:0:0:0:0:8:-6,30:14:50837,1:15:-1,5:15:51900,9:15:-1,
21:15:53655,27:16:61466,42:19:-1,43:19:83153,43:20:-1,1:21:90585,11:22:-1,
17:23:105394,22:24:-1,23:24:111581,0:27:-1,25:27:125621,28:28:-1,3:29:140154=
FRAME1:1:10:17471=FRAME2:11:12:10850,1:28:48285=FRAME3:2:12:21895,
17:15:34992,0:16:38893,6:27:73094=FRAME4:0:0:1335,0:19:47777=FRAME5:0:4:9064=
FRAME6:6:8:16348,0:14:29164=FRAME7:0:10:18385=FRAME8:0:16:34639=
FRAME9:0:4:7002=FRAME10:2:11:25555,15:11:41494,0:24:-1=FRAME11:0:6:88170,
0:7:10335=FRAME13:16:11:20288,0:14:29710=FRAME14:0:16:26506=
FRAME16:0:9:-1,16:11:11103,0:14:17752,18:14:18288,0:16:25394=FRAME17:1:8:11380=
FRAME19:0:0:1234

The slider stops changing the image after Frame 102. Deleting 99-101 fixes it, but I can't figure out why. I am still able to view the isolate frames all the way up to Frame 111 though.

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: PaulNY on 05/22/2014 02:38 am
Wow you guys are making awesome progress!!!!

I know we are trying to replace bad mmb with good mmb. So would it be useful to document the XY and POS of good mmb's? I am thinking if we can map out what we know to be good, figuring out what to replace the bad with may be easier.
Welcome to the forum PaulNY!

From what I understand your idea is to catalog good macro blocks in all the frames. Good as in decided by human beings not by some algorithm I assume.

This may be of some use. I'm not sure. We sort of implicitly rely on our vision to detect good vs bad blocks. We also have a nice feature in the online tool that highlights all error-macroblocks in a frame. There are still plenty of non-error-blocks left that look "bad", but we don't know if they are easily repairable or are really bad. We usually judge their "badness" by the destruction they cause to other blocks.

So depending on what you think 1. what the criteria of "bad" is, or even multiple classes of bad and 2. how you think we can deal with those kinds of "badness", your idea might have some value.

Maybe you can try to "fertilize" your ideas a little bit so it becomes more clear if we can use it. :)

Regards,

arnezami

PS. We refer to mmb as "modify macro block". I believe when you say mmb you actually ment to say "macro block".

Unfortunately work has kept me away for a few days, but let me try to explain my idea more.

Lets say I look at I frame 8. MB 1 (0:0) looks right and is blue water. If we can document the position of this MB in the frame, then lets say some MB (x,y) looks bad and we think it's water we could try replacing it with a known value of a good water MB instead of trying to guess. If we can document all the know good and bad MB values in the I frames, then we can then target our repairs more. I feel like when trying to make a repair the majority of the time I spend is guessing MB pos values or the pos to try a invert / flip.

Maybe I am missing something, but if this is where other people spend a lot of time to it would be great if we could create a way to collaborate on this more and optimize the process.

If we can't document known good or bad positions, maybe we could create a DB of recommended or non recommended pos values for each MB to try and optimize the process.  For example if I try to fix / replace MB (X:Y) and try these 400 values that don't work. If I can document that I can save the next guy from trying the same thing.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/22/2014 03:32 am
Off-topic: Good luck to people who used to make a living out of repairing video files...I guess customers might no longer accept "a 50% chance of recovering one or two frames" as an answer after seeing the latest gifs from this video :))

They  could always point out to the customers that it's taken 40 years to restore the stained glass windows at Sainte Chapelle. I wonder how much the magnificent work over the past month or so would have cost at a mere $20 per hour... :D

(http://si.wsj.net/public/resources/images/AR-AD661_stchap_DV_20130918142628.jpg) (http://online.wsj.com/news/articles/SB10001424127887323308504579082991871355218)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/22/2014 03:38 am
Lets say I look at I frame 8. MB 1 (0:0) looks right and is blue water. If we can document the position of this MB in the frame, then lets say some MB (x,y) looks bad and we think it's water we could try replacing it with a known value of a good water MB instead of trying to guess.

Is this a situation where the encoding mechanisms of MPEG-4 would get in the way? As I understand it, MB(x,y) will depend on inputs from MB(x-1,y) or MB(x,y-1), and if you try to drop it in bit-for-bit elsewhere in the image, it won't look the same because what's feeding it won't be the same. Someone more conversant with the encoding can shed some more light on this.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: meadows.st on 05/22/2014 03:52 am
@PaulNY, theshadow27 created something similar to the "bad/good" marking but it was based on ffmpeg errors and other attributes of each mb.  I agree that @mvpel is correct that a direct substitution of a "good" mb for a "bad" one would create scrambled results but I like where you are going on attempting to prevent duplication of effort.

Ok, first try at auto-marking bad MBs based on ffmpeg log output and primitive analysis.

Blocks which were reported as errors by ffmpeg (modified, with debug on) are marked black. Internally, this tracks:
  --  the -1 and -2 override blocks from MBM,
  -- dc clipped at
  -- ac-tex at
  -- dquant at
  -- stuffing at
  -- Error at MB
although these are all painted black as the colors were confusing.

Blocks that were NOT reported as errors but probably are bad are translucent red.
   -- currently tests for the block having less than 5 individual colors.

(http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=582552)

Next step is to begin (hopefully) intelligent mutation of the MBM string to achieve a better overall score. Sample image is of frame 4 with saliva_sweet MBMs (from wiki).


Edit: Here's the Java source if anyone else wants to play. http://pastebin.com/bKgzCRqD The mammoth inner class at the bottom is for writing the animated gif :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: dorkmo on 05/22/2014 04:05 am
frame 82 and 85 is where the party starts. the iframe before needs a lot of help though :/

my first very bad attempt.
FRAME0:4:1:-1,5:1:-1,20:1:5838,34:2:-1,3:3:-1,0:6:24132,20:10:-1,43:17:66647,
0:18:66647,17:18:-1,32:18:70700,34:18:-1,26:19:75020,31:19:76540=FRAME10:4:16:-1,
18:16:30362=FRAME13:13:15:25802
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mhenderson on 05/22/2014 04:13 am
I am not sure what the process / protocol is for posting -mmb results to the progress wiki, so I will leave it here and let you guys do your magic. 

I put **hundreds** of hours into trying to coax detail out of frame 209. My inner voice tells me that a flash of lightning took out wide swaths of the transport stream. These frames are probably the moment of contact with the water, the legs seem to disappear ... but that could be my fault, too. ;) Perhaps someone else with mohr wikid skillz can do these frames better justice.

I am so proud of this team. As I watch the work in progress I see that really difficult stuff gets done right away. Impossible stuff takes a few hours.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Asmegin on 05/22/2014 04:33 am
I am not sure what the process / protocol is for posting -mmb results to the progress wiki, so I will leave it here and let you guys do your magic. 

I put **hundreds** of hours into trying to coax detail out of frame 209. My inner voice tells me that a flash of lightning took out wide swaths of the transport stream. These frames are probably the moment of contact with the water, the legs seem to disappear ... but that could be my fault, too. ;) Perhaps someone else with mohr wikid skillz can do these frames better justice.

I am so proud of this team. As I watch the work in progress I see that really difficult stuff gets done right away. Impossible stuff takes a few hours.

I noticed you only used -1's for everything. This removed all of the good stuff as well ie the plume. Don't worry about stray pixels and stuff for now. Just try to match as much detail as possible like the exhaust, the legs, the body of the rocket etc. Isolate each p-frame and look for anything recognizable, and use the furthest point possible place the object (if that makes sense). The first few P-Frames of 209 look decent and probably only need minor cleaning :) It only gets really bad @ 212.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: michaelni on 05/22/2014 04:50 am
Hi all
i updated github with some bugfixes and new features for P frames
-1 and -2 should now work like they do for I frames and allow messing with DC values
-3 results in a skiped block (this is what -1 was effectively before it worked so you have to replace -1 by -3 in old mmbs for P frames)
-4 makes it a inter MB with 1 motion vector

example:
24:10:-4:1:1
30:20:-3
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/22/2014 05:05 am
My inner voice tells me that a flash of lightning took out wide swaths of the transport stream. These frames are probably the moment of contact with the water,

Would the exhaust plume be ionized and able to discharge static to ground, or would you need one of these?

(http://www.cablejoints.co.uk/images/gallery/uploads/1382451819_Lind%20Equipment%20Auto-Rewind%20Reels...jpg)

Static Electricity and Aircraft - by Katia Gigliotti (http://onlinelibrary.wiley.com/doi/10.1002/9781118097298.weoc234/abstract)
Quote
The accumulation of electrostatic charge, generating during flight on the outer surfaces of aircraft and inside aircraft piping systems, cannot be considered as an immediate danger for flight safety, but it has to be seriously prevented to avoid upset in flight communications and risks of explosion in fuel areas or during refueling operations.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mhenderson on 05/22/2014 05:10 am
@Asmegin.  Thank you, I suspected the -1 were a blunt instrument. I will take another crack at this tomorrow night.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/22/2014 05:16 am
@Asmegin.  Thank you, I suspected the -1 were a blunt instrument. I will take another crack at this tomorrow night.

I am also working on this set, so be sure to grab my mmbs from the wiki (http://spacexlanding.wikispaces.com/Frames+209-228). I look forward to seeing what you find!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/22/2014 06:20 am
Tweet by Elon: 8)

https://twitter.com/elonmusk/status/469356919979655168

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/22/2014 06:34 am
Tweet by Elon: 8)

https://twitter.com/elonmusk/status/469356919979655168



Was woken up by my e-mail notifications acting like an alarm clock! ;D

That's the recognition you guys absolutely deserve for such fine work on this. Well done again!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: edfishel on 05/22/2014 06:34 am
I loved the Musk recognition of your work NSF and was sure to re-tweet it! ;)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jester on 05/22/2014 07:30 am
Wish I had time to help out, but stuck playing with our spacecrafts.

very impressive job guys !
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Oersted on 05/22/2014 07:35 am
Great encouragement of this TEAM EFFORT. Makes no sense to single out individuals, because no individual could have gotten this far on his own. And that's why it has been cool to follow the cooperation. Anyway, it is probably better to keep the eyes on the ball and focus on the remaining work than to start throwing congrats around. They will arrive in due course!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: FutureSpaceTourist on 05/22/2014 08:06 am
The attention this is getting is very well deserved. Hopefully Elon's tweet will drive even more traffic to NSF  :)

For a new visitor it may not be obvious where to look for this thread? Could there be a non-L2 "What's popular" link from the home page or something?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: FutureSpaceTourist on 05/22/2014 08:22 am
Currently this thread is the 27th most read SpaceX thread

But  I just noticed it's first for likes  8)

Approaching a 1,000 likes on the thread, can't be many threads on any topic that have reached that milestone!

[Edit: drat, actually currently 2nd, SpaceX lawsuit thread has a few more ... but I suspect not for too long]
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Garrett on 05/22/2014 08:55 am
Tweet by Elon: 8)
https://twitter.com/elonmusk/status/469356919979655168
Was woken up by my e-mail notifications acting like an alarm clock! ;D
That's the recognition you guys absolutely deserve for such fine work on this. Well done again!

A massive well done to all involved! Now, get back to work ;)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/22/2014 09:24 am
Approaching a 1,000 likes on the thread, can't be many threads on any topic that have reached that milestone!

There is a lot of intra-workinggroup liking though ;)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/22/2014 09:53 am
Okay, so I spent a not so little while on the p-frames following 52 and I got 1 (one!) frame fixed...and I don't even know if I did this right :))
It's really tedious, because there's almost no difference between frames 52 and 53, so there's almost no information for the lower part of the p-frame (which makes sense, but it makes it a lot more difficult to find markers...). In the lower part of the frame, you can just barely make out the shape of the rocket, but it took a while until I realized that :) I really wish the legs were extended at this point ;)
22:15:36982,30:15:-1,0:16:38358,25:17:-1,41:18:46765

Off-topic: Good luck to people who used to make a living out of repairing video files...I guess customers might no longer accept "a 50% chance of recovering one or two frames" as an answer after seeing the latest gifs from this video :))

That's quite good :)

A few general advices:
- There are usually easy-to-recover p-frames, it's better to begin with these ones, they will then give a reference for the harder-to-correct ones.
- Use the p-frames before and after to edit the actual one, it helps a lot, lots of moving features are at a very close position and help get the MB sequences at the right place.
- Before doing any correction, note the first bit position of all sequences of MB that look "good", then try to put them at the right position.
- If you are not 100% sure of a MB sequence, it's better to fill with 0:0:-1, as no change from the previous frame is better than a wrong change.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: AndyX on 05/22/2014 10:23 am
Well done to all concerned. Great that Elon has recognized the great work by the NSF gang.

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mhenderson on 05/22/2014 11:46 am
My next input on frames 209-215 is attached.  My first submission on pFrames was shameful crap, please ignore it. In this set, frames 209-211 are decent, but it deteriorates pretty quickly after that.

1) @Asmegin - Thank you for the kind coaching. I have now used -1 in the pFrames to disable only single blocks then immediately resume at the next block. I have done only minor tweaking of luminance and color in the pFrames. Are there other tricks?

2) @Wronkview - By all means, lets pool our efforts. There is no I in TEAM. I will look at your stuff today / this evening (US eastern time).

3) This is more addictive than crack. I will be worthless at the office today.

4) The image appears to melt like crayons on the stove. Would it be worthwhile to set the initial iFrame with a "test pattern of horizontal black and grey bars then somehow correct the areas that should stay static?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Untribium on 05/22/2014 02:13 pm
Okay, so I spent a not so little while on the p-frames following 52 and I got 1 (one!) frame fixed...and I don't even know if I did this right :))
It's really tedious, because there's almost no difference between frames 52 and 53, so there's almost no information for the lower part of the p-frame (which makes sense, but it makes it a lot more difficult to find markers...). In the lower part of the frame, you can just barely make out the shape of the rocket, but it took a while until I realized that :) I really wish the legs were extended at this point ;)
22:15:36982,30:15:-1,0:16:38358,25:17:-1,41:18:46765

Off-topic: Good luck to people who used to make a living out of repairing video files...I guess customers might no longer accept "a 50% chance of recovering one or two frames" as an answer after seeing the latest gifs from this video :))

That's quite good :)

A few general advices:
- There are usually easy-to-recover p-frames, it's better to begin with these ones, they will then give a reference for the harder-to-correct ones.
- Use the p-frames before and after to edit the actual one, it helps a lot, lots of moving features are at a very close position and help get the MB sequences at the right place.
- Before doing any correction, note the first bit position of all sequences of MB that look "good", then try to put them at the right position.
- If you are not 100% sure of a MB sequence, it's better to fill with 0:0:-1, as no change from the previous frame is better than a wrong change.

Thanks :) I did most of what you're suggesting, but I guess I was too focused on getting this frame repaired as much as possible. Better to move on to the ones where there's actually something interesting happening :)) (then again, this is video footage of the soft-landing of a rocket stage, so every frame is showing something interesting ;) )
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: michaelni on 05/22/2014 03:04 pm
Hi all
ive added a new option to the custom ffmpeg on github,
-debug mb_pos_bruteforce
This will list all bitstream positions, start, end, size for any correctly decodeable MB.
With this you can quickly find a MB that ends at a specified position or even perform some statistical analysis on the start/end positions, maybe automating the finding of more likely valid chains

Example:
Quote
[mpeg4 @ 0x7f04c0001540] MB pos brute: F#:0, PTS:0, Start:511, End:646, Size:135, dQP:-2
[mpeg4 @ 0x7f04c0001540] MB pos brute: F#:0, PTS:0, Start:512, End:707, Size:195, dQP:0
[mpeg4 @ 0x7f04c0001540] MB pos brute: F#:0, PTS:0, Start:513, End:685, Size:172, dQP:0
[mpeg4 @ 0x7f04c0001540] MB pos brute: F#:0, PTS:0, Start:514, End:690, Size:176, dQP:0
[mpeg4 @ 0x7f04c0001540] MB pos brute: F#:0, PTS:0, Start:515, End:646, Size:131, dQP:0
[mpeg4 @ 0x7f04c0001540] MB pos brute: F#:0, PTS:0, Start:516, End:656, Size:140, dQP:0
[mpeg4 @ 0x7f04c0001540] MB pos brute: F#:0, PTS:0, Start:517, End:646, Size:129, dQP:0
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/22/2014 03:07 pm
To help everyone with the repair job on the p-frames, I wrote a small tutorial:

http://spacexlanding.wikispaces.com/Tutorial+p-frames

Feel free to add your own comments if you have well-working strategies to correct these images!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/22/2014 03:51 pm
Hi all
ive added a new option to the custom ffmpeg on github,
-debug mb_pos_bruteforce
This will list all bitstream positions, start, end, size for any correctly decodeable MB.
With this you can quickly find a MB that ends at a specified position or even perform some statistical analysis on the start/end positions, maybe automating the finding of more likely valid chains

Example:
Quote
[mpeg4 @ 0x7f04c0001540] MB pos brute: F#:0, PTS:0, Start:511, End:646, Size:135, dQP:-2
[mpeg4 @ 0x7f04c0001540] MB pos brute: F#:0, PTS:0, Start:512, End:707, Size:195, dQP:0
[mpeg4 @ 0x7f04c0001540] MB pos brute: F#:0, PTS:0, Start:513, End:685, Size:172, dQP:0
[mpeg4 @ 0x7f04c0001540] MB pos brute: F#:0, PTS:0, Start:514, End:690, Size:176, dQP:0
[mpeg4 @ 0x7f04c0001540] MB pos brute: F#:0, PTS:0, Start:515, End:646, Size:131, dQP:0
[mpeg4 @ 0x7f04c0001540] MB pos brute: F#:0, PTS:0, Start:516, End:656, Size:140, dQP:0
[mpeg4 @ 0x7f04c0001540] MB pos brute: F#:0, PTS:0, Start:517, End:646, Size:129, dQP:0
Nice!

I think what would also be very useful is to able to tell ffmpeg only to decode either luminosity, chrom1 or chrom2. So either only the first 4 dc's are decoded or only the fifth or only the sixth.

Then we can add 3 checkboxes in the online version. That way you can much more clearly see where things go wrong colorwise or intensity wise. Without our eyes being districated by either the intensity or (one of) the two colors.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mhenderson on 05/22/2014 04:06 pm
How do we download / install the tool from github? Are there instructions for installing that app on a Windows box? Do I need a Linux VM? On a scale of 1 to 10, I am a 2 in Linux.

@SwissCheese, @michaelni, @IainCole ... you guys rock ... the pFrame tutorial is just what I needed and the fundamental toolkit is powerful and growing.

I have been using the online tool, but really would benefit from the new bruteforce feature in the github app. It would be soooo much more efficient to use it as a menu of candidate blocks. Most blocks in the water and ship body seem to have a size of 50-110, so being able to see a list of POS values for valid blocks would really help.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/22/2014 04:34 pm
And that's the thread through the 200,000 visit mark!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mhenderson on 05/22/2014 04:35 pm
Answering my own question:

You can run Ubuntu Linux from a USB stick or CD.
http://www.ubuntu.com/download/desktop/try-ubuntu-before-you-install

or if you prefer a VM and have sufficient hardware ...

Download and install VMWare Plus for Windows from
https://my.vmware.com/web/vmware/free#desktop_end_user_computing/vmware_player/6_0
(free for noncommercial use)

Download and install Ubuntu Linux on the VM from http://www.ubuntu.com/download/desktop

Download and install Custom FFmpeg. From the GIT repository at https://github.com/michaelni/FFmpeg/tree/spacexdebug1
Installation instructions for FFmpeg are here: http://forum.nasaspaceflight.com/index.php?topic=34597.msg1193442#msg1193442

System requirements for VMWare Plus:
"For a typical host system, we recommend that you have a 1 GHz or faster processor (2GHz recommended) and 1GB RAM minimum (2GB RAM recommended). You must have enough memory to run the host operating system, plus the memory required for each guest operating system and for applications on the host and guest. See your guest operating system and application documentation for their memory requirements. VMware Player requires approximately 150MB of disk space to install the application. - See more at: http://www.vmware.com/products/player/faqs.html#sthash.smDidWiW.dpuf"

I will give the USB approach a try tonight and publish instructions if it works / a dire warning to noobs if it is way over my head.

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Lourens on 05/22/2014 04:56 pm
How do we download / install the tool from github? Are there instructions for installing that app on a Windows box? Do I need a Linux VM? On a scale of 1 to 10, I am a 2 in Linux.

Ohh, I'm way late to the party, been really busy and I'd overlooked the thread :(. But it's not over yet fortunately :). I think it's possible to build FFmpeg on Windows, but I haven't used Windows in over a decade, so I would have no idea how.

Here's in the simplest possible terms how to do it on Linux though, specifically on Debian GNU/Linux or any derived version (e.g. Ubuntu (http://www.ubuntu.com) or Linux Mint (http://www.linuxmint.com)). I just did it like this on a Linux Mint Debian Edition laptop, but it should work fine in a VirtualBox VM as well. Compared to the install.txt file, this only covers installing FFmpeg, not using it, but it adds a few steps you'll need on a newly installed Linux machine.

I'm not 100% sure that I got everything in that respect, as I've got a lot of development tools already installed. If you get any error messages, PM me and we'll fix it and I'll amend the instructions here.

You're supposed to type the instructions highlighted in blue below into a terminal window, which you should be able to open from the menu, usually under Accessories. The part before the first dollar sign will already be there, so start with sudo on the first one.

First, install some software we'll need. This will ask you for your login password since it's a system administration function.

user@computer:~$ sudo apt-get install git yasm build-essential

Then, we make a working directory and change into it

user@computer:~$ mkdir spacex_landing_video
user@computer:~$ cd spacex_landing_video


Now, we clone the Git repository. This will create a local copy of the modified FFmpeg source code. If you're doing this for the first time, it'll download about 100MB of data, but you'll be able to get only the changes next time Michael updates something, so that it will be much faster then.

user@computer:~/spacex_landing_video$ git clone -b spacexdebug1 --depth 1 https://github.com/michaelni/FFmpeg.git FFmpeg-spacexdebug1

After the download is done, change into the newly created FFmpeg-spacexdebug1 directory:

user@computer:~/spacex_landing_video$ cd FFmpeg-spacexdebug1

Now it's time to build it. First, we'll run the configure program, which will analyse your system and figure out whether everything needed is there, as well as how to build FFmpeg. If this gives any errors, there's probably a library missing from your system. Please PM (to avoid burying the thread in debug output) me the entire configure output, and we'll get it fixed.

user@computer:~/spacex_landing_video/FFmpeg-spacexdebug1$ ./configure

If that completed successfully, you can build FFmpeg using

user@computer:~/spacex_landing_video/FFmpeg-spacexdebug1$ make

This took about 10 minutes on my laptop. You should then be able to run the newly built FFmpeg using

user@computer:~/spacex_landing_video/FFmpeg-spacexdebug1$ ./ffmpeg -version

Now, you can go and download the files from the wiki and play around with them. You'll probably want to put them into a different directory. To run your new ffmpeg from that directory, it's useful to add it to the PATH environment variable. You can then start it from any directory simply by typing ffmpeg (i.e. without the ./ in front). To change the PATH variable, do

user@computer:~/spacex_landing_video/FFmpeg-spacexdebug1$ PATH=$HOME/spacex_landing_video/FFmpeg-spacexdebug1:$PATH

You'll have to run this last command once every time you open a new terminal window, which is a bit cumbersome. If you add it to the end of the file named .bashrc in your home directory, it'll run automatically every time you start a terminal window. Probably the easiest way to do that is by running

user@computer:~$ echo -e '\nPATH=$HOME/spacex_landing_video/FFmpeg-spacexdebug1:$PATH\n' >>~/.bashrc

I hope that helps you get started.

Edit: got the wrong Git repository!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: morningdew76 on 05/22/2014 05:01 pm
Just wanted to share the technique that I've been using to search for single and triple flipped bits-

This is a bash script looking for a single flipped bit between 8800 and 9100, specifically, I am trying to fix 40:03 on frame 52-
Quote
for i in {8800..9100} ; do ffmpeg.exe -threads 1 -debug mb_pos_size -err_detect ignore_err -s:0 704:480 -mmb X:76768:80,X:22120:80,X:$i:80,0:7:-2:-10:-10:-10:-10:8:-5,2:7:15506,9:7:-1,15:7:16391,17:7:-1, 28:7:16704,35:7:-2:-10:-10:-10:-10:8:-5,29:9:21074,34:9:-1,39:9:21626,5:28:-1,10:28:80196 -i iframe52.mpg4-img -f image2 img-%03d-$i.png 2> output-$i.txt ; done

The images are not important.. what I am looking for is any output text file that contains the text of 41:03:8999 , which appears to be a good position.  As such, grep the output text for 41:03:8999 .  No results were found, which tells me that a single bit flip here will not fix MB 40:03.

Here's the bash script for searching for one of the 80C0 triplets-
Quote
for i in {8800..9100} ; do ffmpeg.exe -threads 1 -debug mb_pos_size -err_detect ignore_err -s:0 704:480 -mmb X:76768:80,X:22120:80,X:$i:80,X:$(($i+14)):C0,0:7:-2:-10:-10:-10:-10:8:-5,2:7:15506, 9:7:-1,15:7:16391,17:7:-1,28:7:16704,35:7:-2:-10:-10:-10:-10:8:-5,29:9:21074,34:9:-1,39:9:21626,5:28:-1,10:28:80196 -i iframe52.mpg4-img -f image2 img-%03d-$i.png 2> output-$i.txt ; done

No results were found from that run either.

This setup is running in MinGW32 on Windows.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Swatch on 05/22/2014 05:24 pm
Congratulations to everybody, the tweet from Elon was WELL deserved.  And that was for just 2 seconds worth of video!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/22/2014 06:04 pm
By the way, @SpaceX also retweeted the Elon message. :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/22/2014 07:28 pm
So having an issue with the editor atm. Basically the new -3 instead of -1 thing doesn't seem to work... even though it works when I'm running it on my test rig....

Makes... no... sense.....
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/22/2014 07:45 pm
So having an issue with the editor atm. Basically the new -3 instead of -1 thing doesn't seem to work... even though it works when I'm running it on my test rig....

Makes... no... sense.....

Hi all
i updated github with some bugfixes and new features for P frames
-1 and -2 should now work like they do for I frames and allow messing with DC values
-3 results in a skiped block (this is what -1 was effectively before it worked so you have to replace -1 by -3 in old mmbs for P frames)
-4 makes it a inter MB with 1 motion vector

example:
24:10:-4:1:1
30:20:-3

@michaelni: can you figure this out for us?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/22/2014 07:47 pm
It's seriously weird. I mean, I have it actually working when I run locally through the web app, but it doesn't work when I run EXACTLY the same command through the cmd prompt....

Might just sack this all off and re-write it in something I can run on linux =)

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/22/2014 09:43 pm
here is the "fixed" part_2, at least a version that produces also 19 p-frames now.

This leaves only 4 Pframes missing now , maybe I can get a version out tonight that has finally all frames.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Lourens on 05/22/2014 10:06 pm
Found another bit flip in iframe 52: X:45038:80 relative to iframe52.mpg4-img improves the chroma in the bottom right corner. There are for sure a bunch of other bit flips that are causing the dark spot at bottom right, but it's difficult to find them. I have an idea for potentially automating this search though...maybe on the weekend.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Untribium on 05/22/2014 10:21 pm
Found another bit flip in iframe 52: X:45038:80 relative to iframe52.mpg4-img improves the chroma in the bottom right corner. There are for sure a bunch of other bit flips that are causing the dark spot at bottom right, but it's difficult to find them. I have an idea for potentially automating this search though...maybe on the weekend.

Nice find! X:42196:08 gets rid of some of the dark spot. I'll update the wiki and keep looking for more :)

edit: attached image

edit2: Found another one! X:66234:08 fixes the white gradient thingy at the bottom right. Man, and I just updated the wiki ;)

edit3: And another one...minor color correction using X:50298:08. I should really stop updating the wiki ;)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/22/2014 10:44 pm
Some improvements to iframe 72. More sea and a bit of the good stuff. mmb is massive.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Alkan on 05/22/2014 11:09 pm
It's awesome that you guys got tweeted by Elon himself!

https://twitter.com/elonmusk/status/469356919979655168
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: morningdew76 on 05/22/2014 11:19 pm
Found another bit flip in iframe 52: X:45038:80 relative to iframe52.mpg4-img improves the chroma in the bottom right corner. There are for sure a bunch of other bit flips that are causing the dark spot at bottom right, but it's difficult to find them. I have an idea for potentially automating this search though...maybe on the weekend.

Nice find! X:42196:08 gets rid of some of the dark spot. I'll update the wiki and keep looking for more :)

edit: attached image

edit2: Found another one! X:66234:08 fixes the white gradient thingy at the bottom right. Man, and I just updated the wiki ;)

edit3: And another one...minor color correction using X:50298:08. I should really stop updating the wiki ;)

Change 0:29:81738 to 10:28:80196 to get most of row 28.

I've been attempting to recover the top of the numbers with little success.. there are about 20 positions to put 09:28 that result in 10:28:80196, but no places to put 08:28 that result in a correct 10:28.  I haven't tried flipping bits in this section yet (it's about 4k).
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Untribium on 05/22/2014 11:43 pm
Sweet! Yeah, the timestamp mmbs are huge :D not really surprising, but a pain to check for flips...I just found a one that fixes some stuff, but is "incompatible" with some I found earlier. Not sure whether it's correct or not...If I ignore (-1) the block it's in, the following blocks look exactly the same, but at the same time, the macroblock itself doesn't look quite right...oh well, I'll have to check it tomorrow :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/23/2014 12:01 am
A little update of the landing sequence. I think I will rather post movies after this update, these gif are getting very big!

The sequence goes from frame 169 to frame 211. The missing frames are 178, 201, 202, 206 and 207.

Edit: somehow I cannot put the gif here, but you can see it there: http://spacexlanding.wikispaces.com/Progress+using+multi+frame+MMBs
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/23/2014 12:04 am
Found another bit flip in iframe 52: X:45038:80 relative to iframe52.mpg4-img improves the chroma in the bottom right corner. There are for sure a bunch of other bit flips that are causing the dark spot at bottom right, but it's difficult to find them. I have an idea for potentially automating this search though...maybe on the weekend.

Nice find! X:42196:08 gets rid of some of the dark spot. I'll update the wiki and keep looking for more :)

edit: attached image

edit2: Found another one! X:66234:08 fixes the white gradient thingy at the bottom right. Man, and I just updated the wiki ;)

edit3: And another one...minor color correction using X:50298:08. I should really stop updating the wiki ;)

Very nice!

Could you rather put your updated MMB on this page? http://spacexlanding.wikispaces.com/Frames+52-71
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Untribium on 05/23/2014 12:36 am
Found another bit flip in iframe 52: X:45038:80 relative to iframe52.mpg4-img improves the chroma in the bottom right corner. There are for sure a bunch of other bit flips that are causing the dark spot at bottom right, but it's difficult to find them. I have an idea for potentially automating this search though...maybe on the weekend.

Nice find! X:42196:08 gets rid of some of the dark spot. I'll update the wiki and keep looking for more :)

edit: attached image

edit2: Found another one! X:66234:08 fixes the white gradient thingy at the bottom right. Man, and I just updated the wiki ;)

edit3: And another one...minor color correction using X:50298:08. I should really stop updating the wiki ;)

Very nice!

Could you rather put your updated MMB on this page? http://spacexlanding.wikispaces.com/Frames+52-71

Oh, I meant to do that but forgot about it :) the gif looks awesome, nice work! :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mhenderson on 05/23/2014 02:27 am
Update to iFrame 92.

On the wiki, user mhenderson is replacing the following -mmb
for iFrame 92 by Untribium

PRIOR MMB
X:54419:00,X:53655:00,0:0:550,14:0:-2:0:0:0:0:12:-6,
2:5:14749,39:7:-1,18:9:27558,25:12:39009,
29:12:-1,17:13:42121,18:13:42526,10:14:-1,
18:14:47626,26:14:-2:0:0:0:0:8:-6,30:14:50837,
1:15:-1,5:15:51900,9:15:-1,21:15:53655,
27:16:61466,42:19:-1,43:19:83153,43:20:-1,
1:21:90585,11:22:-1,17:23:105394,22:24:-1,
23:24:111581,0:27:-1,25:27:125621,28:28:-1,
3:29:140154


With the following -mmb for iFrame 92 by mhenderson
NEW MMB
X:54419:00,X:53655:00,X:140154:0C,0:0:550,
14:0:-2:0:0:0:0:12:-6,25:0:2044,27:0:-2:0:0:0:0:12:-6,
0:1:3127,3:1:-2:0:0:0:0:12:-6,4:1:3405,
5:1:-2:0:0:0:0:12:-6,7:1:-2:0:0:0:0:12:-6,
14:1:-2:0:0:0:0:12:-6,24:1:-2:0:0:0:0:12:-6,25:1:5358,
31:1:5258,42:1:-2:0:0:0:0:12:-6,0:2:-2:0:0:0:0:12:-6,
26:2:-2:0:0:0:0:12:-6,0:3:8701,21:3:-2:0:0:0:0:12:-6,
22:3:-2:0:0:0:0:12:-6,0:4:-2:0:0:0:0:12:-6,
12:4:11713:0:0:0:0:1:0:63,22:4:-2:0:0:0:0:12:-6,
23:4:12333,24:4:-2:0:0:0:0:12:-6,30:4:14749,2:5:14749,
39:7:-1,18:9:27558,25:12:39009,29:12:-1,17:13:42121,
18:13:42526,10:14:-1,18:14:47626,22:14:49012,
23:14:49834,25:14:-2:0:0:0:0:-6:-6,
26:14:-2:0:0:0:0:5:-6,28:14:-2,30:14:50837,
1:15:-1,5:15:51900,9:15:-1,19:15:52863,21:15:53655,
16:16:58104,17:16:58324:-8:-8:-8:-5:0:0,
27:16:61466,42:19:-1,43:19:83153,43:20:-1,1:21:90585,
11:22:-1,17:23:105394,22:24:-1,23:24:111581,0:27:-1,
25:27:125621,0:28:-2:-90:-90:-90:-90:0:0,3:29:140154,
10:29:-2:-90:-90:-90:-90:0:0

These changes bring in more detail in the top five rows
and some detail around the legs. Also clean up the
clock bar at the bottom.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/23/2014 02:33 am
Thank you for all the updates everyone!

Remember to post your improvements on the wiki every time, in order to keep the latest versions available for the ones that are working in your same frames!

It's amazing to see we are running 24/7: is really a pleasure to wake up in the morning, and be surprise on the updates from the last 6 or 7 hours!

Keep up with the great work! we are almost there!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mhenderson on 05/23/2014 02:35 am
iFrame 92 image attached
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Lee Jay on 05/23/2014 02:47 am
Amazing work, all.

A suggestion, if I might.

Once in a while, when you are producing videos or animations after repair, create a side-by-side with the original frames.  It will drive home the point of the work done to those watching much less casually than the rest of us.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mhenderson on 05/23/2014 04:58 am
iFrame 92 before and current
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: meadows.st on 05/23/2014 05:16 am
Wow you guys are making awesome progress!!!!

I know we are trying to replace bad mmb with good mmb. (1) So would it be useful to document the XY and POS of good mmb's? I am thinking if we can map out what we know to be good, figuring out what to replace the bad with may be easier.
Welcome to the forum PaulNY!

From what I understand your idea is to catalog good macro blocks in all the frames. Good as in decided by human beings not by some algorithm I assume.

This may be of some use. I'm not sure. (2) We sort of implicitly rely on our vision to detect good vs bad blocks. We also have a nice feature in the online tool that highlights all error-macroblocks in a frame. There are still plenty of non-error-blocks left that look "bad", but we don't know if they are easily repairable or are really bad. (3) We usually judge their "badness" by the destruction they cause to other blocks.

So depending on what you think 1. what the criteria of "bad" is, or even multiple classes of bad and 2. how you think we can deal with those kinds of "badness", your idea might have some value.

Maybe you can try to "fertilize" your ideas a little bit so it becomes more clear if we can use it. :)

Regards,

arnezami

PS. We refer to mmb as "modify macro block". I believe when you say mmb you actually ment to say "macro block".

Unfortunately work has kept me away for a few days, but let me try to explain my idea more.

Lets say I look at I frame 8. MB 1 (0:0) looks right and is blue water. If we can document the position of this MB in the frame, then lets say some MB (x,y) looks bad and we think it's water we could try replacing it with a known value of a good water MB instead of trying to guess. If we can document all the know good and bad MB values in the I frames, then we can then target our repairs more. I feel like when trying to make a repair the majority of the time I spend is guessing MB pos values or the pos to try a invert / flip.

Maybe I am missing something, but if this is where other people spend a lot of time to it would be great if we could create a way to collaborate on this more and optimize the process.

If we can't document known good or bad positions, maybe we could create a DB of recommended or non recommended pos values for each MB to try and optimize the process.  (4) For example if I try to fix / replace MB (X:Y) and try these 400 values that don't work. If I can document that I can save the next guy from trying the same thing.

Added bold/underlined numbers in quoted section to clarify the thoughts I am trying to express. (If this has already been discussed, please let me know and I will remove this post as I am out of my league here and have not had the time to really do my homework and create any proofs of concept yet; I am hoping that these thoughts might trigger a brilliant idea in someone else (even if I am completely off base with this) - I have attempted to follow the thread closely but I may have missed (or forgotten) some posts.)

1) Agreed (as stated before) but the issue is a) the inability to identify a "good" change from a "bad" change and b) the cascading effect of any change.

2) By saying "we ... rely on our vision ... to detect good/bad" I think arnezami means that the ffmpeg-extracted, resulting PNG (or GIF) file is inspected manually after every change to determine the degree of "correctness" of the change. Some changes will make discernible features appear (legs, lines, white-capped waves, the blessed dirt spots, etc) while other changes may remove/alter/"fix" obvious erroneous blocks in the resulting PNG (and thus in what will be the final video).

3) re: Judging "badness" by destruction to other blocks - I understand this to mean that newly resultant cascading failures will become apparent if a "really bad" change is made. I am thinking of a brute-force method (I am sure there are more elegant approaches) initially wherein we could create a reference, fixed integer size per pixel bitmap (or any another format that is a direct representation of each pixel and is essentially "fully" decoded and uncompressed) with a fixed pixel representation so we can compare the resulting file numerically and calculate the degree and number of changes that are created by a changed mb.  We know (reasonably well) that certain colors are not likely to be found in the resulting image and these colors can be detected and compared to a reference. My hypothesis is that we could then script a large number of mmb's and bitflips and numerically compare the resulting bitmaps to narrow down the images that should be inspected by a human.  The analogy I am thinking about is a brute force attack on a cipher (frame by frame) that uses a different key for each "character" and each subsequent "character" is influenced by the key used in each previous "character".  Our advantage (and I think theshadow27 and arnezami and mvpel (and others?) may have already been thinking along these lines) is that we know the ranges of colors that will be highly unlikely and we know large chunks of the clear-text reasonably well.  At the bitmap level, we can numerically calculate a delta from one mmb/bitflip to a reference bitmap.  Obviously the number of calculations increase exponentially due to the cascading errors and even the "known" parts of each image (other than the opaque dirt spots) experience changes from condensation, smoke, etc.

4) From my discussion in 3) above, what I am ultimately thinking about is for each mb(x,y) that looks "bad" and we try a change, that change results in a delta to the original bitmap (i.e. at the discrete pixel level) and a "bad" change would create lots of invalid colors and possibly lots of additional failed cascades and the "degree of goodness/badness" could be described as the number of improbable pixel values (or even the number of changed pixels) in the resulting bitmap. The problem with this approach is that a "good" change may also create a lot of changes until we get down to the final mb that cannot influence any other mb.  I would think that we could use the "prevent inheritance" flag for each mb to then calculate the resulting good/bad value of the change and store this info in a database for each mb (x,y)- obviously if any change results in a ffmpeg error, then the change is (likely?) bad, the second filter would be the comparison to the starting (or reference "best") bitmap.

Added: If this approach is meaningful then we could employ a large(r) army of human eyes to review the resulting candidate images to identify good/better/best/epic-fail images.  If we could script the approach to first target the mb's that result in ffmpeg errors then set the script to attack the remaining "bad but not technically error" mb's and filter out any changes that create new errors or result in improbable colors then hopefully we can significantly reduce the number of human views required and can split up the work to avoid duplication. (obviously this work would require some significant cloud storage (each uncompressed bitmap of 704x520 pixels would result in a >1.5MB filesize) so each frame might require a LOT of storage.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Lourens on 05/23/2014 07:36 am
We're thinking along similar lines I think. The real question is the objective function. The best reference bitmap would of course be the actual frame, which we don't have. Perhaps comparing colour histograms would work (using a good part that we have for reference, and perhaps the earth mover's distance to compare), and I've also been thinking about using the contrast along macroblock edges. We also have some known edges, so another idea is to run an edge detect on the frame and then compute the inner product with a separate image of known edges. On the other hand, there aren't that many edges, and when the legs are moving, we don't know where they are exactly, so maybe that doesn't work so well...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/23/2014 08:17 am
Hi guys,

Since this is the subject I've been spending a lot of time on thinking I will react too. But I havent got a lot of time right now. So I will probably be more elaborate later.

What this whole problem boils down to is three phases (per frame):

1) Extracting: Correcting the predictable streamheaders and I/P-frame headers. This way we (or actually the decoder) can extract all the video data without missing packets.

2) Positioning: Finding all the macroblock-data (that is: finding all bitpositions where each mb-data starts) and then assigning the right macroblocks to the these starting bitpositions.

3) Repairing: Correcting left-over DC-values errors either by bit flipping or by changing the DC-values after decoding. This should implcitly also fix "bad inheritance": where a macro inherents its DC base value from the wrong macroblock (that is: left or top).

I believe that for 2 there is an algorithm that will find all likely correct starting positions of macroblocks, but it will not be able correctly assign macroblocks to these bitpositions. I believe it would be helpful if we auto-generate all the likely starting positions for each frame and make these available as a static file to the online editor. This would allow the user to choose from good starting positions.

As for 3, I think there may be a way to correct DC-values somewhat automatically. However bitflipping is a better way when possible of course. Massively correcting DC-values should only be done if 1 and 2 has been done for a frame. If this works as I hope this will really give it a "finished look".

There is also the issue with the MV's for P-frames. I haven't looked into this too much.

I will go into more detail when I have more time ;).

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: DragonRider on 05/23/2014 12:06 pm
 I have to say, amazing work in this thread, I really did not think it was possible to recover a video like this.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Sohl on 05/23/2014 12:59 pm
I have to say, amazing work in this thread, I really did not think it was possible to recover a video like this.

Yes, the results of the work here greatly eclipse my expectations! Great job everyone.  :D I hope to join in in some way within a week or two.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mhenderson on 05/23/2014 02:11 pm
@arnezami -

In the water, the typical "good" macroblock has a length of 45-100. It isn't too hard to find chains of 4 to 10 valid macroblocks. No error, length of 45-100.

In iFrame 92, for example, the first half dozen rows have an average length of 72. The average gives an estimated starting POS for each Col:Row block.

Fill the gaps between these short chains of good blocks with "estimated water color" Col:Row:-2:0:0:0:0:12:-6. (IMHO, this color should be standardized across the iFrames for consistency / smooth video.)

Badabing.

Similar logic applies to the rocket body.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/23/2014 02:43 pm
I was thinking about the movie, could someone with some artistic talent (not my case :P) make 1-2 title images, and maybe also some credits images at the end (for example a few before/after images, printscreens of the wiki, web application, ...)? It should have the same format as the images, i.e. 704x480.

Edit: a first version of a movie with all recovered frames (the other are not used)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: meadows.st on 05/23/2014 02:52 pm
Hi guys,

Since this is the subject I've been spending a lot of time on thinking I will react too. But I havent got a lot of time right now. So I will probably be more elaborate later.

What this whole problem boils down to is three phases (per frame):

1) Extracting: Correcting the predictable streamheaders and I/P-frame headers. This way we (or actually the decoder) can extract all the video data without missing packets.

2) Positioning: Finding all the macroblock-data (that is: finding all bitpositions where each mb-data starts) and then assigning the right macroblocks to the these starting bitpositions.

3) Repairing: Correcting left-over DC-values errors either by bit flipping or by changing the DC-values after decoding. This should implcitly also fix "bad inheritance": where a macro inherents its DC base value from the wrong macroblock (that is: left or top).

I believe that for 2 there is an algorithm that will find all likely correct starting positions of macroblocks, but it will not be able correctly assign macroblocks to these bitpositions. I believe it would be helpful if we auto-generate all the likely starting positions for each frame and make these available as a static file to the online editor. This would allow the user to choose from good starting positions.

As for 3, I think there may be a way to correct DC-values somewhat automatically. However bitflipping is a better way when possible of course. Massively correcting DC-values should only be done if 1 and 2 has been done for a frame. If this works as I hope this will really give it a "finished look".

There is also the issue with the MV's for P-frames. I haven't looked into this too much.

I will go into more detail when I have more time ;).

Regards,

arnezami

You said it much better than I did. I concur, fix the data 'mechanically' first for the best starting state, then devise algorithms to narrow down the probable solution space.  I am eager to read your more detailed thoughts.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/23/2014 02:59 pm
I was thinking about the movie, could someone with some artistic talent (not my case :P) make 1-2 title images, and maybe also some credits images at the end (for example a few before/after images, printscreens of the wiki, web application, ...)? It should have the same format as the images, i.e. 704x480.

I nominate Def Leppard's "Rocket" (https://www.youtube.com/watch?v=VWg_g7-sIn0) for the soundtrack. And definitely "Landing" by Moby (https://www.youtube.com/watch?v=vD7rqm9OSFs). Start with a few flashed images of the Wright Brothers and Mercury-Redstone, Gemini, Saturn I... some sexy Falcon 9 rollout and countdown shots, the launch, a loving slo-mo of the huge splash of water that gifted us the immensely useful spots on the camera lens and body... then some retro burn video or animations to set the context, and then the landing... ;D It'll be sweet.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: inverted_strawberry on 05/23/2014 03:41 pm
I posted this on reddit (http://www.reddit.com/r/spacex/comments/24bsn2/first_stage_landing_video/) and Destructor1701 suggested to re-post this here. I don't think I will be checking back here a lot so if you have any questions then please get in touch over reddit.

So I may be late to the party but I gave it a serious try. I managed to recover a good number of p-frames, and a cleaned up a few i-frames better than in the original repair1.ts file. Here we go: http://youtu.be/_Fha2q5JF0M extracted i-frames are here: https://imgur.com/a/Fu5by
As you see I was unable to recover most of the earlier frames, so I tried grafting i-frame 6 on top of 3,4, and 5. This means the content of this video may not be what the camera actually recorded but could help in making sense of the motion (p-frames). http://youtu.be/MFPRSrcDKuc
Edit: Here's the .ts in case anyone is interested: https://www.dropbox.com/s/37h85hxxqlgv79z/map1.ts
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mhenderson on 05/23/2014 04:07 pm
scriptmonkey420 on Reddit has posted SwissCheese's latest gif dissected to individual frames here:
http://imgur.com/a/Xv8ln#0 (http://imgur.com/a/Xv8ln#0)

Really cool.

Images 40-43 appear to be out of sequence. I think the order should be 41, 40, 43, 42 (These correspond to frames 206 - 211 with gap). Were they in this order in the GIF?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/23/2014 04:16 pm
Hi all
ive added a new option to the custom ffmpeg on github,
-debug mb_pos_bruteforce

Could you make a new /github.com/mlindner/ffmpeg-spacex/archive/spacexdebug1.tar.gz with these changes. I don't have much hope for this, but frame 72 is causing desperation to set in. I'm grabbing at straws here. Thinking of going down to sub-macroblock level.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: michaelni on 05/23/2014 04:35 pm
I think what would also be very useful is to able to tell ffmpeg only to decode either luminosity, chrom1 or chrom2. So either only the first 4 dc's are decoded or only the fifth or only the sixth.

Then we can add 3 checkboxes in the online version. That way you can much more clearly see where things go wrong colorwise or intensity wise. Without our eyes being districated by either the intensity or (one of) the two colors.

you can achive that by removing chroma or luma after decoding using for example:
-vf mp=eq2=1:0:0:1
-vf mp=eq2=1:1:0:0
or
-vf lutyuv=y=128:u=128
-vf lutyuv=y=128:v=128
-vf lutyuv=v=128:u=128
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: morningdew76 on 05/23/2014 07:25 pm
I've been attempting to tune this bash script, with an eye towards performance.

This script is searching for two bits between 8800 and 9065 on frame 52.  It's a lot of combinations, and I am getting roughly 600 frames per minute.

Quote
for i in {8800..9064} ; do for j in {8801..9065} ; do ffmpeg.exe -debug mb_pos_size -s:0 704:480 -mmb X:76768:80,X:22120:80,X:45038:80,X:42196:80,X:66234:80,X:50298:80,X:$i:80,X:$j:80, 0:7:-2:-10:-10:-10:-10:8:-5,2:7:15506,9:7:-1,15:7:16391,17:7:-1,28:7:16704, 35:7:-2:-10:-10:-10:-10:8:-5,29:9:21074,34:9:-1,39:9:21626,5:28:-1::63,10:28:80196 -i iframe52.mpg4-img 2> output-$i-$j.txt -f image2 /dev/null -y ; done ; done

The output images have been entirely disabled, and the directive to ignore errors has been removed.  I am not 100% sure that the latter is "safe" but it does significantly increase the speed of the process.

Note that there are spaces in the mmb string that would need to be removed to run this.

After the run is done, I first do a grep for 42:03:9061 , then a grep for 00:29:81738 .  The second one is just in case the first "known good" position ends up changing due to correctly flipped bits.  Edit: This second test is probably worthless.

If anybody has suggestions for how to speed this up, I am all ears!  The search space can likely be reduced but I'm really unsure of what would be advisable.

Just sticking to two bits right now... going for three in a search space this size would take something like 21 days at a rate of 600 tries per minute.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: maximlevitsky on 05/23/2014 08:04 pm
I've been attempting to tune this bash script, with an eye towards performance.

This script is searching for two bits between 8800 and 9065 on frame 52.  It's a lot of combinations, and I am getting roughly 600 frames per minute.

Quote
for i in {8800..9064} ; do for j in {8801..9065} ; do ffmpeg.exe -debug mb_pos_size -s:0 704:480 -mmb X:76768:80,X:22120:80,X:45038:80,X:42196:80,X:66234:80,X:50298:80,X:$i:80,X:$j:80, 0:7:-2:-10:-10:-10:-10:8:-5,2:7:15506,9:7:-1,15:7:16391,17:7:-1,28:7:16704, 35:7:-2:-10:-10:-10:-10:8:-5,29:9:21074,34:9:-1,39:9:21626,5:28:-1::63,10:28:80196 -i iframe52.mpg4-img 2> output-$i-$j.txt -f image2 /dev/null -y ; done ; done

The output images have been entirely disabled, and the directive to ignore errors has been removed.  I am not 100% sure that the latter is "safe" but it does significantly increase the speed of the process.

Note that there are spaces in the mmb string that would need to be removed to run this.

After the run is done, I first do a grep for 42:03:9061 , then a grep for 00:29:81738 .  The second one is just in case the first "known good" position ends up changing due to correctly flipped bits.  Edit: This second test is probably worthless.

If anybody has suggestions for how to speed this up, I am all ears!  The search space can likely be reduced but I'm really unsure of what would be advisable.

Just sticking to two bits right now... going for three in a search space this size would take something like 21 days at a rate of 600 tries per minute.


you could pipe output of the for loop to grep so you don't create temp files. dunno how much will this save in terms of performance.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Lourens on 05/23/2014 08:15 pm
For what it's worth, I'm running at about 750 calls per minute, with all output to a RAM disk and subsequent call to a separate program to analyse the resulting image. This is on GNU/Linux on a three year old laptop, with ignore errors enabled. Putting intermediate output on a RAM disk may help a bit, since disk writes are slow. Beyond that I think it's probably mostly the overhead of starting up FFmpeg all the time.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: morningdew76 on 05/23/2014 08:23 pm
you could pipe output of the for loop to grep so you don't create temp files. dunno how much will this save in terms of performance.

That's a great idea, I'll try to figure that out (my Linux skills.. not so great).

I also realized that the nested loops weren't running efficiently-

Quote
for i in {8800..9009} ; do for j in $(eval echo "{$(($i+1))..9010}") ; do ffmpeg.exe -debug mb_pos_size -s:0 704:480 -mmb X:76768:80,X:22120:80,X:45038:80,X:42196:80,X:66234:80,X:50298:80,X:$i:80,X:$j:80, 0:7:-2:-10:-10:-10:-10:8:-5,2:7:15506,9:7:-1,15:7:16391,17:7:-1,28:7:16704, 35:7:-2:-10:-10:-10:-10:8:-5,29:9:21074,34:9:-1,39:9:21626,5:28:-1::63,10:28:80196 -i iframe52.mpg4-img 2> output-$i-$j.txt -f image2 /dev/null -y ; done ; done

This change halves the number of combinations to search.

For what it's worth, I'm running at about 750 calls per minute, with all output to a RAM disk and subsequent call to a separate program to analyse the resulting image. This is on GNU/Linux on a three year old laptop, with ignore errors enabled. Putting intermediate output on a RAM disk may help a bit, since disk writes are slow. Beyond that I think it's probably mostly the overhead of starting up FFmpeg all the time.

I had thought about using a RAM disk, and I'm probably losing a lot to Windows/MinGW overhead.  Too bad I can't run Linux on this machine.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jamsta on 05/23/2014 08:24 pm
From L2:

<snip>

You've also got a fan in Elon. He's been full of praise for the video work.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/23/2014 08:31 pm
I was thinking about the movie, could someone with some artistic talent (not my case :P) make 1-2 title images, and maybe also some credits images at the end (for example a few before/after images, printscreens of the wiki, web application, ...)? It should have the same format as the images, i.e. 704x480.

Edit: a first version of a movie with all recovered frames (the other are not used)


When you compare that to the initial video provided, it's absolutely stunning. Amazing work guys.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/23/2014 08:33 pm
Legs for days....

http://spacexlanding.wikispaces.com/Frames+72-91
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: aero on 05/23/2014 09:12 pm
Wonder if the individual movie or gif frames should be trademarked? "SpaceX?"
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: AncientU on 05/23/2014 09:35 pm
Legs for days....

http://spacexlanding.wikispaces.com/Frames+72-91
Can you animate these frames?  Legs are starting to open frames 85-90...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Asmegin on 05/23/2014 09:49 pm
Is there a way to add a section for each Frameset page on the wiki that automatically creates a 'Entire Segment' input from the data entered?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: princess on 05/23/2014 10:33 pm
Newbie here - I have experience working with MPEG-TS streams and I've written a few custom tools that can fix corruption at the transport stream layer. Is it worth me working on the split TS files that shanuson created from the raw edit8.ts?

I've run his files through the tools, and they fix a lot of stuff like bad PIDs, bad program continuity counters, wrong adaptation field lengths, things like that. The start positions of various badly-corrupted P-frames become visible, but they're still quite damaged.

I guess I'm asking, would the recovery effort benefit from this?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: dorkmo on 05/23/2014 10:39 pm
heyo,
seems like theres some sort of small flame event on frames 74 and 75. is from that the engines or perhaps do the legs have some sort of pyrotechnic release system? thought it was odd since it was so brief.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: AncientU on 05/23/2014 10:51 pm
heyo,
seems like theres some sort of small flame event on frames 74 and 75. is from that the engines or perhaps do the legs have some sort of pyrotechnic release system? thought it was odd since it was so brief.
Post link or images?
Thanks.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: deruch on 05/23/2014 10:57 pm
Newbie here - I have experience working with MPEG-TS streams and I've written a few custom tools that can fix corruption at the transport stream layer. Is it worth me working on the split TS files that shanuson created from the raw edit8.ts?

I've run his files through the tools, and they fix a lot of stuff like bad PIDs, bad program continuity counters, wrong adaptation field lengths, things like that. The start positions of various badly-corrupted P-frames become visible, but they're still quite damaged.

I guess I'm asking, would the recovery effort benefit from this?

Excellent, I've been trying to clean them up some.  But I'm totally in the dark.  I've done what I can for parts _11 and _12.  But I'm only really playing with a hex editor.  So I don't even know if what I'm doing is helpful, beyond the fact that it's cleaned up a bunch of the continuity errors.  It did return a few additional frames, but I can't see them (no computer Fu, i'm capable of following exact directions but that's it).  I've attached my attempts.  Someone let me know if they're any improvement or if I'm just wasting my time.  Not that that will necessarily keep me from goofing around with them.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: 411rocket on 05/23/2014 11:36 pm
Newbie here - I have experience working with MPEG-TS streams and I've written a few custom tools that can fix corruption at the transport stream layer. Is it worth me working on the split TS files that shanuson created from the raw edit8.ts?

I've run his files through the tools, and they fix a lot of stuff like bad PIDs, bad program continuity counters, wrong adaptation field lengths, things like that. The start positions of various badly-corrupted P-frames become visible, but they're still quite damaged.

I guess I'm asking, would the recovery effort benefit from this?

In my way of thinking, anyone with skills that can help the effort, should be encouraged to do so. Some, seem to enjoy the challenge of this task, but remember to take breaks, to prevent burnout (or showing up at work in a daze). As life still happens, regardless of this fantastic group effort. 
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: princess on 05/24/2014 12:38 am
Excellent, I've been trying to clean them up some.  But I'm totally in the dark.  I've done what I can for parts _11 and _12.  But I'm only really playing with a hex editor.  So I don't even know if what I'm doing is helpful, beyond the fact that it's cleaned up a bunch of the continuity errors.  It did return a few additional frames, but I can't see them (no computer Fu, i'm capable of following exact directions but that's it).  I've attached my attempts.  Someone let me know if they're any improvement or if I'm just wasting my time.  Not that that will necessarily keep me from goofing around with them.

That's great, thanks for posting those. I've had a quick look at part 11 and what you've done to it - I've attached the output of my TS fix tool when run against shanuson's part 11.

The good part is that it's managed to fix a few more TS-level issues! It deliberately doesn't change the contents of the data stream, so it hasn't fixed some of the MPEG4 headers that you'd got. I'll see if I can update the tool so it can incorporate your MPEG4 changes.

One thing I did notice is that sometimes you removed the adaptation field from an 0x03e8 packet when it looks too long, but as far as I can see it is sometimes legitimately long. For example, packet 87 at file offset 0x00003fe4 is marked as having a 69-byte adaptation field. It's human nature to think "whoa, that can't be right" and zero the AF, but if you do this and then look at the data it has a huge run of 0xff bytes before the data starts (from deruch's edit):

Packet 87 at 0x00003fe4: PID 0x03e8 Pay3:4500ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff

I believe the correct thing to do in this case is to leave the AF at 69 bytes (from my edit):

Packet 87 at 0x00003fe4: PID 0x03e8 AF[69] Pay3:ff01c0d80f09017b09418cde36def8f3fb90982fa7bf8dbee58fbdef1bcf7c6d

However, in other cases you can see that a packet header has become corrupted so that it adds a freaky AF, sometimes one that's longer than the 188-byte TS packet itself! So it is hard to automate this kind of checking, it has to be done manually. Just be super-careful when removing the AF from a packet, as you might introduce a huge run of padding into the MPEG4 datastream.

If we sort out the lower-level TS problems, your MPEG4 header fixes will be very useful, so we'll probably need to work together on this one!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Req on 05/24/2014 01:13 am
I've been attempting to tune this bash script, with an eye towards performance.

This script is searching for two bits between 8800 and 9065 on frame 52.  It's a lot of combinations, and I am getting roughly 600 frames per minute.

Quote
for i in {8800..9064} ; do for j in {8801..9065} ; do ffmpeg.exe -debug mb_pos_size -s:0 704:480 -mmb X:76768:80,X:22120:80,X:45038:80,X:42196:80,X:66234:80,X:50298:80,X:$i:80,X:$j:80, 0:7:-2:-10:-10:-10:-10:8:-5,2:7:15506,9:7:-1,15:7:16391,17:7:-1,28:7:16704, 35:7:-2:-10:-10:-10:-10:8:-5,29:9:21074,34:9:-1,39:9:21626,5:28:-1::63,10:28:80196 -i iframe52.mpg4-img 2> output-$i-$j.txt -f image2 /dev/null -y ; done ; done

The output images have been entirely disabled, and the directive to ignore errors has been removed.  I am not 100% sure that the latter is "safe" but it does significantly increase the speed of the process.

Note that there are spaces in the mmb string that would need to be removed to run this.

After the run is done, I first do a grep for 42:03:9061 , then a grep for 00:29:81738 .  The second one is just in case the first "known good" position ends up changing due to correctly flipped bits.  Edit: This second test is probably worthless.

If anybody has suggestions for how to speed this up, I am all ears!  The search space can likely be reduced but I'm really unsure of what would be advisable.

Just sticking to two bits right now... going for three in a search space this size would take something like 21 days at a rate of 600 tries per minute.

Farm it out to multiple cores...

A quick paste(and edited a bit for your case) from something I made for a friend to run N rsyncs at once - in this case it's looking for "ffmpeg" in the processlist to track threads.

# use command-line argument for threads if specified
if [ "$1" == "" ]
then
  threads=3
else
  threads=$1
fi

..<snip>..

for i in {8800..9064}
do
  # check to see if enough threads are running and wait if so
  while [ `ps aux | grep ffmpeg | grep -v grep | wc -l` -ge $threads ]
  do
    sleep .1
  done

  # spawn a worker on the current offset
  /root/process_one $i &
  sleep .1
done

# don't exit back to the prompt until all of the workers are finished
while [ `ps aux | grep ffmpeg | grep -v grep | wc -l` -gt 0 ]
do
  sleep .1
done


Maybe farm out the "for j in" part this way.  Use however many logical cores (this includes hyper-threading) that your system has +1 (so for a 4-core i7 you'd want to use 9).  You'll have to eliminate your temp file per an earlier recommendation or use $i for it's filename if you want to run several at once.  Also, yes, as previously mentioned by another poster, if you're going to be using a temp file you want to make sure that it's on a ramfs mount with atime disabled(if applicable in your distro).

Even if ffmpeg is already using multiple cores, you are likely to see at least double your current performance.  If ffmpeg is not using multiple cores, you should see a fairly linear increase relative to number of cores used.  If ffmpeg is already using multiple cores, you should probably run the script with just 2 or 3 threads, not cores+1.

This should work well since you have a decent amount of work to farm out per worker if the farmed out work is the "for j in" part.  If you needed to farm out tons of really short workers you'd want to use memcached listening on a socket to track the number of workers running to eliminate the time of the "ps" call and invoke them from compiled code like C++ or hiphop'd php or something instead of a shell script.

Also, I'm not sure what the output of this looks like, but you should probably append whatever useful output this stuff generates into a logfile and run the main script in a screen, since it may take a while, and you don't want it aborting just because you got disconnected from your SSH session.  You want the logfile because it's a pain in the ass to scroll up in a screen.  This also allows you to detach the screen and just tail -f the logfile.  Note that >> appends stdout and 2>> appends errout, so you'll need to use both if you also want error output in the log.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: deruch on 05/24/2014 01:15 am
Good catch, Princess.  There's probably a couple of other places where I made the same mistake.   :'( 

Have you been to the http://spacexlanding.wikispaces.com (http://spacexlanding.wikispaces.com) yet?  There's a section there, towards the bottom of the active work that deals with the headers and .ts files.   
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Req on 05/24/2014 02:24 am
Actually, it occurs to me that the straight up "ps" approach to tracking workers may have problems with the above script, because the workers are constantly starting and stopping ffmpeg, which ps is looking for.

Another easy trick to track spawned workers is just to use PHP.

To check workers you can use:
while [ `ps aux | grep process_one\.php | grep -v grep | wc -l` -ge $threads ]

And to spawn workers you can use:
# spawn a worker(via php passthrough for easy thread tracking) on the current offset
php /root/process_one.php $i &

process_one.php looks like this:

<?

function parseArgs($argv)
{
        array_shift($argv);
        $out = array();
        $i = 0;
        foreach ($argv as $arg)
        {
                $out[$i] = $arg;
                $i = $i + 1;
        }
        return $out;
}

// parse arguments from commandline
$args = parseArgs($argv);
if ($args[0])
{
        $offset = $args[0];
}

exec('/root/process_one ' . $offset);

?>

Again, the overhead php adds to spawning a process in this case is negligible because the workers will be running for a good while each and you're spawning thousands, not millions or billions.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/24/2014 02:40 am
Try using HTCondor - when you install it on your Linux box after downloading from http://research.cs.wisc.edu/htcondor/ as an RPM, it will set itself up to manage jobs on the local machine. You submit the jobs and it will feed them into however many cores you have as needed.

Edit: Sorry, typo in the URL.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Req on 05/24/2014 02:50 am
Try using HTCondor - when you install it on your Linux box after downloading from http://research.cs.uwisc.edu/htcondor/ as an RPM, it will set itself up to manage jobs on the local machine. You submit the jobs and it will feed them into however many cores you have as needed.

Link is broken.

I can tell you right now though, it'll be quite easier to make 3 10-20 line files in this case.

My background is hosting extremely large scale sites and applications(100,000+ concurrent users), and I also had a sole source contract with the state department for several years to aggregate and analyze "sources of interest" to the tune of >150GB/day(of text) including twitter, facebook, forums, etc for sentiment analysis/event prediction and notification/outlier detection and characterization/personal network visualization and characterization/etc to provide intelligence for select embassies in the middle east.  To complicate matters, it had to "understand" all of the various dialects of arabic/farsi/urdu/kurdish/turkish/etc along with the standard english/german/spanish/etc obviously.  I had to develop a lexicon system that had native speakers characterizing words in the order of their significance in the dataset.  I do have some sense of what it looks like when you want to spread a task across a few cores, or 14 cabinets full of servers.

Back on topic... Implementation isn't even really the crux of the matter, most of the thought and design goes into HOW you will stage the data and scale the task to operate efficiently given your dataset and task.  This particular task can be readily scaled just using the two loops he already uses.  Implementation on the shell won't have a noticeable impact on performance for this data set, and 30-60 lines of code which is mostly copy and paste at this point is pretty ridiculously easy implementation.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Untribium on 05/24/2014 04:16 am
Actually, it occurs to me that the straight up "ps" approach to tracking workers may have problems with the above script, because the workers are constantly starting and stopping ffmpeg, which ps is looking for.

Another easy trick to track spawned workers is just to use PHP.

To check workers you can use:
while [ `ps aux | grep process_one\.php | grep -v grep | wc -l` -ge $threads ]

And to spawn workers you can use:
# spawn a worker(via php passthrough for easy thread tracking) on the current offset
php /root/process_one.php $i &

process_one.php looks like this:

-snip-

Again, the overhead php adds to spawning a process in this case is negligible because the workers will be running for a good while each and you're spawning thousands, not millions or billions.

Looks like this: http://coldattic.info/shvedsky/pro/blogs/a-foo-walks-into-a-bar/posts/7 (http://coldattic.info/shvedsky/pro/blogs/a-foo-walks-into-a-bar/posts/7) might be an option as well. Generate the addresses using seq and then pipe it to xargs which then runs a fixed number of ffmpeg processes at a time. I'll check it out tomorrow, should get some sleep first :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Req on 05/24/2014 04:40 am
Actually, it occurs to me that the straight up "ps" approach to tracking workers may have problems with the above script, because the workers are constantly starting and stopping ffmpeg, which ps is looking for.

Another easy trick to track spawned workers is just to use PHP.

To check workers you can use:
while [ `ps aux | grep process_one\.php | grep -v grep | wc -l` -ge $threads ]

And to spawn workers you can use:
# spawn a worker(via php passthrough for easy thread tracking) on the current offset
php /root/process_one.php $i &

process_one.php looks like this:

-snip-

Again, the overhead php adds to spawning a process in this case is negligible because the workers will be running for a good while each and you're spawning thousands, not millions or billions.

Looks like this: http://coldattic.info/shvedsky/pro/blogs/a-foo-walks-into-a-bar/posts/7 (http://coldattic.info/shvedsky/pro/blogs/a-foo-walks-into-a-bar/posts/7) might be an option as well. Generate the addresses using seq and then pipe it to xargs which then runs a fixed number of ffmpeg processes at a time. I'll check it out tomorrow, should get some sleep first :)

I am interested to see the results.  I'm just so busy that my "recreation" basically involves this thread and a few other news sites at the moment.  Maybe you have found something that will be very useful to my endeavors in the future, at least on one level of scale!  Although I must admit, I do enjoy coding this type of thing.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/24/2014 04:53 am
I've used my bvi/wireshark approach to fix the transport stream headers on fixed_edit8_part_229.ts, and the "clean47" version is attached below. There were good-sized chunks of bad headers at the outset of the file, and I'm hoping that it can reveal something more of the top half of that frame, though I'm not particularly hopeful.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: princess on 05/24/2014 09:19 am
I've used my bvi/wireshark approach to fix the transport stream headers on fixed_edit8_part_229.ts, and the "clean47" version is attached below. There were good-sized chunks of bad headers at the outset of the file, and I'm hoping that it can reveal something more of the top half of that frame, though I'm not particularly hopeful.

You did really great! Those headers were totally mangled, but after your fixes your result file is pretty clean from a TS point of view.

I hope you don't mind but I've processed it a little more and fixed a couple of PCC discontinuities that were remaining. I've also removed the AF from any packets where either the AF is longer than the packet, or when the AF shows a wildly invalid PTS time. This is hopefully the correct thing to do, but please take a look and let me know what you think.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/24/2014 10:43 am
I've used my bvi/wireshark approach to fix the transport stream headers on fixed_edit8_part_229.ts, and the "clean47" version is attached below. There were good-sized chunks of bad headers at the outset of the file, and I'm hoping that it can reveal something more of the top half of that frame, though I'm not particularly hopeful.

You did really great! Those headers were totally mangled, but after your fixes your result file is pretty clean from a TS point of view.

I hope you don't mind but I've processed it a little more and fixed a couple of PCC discontinuities that were remaining. I've also removed the AF from any packets where either the AF is longer than the packet, or when the AF shows a wildly invalid PTS time. This is hopefully the correct thing to do, but please take a look and let me know what you think.

what do you mean be remoced the AF? not every AF has a PTS.


Yet the file looks really really clean, only at 2 points there was a 47 23 e8 1x instead of 4703e81x
I did not check if the AF length is correct. looked ok. In the end if the AF length is to small and some FFs get in the img file, that will be ok and only force a reassignment of one MB or so and be handled by the nice group that is fixed P-frames.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/24/2014 10:56 am
here is cleaned up part 7.

I will redo part 1 and 2 to also fixed all TS-headers.

The end result should be 15 fixed parts that only have to be put together to get a final ts file to which no tsfix etc has to be applied.

Cheers
Shanuson
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: princess on 05/24/2014 11:37 am
I've also removed the AF from any packets where either the AF is longer than the packet, or when the AF shows a wildly invalid PTS time. This is hopefully the correct thing to do, but please take a look and let me know what you think.

what do you mean be remoced the AF? not every AF has a PTS.

I'm working on the theory that if a packet has a "long" AF, and it indicates it has a PTS in the AF header, and the PTS is obviously completely wrong, then probably what's happened is that the "AF" bit has been flipped on a normal data packet that shouldn't contain an AF.

The aim is to try to recover more of the MPEG4 data. If an error flips the "AF present" bit, then the normal MPEG4 data gets interpreted as an AF length, and a number of bytes get removed from the MPEG4 data stream. If we can detect when this has happened, we can recover more of the MPEG4 data, which might result in more valid blocks popping into place in I-frames and P-frames.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: princess on 05/24/2014 11:57 am
here is cleaned up part 7.

The data packets all look good to me - great job! I've cleaned up the big runs of 0x1fff padding packets so that there are no more invalid PIDs or invalid packets in the stream. It won't make any difference to the MPEG4 data (as your edits have already done that), but it means that the file is clean at the TS level.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Oersted on 05/24/2014 12:19 pm
My background is hosting extremely large scale sites and applications(100,000+ concurrent users), and I also had a sole source contract with the state department for several years to aggregate and analyze "sources of interest" to the tune of >150GB/day(of text) including twitter, facebook, forums, etc for sentiment analysis/event prediction and notification/outlier detection and characterization/personal network visualization and characterization/etc to provide intelligence for select embassies in the middle east.  To complicate matters, it had to "understand" all of the various dialects of arabic/farsi/urdu/kurdish/turkish/etc along with the standard english/german/spanish/etc obviously.

Cool to have the NSA contributing to this video clean-up now...  ;-)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/24/2014 01:22 pm
here is cleaned up part 7.

The data packets all look good to me - great job! I've cleaned up the big runs of 0x1fff padding packets so that there are no more invalid PIDs or invalid packets in the stream. It won't make any difference to the MPEG4 data (as your edits have already done that), but it means that the file is clean at the TS level.


Yes, i clean it by hand so i don't worry much about the 1fff parts. only look it up for CC if there is some ambiguity if a TS packet is data or something else. 
There are 2 cases where the AF bit should be set for a packet with PID 3e8: The first packet of a frame, and the last packet of a frame. The frist contains a PTS and stuff. the last only stuffing FF's in front of the last data bytes. The frist one is always there, but some frames dont need stuffing at the end.

I'm half way through part1 fixing the TS packets. :D More tonight.
Cheers,
Shanuson
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: princess on 05/24/2014 01:43 pm
Here is my cleaned version of the part14 TS file. Comments gratefully received!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Lourens on 05/24/2014 03:34 pm
Question: when we use -mmb X:offset:pattern, are we flipping bits before or after the entropy decoding stage? Is this simply equivalent to flipping bits in the input file? Or is it essentially just a low-level way of editing the MB directly?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/24/2014 03:45 pm
Out of sheer desperation I decided to go deeper into the macroblock data to see if there's any way to find macroblock start positions, which would be a tremendous help for flipping bits. No amount of flipping can save a block if its start position is wrong. Unfortunately, but also unsurprisingly I did not find much signal in the bitstream to do this. But here are my results.

I took the first 623 macroblocks from frame 169. They are known to be good and haven't been tampered with. The sample is a bit small and mostly represents ocean data though. MB size range is 23 - 725 bits, mean 89, median 75. Data isn't random as expected, there is a slight tendency for ones to follow ones and zeros to follow zeros, but it's only like 2% over 50/50 so no use for flipping.

All macroblocks are type 3, not sure if this is due to small sample and rarity of type 4 macroblocks or they are not used in live encoding. This is about the only concrete result, but all it tells us is that a macroblock does not start with more than 2 zeros.

69% of macroblocks start with a one. 71% of macroblocks end with a one. The first part of macroblock is mcbpc, the ones used are (mcbpc:count):
001 : 37
011 : 32
010 : 123
1 : 431
This is followeb by ac_pred_flag (fixed length 1 bit), it's set to zero in 89% of macroblocks
Then comes cbpy. Here are the ones I saw with counts.
cbpy: 1010 : 43
cbpy: 000011 : 14
cbpy: 0011 : 4
cbpy: 0110 : 65
cbpy: 1000 : 45
cbpy: 1001 : 24
cbpy: 0111 : 20
cbpy: 1011 : 51
cbpy: 0100 : 20
cbpy: 00101 : 13
cbpy: 00011 : 11
cbpy: 0101 : 20
cbpy: 00100 : 6
cbpy: 000010 : 12
cbpy: 11 : 269
cbpy: 00010 : 6
It's set to 11 in 43% of macroblocks. The first bit of block data (1st bit of dct_dc_size_luminance code) is set to 1 in 65% of macroblocks. I didn't go further at this time. On the whole the best signature based on these data for macroblock start was thus 1|10111, where | is the macroblock border. 11% of macroblocks start with this pattern (about 1 in 9). So it could in principle provide a 64X reduction in search space at the cost of 89% of sensitivity. Not sure if that's worthwhile.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/24/2014 04:10 pm
Hi guys,

Now that we have come so far already and developed new techniques I wanted to express my thoughts about why I think we are succeeding and how we can best proceed.

Why are we succeeding?

I think there are two main reasons why we are succeeding in what was deemed to be impossible. The first is that we really went outside-the-box with our approach. The second is that our techniques are such that we can collaborate in our efforts. Let me elaborate on both.

Outside-the-box thinking:

The "conventional" way of thinking when repairing a video is (1) to try to repair all predictable header information and (2) try to repair each decoding error one at the time. The thinking is: since you cannot decode something if you haven't decoded everything before it (from the point of a key frame) you can only fix the error where the decoder halts.

Keep in mind that everything in mpeg-4 compressed video is highly dependent on each other: if decoding of a block fails you lose all data in three dimensions! You lose every block to the right, every row of blocks to the bottom and every block of every frame after this frame in this right-lower region. This is horrible. Especially when the error occurs near the top of the key frame. And when you can't repair an error you're stuck. This is the situation we started in. On the blog of Benoît Joossen (http://aeroquartet.com/wordpress/2014/05/07/spacex-falcon-first-stage-landing-pictures-are-from-us/) he describes what he did to extract parts of the two key frames and could not do much more than that, not without enormous effort.

This situation looks very similar to having an error in encrypted data where the encryption algorithm used the "Propagating cipher-block chaining (http://en.wikipedia.org/wiki/Block_cipher_mode_of_operation#Propagating_cipher-block_chaining_.28PCBC.29)"-mode. Meaning that if you have a single error in the data everything after that consists of seemingly random bits when you decrypt. However compression is very different from encryption: while the end-result of both encrypted and compressed data may look similar - they both look like random bits - the goal of encryption is to prevent decoding without a key while the goal of compression is to lower the amount of bits needed.

We can learn something very useful though from the area of encryption: if you want to break a cipher you ultimately want to recover it's "internal state". And if a cipher is weak this is actually possible. Since a compression algorithm can always be considered a weak cipher algorithm (it's not designed to be protecting it's "internal state") there might be a way to recover it's "internal state" after an error has occurred.

The mpeg-4 decoder algorithm has something which you could call an "internal state". It contains the current x and y of the macroblock, the bitposition of that block, the DC values from either the top or left block and some other values (like MV values). If you have these values at a certain x,y then you can decode a video from there.

What has happened is that after some time struggling with the problem and realizing the bad situation we were in, we had two major breakthroughs in this regard. One was that we told the decoder to ignore errors (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1192555#msg1192555). This allowed us to see what the decoder would do after the error, if it could somehow self-recover somewhat. And this appeared to be the case: parts of it's "internal state" were apparently restored because it was clearly producing recognizable parts in the picture (albeit in the wrong color, brightness and x,y-position). The clock (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1192775#msg1192775) was first found this way. :)

The other breakthrough was that we managed to change parts of the "internal state": by setting the bitposition of a macroblock the best effort by Benoît Joossen was essentially recreated (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1193177#msg1193177). But in our case there was no brute forcing needed (which is what Benoît Joossen used). We could simply set the bitposition in the "internal state" at the moment a certain macroblock was supposed to be decoded. Later the tool was released (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1193442#msg1193442) so that everybody could start trying mmb-commands.

Which brings us to the second reason why we are succeeding. ;)

Collaboration becomes possible:

There is no substitute for the human eye! :) And brain!

It is simply incredible how much time and effort people are putting into this major task! After the mmb-command became usable people started to play around with it and were quickly producing nice I-frame results. This lead to more added features and more use of those features, leading to more progress. Within a few days we had a wiki, a repository, command line repair tools and a really cool online repair tool! The amount of constant updates on the wiki is also impressive. Some people seem absolutely dedicated making this video work. Almost every day I start reading this thread and the forum simply won't let me "like" fast enough (120 seconds delay). Thats how good it is.

All this has personally inspired me to do even more. It was really nice to see the first P-frames coming out. I'm so proud of everyone here :). The latest video was also awesome. And the tweet from Elon made it all worth it for me.

Anyway. It turns out this problem was very fit to be crowd-sourced. NSF rocks!

How should we proceed?

Here are the three phases I mentioned earlier for reference:

Quote
What this whole problem boils down to is three phases (per frame):

1) Extracting: Correcting the predictable streamheaders and I/P-frame headers. This way we (or actually the decoder) can extract all the video data without missing packets.

2) Positioning: Finding all the macroblock-data (that is: finding all bitpositions where each mb-data starts) and then assigning the right macroblocks to the these starting bitpositions.

3) Repairing: Correcting left-over DC-values errors either by bit flipping or by changing the DC-values after decoding. This should implcitly also fix "bad inheritance": where a macro inherents its DC base value from the wrong macroblock (that is: left or top).


I think we are doing a very good job at 1 and 2 and we are also doing a bit of bitflipping (3). As I mentioned last time I've got some ideas how to make things even better. Here is a description of the tools I am thinking about:

Macroblock data finder:

In a lot of cases I see that there are still quite large swaths of blocks that are "turned off" (as in a -1 somewhere before it) because of some error that occurred earlier. It's not always easy to know where actual non-error data begins again. Especially the last row of the I-frames. And I think I have found out a way to find most missing data for macroblocks.

The algorithm goes like this. You iterate through every possible starting position (begin-bit to end-bit of the frame data) and record at which bitposition it begins and at which bitposition it ends (which is either an error or the end of the frame data) of each of these starting positions. You count the amount of bits and macroblocks are decoded for each of these starting positions until it hits an error (or the end of the frame data). So a really good part for I-frame 169 would start at bitposition 550: it decodes many (thousands!) of bits and many blocks from bit position 550 before it hits an error.

Now you sort all these bitsequences by the length they have (the total number of bits they decode from that bitposition). You take the first of that list because that is obviously the best sequence. Then you remove it from the list and also remove all sequences that either end or start within the bitrange of the "winning" sequence. They are all considered to be of less value. And you keep doing this: taking the first sequence, removing all sequences that lie in its coverage area. Until you have no sequences left.

I believe at the end you will have all the longest bitsequences that are available in the frame data. You "only" have to assign the right macroblocks to the starting bitpositions of these sequences in this set. And as said before I believe it would be helpful if we auto-generate all the likely starting positions for each frame and make these available as a static file to the online editor. This would allow the user to choose from good starting positions.

Brute force bit flipper inside the decoder:

Another idea is to do the bruteforce bitflipping inside the decoder. The main reason to do the brute forcing inside the code of the decoder is that it is likely to be much faster. Lets say you have four blocks of -1 (for example 15:12, 16:12, 17:12 and 18:12 in I-frame 150). To the left and right of it is are good macroblocks (14:12 at 45301 and 19:12 at 46628). You remove the four -1s so it will give an error again. Now you say to the decoder (in broteforce mode) try bitflipping 1-3 bits between the starting position of 15:12 (45472) and the starting position of 19:12 (46628) and each time you do this try to decode from 15:12 and see if you find 4 macroblocks and end at exactly at 19:12 (46628).

The advantage of this would be that there is a very clear end goal which is not likely to happen by chance. The decoder doesn't have to decode the entire frame over and over again (only 4 macroblocks decoding each time). And you could even reduce that to close to just one macroblock decoding each try. And if it found a candidate it would simply print it out.

I think that would be the fastest way of bitflipping. There may be smarter variations of it. It is however not trivial (at least not for me ;)) to "reset" an already decoded macroblock. @michaelni: do you know how to do this? Because that would be required.

DC value fixer:

Changing the DC values turns out to be very tricky. Not only can you get some unpredictable behaviour to the right and bottom of a block, it is also possible that everything gets messed up if a block-position is changed above and to the left. And it's also very tedious I think. Hard to get it perfect.

There may be a way to (semi-)automatically fix the DC values (here I am less certain though).  We are assuming all marcoblock have been given a position or are marked as -1. So no error blocks. And we have done all the bitflipping we could imagine. So this is basically something you do at the very end: "post-processing" if you will.

This is roughly how it would go. You disable color output. So only luminance comes out. You set 0,0 to -1 and you set 42:0 it designated bitposition. To the right of 42:0 is the last block of the row. It is affected by DC values of 42:0. If you see a horizontal "line" on the 8th pixel from above (as in: consistently lighter or darker pixels between 8 pixels and 9 pixels from above) then you know that the DC values of your block is off and change it. Then you go do the same with 41:0. But now you can check a longer (more reliable) horizontal line.

There are still some problems with that idea though. I will have to experiment and think about it longer.

Have fun! :)

Regards,

arenzami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/24/2014 04:16 pm
You did really great! Those headers were totally mangled, but after your fixes your result file is pretty clean from a TS point of view.

Just that one duplicate continuity counter - it was odd, the lead-in and tail-out of that section were quite clean but the numbers just didn't line up. Maybe that's a side effect of earlier alignment efforts.

Quote
I hope you don't mind but I've processed it a little more and fixed a couple of PCC discontinuities that were remaining. I've also removed the AF from any packets where either the AF is longer than the packet, or when the AF shows a wildly invalid PTS time. This is hopefully the correct thing to do, but please take a look and let me know what you think.

Why would I mind? :D

I've been putting some thought into parsing code to deal with the PES headers and the frame headers, but haven't had time to deal with it. Had a big deadline yesterday for a symposium presentation I'll be doing the second week of June and have a pile of essays to write for a course I'm taking - the work I did yesterday was to wind down after a 60-hour workweek.

I agree clearing the AF bit in packets where it's clear that its definitely not an adaptation field that follows is the right thing to do. As Shanuson points out there's a clear pattern to AFs which we can use to figure them out. There were sections where there were a really wild number of flipped bits, and figuring out whether or not an AF is valid wasn't going to be a "wind down" sort of thing for me to think about last night. There were one or two that showed up clearly in Wireshark that I fixed, but I'm glad you were able to take care of the rest of it.

I got the info I needed to extract the frames with ffmpeg, but I wound up with a completely blank iframe for some reason that I didn't have time to investigate, and I haven't downloaded MichaelNi's patched ffmpeg yet. It did deliver a full 20 frames, though.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/24/2014 04:39 pm
Question: when we use -mmb X:offset:pattern, are we flipping bits before or after the entropy decoding stage? Is this simply equivalent to flipping bits in the input file? Or is it essentially just a low-level way of editing the MB directly?
From what I understand it is equivalent to flipping the bits in the input file. It happens well before the decoding of any blocks start.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/24/2014 04:50 pm
Legs for days....

http://spacexlanding.wikispaces.com/Frames+72-91
Can you animate these frames?  Legs are starting to open frames 85-90...

If you paste the attached MMB into the http://spacex2.slapbet.com/ frame 72 entire-segment box, and then hit the play button at the right end of the slider bar below the image, you'll see the beginning of the legs opening - this should not be used for ongoing work, as I simply nulled out some of the intervening pframes which were making it difficult to see the leg movement and combined all the work up to this point.

It's VERY cool to see!!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: morningdew76 on 05/24/2014 05:19 pm
Brute force bit flipper inside the decoder:

Another idea is to do the bruteforce bitflipping inside the decoder. The main reason to do the brute forcing inside the code of the decoder is that it is likely to be much faster. Lets say you have four blocks of -1 (for example 15:12, 16:12, 17:12 and 18:12 in I-frame 150). To the left and right of it is are good macroblocks (14:12 at 45301 and 19:12 at 46628). You remove the four -1s so it will give an error again. Now you say to the decoder (in broteforce mode) try bitflipping 1-3 bits between the starting position of 15:12 (45472) and the starting position of 19:12 (46628) and each time you do this try to decode from 15:12 and see if you find 4 macroblocks and end at exactly at 19:12 (46628).

The bash script that I have been using now spits out roughly 900 combinations per minute in, using one thread.  This produces a rate sufficient for a 4-bit search over a 50-bit area in a matter four hours, and works pretty much exactly how you described.. The difference being that I am grepping the output of FFmpeg.  A few hundred candidates were produced, of which 40 or so look good.  My next step is to add generation of the following P-frame to cross-reference these 40 candidates.

The size of the search space follows "bit-space choose bit-size", which worked out to "50 choose 4" for the 4-bit run.

Moving this searching into FFmpeg is an obvious next step, I sincerely hope it's possible!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Untribium on 05/24/2014 05:22 pm
Actually, it occurs to me that the straight up "ps" approach to tracking workers may have problems with the above script, because the workers are constantly starting and stopping ffmpeg, which ps is looking for.

Another easy trick to track spawned workers is just to use PHP.

To check workers you can use:
while [ `ps aux | grep process_one\.php | grep -v grep | wc -l` -ge $threads ]

And to spawn workers you can use:
# spawn a worker(via php passthrough for easy thread tracking) on the current offset
php /root/process_one.php $i &

process_one.php looks like this:

-snip-

Again, the overhead php adds to spawning a process in this case is negligible because the workers will be running for a good while each and you're spawning thousands, not millions or billions.

Looks like this: http://coldattic.info/shvedsky/pro/blogs/a-foo-walks-into-a-bar/posts/7 (http://coldattic.info/shvedsky/pro/blogs/a-foo-walks-into-a-bar/posts/7) might be an option as well. Generate the addresses using seq and then pipe it to xargs which then runs a fixed number of ffmpeg processes at a time. I'll check it out tomorrow, should get some sleep first :)

I am interested to see the results.  I'm just so busy that my "recreation" basically involves this thread and a few other news sites at the moment.  Maybe you have found something that will be very useful to my endeavors in the future, at least on one level of scale!  Although I must admit, I do enjoy coding this type of thing.

Okay, I looked up some xargs stuff...
The single flip should be pretty simple, something like this should do it:

Quote
seq 8800 9100 | xargs -n 1 -I BITPOS --max-procs=4 ffmpeg -threads 1 -debug mb_pos_size -err_detect ignore_err -s:0 704:480 -mmb ...

In the mmb you can now use BITPOS as the parameter for example X:BITPOS:08. Unfortunately, I can't check whether it works and actually runs faster right now...
Also, I'm having some trouble when it comes to multiple flips. The problem is, that you can only -I one parameter in xargs, so some bash trickery (like creating an array with the params in) will be necessary. Now if we get the bitflipping built into the decoder as arnezami suggested (which would be awesome!), that will be kinda pointless, but I least I now know how to move and rename a bunch of numbered files at once (using multiple processes even!) :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: The Roadie on 05/24/2014 06:37 pm
Why are we succeeding?

I think there are two main reasons why we are succeeding in what was deemed to be impossible. The first is that we really went outside-the-box with our approach. The second is that our techniques are such that we can collaborate in our efforts.
I may be a noob here, but as an old-timer on IRC, Usenet, and forums, I've been over-the-top impressed with this project. Every forum with a critical mass of talent and altruistic members seems to adopt a project sooner or later that defines the spirit of the group. And it's often called the Epic Thread. And mention of it can go viral, and it's talked about at trade meetings, cocktail parties, and campfires for a generation.

I'm certain I missed a few here in the past, but this one seems to be the Thread of 2014 in my opinion. Carry on, and thanks!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: princess on 05/24/2014 08:52 pm
OK, here's my fixed version of the part 15 TS file. As far as I can see there's a damaged P-frame starting in packet 108 (offset 0x4f50 bytes into the file), but it's damaged at the MPEG4 level and I'm only fixing TS-level things at the moment.

I've set its AF to 7 bytes so that the MPEG4 data will be aligned properly, but I've not put a PCR in it because it's very damaged (and as far as I can see, ffmpeg doesn't seem to mind the lack of a PCR).

It should be ready for MPEG4 experts to start playing with it - if anyone wants to correct the header of this packet, a new P-frame should hopefully emerge. Let us know how you go!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/24/2014 10:34 pm
OK, here's my fixed version of the part 15 TS file. As far as I can see there's a damaged P-frame starting in packet 108 (offset 0x4f50 bytes into the file), but it's damaged at the MPEG4 level and I'm only fixing TS-level things at the moment.

I've set its AF to 7 bytes so that the MPEG4 data will be aligned properly, but I've not put a PCR in it because it's very damaged (and as far as I can see, ffmpeg doesn't seem to mind the lack of a PCR).

It should be ready for MPEG4 experts to start playing with it - if anyone wants to correct the header of this packet, a new P-frame should hopefully emerge. Let us know how you go!

Here how the header looks like in rawsplit_part_15.ts:
47 65 28 20 87 80 00 38 12 2A 53 00 9E 00 5E E7 02 00 81 80 07 21 09 BF FA B9 FF 47 00 50 05 96 4D

Here is the P-frame header how it should look like:
47 43 E8 3E 07 10 00 38 12 2a 7E 00 00 00 01 E0 00 00 81 80 07 21 01 BF FA 89 FF FF 00 00 01 B6 5D

34 bits flipped between them (not counting the CC and the last half byte D)
Actually, i never checked if there is more header after the  00 00 01 b6 5 which we should repair.
I will do that when the rest is done.

Cheers Shanuson

Edit: Ok I looked at your fixed part15 file. There were still many unfixed Ts-packets, most with the correct PID and CC but wrong flags set. The TS-Header of a data packet should really only look like 47 03 e8 1X, with 2 exceptions, the first and last of a frame which should be easily identified.

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: princess on 05/24/2014 11:51 pm

Edit: Ok I looked at your fixed part15 file. There were still many unfixed Ts-packets, most with the correct PID and CC but wrong flags set. The TS-Header of a data packet should really only look like 47 03 e8 1X, with 2 exceptions, the first and last of a frame which should be easily identified.

That's great, thank you for taking a look. Your changes all look good! I'd missed that packet with the huge AF at offset 0x52fc, good catch.

For the other "unfixed" packets I'd just left the erroneous Payload Unit Start Indicator bits in there. I agree with you that they should be removed - I had left them in there because (as far as I can see) they don't trip ffmpeg up, and if I'm in doubt I like to change as little as possible. I'm always worried about changing too much, it always feels nicer to make as small a change as I can. However, in future I can remove these too if you think it makes it better.

Thank you for your feedback, much appreciated!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/25/2014 12:13 am
For the other "unfixed" packets I'd just left the erroneous Payload Unit Start Indicator bits in there. I agree with you that they should be removed - I had left them in there because (as far as I can see) they don't trip ffmpeg up, and if I'm in doubt I like to change as little as possible. I'm always worried about changing too much, it always feels nicer to make as small a change as I can.

It's not so much the size of the change, but the quality of the change that's important. There's a true and original state underneath all that corruption, and that's where we should aim as long as we can aim with confidence and precision.

(http://www.cornercottageframeshop.com/pics/restoration/fine-art-restoration-02.jpg)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: princess on 05/25/2014 12:18 am
There's a true and original state underneath all that corruption, and that's where we should aim as long as we can aim with confidence and precision.

That's a good way to look at it! OK, I will make sure to fix the Payload Unit Start Indicators in future ;)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/25/2014 07:57 am
This is just a proof of concept. I deciphered a few of the timestamps on the images and interpolated the rest. Due to interlacing and propagating errors, even if we get good mbs in the pframes the timestamps are still going to be gibberish most of the time. So, I propose we just redraw them. Any concerns about this?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jakusb on 05/25/2014 08:18 am

This is just a proof of concept. I deciphered a few of the timestamps on the images and interpolated the rest. Due to interlacing and propagating errors, even if we get good mbs in the pframes the timestamps are still going to be gibberish most of the time. So, I propose we just redraw them. Any concerns about this?

Seems like a very good idea, with a few constraints:
- they need to be correct
- they are only applied at time of generating the movie file, so not to mess up de frame reconstruction process. In the end it is much better to have them fixed instead of reapplied.

But if you are at it, is it easy for you to also add a side view impression of how the stage would look at a every frame?

I think it would really help the viewers better understand what they are looking at. Including frames that are too corrupted. Also it would trigger a discussion on what happens in every frame.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: deruch on 05/25/2014 08:31 am
This is just a proof of concept. I deciphered a few of the timestamps on the images and interpolated the rest. Due to interlacing and propagating errors, even if we get good mbs in the pframes the timestamps are still going to be gibberish most of the time. So, I propose we just redraw them. Any concerns about this?

Another route to go would be to put the frame# in the upper corner (I remember one of the previous gifs posted did this).
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Lourens on 05/25/2014 08:53 am
Just a quick note before I have to go do other stuff. I'm working on brute-force bit flipping a bit. As I expected, making a good objective function is not so easy, looking at the sum of squared errors across block edges is a good first stab, but it needs more work for sure. The combinatorics is the second problem, there are just way too many combinations of bits to flip, and some sort of clever search algorithm needs to be devised. Also it would really help to know a bit more about the error correction method used by the telemetry stream.

I have one idea for speeding things up though, which I haven't tried yet but will. What if you concatenate n copies of the I-frame together into a single file, each with a different bit flipped, and then put that into FFmpeg. Would that yield n separate frames in a single call? If so, that would amortise the overhead I think.

Excellent work on the TS repair by the way, that will really help because it will allow us to take things one frame at a time. That has to be easier than solving the whole problem in one go.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: CuddlyRocket on 05/25/2014 09:21 am
This is just a proof of concept. I deciphered a few of the timestamps on the images and interpolated the rest. Due to interlacing and propagating errors, even if we get good mbs in the pframes the timestamps are still going to be gibberish most of the time. So, I propose we just redraw them. Any concerns about this?

Reminds me of the Doctor Who Restoration Team who fix old (1960s onwards) episodes. With the credits, they know what they're supposed to say; they know what font etc they were in, so they simply make crisp new sets and slot them in. They use another trick for the titles, most of which would have been identical for each episode. They make a compilation of the best bits from individual episodes and use that for all the episodes.

However, ...

Seems like a very good idea, with a few constraints:
- they need to be correct
- they are only applied at time of generating the movie file, so not to mess up de frame reconstruction process. In the end it is much better to have them fixed instead of reapplied.

IMO this is an historical document. First, you should obtain the best version of the data that is present. Ideally, someone should detail all the individual steps to get from the original data to the present so that future historians can verify its correctness (hope you're all keeping notes! :) ). Once that has been done and archived, then you can start on these and similar techniques to improve the appearance of the video.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mhenderson on 05/25/2014 03:14 pm
For the final work product (if there is such a thing) I suggest a video that repeats the sequence several times interleaved with some text screens that explain who / what / why at a high level.

1) Begin with a brief paragraph about the circumstances and the challenge.
2) Full video first pass, show the original video before restoration.
3) Another paragraph, introduce the NSF forum and crowdsourced effort.
4) A series of before / after shots of individual frames to indicate progress with captions highlighting tools used, round-the-globe contributions, people contributing from their own skillsets....
5) Full video, showing a "true" restoration (no enhancements)
6) Full video, show a side-by-side presentation. in the top left corner of screen (20%), show a graphic animation of the landing process in the main frame bottom right (80%) show the restored video again. For the graphic animation, perhaps  the YouTube video by Oberg N titled "F9R in KSP" is pretty close to what we would need. http://youtu.be/jU2Bbkeq4ds (http://youtu.be/jU2Bbkeq4ds)  Perhaps this can be captioned with the interpretation.
7) A final pass, full screen of the team's video including enhancements such as a frame counter at the bottom right and wronkiew's nice clock in the bottom left.
8) End with credits thanking core people - slow scroll - and every username on the forum - fast scroll.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: MP99 on 05/25/2014 03:18 pm
This is just a proof of concept. I deciphered a few of the timestamps on the images and interpolated the rest. Due to interlacing and propagating errors, even if we get good mbs in the pframes the timestamps are still going to be gibberish most of the time. So, I propose we just redraw them. Any concerns about this?

As just an interested onlooker, my opinion is just to correct as much as possible, but not to overwrite anything like timestamps.

Otherwise, someone somewhere is gonna do a "moon landings were fake" meme on this, highlighting the bits that don't match the rest of the stream. (Yes, yes - but you know that they will.)

Cheers, Martin
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: meadows.st on 05/25/2014 04:45 pm
This is just a proof of concept. I deciphered a few of the timestamps on the images and interpolated the rest. Due to interlacing and propagating errors, even if we get good mbs in the pframes the timestamps are still going to be gibberish most of the time. So, I propose we just redraw them. Any concerns about this?

I agree with a number of other posters (Jakusb, CR) that this is an historic document and should this be restored as carefully as possible, however, I also agree that this timestamp information is important. I would like to see any intensional additions (other than bitflips) to be included not as overlays but rather as additions in a larger screen such that the restored video is the "picture in picture" or as an overlay in an extended sequence such as mhenderson suggests here.

For the final work product (if there is such a thing) I suggest a video that repeats the sequence several times interleaved with some text screens that explain who / what / why at a high level.

1) Begin with a brief paragraph about the circumstances and the challenge.
2) Full video first pass, show the original video before restoration.
3) Another paragraph, introduce the NSF forum and crowdsourced effort.
4) A series of before / after shots of individual frames to indicate progress with captions highlighting tools used, round-the-globe contributions, people contributing from their own skillsets....
5) Full video, showing a "true" restoration (no enhancements)
6) Full video, show a side-by-side presentation. in the top left corner of screen (20%), show a graphic animation of the landing process in the main frame bottom right (80%) show the restored video again. For the graphic animation, perhaps  the YouTube video by Oberg N titled "F9R in KSP" is pretty close to what we would need. http://youtu.be/jU2Bbkeq4ds (http://youtu.be/jU2Bbkeq4ds)  Perhaps this can be captioned with the interpretation.
7) A final pass, full screen of the team's video including enhancements such as a frame counter at the bottom right and wronkiew's nice clock in the bottom left.
8) End with credits thanking core people - slow scroll - and every username on the forum - fast scroll.

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/25/2014 04:53 pm
This is just a proof of concept. I deciphered a few of the timestamps on the images and interpolated the rest. Due to interlacing and propagating errors, even if we get good mbs in the pframes the timestamps are still going to be gibberish most of the time. So, I propose we just redraw them. Any concerns about this?

I also agree that putting a clear timestamp on a ribbon outside of the original frame, rather than overlapping it, would be the way to go.

I agree with CuddlyRocket that this is a historical document, because I believe that the ability to reuse at least 70% of the cost of the launch vehicle - which is what this video represents - will lead to a revolutionary reduction in the cost of access to space, leading to a new era of human spaceflight and a permanent human presence not only in LEO, but on the Moon, at various Lagrange points, and ultimately Mars and beyond. In that future, these few frames of video will be treasured as the first steps down that path, and and even the gibberish will have historic significance in the decades and centuries to come. The Star Spangled banner is in tatters, but even the bits of shredded string were meticulously preserved and carefully displayed.

(http://amhistory.si.edu/starspangledbanner/images/3600_04.jpg)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/25/2014 08:00 pm
Thanks for the feedback on the timestamp reconstruction. It sounds like adding mattes to the video and drawing timestamps and/or simulation graphics there would be preferable.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: AncientU on 05/25/2014 08:19 pm
For the final work product (if there is such a thing) I suggest a video that repeats the sequence several times interleaved with some text screens that explain who / what / why at a high level.

1) Begin with a brief paragraph about the circumstances and the challenge.
2) Full video first pass, show the original video before restoration.
3) Another paragraph, introduce the NSF forum and crowdsourced effort.
4) A series of before / after shots of individual frames to indicate progress with captions highlighting tools used, round-the-globe contributions, people contributing from their own skillsets....
5) Full video, showing a "true" restoration (no enhancements)
6) Full video, show a side-by-side presentation. in the top left corner of screen (20%), show a graphic animation of the landing process in the main frame bottom right (80%) show the restored video again. For the graphic animation, perhaps  the YouTube video by Oberg N titled "F9R in KSP" is pretty close to what we would need. http://youtu.be/jU2Bbkeq4ds (http://youtu.be/jU2Bbkeq4ds)  Perhaps this can be captioned with the interpretation.
7) A final pass, full screen of the team's video including enhancements such as a frame counter at the bottom right and wronkiew's nice clock in the bottom left.
8) End with credits thanking core people - slow scroll - and every username on the forum - fast scroll.
The full restoration should be edited in with launch video, separation of first stage, supersonic retro burn, and then this landing. SpaceX has much unreleased footage and might be willing to collaborate on the effort.  The film of Wright brother's first flight in 1903 (link below) has spooky parallels with the SpaceX effort.  Even some of the phraseology is directly applicable.  Maybe this should be a separate product, but one or both will be seen by millions (billions?).
https://www.youtube.com/watch?v=Wfyvspnko04
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/25/2014 09:07 pm
Decoding parts of the I-Frame and P-Frame Header to find out how the part after 00 00 01 b6 should look like for P-Frames.
Summary at the end!!!!

First row is  hex, second row is binary value. Then a bunch of explanation.

Video Object Layer Part: This sets certain flags used later in decoding the  video object plane header:
   00       00       01       20       00       c4       8d   
00000000 00000000 00000001 00100000 00000000 11000100 10001101

video object layer start code
random_accessible_vol
video object type indication -> simple object type
is object layer identifier
object layer verid -> 0001 -> newpred_enable is 0, also reduced_resolution_vop_enable is 0
object layer priority
aspect ratio info -> 1:1
vol control parameter
chroma format -> 4:2:0

   c0       00       46       56       c0       1e       c0       01       xx       xx       
11000000 00000000 01000110 01010110 11000000 00011110 11000000 00000001 xxxxxxxx xxxxxxxx

low delay
vdv_parameters -> yes
first half bit rate
marker bit
latter half bit rate
marker bit
first half vbv buffer size
marker bit
latter half vbv buffer size
first half vbv occupancy
marker bit
latter half vbv occupancy

   9a       fc       7c       2e       ee       2c       08       78       28       c7
10011010 11111100 01111100 00101110 11101110 00101100 00001000 01111000 00101000 11000111

marker bit
video_object_layer_shape -> rectangular
marker bit
vop_time_increment_resolution = 44999
marker_bit
fixed_vop_rate
fixed_vop_time_increment = 3003
marker_bit
video_object_layer_width = 704
marker_bit
video_object_layer_height = 480
marker_bit
interlaced -> no (it is true that this bit is 0 for all 15 I-Frames)
obmc_disable  -> yes
sprite_enable  -> no
not_8_bit -> no ( is 8 bit?) -> quant_precision = 5
quant_type -> no
complexity_estimation_disable -> yes
resync_marker_disable -> yes
data_partitioned -> no
scalability -> no
stuffing
Next comes userdata until we hit the vop_start_code 0x000001b6.

P-Frame VOP header part:
(1/5)x       xx       x(e/c)    0(even x) (4/5)x
 0x01xxxx xxxxxxxx xxxx11x0  0000xxx0      010xxxxx

vop_coding type - 00 = I-Frame ; 01 = P-frame
modulo time base = 0
marker bit
vop_time_increment <- this is increasing from frame to frame by 3003 (fixed_vop_time_increment) modulo 44999 (vop_time_increment_resolution), so it is easy to calculate what it should be.
marker bit
vop_coded -> yes
vop_rounding_type -> alternating between 1 and 0
intra_dc_vlc_thr = 000
vop_quant 00xxx
vop_fcode_forward  <- could this always be 001? my guess but i am not sure.
ref_select_code - either 01 or 00?? how do we know which?
and now follow the makroblocks....

Edit:
I-Frame VOP header part:
(1/5)x       xx       xc    (0/1)x     x
 0x01xxxx xxxxxxxx xxxx1100  000xxx0x xxxx

vop_coding type - 00 = I-Frame ; 01 = P-frame
modulo time base = 0
marker bit
vop_time_increment <- this is increasing from frame to frame by 3003 (fixed_vop_time_increment) modulo 44999 (vop_time_increment_resolution), so it is easy to calculate what it should be.
marker bit
vop_coded -> yes
vop_rounding_type -> alternating between 1 and 0
intra_dc_vlc_thr = 000
vop_quant 00xxx
vop_fcode_forward  <- could this always be 001? my guess but i am not sure.
ref_select_code - either 01 or 00?? how do we know which?
and now follow the makroblocks....


Summary:
I am surprised that the interlace flag is 0!!
I can restore parts of the VOP-header. Some things I am not sure what they should be.
Especially vop_quant, vop_fcode_forward and ref_select. Somebody with more knowledge about the decoding  of makroblocks could help here maybe?
Maybe I can see patterns when I checked all P-Frames.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Lar on 05/25/2014 10:32 pm
For the final work product (if there is such a thing) I suggest a video that repeats the sequence several times interleaved with some text screens that explain who / what / why at a high level.

1) Begin with a brief paragraph about the circumstances and the challenge.
2) Full video first pass, show the original video before restoration.
3) Another paragraph, introduce the NSF forum and crowdsourced effort.
4) A series of before / after shots of individual frames to indicate progress with captions highlighting tools used, round-the-globe contributions, people contributing from their own skillsets....
5) Full video, showing a "true" restoration (no enhancements)
6) Full video, show a side-by-side presentation. in the top left corner of screen (20%), show a graphic animation of the landing process in the main frame bottom right (80%) show the restored video again. For the graphic animation, perhaps  the YouTube video by Oberg N titled "F9R in KSP" is pretty close to what we would need. http://youtu.be/jU2Bbkeq4ds (http://youtu.be/jU2Bbkeq4ds)  Perhaps this can be captioned with the interpretation.
7) A final pass, full screen of the team's video including enhancements such as a frame counter at the bottom right and wronkiew's nice clock in the bottom left.
8) End with credits thanking core people - slow scroll - and every username on the forum - fast scroll.
The full restoration should be edited in with launch video, separation of first stage, supersonic retro burn, and then this landing. SpaceX has much unreleased footage and might be willing to collaborate on the effort.  The film of Wright brother's first flight in 1903 (link below) has spooky parallels with the SpaceX effort.  Even some of the phraseology is directly applicable.  Maybe this should be a separate product, but one or both will be seen by millions (billions?).
https://www.youtube.com/watch?v=Wfyvspnko04

Good list!  I support the notion of boxing the original in new larger resolution with the edges used for things. Also nice find on the Wright Brothers.  As a trivia  note, that footage is actually from their flight in 1908. The first flight was far shorter, was linear (no circling back and buzzing the spectators) and didn't have spectators.

http://en.wikipedia.org/wiki/Wright_Flyer:

Quote
The airplane left the rail, but Wilbur pulled up too sharply, stalled, and came down in about three seconds with minor damage.

Repairs after the abortive first flight took three days. When they were ready again on December 17, the wind was averaging more than 20 mph, so the brothers laid the launching rail on level ground, pointed into the wind, near their camp. This time the wind, instead of an inclined launch, helped provide the necessary airspeed for takeoff. Because Wilbur already had the first chance, Orville took his turn at the controls. His first flight lasted 12 seconds for a total distance of 120 ft (36.5 m) – shorter than the wingspan of a Boeing 747, as noted by observers in the 2003 commemoration of the first flight.

But none of that matters. This video and its restoration is historic. Everyone involved has something to be quite proud of.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Llian Rhydderch on 05/25/2014 10:50 pm
Agree with mhenderson, meadows.st and mvpel:  how we put together the final/mongo historical document video, with appropriate video meta-text (seqrchable) and on-screen text, time counter added in the matte, etc. are all very important.

I also think we should include the bit-for-bit identical original file, plus attach appropriate source code for the correction input stream to ffm... program (as it has been enhanced for this effort), plus any other supplemental files that might be useful to archive.  (those could be hosted on the spacexlanding wiki, or some other suitable archive location).

One other matter to consider:  I'm hopeful we will be able to put on a Creative Commons license (perhaps share and share alike) on the finished product that would make the end video product widely shareable, including in public encyclopedias like Wikipedia.  I think this effort is sufficiently notable to be it's own Wikipedia article, with sources from several reliable sources that have discussed the effort.

I put something about the license question on the thread two weeks ago, but we were all too heads down on video recovery to think video licensing then.  Here's the link to that post (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1199979#msg1199979). 

At any event, getting a clean license on this repaired file, and all the associated work, will be an important consideration that we should discuss/consider also.

Cheers,
 Llian
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/25/2014 11:04 pm
Interlaced == 0?  :o For all iframes?  :o :o But then again...

Since we've seen the clear and unmistakable evidence that it the images are interlaced right there in the macroblocks, it must be that it was captured interlaced at 30 fields per second and then encoded for transmission non-interlaced at 15 frames per second by combining the two fields. This is supported in a way by the presentation time stamp numbers - you point out that they increase by 3003 between each frame and by 60060 (20x3003) between iframes, (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1201981#msg1201981) which given the PTS rate of 90kHz (https://en.wikipedia.org/wiki/Presentation_timestamp), comes out to 0.03336 seconds per frame, and multiplying that by the NTSC frame rate of 29.97 fps gives you 0.99 - or as close to exactly one second as you can get in this universe.

What's also interesting here is that the presentation timestamp value is claiming to be an NTSC frame rate of 29.97 fps, but real time is advancing at twice that rate as we can see on the clock - the time between iframes is 1.3346 seconds rather than 60060 / 90000 seconds. I'm sure someone familiar with video capture and encoding will be able to enlighten us about this.

As to the question of licensing, this page about "Copyfree" is informative. (http://copyfree.org/policy/public) One could say that SpaceX has released the raw video into the public domain by virtue of their statements and actions, but apparently that's not always sufficient depending on the jurisdiction.

Quote
Provision is identified here for an author assigning copyright to another entity, but not for dedication to the public domain -- simply relinquishing, rather than transfering, copyright. More nuanced legal analyses are generally incomplete and/or contradictory on this subject. Public domain dedication in the United States tends to "work" in practice because the author attempting to dedicate the work to the public domain is not likely to pursue legal action against someone who uses the work as though it were in the public domain, but the question of whether it is actually a public domain work is still largely unanswered. If it is not, the author's heirs may assert copyright protections in the future, or the author might if he or she has a change of heart. Furthermore, any copyright the author of a derivative work might think he or she can claim over that derivative work might be null and void, leading to other unexpected legal problems that are too complex to address here at this time.

So I think you're right, here, Llian - it's going to be important for SpaceX to explicitly designate a CC-BY-SA or something comparable in the near future, especially considering the global character of this effort of creating a "derivative work."
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Lar on 05/25/2014 11:06 pm
At any event, getting a clean license on this repaired file, and all the associated work, will be an important consideration that we should discuss/consider also.

Very important. This can be considered a derivative work (http://en.wikipedia.org/wiki/Derivative_work ) so the copyright of the original source is important too. Was that ever ascertained?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/25/2014 11:22 pm
Ok, finally a rawsplit_part_1_fixed.ts and  rawsplit_part_7_fixed.tsfile that gives no "important" errors with ffmpeg -debug -vstats ...
All PID 3e8 CC's are corrected, all P and I-Frame headers are corrected as far as possible (vop_quant, vop_fcode_forward and ref_select are probably wrong for some Frames)

Cheers
Shanuson

PS, somebody could add those to the online Editor.
PPS, also attached a python script that I use to check for the continuity of the CC of PID 3e8, and for the time_increment value of the VOP header.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: michaelni on 05/26/2014 12:37 am
Summary:
I am surprised that the interlace flag is 0!!
I can restore parts of the VOP-header. Some things I am not sure what they should be.
Especially vop_quant, vop_fcode_forward and ref_select. Somebody with more knowledge about the decoding  of makroblocks could help here maybe?
Maybe I can see patterns when I checked all P-Frames.

vop_quant is the initial QP, a wrong value there would lead to artifacts but the image should be recognizeable
vop_fcode is related to the VLC coding of motion vectors its value must be correct or MBs with motion vectors wont decode correctly
larger fcode values allow longer motion vectors so they might be related to more "active" frames. A simple encoder might also simply use the same fcode for all frames
ref_select is used in scalability, it should not be in the header as this should be a non scaleable stream
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mhenderson on 05/26/2014 01:01 am
@Shanuson -

Thank you.
What do we need to know / be aware of when using this .ts?
How will this affect the legacy mmb work? Will this shift have the same effect as some of the previous changes to the .ts when there were POS changes in iFrames?
What do "part_1" and "part_7" refer to?

Thank you
 
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Llian Rhydderch on 05/26/2014 01:22 am
At any event, getting a clean license on this repaired file, and all the associated work, will be an important consideration that we should discuss/consider also.

Very important. This can be considered a derivative work (http://en.wikipedia.org/wiki/Derivative_work ) so the copyright of the original source is important too. Was that ever ascertained?

No, I don't think SpaceX ever made that explicit.  Although I doubt they would have much trouble with clarifying that license, given they've put it out to the world for crowdsourced improvement of the video file, as mvpel noted.

I have one contact in the communications/PR group there, and will connect with them to see if we can get SpaceX to formally clarify the original file on the raw.ts file that they made available, and which is serving as the master file for all the great video tech-heads who are doing so much to cleanup the stream and fix errors so we all get the historical view into this video data that this first in the history of human technology deserves:  a successful vertical soft-landing of an orbital booster stage rocket.

Edit:  I have sent the request.  Asked specifically for CC-BY-SA as Lar suggested. 
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Lar on 05/26/2014 01:29 am

I have one contact in the communications/PR group there, and will connect with them to see if we can get SpaceX to formally clarify the original file on the raw.ts file that they made available, and which is serving as the master file for all the great video tech-heads who are doing so much to cleanup the stream and fix errors so we all get the historical view into this video data that this first in the history of human technology deserves:  a successful vertical soft-landing of an orbital booster stage rocket.

That would be awesome. Please ask your contact to see if the original can be released CC-BY-SA ... please avoid NC and ND!!!! Version 4 would be best I think. Here's more on the license.

http://en.wikipedia.org/wiki/Creative_Commons_license

I don't know if a barrage of requests would be useful but maybe asking via another channel or two might be good.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: michaelni on 05/26/2014 03:13 am
Hi all
ive added some simple bitflip brute forcing support to ffmpeg at github
example: -bitflip_brute 0:28:5:3:19771:20055:2

this will try to flip all combinations of 2 bits between 19771 and 20055 and try to decode this area as 3 MBs starting at 28:5 of frame 0

that is framenum:mb_x:mb_y:mb_count:bitstart:bitend:bitcount

from the 2 experiments i did, it succeeded to find a 1 and a 2 bitflip that i intentionally introduced, in the case of the 2bits it also found 63 alteratives, the 1 flip case was uniquely found though
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/26/2014 05:25 am
For those of us working on the transport stream headers, it appears that the final TS header before a new frame starts, often followed by a series of null TS packets leading into the 0x4743e83_ header, uses the adaptation field to push the final bytes of the frame out to the end of the TS packet, presumably for alignment purposes. Thus, rather than

0x4703e81_

... you will see:

0x4703e83_

This indicates that there is both an adaptation field and a payload. Immediately after the header, you'll see an adaptation field length that is less than 184, followed by a "00" indicating a "null" adaptation field:

47 03 e8 39 5e 00 ff ff ff ...

This one is 94 bytes. At the end of the 94 bytes, you'll see some payload data that goes to the end of the TS packet.

So this gives us a couple of things to check - we should be able to figure out whether or not the AF length is correct based on where the ff's end, and insure that the AF bit is set in the last packet of the frame if it doesn't end on a clean TS packet boundary.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: deruch on 05/26/2014 05:53 am
What do "part_1" and "part_7" refer to?

The work on .ts headers is being done on raw_edit8.ts that was split up into 15 parts by Shanuson in a previous post.  The wiki page http://spacexlanding.wikispaces.com/Extraction+of+p-frames (http://spacexlanding.wikispaces.com/Extraction+of+p-frames) explains.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/26/2014 06:01 am
I cleaned the transport stream headers on part 131, attached below.

My notes during the cleaning:

Packet 163 at offset 30644 is a VOP start frame, but it needs a bit of attention. Unlike the other start packets, it has an 11-byte adaptation field and the 000001e0 starts 4 bytes later in the ts packet at offset 0x7708. I adjusted the AF length to allow the 0x000001e0 to line up as needed, but I'm not sure why this one is different and what the extra four bytes are for.

The four bytes in question, 0x181803c0, register as padding based on the adaptation field's bitmap, so theoretically should be 0xff, but I'm not sure if it might be something else that's supposed to fall in those four bytes. But it does appear that a reasonable-looking pframe came out.

At offset 0x12d18, this appears to be a null packet, but the continuity counter is off, the count goes from 0x1 to 0x2 across this gap in raw.ts as well:    null packet at 0x24a7ec and at 0x251324  However in raw.ts, this packet at 0x24ea04 reads 0x2e3ffe2f - much closer to 0x471fff1x than a 0x3e8 packet. Setting it to 0x471fff1f at 0x12d18 in fixed_edit8_part_131.ts - looks like it turned out okay.

Offset 0x21004 - a clear null packet with an incorrect CC - 0xd to 0x0 and later 0xe - perhaps an alignment packet inserted by Shanuson?

Offset 0x401ec - the adapation field length 0x77 (119) may be wrong here, but can't tell due to corruption. The last ff is at 0x4021A, which is 42 length, 0x2a, but we then have 0x7F (1 bit) 0xfc (1 bit) then a few more ff's. Leaving it at 0x77 since it doesn't exceed the packet length. The data at this AF length starts with 0x95591f6b.

The pframes look  promising, I've attached one of the better ones for your enjoyment - it's flames lapping out to the left side, and you can see the outline of the legs as the lighting changed from the previous frame.

This still needs work in the MPEG headers, but we should have a solid transport stream.

Edit: (Depressingly, the pframes that emerge from ffmpeg checksum as identical to the pframes out of the original ts file, but I guess you have to actually do the cleaning in order to find that out.)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/26/2014 07:53 am
Correcting the p-frames is often quite tricky because we don't see very well where we have "correct" information. We can also easily put some sequences at a slightly incorrect position. Would it be possible to add a slider enhancing the image contrast? If we could see a little better, it would already help a lot.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/26/2014 08:31 am
Summary:
I am surprised that the interlace flag is 0!!
I can restore parts of the VOP-header. Some things I am not sure what they should be.
Especially vop_quant, vop_fcode_forward and ref_select. Somebody with more knowledge about the decoding  of makroblocks could help here maybe?
Maybe I can see patterns when I checked all P-Frames.

vop_quant is the initial QP, a wrong value there would lead to artifacts but the image should be recognizeable
vop_fcode is related to the VLC coding of motion vectors its value must be correct or MBs with motion vectors wont decode correctly
larger fcode values allow longer motion vectors so they might be related to more "active" frames. A simple encoder might also simply use the same fcode for all frames
ref_select is used in scalability, it should not be in the header as this should be a non scaleable stream

Thanks for the explanation. I was wondering about the ref_select, I overlooked a ! when i read the ISO. Thank for pointing it out. I will correct my post.
So one has to fix vop_quant first, before one works on a P-Frame? Will the correct value be recognizable from a wrong one?

For fcode i would assume it is 001 for all P-frames at the moment. Will check this for more active and correct parts of the video.


@Shanuson -

Thank you.
a)What do we need to know / be aware of when using this .ts?
b) How will this affect the legacy mmb work? Will this shift have the same effect as some of the previous changes to the .ts  when there were POS changes in iFrames?
c) What do "part_1" and "part_7" refer to?

Thank you

a) that this is most like the most correct raw file you will get. That for parts where I fixed the P-Frame header (same is true for the I-frames work i did before) vop_quant could be wrong. Maybe that is why 0:0:-1 helps sometimes a lot? When all 15 parts are fixed to my satisfaction, I will combine them back together and we have the best possible raw-file.
b) Some will still work. some wont work without shifting some things. Sry.
c) Part of the Stream following I-Frame 1 and 7. They still contain one overlapping TS packet at start and end. 

We should also look at the parts where it already found 19 P-Frames. Tsfix does not find all the right PIDs i think. so maybe there is still some stuff hidden somewhere.

For those of us working on the transport stream headers, it appears that the final TS header before a new frame starts, often followed by a series of null TS packets leading into the 0x4743e83_ header, uses the adaptation field to push the final bytes of the frame out to the end of the TS packet, presumably for alignment purposes.
<snip>

Thanks for pointing it out again and more clearly than I did before. It could be that the last few MB's of a frame are part of that last packet. So identifying this is important for the last row of a file.

Cheers,
Shanuson
 
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Microphobe on 05/26/2014 09:23 am
For the final work product (if there is such a thing) I suggest a video that repeats the sequence several times interleaved with some text screens that explain who / what / why at a high level.

1) Begin with a brief paragraph about the circumstances and the challenge.
2) Full video first pass, show the original video before restoration.
3) Another paragraph, introduce the NSF forum and crowdsourced effort.
4) A series of before / after shots of individual frames to indicate progress with captions highlighting tools used, round-the-globe contributions, people contributing from their own skillsets....
5) Full video, showing a "true" restoration (no enhancements)
6) Full video, show a side-by-side presentation. in the top left corner of screen (20%), show a graphic animation of the landing process in the main frame bottom right (80%) show the restored video again. For the graphic animation, perhaps  the YouTube video by Oberg N titled "F9R in KSP" is pretty close to what we would need. http://youtu.be/jU2Bbkeq4ds (http://youtu.be/jU2Bbkeq4ds)  Perhaps this can be captioned with the interpretation.
7) A final pass, full screen of the team's video including enhancements such as a frame counter at the bottom right and wronkiew's nice clock in the bottom left.
8) End with credits thanking core people - slow scroll - and every username on the forum - fast scroll.

Hi, I think you are correct to doubt that there will be a "final" version, my 2c is to plan a "release" sometime prior to the next Falcon 9 mission, but also plan for later releases.

I guess what I am trying to say is, it might be best to assemble video releases in the most automated way possible, because it probably won't be the last time it has to be done.

Disclaimer: I don't really know how much work it involves to assemble a polished video :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: MTom on 05/26/2014 09:45 am

Hi, I think you are correct to doubt that there will be a "final" version, my 2c is to plan a "release" sometime prior to the next Falcon 9 mission, but also plan for later releases.


BTW: The next launch attempt is Juni 11, Window is 2130-2224 Local.
Likely there will be less to see on the video from the descent.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/26/2014 10:24 am
I cleaned the transport stream headers on part 131, attached below.

My notes during the cleaning:

Packet 163 at offset 30644 is a VOP start frame, but it needs a bit of attention. Unlike the other start packets, it has an 11-byte adaptation field and the 000001e0 starts 4 bytes later in the ts packet at offset 0x7708. I adjusted the AF length to allow the 0x000001e0 to line up as needed, but I'm not sure why this one is different and what the extra four bytes are for.

The four bytes in question, 0x181803c0, register as padding based on the adaptation field's bitmap, so theoretically should be 0xff, but I'm not sure if it might be something else that's supposed to fall in those four bytes. But it does appear that a reasonable-looking pframe came out.


Thanks for your work:
First this is part_8.

There is no 4 bit offset. but some nice bitflips that make it look like there is:
               from raw: 8F 9F D0 74 0E 20 00 63 67 52 F8 64 18 18 03 C0 00 01 01 E0 00 02 13 2A E1 DD FB 43 8C 00 06 D9 49
and shifted 4 byte left: 0E 20 00 63 67 52 F8 64 18 18 03 C0 00 01 01 E0 00 02 13 2A E1 DD FB 43 8C 00 06 D9 49 19 B0 11 0A
    and 4 bytes cut out: 8F 9F D0 74 0E 20 00 63 67 52 F8 C0 00 01 01 E0 00 02 13 2A E1 DD FB 43 8C 00 06 D9 49 19 B0 11 0A
      what it should be: 47 43 E8 3x 07 10 00 31 b3 a1 7E 00 00 00 01 E0 00 00 81 80 07 21 01 8D 80 65 FF FF 00 00 01 B6 5x

Comparing the first 3 to the last it shows that there is no 4 byte shift. Also the next packet is aligned with no 4byte shift.

Quote

At offset 0x12d18, this appears to be a null packet, but the continuity counter is off, the count goes from 0x1 to 0x2 across this gap in raw.ts as well:    null packet at 0x24a7ec and at 0x251324  However in raw.ts, this packet at 0x24ea04 reads 0x2e3ffe2f - much closer to 0x471fff1x than a 0x3e8 packet. Setting it to 0x471fff1f at 0x12d18 in fixed_edit8_part_131.ts - looks like it turned out okay.
That is ok.

Quote
Offset 0x21004 - a clear null packet with an incorrect CC - 0xd to 0x0 and later 0xe - perhaps an alignment packet inserted by Shanuson?
Sometimes there are single null packets inserted with a CC = 0 that break the CC for null packets. I tried to explain this before.
Quote
Offset 0x401ec - the adapation field length 0x77 (119) may be wrong here, but can't tell due to corruption. The last ff is at 0x4021A, which is 42 length, 0x2a, but we then have 0x7F (1 bit) 0xfc (1 bit) then a few more ff's. Leaving it at 0x77 since it doesn't exceed the packet length. The data at this AF length starts with 0x95591f6b.

The pframes look  promising, I've attached one of the better ones for your enjoyment - it's flames lapping out to the left side, and you can see the outline of the legs as the lighting changed from the previous frame.

This still needs work in the MPEG headers, but we should have a solid transport stream.

Edit: (Depressingly, the pframes that emerge from ffmpeg checksum as identical to the pframes out of the original ts file, but I guess you have to actually do the cleaning in order to find that out.)

The 77 looks ok for me. The corruption starts later in that packet.

I had to clean up a few TS-headers where the CC was not correct. Fixed the time_increment in the VOP headers.

Edit: fixed 4 minor things in the ts file, thanks mvpel for pointing them out.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mhenderson on 05/26/2014 02:56 pm
Narrative inserts for the final video.

These tell the background story and are meant to be interleaved with before / after video. They are in .html format, so anyone who cares to integrate with video can play in your browser and screen capture to record.

Wouldn't it be great to get somebody awesome to record this as a voiceover?  (My candidates include Tom Hanks, James Earl Jones, Leonard Nimoy, Bill Shatner, George Takei, Morgan Freeman, ... and Elon Musk) I would drop the scrolling graphics in a blink for a celebrity voice. Does anybody have an "in" with one of them?

Downlad and drop the five files in any folder, change the .txt extensions to .html, open the .html files with your browser.

(No javascript, just basic html.)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/26/2014 03:12 pm
Thanks for pointing it out again and more clearly than I did before. It could be that the last few MB's of a frame are part of that last packet. So identifying this is important for the last row of a file.

I found an instance of this that I missed in the file you attached above:

0x6D6C : 47 03 E8 19 3E 00 FF ...

...should be...

0x6D6C : 47 03 E8 39 3E 00 FF ...

And there's another at 0x256FC:

0x257FC : 47 03 E8 1A 1E 16 ...

Looks like it should be:

0x257FC : 47 03 E8 3A A9 FF ...

... as it appears that the stuffing is corrupted for a little while but then clears up by the time it gets to 86 60 E0 C8 which appears to be the payload.

Also, should the next packet after this one, at 0x258B8, be 0x4740201D instead of 0x4740001D?

At offset 0x19EEC, the 0x4783E81F should be 0x4703E81F.

At offset 0x 2DCE8, there's a 0x4723E81F which should be 0x4703E81F.

The final packet of a frame at 0x351AC appears to not need to be aligned to the TS packet so it doesn't have an adaptation field to push the remaining payload over - it happens to end at the end of the packet.

Everything else looks good. Do you want to apply those changes to your file and modify the post above so we save some disk space?

Edit: Updates applied to the attachment in Shanuson's post, above (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1204247#msg1204247).
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/26/2014 04:04 pm
For voice over, how about James Oliver Cromwell, who played Zefram Cochrane in Star Trek: First Contact?
http://en.m.wikipedia.org/wiki/James_Cromwell

The video's soundtrack would need "Magic Carpet Ride" in that case. :D
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mhenderson on 05/26/2014 04:42 pm
Cromwell! That'll do, pig. That'll do. ;)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/26/2014 05:21 pm
Here is rawsplit_part_2_fixed.ts again this time with all TS-data packets headers and stuff fixed.

I will now check all those parts that are submitted by others,

Done is part: 1,2,7,8,15
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/26/2014 06:55 pm
mvpel asked about this and probably a few others are also interested:

here are my 2 python scripts i use:

getheaderbitdistance.py:

This tries to find all P-Frame header positions in a file by comparing the data to a mask and counting bit discrepancies.
You have to un-comment the right I-Frame, so it can calculate the right PTS for each P-Frame:

The input is the TS file name

Output will be something like this:

read byte : 0
read byte : 50000
read byte : 100000
read byte : 150000
read byte : 200000
read byte : 250000
['47', '43', 'E8', '3x', '07', '10', '00', 'xx', 'xx', 'xx', '7E', '00', '00', '00', '01', 'E0', '00', '00', '81', '80', '07', '21', '01', 'xx', 'xx', 'xx', 'FF', 'FF', '00', '00', '01', 'B6', '5x']
trying to find header: 1
          -> ['47 43 E8 3x 07 10 00 34 67 ba 7E 00 00 00 01 E0 00 00 81 80 07 21 01 A3 50 C9 FF FF 00 00 01 B6 5x ']
0x415c 0 -> ['47 43 e8 34 07 10 00 34 67 ba 7e 00 00 00 01 e0 00 00 81 80 07 21 01 a3 50 c9 ff ff 00 00 01 b6 51 '] 0
0x21238 13 -> ['47 43 e8 32 07 10 00 34 c5 92 7e 00 00 00 01 e0 00 00 81 80 07 21 01 a5 c8 29 ff ff 00 00 01 b6 57 '] 48048

followed by a repetition of the last 4 lines for all 19 P-Frames.
Of this last 4 lines the first gives the P-Frame number, the next how it should look like, and finally the 2 closest data packets.
There the output is: position, bit distance, data , PTS distance to the right PTS in decimal.
In the end the right P-Frame header should have bit distance and PTS distance = 0, like it is the case here.

and the secound script is
tsheaderCCcheck.py
It also takes the file name as an argument.

it gives 4 blocks of information:
First the readin part like above
Next is about the time_increment_bits(TTB) part of the VOP-header.
It list for both I-Frame and P-Frame the position of the Header start; the first 5 byte after the startcode 0x000001b6; TTB in dec of the current Frame, the TTB in dec and then hex of what the next frame should be, TTB in dec and hex of what the last frame should be.

The last 2 parts are print outs of first the headers, and finally the tails of a Frame, at least that should be it. sometimes normal data packets are hidden in here if some ts-headers were not fixed correctly. So one has to look if there are only 20/21 headers and only tail packets with AF fields printed out.

Cheers
Shanuson
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/26/2014 09:40 pm
Some slight improvements to iframe 72. My relationship with this frame is a case of stockholm syndrome by now.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/26/2014 10:09 pm
I need some help.

I was working on part_13, the file from princess and mvpel.

There is one P-frame header that I can't find. (between 0x2f468 and 0x37374 in their file)

Attached you find the part of raw.ts that should include the missing P-frame. I starts with the previous P-frame (the one at  0x2f468 in the part file) and ends where the next P-frame begins (0x37374). Both PTS and the counter shows that there has to be an other P-frame between those two.
That part also includes a 56 byte shift, which i fixed by including 56 FFs and declaring one ts-packet a null packet. (it was actually 132 bytes too many for the CC of all PID 0x3e8 packets). The attached file does not include this fix.

The missing P-frame header is:
47 43 E8 3x 07 10 00 36 d5 71 7E 00 00 00 01 E0 00 00 81 80 07 21 01 B7 07 A5 FF FF 00 00 01 B6 5x

There is a bunch of FF's in the middle of the file (pos 0x4150) which i thought are part of the null packet. ( I added the 56 FFs here, so they are not part of a data-packet.) but maybe not.

Maybe someone of you can point out where that header should be.

Cheers
Shanuson
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/26/2014 10:21 pm
Also some good news:

Here are part 11 and part 14 based on the work of deruch and princess.

Cheers
Shanuson
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/27/2014 12:36 am
We are slowly adding frames to the movie, so I put an updated version. Big thanks in particular to the anonymous people who cleaned part 248!

97 frames (some of them only very partially recovered) out of ~270 were used.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: S.Paulissen on 05/27/2014 01:08 am
Wow, Swiss. That video is looking pretty awesome.  Those legs coming out is getting silky smooth, as is the engine fire/splashdown.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/27/2014 01:11 am
The missing P-frame header is:
47 43 E8 3x 07 10 00 36 d5 71 7E 00 00 00 01 E0 00 00 81 80 07 21 01 B7 07 A5 FF FF 00 00 01 B6 5x

The packet at the end of the range you mention, at 0x37374, has a PTS of 79.67960, while the earlier one has a contiguous PTS of 79.612866.

I'm looking over what I uploaded using Wireshark, filtering on mp2t.af.pcr_flag == 1. It's recognizing the following PTS values:

0 78.81206 PCR base = 2132727600
 1 78.87880
 2 78.94553
 3 79.01226
 4 79.0790
 5 79.14573
 6 79.21246
 7 79.27920
 8 79.34593
 9 79.41266
10 79.47940
11 79.54613
12 79.61286
13 79.67960
14
15 79.81306
16 79.87980
17 79.94653
18 80.01326


The only packet between 13 and 15 which has the PCR bit turned on looks like this:

0x37664 : 47 03 e8 37 80 b2 f1 13 0c 83 2b 2d 50 b5 e9 12 34 34 78

This is obviously wrong, since it only appears four TS packets after the previous PTS of 79.6796 and I'm pretty sure a pframe can't be that small, aside from the fact that it has ES priority and transport-private bits turned on which is incorrect. So that's not the one. And there doesn't seem to be enough room in that space to fit another pframe. Are the PTS numbers correct across this file and into the next and prior ones? Maybe something tripped over this packet, or the missing one, and got confused.

Anyway, based on the rough number of TS packets per pframe, I'm thinking that our most likely candidate is the TS packet at immediately following the null packet, at offset 0x33680. I know that null packets sometimes appear in the midst of pframes, but this one is far enough down that it seems like it might be padding between frames. And here's the actual bytes as compared to the expected pframe header:

47 03 E8 10 03 B2 05 5E 38 49 FB C9 2F 0E BC D0 73 CF E1 F8 92 0D 44 BE 5C 8B F8 F7 F1 F0 F4 DC 02
47 43 E8 3x 07 10 00 36 d5 71 7E 00 00 00 01 E0 00 00 81 80 07 21 01 B7 07 A5 FF FF 00 00 01 B6 5x


It's kind of a thin reed, I know, but perhaps mashing the header in there is worth a try to see if a pframe pops out. The PTS value of the next frame would need to be adjusted, as I discussed above, so that this one would get 79.67960 and 13 (above) would get 79.74633.

Edit: It also occurs to me that the bytes at the beginning of the null frame are far enough off from FF that they might be the end bytes of the previous pframe, and so the null packet there should be converted into a 0x4703e83x header and the bytes pushed to the end of the packet with an appropriate amount of stuffing.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/27/2014 01:15 am
97 frames (some of them only very partially recovered) out of ~270 were used.

Wow, fantastic!

Since SpaceX actually designed and built the rocket, I'm thinking they should get top billing, right?  ;D
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: deruch on 05/27/2014 02:12 am

The missing P-frame header is:
47 43 E8 3x 07 10 00 36 d5 71 7E 00 00 00 01 E0 00 00 81 80 07 21 01 B7 07 A5 FF FF 00 00 01 B6 5x


I've looked, but can't seem to find it either.  Though I think PTS should be B6 07 A5.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: dorkmo on 05/27/2014 02:42 am
97 frames (some of them only very partially recovered) out of ~270 were used.

Wow, fantastic!

Since SpaceX actually designed and built the rocket, I'm thinking they should get top billing, right?  ;D

shoutout to wikispaces.com too. i bet theyre seeing quite a load of traffic.

ps is anyone backing up all the data?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/27/2014 08:25 am

The missing P-frame header is:
47 43 E8 3x 07 10 00 36 d5 71 7E 00 00 00 01 E0 00 00 81 80 07 21 01 B7 07 A5 FF FF 00 00 01 B6 5x


I've looked, but can't seem to find it either.  Though I think PTS should be B6 07 A5.

The last bit of the B7 byte has to be a 1 (it is a marker bit) so it is a 7.

I think i will take mvpels suggestion for the location.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: princess on 05/27/2014 11:46 am
Here is my cleanup of part 10 of the split TS file. If Shanuson, mvpel and the others can have a look and let me know if I've missed anything, that would be great.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/27/2014 12:13 pm
Here is my cleanup of part 10 of the split TS file. If Shanuson, mvpel and the others can have a look and let me know if I've missed anything, that would be great.

Pretty good so far!

It looks like you may be leaving some of the bits in the header on - at offset 0x5F78  and 0x15C18, for example, the header is 0x4723e81x, which indicates that the transport priority bit is on. There's bits in the TS header which we know should always be turned off: the transport error indicator, transport priority, and scrambling control. Insuring that these are always off not only avoids problems if a decoder takes the TEI or scrambling seriously and skips the packet as a result, but it also makes it easier to visually scan the file in an editor to see that the headers are consistent, and makes pattern matching more straightforward.

This page shows the exact layout of the header: https://en.wikipedia.org/wiki/MPEG_transport_stream
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/27/2014 12:54 pm
Here is rawsplit_part_2_fixed.ts again this time with all TS-data packets headers and stuff fixed.

I will now check all those parts that are submitted by others,

Done is part: 1,2,7,8,15

Part 7 now has a (nearly empty) p-frame at the end (thus 1 i-frame and 20 p-frames in the file!), otherwise if we discard it, it looks good, part 8 also has now the right number of p-frames and could directly be put in the online editor.

Could you check a last time the end of part 7?

Thank you for everything!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/27/2014 02:45 pm
Yes some parts still have one overlapping TS packet at the beginning and the end.
Dont worry about that.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/27/2014 03:34 pm
We are slowly adding frames to the movie, so I put an updated version. Big thanks in particular to the anonymous people who cleaned part 248!

97 frames (some of them only very partially recovered) out of ~270 were used.

Wow. Leg deploy!

Amazing work guys. Every day is a fun day when visiting this thread!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/27/2014 03:36 pm
Hi guys!

Can I suggest us to take a minute in order to outline our roadmap ahead? My feeling is that there is a lot of clarity on the current tasks, but little has been discussed on what to do when we are done with the macroblock fixes (besides some discussion about how to edit and release the video)

This is my (very preliminary) proposal:

Phase 1: .ts file and MMB fixes (ongoing)
- Finishing the process of fixing frame headers and Extracting the missing p-frames (thanks to Shanuson,  princess, deruch and mvpel for the amazing work so far!)
- Fixing frames (thanks to everyone that has contributed. And in particular to SwissCheese that has been delivering inputs every day). 

Phase 2: Further improvements (the next logical step: video experts, help me out!)
- I'm assuming there are many other things we could do to improve the video even further. Possible Interpolation of frames in order to fix frames or obtain a smother video? Maybe even a 24fps one?

Phase 3: Release and Distribution
- Define license for distribution
- Edit final version of the video (previous suggestions would go here).
- Write articles to go with the video: Maybe one general one to present the results, and a technical one to describe the video recovery process
- Formally release the video (possibly by publishing the articles as if they where a press release, and the video in the may NSF homepage).


Comments? Suggestions? Ideas?  ;)

Edit: Added a Roadmap Page in the Wiki: http://spacexlanding.wikispaces.com/Roadmap
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: gospacex on 05/27/2014 03:43 pm
Phase 3: Release and Distribution
- Define license for distribution

I think Chris needs to ask his SpaceX contacts for a reasonably permissive license for the video.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/27/2014 03:48 pm
Phase 3: Release and Distribution
- Define license for distribution

I think Chris needs to ask his SpaceX contacts for a reasonably permissive license for the video.

I can ask about that. If someone wants to post a question to send them, I'll fire that their way.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: CraigLieb on 05/27/2014 03:55 pm
Many of the lurkers on the thread would love to contribute.
 
Is there some possibility of expanding the eyeball set so that we can "vote" for what constitutes a better frame out of tens or hundreds of possibilities. For example you smart people coders would generate images but would not have to analyze each of them. Instead feed the large collection  to a bunch of us of folks volunteering as "lookers" who could vote on the best looking versions a particular frame ( maybe with side by side comparisons).  you free yourselves up to generate the pictures, and then the top vote-getter images could go back to the author for a definitive version pick.

 Not sure this is helpful, but if it makes the process quicker and leverages more brain and eye power, it could help. Yes?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: gospacex on 05/27/2014 03:59 pm
Phase 3: Release and Distribution
- Define license for distribution

I think Chris needs to ask his SpaceX contacts for a reasonably permissive license for the video.

I can ask about that. If someone wants to post a question to send them, I'll fire that their way.

That *is* the question: "Please define under what license do you allow the repaired video to be distributed".
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/27/2014 04:00 pm
Phase 3: Release and Distribution
- Define license for distribution

I think Chris needs to ask his SpaceX contacts for a reasonably permissive license for the video.

I can ask about that. If someone wants to post a question to send them, I'll fire that their way.

That *is* the question: "Please define under what license do you allow the repaired video to be distributed".

Oh, ok! :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Eer on 05/27/2014 04:15 pm

I have one contact in the communications/PR group there, and will connect with them to see if we can get SpaceX to formally clarify the original file on the raw.ts file that they made available, and which is serving as the master file for all the great video tech-heads who are doing so much to cleanup the stream and fix errors so we all get the historical view into this video data that this first in the history of human technology deserves:  a successful vertical soft-landing of an orbital booster stage rocket.

That would be awesome. Please ask your contact to see if the original can be released CC-BY-SA ... please avoid NC and ND!!!! Version 4 would be best I think. Here's more on the license.

http://en.wikipedia.org/wiki/Creative_Commons_license

I don't know if a barrage of requests would be useful but maybe asking via another channel or two might be good.

Chris - see the above message for license discussion and expressed preferences.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mme on 05/27/2014 04:16 pm
...
I can ask about that. If someone wants to post a question to send them, I'll fire that their way.

That *is* the question: "Please define under what license do you allow the repaired video to be distributed".

Oh, ok! :)

As has been discussed earlier (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1199979#msg1199979), it may be a good idea to suggest a license, and as suggested creative commons (http://creativecommons.org/choose/) is nice and permissive.

Edit: Fixed quotes.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Lar on 05/27/2014 04:30 pm
Phase 3: Release and Distribution
- Define license for distribution

I think Chris needs to ask his SpaceX contacts for a reasonably permissive license for the video.

I can ask about that. If someone wants to post a question to send them, I'll fire that their way.

That *is* the question: "Please define under what license do you allow the repaired video to be distributed".

Oh, ok! :)

Please be more specific... "can we have it licensed under CC-BY-SA version 4, please" and "WITHOUT the NC or ND clauses attached, please"

I have some expertise in this licensing area having worked with a lot of images on Wikimedia Commons, and the Creative Commons license family is probably the best.

- we want BY to allow us to modify the original (recovery is a kind of modification) and redistribute derivative works
- we want SA to require that any derivative work continue to carry the same license (this protects SpaceX and NSF rights in case someone makes a fake) and that it be identified as to the source.
- we do NOT want NC because that restricts commercial ... arguably "commercial" is rather broad and NSF itself is "commercial" so we want to avoid NC
- we do NOT want ND because that restricts redistribution, we want to be able to redistribute and enable others to redistribute too... SA means that attribution info HAS to be carried along


Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/27/2014 04:50 pm
However, "NC" may be desirable because we wouldn't necessarily want, say, Lockheed or Boeing using this SpaceX footage in an advertisement, even if they did attribute it to SpaceX in some way.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Lar on 05/27/2014 04:54 pm
However, "NC" may be desirable because we wouldn't necessarily want, say, Lockheed or Boeing using this SpaceX footage in an advertisement, even if they did attribute it to SpaceX in some way.

Using NC to control that is a very blunt instrument... as I said someone could argue that NSF posting it, is itself a commercial use.

Wikipedia avoids tagging things NC

Better for SpaceX to go back at this putative competitor in other ways. The truth usually works best.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/27/2014 05:02 pm
Looks like the current version of online editor for p frames cannot handle very long mmbs (like iframe72). That's unfortunate.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/27/2014 05:45 pm
Swiss and/or Shanuson, can you tell us which of the .ts files are currently "stable" and ready for mb fixing? I took a look at frame 131. The fixed_edit8_part_131.ts presumably gives < 19 frames. I downloaded rawsplit_part_8_fixed.ts and found that the mb fixes on http://spacexlanding.wikispaces.com/Frames+131-149 resulted in a scrambled image. I can go ahead and port the fixes to rawsplit_part_8_fixed.ts, but I want to make sure that is our more-or-less final version before I put more time into it.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/27/2014 06:00 pm
This is just a proof of concept. I deciphered a few of the timestamps on the images and interpolated the rest. Due to interlacing and propagating errors, even if we get good mbs in the pframes the timestamps are still going to be gibberish most of the time. So, I propose we just redraw them. Any concerns about this?

A bit late, but I really like this idea. How would you proceed wronkiew? Is it something that can be done in parallel with the MMB task? or we should wait until we have a full reconstructed video?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/27/2014 06:11 pm
Swiss and/or Shanuson, can you tell us which of the .ts files are currently "stable" and ready for mb fixing? I took a look at frame 131. The fixed_edit8_part_131.ts presumably gives < 19 frames. I downloaded rawsplit_part_8_fixed.ts and found that the mb fixes on http://spacexlanding.wikispaces.com/Frames+131-149 resulted in a scrambled image. I can go ahead and port the fixes to rawsplit_part_8_fixed.ts, but I want to make sure that is our more-or-less final version before I put more time into it.

rawsplit_part_8_fixed.ts will be used to create a final raw file. at least that is my idea.
Also all files currently active could still/will be modified before put back into a final raw file.
Working on the 6 Ts files I assigned _fixed to should be save.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: somepitch on 05/27/2014 06:23 pm
Many of the lurkers on the thread would love to contribute.
 
Is there some possibility of expanding the eyeball set so that we can "vote" for what constitutes a better frame out of tens or hundreds of possibilities. For example you smart people coders would generate images but would not have to analyze each of them. Instead feed the large collection  to a bunch of us of folks volunteering as "lookers" who could vote on the best looking versions a particular frame ( maybe with side by side comparisons).  you free yourselves up to generate the pictures, and then the top vote-getter images could go back to the author for a definitive version pick.

 Not sure this is helpful, but if it makes the process quicker and leverages more brain and eye power, it could help. Yes?

I second this, would love to help in any way I can but don't have the technical skills for the current methods
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/27/2014 06:47 pm
This is just a proof of concept. I deciphered a few of the timestamps on the images and interpolated the rest. Due to interlacing and propagating errors, even if we get good mbs in the pframes the timestamps are still going to be gibberish most of the time. So, I propose we just redraw them. Any concerns about this?

A bit late, but I really like this idea. How would you proceed wronkiew? Is it something that can be done in parallel with the MMB task? or we should wait until we have a full reconstructed video?

My current thinking on this is that rather than putting a duplicate timestamp in a matte, I'll redraw the known timestamps using the bitmap numerals already displayed. No interpolated timestamps added.

When I get a chance I also want to put together a system that automatically generates png sequences from the mmbs, zips them up, and uploads them to the wiki. That way anyone can download the frames and do their own batch processing on them.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Lourens on 05/27/2014 07:11 pm
Swiss and/or Shanuson, can you tell us which of the .ts files are currently "stable" and ready for mb fixing? I took a look at frame 131. The fixed_edit8_part_131.ts presumably gives < 19 frames. I downloaded rawsplit_part_8_fixed.ts and found that the mb fixes on http://spacexlanding.wikispaces.com/Frames+131-149 resulted in a scrambled image. I can go ahead and port the fixes to rawsplit_part_8_fixed.ts, but I want to make sure that is our more-or-less final version before I put more time into it.

rawsplit_part_8_fixed.ts will be used to create a final raw file. at least that is my idea.
Also all files currently active could still/will be modified before put back into a final raw file.
Working on the 6 Ts files I assigned _fixed to should be save.

Would it be possible to have a fixed .ts that is directly comparable to the original raw.ts? Or very specifically, on the one hand our best guess at the correct stream so far, and on the other hand the original raw.ts with apparent holes (the 56-byte gaps that were found here and there) filled with zeros?

I've been thinking about any error correction that may have been applied to the stream, inspired by ajmartin's post (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1201383#msg1201383). There's a Reed-Solomon code that packs 28-byte chunks of data into 32-byte code words and can repair up to two broken bytes, and detect more than that. That might explain the 28-byte or 56-byte pattern and the gaps (error detected, packet dropped), but not the bit-pattern ajmartin mentioned. It seems to be pretty common to have two nested codes however, so perhaps there's a bit-level code within that. Anyway, detecting these kinds of patterns would be easier to attempt if we had a raw and a fixed stream side-by-side to compare. I've tried with rawsplit_part_2.ts and rawsplit_part_2_fixed.ts, but as I understand it the rawsplit_part_2.ts is already an extract from a patched-up file.

Alternatively Chris, if SpaceX can answer the question "Which (forward) error correction mechanisms were used in getting the TS stream from the camera to the web?" that might be helpful too.

The reason that this is important is that we'll get to a point where we have all the undamaged macroblocks located and put into the right place, and we want to try to improve the broken ones. That amounts to flipping bits and seeing what the result looks like, but there are problems with that:

1) There are a lot of possible combination to try. If an error correction code was used then that information could possibly allow us to discard some of them (e.g. if a single-error correcting code was used at the bit level, then any single-bit flips will have already been fixed and don't need to be tried, and there may be a more limited number of combinations of flipped bits to check).

2) It's difficult to figure out whether the result is any good. Of course it should decode without errors, but there are lots of combinations of bits that give a correctly decodable block, and only one correct one. We can weed out some bad ones by seeing if the block matches up with neighbouring blocks (I call it the jigsaw-check), and even better is just looking at it and seeing if it makes sense. Having a lot of people willing to judge potential solutions will be very helpful in that stage.

3) If the block is damaged badly enough, you end up flipping so many bits that you're effectively just painting in the gaps. If it gets to that, then there are much better ways of doing so than flipping bits and digging through the results. Knowing the FEC mechanism may make it easier to limit the search to small errors only, so that if something usable pops out there's still a reasonable chance that it's really the original block.

Finally, maybe there should be a separate thread for how to release the results, licensing, presentation, etc.? It's becoming a fairly big topic, and I'm sure it's only going to grow. Then we can keep this one for technical discussion on the repair process.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/27/2014 07:22 pm
Swiss and/or Shanuson, can you tell us which of the .ts files are currently "stable" and ready for mb fixing? I took a look at frame 131. The fixed_edit8_part_131.ts presumably gives < 19 frames. I downloaded rawsplit_part_8_fixed.ts and found that the mb fixes on http://spacexlanding.wikispaces.com/Frames+131-149 resulted in a scrambled image. I can go ahead and port the fixes to rawsplit_part_8_fixed.ts, but I want to make sure that is our more-or-less final version before I put more time into it.

I hope it's the final version, but you can never be sure it seems :D

I also started earlier fixing the i-frame (there is more data, so the bit positions have mostly just to be shifted), but without the data available on the online editor it takes much more time (counting the squares to find the good positions instead of just clicking on the image...), so I stopped and went back to the other parts :P

I attach the mmb sequence where I stopped, if you want to continue fixing it.

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: princess on 05/27/2014 08:03 pm
OK, here is my second attempt at the part 10 TS file after feedback from mvpel. I've managed to recover an extra P-frame (starting at offset 0x00021528 into the file) and I've set the PCR, PTS and MPEG4 headers correctly as far as I can see. Feedback would be much appreciated!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: princess on 05/27/2014 08:12 pm
And as an extra bonus for all you lovely people, here is my fixed up Part 9 TS file. I've recovered two P-frames, corrected their PCR and PTS timestamps, and set the MPEG4 headers so that ffmpeg can find them. As usual, let me know your feedback!

It would be great if one of the -mmb wizards (SwissCheese?) could take a look and see if it makes a difference to their efforts, as that's a bit out of my league...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: princess on 05/27/2014 08:38 pm
Here's my fixed Part 6 TS file. This one was fairly straightforward with no new P-frames uncovered, so this is less of a priority.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mhenderson on 05/27/2014 08:49 pm
My $.02. Perhaps I am being naive, but here's the case for NOT requesting a license:

The original post at spacex.com reads as follows:
"We welcome anyone to download the raw file to make improvements directly—if you are successful, please feel free to post your video at: http://www.reddit.com/r/spacex/comments/24bsn2/first_stage_landing_video/. "

The source files have been downloaded with no license restriction thousands of times.

Reddit.com is a commercial website. Republishing and some commercial use of derivative work is allowed.

Folks from Hawthorne have actively participated in this forum. NSF is a commercial website. (See those Site Booster ads >>>)

Work in progress is openly posted to the wiki. No cease and desist letter so far.

Requesting a license == inviting lawyers to the party. Not a good idea without deep pockets.

This ship set sail five weeks ago and has met / exceeded the highest hopes and expectations. Nobody's gonna risk a Streisand Effect by squelching this little exercise in fan fixin'. 

Edit:  Oh yeah, I neglected to mention the CEO of SpaceX tweeted a link to a copy of our derivative work and publicly said "Great progress by @NASASpaceflight members repairing the Falcon 9 ocean landing video". I think we got a green light there.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: princess on 05/27/2014 09:05 pm
I'm on a roll tonight! Here is my fixed Part 5 TS file. One of the P-frame headers was slightly corrupt and the PTS timestamp was wrong, so I've corrected that too. It probably won't affect ffmpeg that much but it's nice to have the timestamps correct anyway.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: princess on 05/27/2014 10:14 pm
Another TS file! Here's part 4 fixed up. A few P-frame fixes, mainly timestamps and the odd bit of header corruption.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: princess on 05/27/2014 10:59 pm
Finishing up the unclaimed TS parts from the wiki, here is part 3 fixed up. It had a lot of corruption and a few P-frames have had timestamps and headers corrected.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: princess on 05/27/2014 11:29 pm
Two new P-frames in part 11!

After tonight's fixing marathon, I decided to go back over my part 11 attempt and I found the two extra P-frames that I was missing before. I've attached the fixed part 11 below. If the mmb / ffmpeg experts could give it a look, I'd be most grateful.

I hope everyone didn't mind me posting lots of results - I've just written some custom tools in C++ that help me find the problems, calculate timestamps and so on. It's made it a lot quicker to fix up the TS problems and find P-frames hiding in the corruption. I'm going to turn in for the night now, so it'll give you all a chance to have a look at all the fixed up things and see what you can make from them!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/28/2014 01:13 am

Folks from Hawthorne have actively participated in this forum. NSF is a commercial website. (See those Site Booster ads >>>)


Kinda. No one is earning any money off this thing (as much as we should be given the size of the place now). I think I'm personally $30K down since starting it....don't worry folks, we're self sustaining now, just about.

Oh and I'd do it all again if I was given a time machine. Though probably not by starting it by maxing out a card to get it built! ;D

Quote

Edit:  Oh yeah, I neglected to mention the CEO of SpaceX tweeted a link to a copy of our derivative work and publicly said "Great progress by @NASASpaceflight members repairing the Falcon 9 ocean landing video". I think we got a green light there.

Yep, that's a good call. Focus should continue to be on the fine work in fixing the video. I'll ensure the rest, but the non official response I got first of all today was "LOL! You're doing us a favor!" <---seriously! :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Lee Jay on 05/28/2014 01:26 am
Exactly.  When I thought of SpaceX exerting some sort of rights over this effort, the expression that came to my mind was "biting the hand that feeds you".  So glad to hear they are being entirely rational about the whole thing!

The effort is amazing, by the way.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/28/2014 02:09 am
Wow, many updates!
Guys: if you need to add new files to the online editor, just let me know :) I can gladly take care of all the arrangements :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: deruch on 05/28/2014 03:25 am
Wow Princess!  Charger day, huh?

(http://i.imgur.com/sbgg3z2.png)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/28/2014 03:35 am
Two new P-frames in part 11!

After tonight's fixing marathon, I decided to go back over my part 11 attempt and I found the two extra P-frames that I was missing before. I've attached the fixed part 11 below. If the mmb / ffmpeg experts could give it a look, I'd be most grateful.

I hope everyone didn't mind me posting lots of results - I've just written some custom tools in C++ that help me find the problems, calculate timestamps and so on. It's made it a lot quicker to fix up the TS problems and find P-frames hiding in the corruption. I'm going to turn in for the night now, so it'll give you all a chance to have a look at all the fixed up things and see what you can make from them!
Hi princess,

I would like to wish you a good sleep. I liked your marathon. :)

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/28/2014 03:48 am
Hi guys,

My "macroblock data finder" kinda works, but it doesn't seem to be much more effective than what we have already done.

You can try it out (its attached to the post). You will have to edit  the variables $filename_ts and $ffmpeg_binary in order for it to do what you want. It outputs an mmb and puts -2s between the "winner" mb-sequences it found. Notice this x,y positioning is just so the sequences dont overlap and many times the y goes over 29. This is only useful for finding new mb-data, not for positioning them or for auto generating mmbs. Its fixed to frame 0 (I-frame) but you can change that in the code.

So back to the drawing board...

I'm currently working on ways to recover bad macroblocks. Or at least finding their starting positions. I've still got some ideas. ;)

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Llian Rhydderch on 05/28/2014 03:54 am

Folks from Hawthorne have actively participated in this forum. NSF is a commercial website. (See those Site Booster ads >>>)


Kinda. No one is earning any money off this thing (as much as we should be given the size of the place now). I think I'm personally $30K down since starting it....don't worry folks, we're self sustaining now, just about.

Oh and I'd do it all again if I was given a time machine. Though probably not by starting it by maxing out a card to get it built! ;D

Quote

Edit:  Oh yeah, I neglected to mention the CEO of SpaceX tweeted a link to a copy of our derivative work and publicly said "Great progress by @NASASpaceflight members repairing the Falcon 9 ocean landing video". I think we got a green light there.

Yep, that's a good call. Focus should continue to be on the fine work in fixing the video. I'll ensure the rest, but the non official response I got first of all today was "LOL! You're doing us a favor!" <---seriously! :)

I absolutely agree, Chris, that the NSF team here are doing SpaceX a very large favor.  And wonderfully, and somewhat counter-intuitively, the way this works is that those people volunteering so much good time here are also receiving benefits of various sorts from doing the work.*

But on the license question, I do thinkit is a very good idea to "make the implicit, explicit".  They know, and we know, (and channeling Dick Cheney) they know that we know and we know that they know.  But all of that is in our heads; it's implicit.

Best to make it explicit with the license Lar suggested earlier:  "can we have it licensed under CC-BY-SA version 4, please" and "WITHOUT the NC or ND clauses attached, please" — and for the reasons he articulated so well.

Best,
 Llian

* fun, bragging rights, the opportunity to contribute to something that is meaningful, etc.—heck, I see some of the video guys able to write a book about the topic based on all that was learned here in this project.  All of this is common in the open source world, and literally millions of person hours are donated to such efforts each month.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/28/2014 05:52 am

My current thinking on this is that rather than putting a duplicate timestamp in a matte, I'll redraw the known timestamps using the bitmap numerals already displayed. No interpolated timestamps added.

When I get a chance I also want to put together a system that automatically generates png sequences from the mmbs, zips them up, and uploads them to the wiki. That way anyone can download the frames and do their own batch processing on them.

Ok, the first part of this is done. Unfortunately in PNG format the zip file is 85 MB, so trying to auto-upload it to the wiki is probably pushing it. No pun intended. Instead, I put it on Google Drive at this url:

https://drive.google.com/file/d/0B_wTAwfYg03FQ3p2UVlkZE9MckU/edit?usp=sharing

Also on BTSync at B76OWSNRL5HWE47N7SHT4ZIX2B3B63IXT

This is ~250 frames in PNG format generated by scraping the wiki and then running it through ffmpeg. Why is this useful? Well, you can turn these into a movie file with this command:

ffmpeg -y -start_number 40 -i landing_base_set/frame-%03d.png -c:v libx264 -r 15 -pix_fmt yuv420p out.mp4

Also, you can run a batch process on these images to do post-processing or add annotations.

The zip archive will be regenerated every hour if the wiki has seen any updates.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/28/2014 06:00 am

Ok, the first part of this is done. Unfortunately in PNG format the zip file is 85 MB, so trying to auto-upload it to the wiki is probably pushing it. No pun intended. Instead, I put it on Google Drive at this url:

I should mention that I padded the frame sequence to reserve space for extra p-frames. Here is the mapping:

32 -> 40
52 -> 60
72 -> 80
92 -> 100
112 -> 120
131 -> 140
150 -> 160
169 -> 180
189 -> 200
209 -> 220
229 -> 240
248 -> 260
268 -> 280
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/28/2014 06:32 am

Best to make it explicit with the license Lar suggested earlier:  "can we have it licensed under CC-BY-SA version 4, please" and "WITHOUT the NC or ND clauses attached, please" — and for the reasons he articulated so well.

I think I would like to see the original video licensed CC-BY-SA. The reason is that this would allow Wikipedia to use the final video for educational purposes. Wikimedia has sample letters to use for requesting a free license for content. I have used this to successfully request CC licenses from SpaceX in the past, though they were a smaller company then.

See http://commons.wikimedia.org/wiki/Commons:Email_templates
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: princess on 05/28/2014 07:45 am
I would like to wish you a good sleep. I liked your marathon. :)

Thank you (and deruch!) for your kind words! My plan now is to go through all the fixed TS files we have and see if we're missing any more P-frames. The two timestamps are very regular: the PTS always lags the PCR by exactly 10,000 ticks, and each frame (I-frame or P-frame) is always 6006 ticks after the last one. We can use this to see if there are any "holes" in the timestamp sequencing, which would imply a missing P-frame.

Once we've got every P-frame and set their timestamps correctly, Shanuson should then be able to give the whole lot a final check and hopefully declare the TS files stable. Then we will need the ffmpeg wizards to port their -mmb changes to the new files.

The -mmb lines will need changing because part of the TS recovery work allows us to pull out more of the MPEG4 stream. In some places, TS-level corruption causes part of the valid data stream to be skipped as padding. Once we have the fixed TS files, you'll all have more macroblocks (and more corruption!) to fix.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/28/2014 07:50 am
Two new P-frames in part 11!

After tonight's fixing marathon, I decided to go back over my part 11 attempt and I found the two extra P-frames that I was missing before. I've attached the fixed part 11 below. If the mmb / ffmpeg experts could give it a look, I'd be most grateful.

I hope everyone didn't mind me posting lots of results - I've just written some custom tools in C++ that help me find the problems, calculate timestamps and so on. It's made it a lot quicker to fix up the TS problems and find P-frames hiding in the corruption. I'm going to turn in for the night now, so it'll give you all a chance to have a look at all the fixed up things and see what you can make from them!

Hi Princess, thank you for all that work.

i looked into part_11:
The following P-Frame headers are not 100% correct:

Pos - bit distance from the correct header - how the header looks lile - PTC distance in dec
bit distance and PTC distance should be 0.

You identified all correct positions, but correct the complite header it seems. We now what it should look like since all those bits in the header have to be the same for all headers. (except the counter and PTS part - but you got that one correct).

trying to find header: 2
          -> ['47 43 E8 3x 07 10 00 34 73 75 7E 00 00 00 01 E0 00 00 81 80 07 21 01 A3 7F B5 FF FF 00 00 01 B6 5x ']
0x7870 6 -> ['47 43 e8 37 07 10 00 34 73 75 00 00 00 00 01 e0 00 00 81 80 07 21 01 a3 7f b5 ff ff 00 00 01 b6 52 '] 0
0xb330 11 -> ['47 43 e8 3f 07 10 00 34 7f 30 7e 00 00 00 01 e0 00 00 81 80 07 21 01 a3 ae a1 ff ff 00 00 01 b6 53 '] 6006


trying to find header: 13
          -> ['47 43 E8 3x 07 10 00 34 f4 7e 7E 00 00 00 01 E0 00 00 81 80 07 21 01 A7 83 D9 FF FF 00 00 01 B6 5x ']
0x2f98c 11 -> ['47 43 e8 3c 07 1b 00 34 f4 7e 00 00 00 00 01 e0 00 00 81 80 07 20 01 a6 83 d9 ff ff 00 00 01 b6 5a '] 0
0x415c 14 -> ['47 43 e8 34 07 10 00 34 67 ba 7e 00 00 00 01 e0 00 00 81 80 07 21 01 a3 50 c9 ff ff 00 00 01 b6 51 '] -72072


trying to find header: 14
          -> ['47 43 E8 3x 07 10 00 35 00 39 7E 00 00 00 01 E0 00 00 81 80 07 21 01 A7 B2 C5 FF FF 00 00 01 B6 5x ']
0x162b4 13 -> ['47 43 e8 34 07 10 00 34 a2 61 7e 00 00 00 01 e0 00 00 81 80 07 21 01 a5 3b 65 ff ff 00 00 01 b6 55 '] -48048
0x3344c 13 -> ['47 43 e8 34 07 d0 00 35 00 39 00 00 00 00 01 e0 00 00 81 80 0a 21 01 a7 b2 c5 ff ff 00 00 01 b6 68 '] 0


also my CC continuity check failed at pos: 0x2F98C (CC is = C in your file but should be a 4)

and finally there are 56 times where a TS header that starts with 4703e8 continues not with 1X but something else. This is ok for the last TS-packet of a frame when the frame data does not fill the 184 byte completely.
I attached a file where all those 56 parts are listed and marked the correct tails. all the other packets should be 4703e81x.

I hope this helps to improve your tools further.
Btw. I put a part_11.ts file online yesterday or the day before that was fixed. You can use that to compare your version to it.

Thanks again for all the hard work. Even the final edit on your versions will be much faster than starting from raw... So your work is much appreciated anyways.

I will check the rest of your upload tonight. Have to go to work now.

Cheers
Shanuson
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/28/2014 07:53 am

Best to make it explicit with the license Lar suggested earlier:  "can we have it licensed under CC-BY-SA version 4, please" and "WITHOUT the NC or ND clauses attached, please" — and for the reasons he articulated so well.

I think I would like to see the original video licensed CC-BY-SA. The reason is that this would allow Wikipedia to use the final video for educational purposes. Wikimedia has sample letters to use for requesting a free license for content. I have used this to successfully request CC licenses from SpaceX in the past, though they were a smaller company then.

See http://commons.wikimedia.org/wiki/Commons:Email_templates

That's a good idea! Because all this is really educational.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: CuddlyRocket on 05/28/2014 08:28 am
Focus should continue to be on the fine work in fixing the video. I'll ensure the rest, but the non official response I got first of all today was "LOL! You're doing us a favor!" <---seriously! :)

I wonder if Elon will mention this at the new Dragon reveal Thursday night? Or perhaps they'll show the latest version to the crowd whilst they're waiting for the main event of the evening to start?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Oersted on 05/28/2014 10:55 am
Thanks again for all the hard work. Even the final edit on your versions will be much faster than starting from raw... So your work is much appreciated anyways.

It is a normal part of any work process that error checkers comb through the large job of any participant. It is good to see that you guys (and princesses) have informally set up a good-natured, friendly system for improving on each others' work. A new set of eyes always see a few surprises... 
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/28/2014 11:53 am
Focus should continue to be on the fine work in fixing the video. I'll ensure the rest, but the non official response I got first of all today was "LOL! You're doing us a favor!" <---seriously! :)

I wonder if Elon will mention this at the new Dragon reveal Thursday night? Or perhaps they'll show the latest version to the crowd whilst they're waiting for the main event of the evening to start?

Thursday is going to be more of a Hollywood style event. Some of the rumors I'm hearing are absolutely crazy for this reveal. I don't think a technical effort on an ocean landing video would fit with Robert Downey Jr using robots and lasers to pull a sheet off a new spaceship. ;)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mhenderson on 05/28/2014 12:17 pm
Proof of concept, please?

I applaud the "header work" by mvpel, Shanuson, princess, et al. Bravo / Brava, my hat is off to you!

Progress is not always linear and forward. Sometimes it is necessary to step backward to find the best solution.

Will the updated .ts now cause rework of the mmp effort? We have seen remarkable progress and the cumulative result (per the mp4 that SwissCheese posted yesterday) is damn fine. Watching those legs bounce teaches a lot about the turbulent environment at the base of a falling rocket! And yet the vide leaves me begging for more / better. Color consistency, filling in the gaps, wronkiew's clock overlay...

Am I correct by inferring that header fixes - finding new macroblocks, removing errant "ignore this part" flags, and other logical corrections - may change POS values and render manual X: bitflips redundant?

If so, let's get a decision on the direction forward and switch horses. Meanwhile, stop any work on the legacy raw_edit8.ts and point the resources at the new, mobettah .ts.

A proof of concept comparing a few frames from each .ts would go along way to re-energizing an effort on the new stuff.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/28/2014 02:24 pm
Hi everyone,

One p-frame looks very strange, with "too intense" colors, could anyone come with an explanation and maybe a solution to this?

It's frame 227 / part 12-19

mmb: 0:0:-1,30:1:10681,10:3:-1,31:3:19630
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/28/2014 02:41 pm
Hi everyone,

One p-frame looks very strange, with "too intense" colors, could anyone come with an explanation and maybe a solution to this?

It's frame 227 / part 12-19

mmb: 0:0:-1,30:1:10681,10:3:-1,31:3:19630

I saw that too and wondered. Could it be a corrupted VOP quant in the frame header?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/28/2014 02:59 pm
One p-frame looks very strange, with "too intense" colors, could anyone come with an explanation and maybe a solution to this?

To me, looks too smooth and consistent to me to be a MPEG artifact. It seems that if it were an error in the encoding that it would be more blocky and vary more wildly, but it's a fairly steady sprinkle of red, green, and blue across the frame.

I'm wondering if it might be a burst of noise that affected the electronics of the NTSC camera itself and the RGB sensor, prior to the video signal reaching the digitizer and MPEG encoder.

If that's the case, and someone with more familiarity with video capture and encoding will want to comment on this, then it would seem that the only feasible workaround would be to ignore that pframe.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: princess on 05/28/2014 05:00 pm
my CC continuity check failed at pos: 0x2F98C (CC is = C in your file but should be a 4)
and finally there are 56 times where a TS header that starts with 4703e8 continues not with 1X but something else. This is ok for the last TS-packet of a frame when the frame data does not fill the 184 byte completely.
I attached a file where all those 56 parts are listed and marked the correct tails. all the other packets should be 4703e81x.
I hope this helps to improve your tools further.

Shanuson, thank you very much for that feedback, I have updated my tool to fix the issues you found, and I now feel it's ready for release.

I've decided to post my "tsrepair" program here for a few reasons - firstly, I feel bad about the amount of data we're uploading to the server here! Also it is hard to see the difference between edits, and it's hard to merge them too. It also makes it hard for us to crowd-source the work.

The tsrepair program accepts a "-fix" command line argument in a similar way as the "-mmb" option in ffmpeg. It allows people to take one of the TS files and apply some changes to it using a simple text string that can be posted here or on the wiki. Example usage is like this:

tsrepair rawsplit_part_9.ts -fix:203,noaf/205,pay,8 rawsplit_part_9_fixed.ts > fix.txt

The first filename is the name of the input file, the second filename is the name of the output TS file. The program produces one line of output per packet, which in the example above is redirected to the file "fix.txt".

The program will take the input file, apply any manual fixes that you specify, and then run some automatic fix passes to do the bulk of the heavy lifting. This can be suppressed with the "-nofix" option. In fact, the way I work is to use the -nofix option to analyse the file first:

tsrepair rawsplit_part_9.ts -nofix > nofix.txt

Then run a repair pass (as detailed above), and show the differences between the two log files using a visual diff program like meld (on Linux) or WinDiff or something:

meld nofix.txt fix.txt

The program is written in C++ and I've developed it in Linux, but it should port easily to Windows for those people that need it.

The fix options that I currently have are:

Part 3:
-fix:114,pid,3e8/114,pay,5/114,noaf/133,noaf/160,pay,2/162,
pcr,5913898/162,ptsauto/162,pes/162,pframe/178,noaf/199,pid,3e8/199,pay,7/199,af,66/
199,nopcr/200,pid,1fff/200,pay,4/243,pid,1fff/243,pay,15/
255,noaf/257,noaf/321,pcr,5925910/321,ptsauto/323,pay,12/349,
noaf/364,pay,11/405,pes/405,ptsauto/405,pframe/
720,pes/720,pframe/815,noaf/945,noaf/947,pay,2/1124,ptsauto/1245,noaf[

Part 4:
-fix:195,noaf/326,pes/326,pcr,6046030/326,pframe/361,noaf/364,noaf/
402,pes/402,pframe/818,noaf/863,noaf/907,noaf/955,pid,1fff/956,pcr,6094078/1275,pes/1275,pcr,6118102/
1316,noaf/1387,noaf/1415,pid,1fff/1415,pay,7/1511,pay,15/1529,noaf/1530,noaf/1535,noaf

Part 5:
-fix:175,pid,3e8/175,pay,11/176,pay,12/311,pay,5/366,pid,3e8/419,noaf/
451,noaf/506,pid,3e8/506,pay,0/506,noaf/508,pay,1/535,af,137/654,noaf/715,ptsauto/
715,pframe/767,af,43/801,noaf/828,noaf/829,noaf/959,noaf/1047
,noaf/1057,noaf/1247,pid,1fff/1247,pay,8/1371,noaf/1434,noaf/1539,noaf

Part 6:
-fix:4,noaf/39,noaf/500,noaf/1042,pay,7/1042,pid,3e8/1043,pay,8/1043,pid,1fff/
1044,pid,3e8/1044,noaf/1044,pay,8/1046,noaf/1272,pay,15/1274
,pay,1/1462,noaf/1505,pid,1fff/1507,pid,3e8/1507,pay,12/1604,noaf/1622,noaf/1644,noaf


Part 7:
-fix:44,noaf/131,pay,7/131,pid,3e8/132,pay,8/351,noaf/419,pay,12/451,pid,1fff/
547,noaf/683,pay,9/741,noaf/759,noaf/789,noaf/826,noaf/955,pid,3e8/955,pay,5/956,pid,1fff/
956,pay,13/978,pid,1fff/978,pay,3/979,pid,3e8/
979,pay,6/979,pes/979,pcr,6460444/979,ptsauto/
979,pframe/1056,pid,1fff/1056,pay,11/1173,noaf/1328,pid,3e8/1328,pay,3/1329,pid,1fff/
1329,pay,1/1403,noaf/1466,noaf/1474,noaf/1493,noaf/1534,pid,1fff/1534,pay,9

Part 8:
-fix:0,pid,1fff/105,pid,03e8/106,pid,1fff/163,pay,10/163,pes/163,
pcr,6514498/163,ptsauto/163,pframe/187,noaf/412,pid,03e8/412,pay,4/412,noaf/601,noaf/816,pid,03e8/
816,pay,8/817,pid,03e8/818,pid,03e8/898,noaf/907,noaf/977,noaf/1158,pay,11

Part 9:
-fix:203,noaf/205,pay,8/227,noaf/371,noaf/400,pes/400,pcr,6652636/400,
ptsauto/400,pframe/478,pid,1fff/576,pay,2/576,noaf/691,pid,1fff/691,noaf/702,pid,1fff/
1115,pes/1115,ptsauto/1115,pframe/1119,noaf/1238,noaf/1285,noaf/1418,noaf

Part 10:
-fix:75,noaf/474,noaf/710,noaf/726,af,7/726,pcr,6796780/726,ptsauto/726,pes/
726,pframe/765,noaf/755,noaf/1037,pframe/1065,pay,6/1077,noaf

Part 11:
-fix:103,noaf/164,pes/164,pcr,6874858/164,pframe/887,noaf/920,noaf/962,noaf
/966,noaf/980,noaf/1037,pay,4/1037,pes/1037,pcr,6940924/1037,ptsauto/1037,pframe/
1065,noaf/1113,pid,1fff/1114,pay,7/1116,pay,9/1117,pes/1117,
pcr,6946930/1117,ptsauto/1117,pframe/1141,noaf/1322,pid,3e8/1324,pay,2/1324,noaf/1515,ptsauto

Part 14:
-fix:718,pes/718,ptsauto/719,pay,11/912,pid,3e8/912,pay,9/913,pid
,3e8/915,pay,12/1214,noaf/1240,pid,3e8/1241,pay,15/1274,pay,0/1322,pay,15/1505,pay,8/1506,pay,9

Part 15:
-fix:22,pid,3e8/27,pay,15/108,pid,3e8/108,pay,14/108,pes/
108,pcr,7349332/108,ptsauto/108,pframe/124,pid,3e8/127,pid,3e8

I'll update the Wiki and if anyone needs me to describe the -fix options I can do that too. Please take a look and let me know what you think. The code is in the public domain, no copyright is asserted and you're free to do what you like with the code.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/28/2014 05:01 pm
We are slowly adding frames to the movie, so I put an updated version. Big thanks in particular to the anonymous people who cleaned part 248!

97 frames (some of them only very partially recovered) out of ~270 were used.

So, based on where we are right this moment, is this still a very good representation of where the progress currently is?

There's a very good reason for asking :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/28/2014 05:07 pm
We are slowly adding frames to the movie, so I put an updated version. Big thanks in particular to the anonymous people who cleaned part 248!

97 frames (some of them only very partially recovered) out of ~270 were used.

So, based on where we are right this moment, is this still a very good representation of where the progress currently is?

There's a very good reason for asking :)

Yes, I think that is still a good representation of our progress to date. There have been some minor improvements since then, but SwissCheese's video has more polish. I posted my most recent auto-build for comparison.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/28/2014 05:08 pm
PS Sorry Princess. The forum simply can't handle those mega long strings, so I've had to cut them....but I'm sure people can just merge them when using (they are otherwise intact).
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/28/2014 05:09 pm
We are slowly adding frames to the movie, so I put an updated version. Big thanks in particular to the anonymous people who cleaned part 248!

97 frames (some of them only very partially recovered) out of ~270 were used.

So, based on where we are right this moment, is this still a very good representation of where the progress currently is?

There's a very good reason for asking :)

Yes, I think that is still a good representation of our progress to date. There have been some minor improvements since then, but SwissCheese's video has more polish. I posted my most recent auto-build for comparison.

Very good, thanks!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: princess on 05/28/2014 05:10 pm
PS Sorry Princess. The forum simply can't handle those mega long strings, so I've had to cut them....but I'm sure people can just merge them when using (they are otherwise intact).

No problem Chris, thanks for fixing it up. It looked OK in preview!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/28/2014 05:29 pm
And here's the youtube version of SwissCheese's video, uploaded to a clean account with nothing else in it, etc. This is a requested version (as opposed to the version that is attached as a file on the thread - which is over 4,000 downloads already :o)

This is easier to share than the link.

https://www.youtube.com/watch?v=XoufUV5oGTo
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Joffan on 05/28/2014 05:56 pm
Tweeted this  video out
https://twitter.com/Joffan7/status/471710560006848512
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/28/2014 06:14 pm
We are slowly adding frames to the movie, so I put an updated version. Big thanks in particular to the anonymous people who cleaned part 248!

97 frames (some of them only very partially recovered) out of ~270 were used.

So, based on where we are right this moment, is this still a very good representation of where the progress currently is?

There's a very good reason for asking :)

Yes, I think that is still a good representation of our progress to date. There have been some minor improvements since then, but SwissCheese's video has more polish. I posted my most recent auto-build for comparison.

You should set the frame rate manually (14.98 fps) when reconstructing the movie with option -framerate 44999/3003, it seems to be too quick now.

(my command line: ffmpeg -framerate 44999/3003 -i img_spx_%03d.png -c:v mpeg4 -q:v 3 spacex_landing.mp4)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/28/2014 06:16 pm
One p-frame looks very strange, with "too intense" colors, could anyone come with an explanation and maybe a solution to this?

It's frame 227 / part 12-19

mmb: 0:0:-1,30:1:10681,10:3:-1,31:3:19630

I checked the VOP quants in this section and that frame does look like a bit of an outlier. Here are the VOP quants in fixed_edit8_part_209.ts:
1  00100
2  00010
3  00001
4  00001
5  00010
6  00010
7  00010
8  00001
9  00001
10 00000
11 00001
12 00001
13 00001
14 00001
15 00001
16 00001
17 00001
18 00001
19 00111
20 00001

My first guess is that the correct VOP quant is 00001, and there's a double flip, but other values are possible. I attach a ts file where that VOP quant is set to 00001. It's at offset x43A9B so other values can be tried. I can't check it myself as I'm yet to establish a proper setup for multiframe work. Let me know how it looks.

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/28/2014 06:24 pm
One p-frame looks very strange, with "too intense" colors, could anyone come with an explanation and maybe a solution to this?

It's frame 227 / part 12-19

mmb: 0:0:-1,30:1:10681,10:3:-1,31:3:19630

I checked the VOP quants in this section and that frame does look like a bit of an outlier. Here are the VOP quants in fixed_edit8_part_209.ts:
1  00100
2  00010
3  00001
4  00001
5  00010
6  00010
7  00010
8  00001
9  00001
10 00000
11 00001
12 00001
13 00001
14 00001
15 00001
16 00001
17 00001
18 00001
19 00111
20 00001

My first guess is that the correct VOP quant is 00001, and there's a double flip, but other values are possible. I attach a ts file where that VOP quant is set to 00001. It's at offset x43A9B so other values can be tried. I can't check it myself as I'm yet to establish a proper setup for multiframe work. Let me know how it looks.

Waouh very nice! It solves the issue!!!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/28/2014 06:40 pm
Tweeted this  video out
https://twitter.com/Joffan7/status/471710560006848512


So did I! ;D

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/28/2014 06:47 pm
One p-frame looks very strange, with "too intense" colors, could anyone come with an explanation and maybe a solution to this?

It's frame 227 / part 12-19

mmb: 0:0:-1,30:1:10681,10:3:-1,31:3:19630

I checked the VOP quants in this section and that frame does look like a bit of an outlier. Here are the VOP quants in fixed_edit8_part_209.ts:
<snip>
10 00000
<snip>

My first guess is that the correct VOP quant is 00001, and there's a double flip, but other values are possible. I attach a ts file where that VOP quant is set to 00001. It's at offset x43A9B so other values can be tried. I can't check it myself as I'm yet to establish a proper setup for multiframe work. Let me know how it looks.

VOP-quant should never be 0.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/28/2014 06:56 pm
VOP-quant should never be 0.

Doublechecked. Frame 218 has VOP quant 0 (offset in file x21027) and it seems to look fine.
Edit: Maybe it should be set to 1 still for the sake of correctness.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: princess on 05/28/2014 07:02 pm
My latest fix for the part 12 TS file (command line options for the tsrepair program posted earlier):

-fix:55,noaf/127,pid,20/405,pes/405,ptsauto/405,pframe/423,pay,3/426,noaf/
430,noaf/431,noaf/486,noaf/487,noaf/657,pid,1fff/658,noaf/658,pay,0/
658,pid,3e8/660,noaf/660,pay,2/661,pay,3/971,noaf/996,noaf/1046,pid,1fff/
1072,noaf/1122,pay,10/1124,noaf/1124,pay,12/1439,noaf/1450,noaf/
1475,ptsauto/1477,pid,3e8/1477,pay,4/1480,pay,6

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: princess on 05/28/2014 07:22 pm
If people want to help out with the TS file fixing that would be great. The -fix option in the tsrepair program is split into subfields with the forward slash character '/'. Each subfield takes a packet number (starting from 0), a command, and an optional parameter.

Commands are:

af <number> - Set the length of the AF (adaptation field). This adds an AF if none is present.

noaf - Remove an existing adaptation field.

nopcr - Remove a PCR timestamp.

nopusi - Remove a Payload Unit Start Indicator flag.

pay <number> - Set the "contains payload" flag, and set the PCC (payload continutity counter) to the given value.

pcr <ticks> - Set the PCR timestamp to the given number of ticks (in decimal).

pes - Indicate that it's a PES header packet. This is used for the start of I-frames and P-frames. It sets the AF to 7 bytes, adds a Payload Unit Start Indicator, sets the PID to the SpaceX PID of 0x03e8, sets the "contains payload" flag, and sets the PES header bytes correctly for the SpaceX video.

pframe - Corrects the 4-byte MPEG4 P-frame header.

pid <number> - Sets the PID to the given number. The number is specified in hexadecimal, and is usually either 3e8 for data packets or 1fff for padding packets.

ptsauto - Sets the PTS timestamp by lagging the existing PCR timestamp by 10,000 ticks. The PCR must be set correctly for this to work.

pusi - Sets the Payload Unit Start Indicator. This is usually only set on PES header packets (the start of I-frames or P-frames).

If you need any more information please send me a message!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/28/2014 08:16 pm
Hi, since we are really close to have a finished fixed ts-file version (i am waiting for princess to send me her ts-parts 3,4,5,6,9,10 and 12 (those are the ones I still need fixed, rest is done)
Do you guys think we should keep the P-frames split up for each i-frame or merge them back together to one large file?
What is easier to work with?

Cheers,
Shanuson
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/28/2014 09:04 pm
Long MMBs should be handled by the editor now.... max query string length =(

Will be updating the editor with all of the latest ts files once the people working on that have finished the whole set, hopefully pretty soon =)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: princess on 05/28/2014 09:28 pm
I am waiting for princess to send me her ts-parts 3,4,5,6,9,10 and 12 (those are the ones I still need fixed, rest is done)

Here are the files you need, let me know if you need anything else!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/28/2014 09:36 pm
I am waiting for princess to send me her ts-parts 3,4,5,6,9,10 and 12 (those are the ones I still need fixed, rest is done)

Here are the files you need, let me know if you need anything else!
Thanks
I will have a final version in a few hours.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/28/2014 09:51 pm
I made some changes to iframe 72. I think this one behaves better with the subsequent P frames. Unfortunately this ruins a lot of great work by someone who meticulously fixed all chroma/luma issues. It also looks very harsh and high contrast.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/28/2014 09:57 pm
Hi, since we are really close to have a finished fixed ts-file version (i am waiting for princess to send me her ts-parts 3,4,5,6,9,10 and 12 (those are the ones I still need fixed, rest is done)
Do you guys think we should keep the P-frames split up for each i-frame or merge them back together to one large file?
What is easier to work with?

Cheers,
Shanuson

I have found the split files to be helpful. For example if I want to try a number of different mmbs it takes less time if fewer frames have to be processed.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/28/2014 10:26 pm
<snip>

Summary:
I am surprised that the interlace flag is 0!!
I can restore parts of the VOP-header. Some things I am not sure what they should be.
Especially vop_quant, vop_fcode_forward and ref_select. Somebody with more knowledge about the decoding  of makroblocks could help here maybe?
Maybe I can see patterns when I checked all P-Frames.
I have to update part of what I posted above:
First like michaelni said, there is no ref_select_code

And I overlooked that if the VOP_time_increment gets modulod (is smaller than 3003) there is one bit added before it (I marked it purple below). so everything shifts by 1 bit.

P-Frame VOP header part:
(1/5)x       xx       x(e/c)    0(even x) (4/5)x
 0x101xxxx xxxxxxxx xxxx11x0  0000xxx0      01xxxxxx

vop_coding type - 00 = I-Frame ; 01 = P-frame
modulo time base = 0  (if time_increment is smaller than 3003, there will be single 1 before that 0)
marker bit
vop_time_increment <- this is increasing from frame to frame by 3003 (fixed_vop_time_increment) modulo 44999 (vop_time_increment_resolution), so it is easy to calculate what it should be.
marker bit
vop_coded -> yes
vop_rounding_type -> alternating between 1 and 0
intra_dc_vlc_thr = 000
vop_quant 00xxx
vop_fcode_forward  <- could this always be 001? my guess but i am not sure.
and now follow the makroblocks....

Edit:
I-Frame VOP header part:
(1/5)x       xx       xc    (0/1)x     x
 0x101xxxx xxxxxxxx xxxx1100  000xxx0x xxxx

vop_coding type - 00 = I-Frame ; 01 = P-frame
modulo time base = 0 (if time_increment is smaller than 3003, there will be single 1 before that 0)
marker bit
vop_time_increment <- this is increasing from frame to frame by 3003 (fixed_vop_time_increment) modulo 44999 (vop_time_increment_resolution), so it is easy to calculate what it should be.
marker bit
vop_coded -> yes
vop_rounding_type -> alternating between 1 and 0
intra_dc_vlc_thr = 000
vop_quant 00xxx
vop_fcode_forward  <- could this always be 001? my guess but i am not sure.
and now follow the makroblocks....

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/29/2014 01:49 am
This is why we needed a youtube version ;D

Elon Musk ‏@elonmusk  4m 
 More
 Amazing repair job of Falcon 9 ocean landing vid by @NASASpaceflight forum. Now shows leg deploy http://www.youtube

Totally distracted me during Soyuz docking. I hope he doesn't do that again! (;D Joke!)

Again, that's all deserved for what you guys are doing. It's direct praise from one of the most important people on the planet right now. You're a motivated bunch, but I'm sure that helps while you dig through all these bytes of data :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/29/2014 02:05 am
Thanks to princesses really good work, it was relatively easy to finally put together a raw_final_fixed.ts (well final for me, everything important/necessary  is fixed in there I hope).
It produces 283 Frames, and it has no! missing ts-packets (CC - discontinuities) for PID 1000.
For that one P-frame i talked about before (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1204583#msg1204583). I added the relevant data a 2nd time into the file and put the 2nd header there. They are frame 254 and 255.  Frame 254 was shorten at the end by a few packets to make the CC continuous without too much editing (those packets should belong to Frame 255 anyways so it does not matter if they are there or not). So P-Frame 254 has a lot of waste data at the end and P-frame 255 has even a bit more waste data before the first MB. I was not happy with adding the header at the suggested point because it would most like mean that one P-frame has too much and the next not enough data.


I add the ts.file here.
iframe png's and parts follow.

IainCole already got the parts to add them to the online editor separately.

And i hope it does not require to much work to get a nice new video up like we had before.

Also a few Iframes contain a bit more data, most likely at the end!

Cheers
Shanuson
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/29/2014 02:11 am
The first 40 Frames (2 I-frames) are most likely really hard to be repaired. A lot of messed up data in there.

Here are the parts.
Next will be the pngs.
Here are the first few:
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/29/2014 02:13 am
PNGs for I-Frames 7-15
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Lar on 05/29/2014 02:45 am
PS Sorry Princess. The forum simply can't handle those mega long strings, so I've had to cut them....but I'm sure people can just merge them when using (they are otherwise intact).

No problem Chris, thanks for fixing it up. It looked OK in preview!

A suggestion, make your code accept a space after the comma, without treating it as data. (if you're using any kind of canned argument parsing code that should be very easy)... Then when you or anyone posts strings, spaces will allow for good wrap points without any fuss.

(your code may well actually already accept spaces, I didn't actually check.. heck it probably does)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/29/2014 03:58 am
I made some changes to iframe 72. I think this one behaves better with the subsequent P frames. Unfortunately this ruins a lot of great work by someone who meticulously fixed all chroma/luma issues. It also looks very harsh and high contrast.

72 looked pretty good already - do you have an animation of how this one fits the pframes better? I'm not sure what you mean.

(http://spacexlanding.wikispaces.com/file/view/frames_72_91.gif/511813810/490x336/frames_72_91.gif)

Looks like a ray of sunshine on the body as it breaks the clouds.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/29/2014 05:40 am
Here's an example of frame 94 with redrawn timestamps. The top part is the isolated pframe, the middle is the output from ffmpeg, and the bottom is repainted.

Only the digits which are known with certainty are modified, so this should be a faithful representation of those pixels. The 1/10 second place is interlaced between 6 and 7. The last digit is not known, so it is left scrambled.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/29/2014 09:23 am

72 looked pretty good already - do you have an animation of how this one fits the pframes better? I'm not sure what you mean.

Hard to be sure of anything in this region, but I feel this alignment meshes better with frame 74. And the gif SwissCheese has made doesn't look bad either. Your img link is pointing to the new version.

Someone has redone the hard work with chromas and lumas too (Quialiss?). Great stuff.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/29/2014 10:29 am
Just want to say this is an amazing job you guys have done! I had to drop out from doing it because real life intervened in my ability to have time to assist. Hopefully I'll have time to catch up on the some 30 pages that have been posted since I last helped.

Just as a quick question, is there still need to repair I-frames? I notice that many of the I-frames have not advanced much in repair state from when I was last involved.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/29/2014 10:43 am
Hi guys,

Little update.

I'm experimenting with logging detailed info about all the decoded bits in the macroblocks (and their interpretation).

Attached is a preliminary image showing the first few marcroblocks of frame 169.

Red is macroblock-header. The cbpy represents which lum-blocks will contain DCTs, cbpc does for chrom-blocks.
Yellow is DC-values (4 lum, 2 chrom). So this is either 8x8px info or 16x16px info. Bascicaly ranges from 0 to 255 each.
Green is DCT-values inside each of these 6 blocks. So this is sub 8x8px or 16x16px info.

You can see that not every block has transformation data (the green sub-8/16-pixel data).

The bits themselves are not logged yet, but that should be relatively easy to do.

Just wanted to share. :)

Regards,

arnezami

PS. I am considering a macroblock editor/analyser of sorts. Where you can see each bit and where it belongs to given the current starting bitposition of a (macro)block. Then you can see what a flipped bit will actually do to the interpretation by the decoder. Not sure yet how best to do that.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/29/2014 10:56 am
Just want to say this is an amazing job you guys have done! I had to drop out from doing it because real life intervened in my ability to have time to assist. Hopefully I'll have time to catch up on the some 30 pages that have been posted since I last helped.

Just as a quick question, is there still need to repair I-frames? I notice that many of the I-frames have not advanced much in repair state from when I was last involved.
Hi mlindner! Glad you're back :)

There is definitely still a need for work on the I-frames. I think many moved their focus to the P-frames because making video is so tempting and rewarding ;)

But improving the I-frames clearly makes the 19 P-frames that follow much better!

Your excellent work on frame 169 shows that!

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/29/2014 11:21 am
@arnezami
Would it be possible to add a option to manipulate the VOP-quant and vop_fcode_forward part of a p-frame header?
Some seem to be messed up, and at least vop-quant seems to have a large effect on the whole frame.

Cheers
Shanuson
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: FutureSpaceTourist on 05/29/2014 11:42 am
Thread has just passed 250,000 views!

Keep up the amazing work - seeing the legs deploy on the latest video is a wonderful result.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/29/2014 12:18 pm
@arnezami
Would it be possible to add a option to manipulate the VOP-quant and vop_fcode_forward part of a p-frame header?
Some seem to be messed up, and at least vop-quant seems to have a large effect on the whole frame.

Cheers
Shanuson
Hi Shanuson,

Maybe. Currently the mmb-command work in the "scope" after the VOP-quant etc have been decoded. Maybe/probably we are too late there. So I'm not sure if it's possible using the mmb-commands. But maybe using a different command. @michaelni: do you know how to do this?

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Lourens on 05/29/2014 12:39 pm
arnezami, could you post a patch? I was about to start doing the exact same thing :). Hopefully the additional output will allow a kind of guided bit flipping, where we can quickly discard flips that yield obvious nonsense.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/29/2014 01:08 pm
I adapted part 10 to the new ts, wasn't too difficult. I made a few correction on the way :)

I think we should keep all the work put in the wiki with the previous ts files as an archive and create new pages with the new ts. Meanwhile I'll put the modification here.

edit: forum does not like big gif it seems...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/29/2014 03:27 pm
And the same for part 9 (13/20 frames corrected) :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/29/2014 03:40 pm
arnezami, could you post a patch? I was about to start doing the exact same thing :). Hopefully the additional output will allow a kind of guided bit flipping, where we can quickly discard flips that yield obvious nonsense.

Yes we can do that. I'll put the current pages on archive and create new empty ones. :)

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Comga on 05/29/2014 04:07 pm
Here's an example of frame 94 with redrawn timestamps. The top part is the isolated pframe, the middle is the output from ffmpeg, and the bottom is repainted.

Only the digits which are known with certainty are modified, so this should be a faithful representation of those pixels. The 1/10 second place is interlaced between 6 and 7. The last digit is not known, so it is left scrambled.

If the first decimal place is interlaced between 6 and 7 one would expect that the second decimal place is interlaced between 9 and 0.  It does look like that.  With the preceding or following frames is that not sufficiently certain?

edit: Forgot to say that I am amazed at the terrific work people are doing here.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/29/2014 04:12 pm
arnezami, could you post a patch? I was about to start doing the exact same thing :). Hopefully the additional output will allow a kind of guided bit flipping, where we can quickly discard flips that yield obvious nonsense.

Yes we can do that. I'll put the current pages on archive and create new empty ones. :)

@moralec: I think you meant to reply on this post by SwissCheese (quoted below) right?  (instead of Laurens post)
@moralec/SwissCheese: would it be a good idea to number the I-frames to 1,21,41,61 etc? And the P-frames in between? Since we expect each I-frame to contain exactly 19 P-frames right?

I adapted part 10 to the new ts, wasn't too difficult. I made a few correction on the way :)

I think we should keep all the work put in the wiki with the previous ts files as an archive and create new pages with the new ts. Meanwhile I'll put the modification here.

edit: forum does not like big gif it seems...

@Lourens: I will try but there are still quite a few problems with this version I noticed. For example you can see that bits are missing. Thats not good ;). It's also very hacky atm. Will keep you updated.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/29/2014 04:19 pm
I adapted part 10 to the new ts, wasn't too difficult. I made a few correction on the way :)

It looks great! Does the stability of non-moving parts of the image mean that you guys were able to fix up bad motion vectors, or is this just an unusually good sequence?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mhenderson on 05/29/2014 04:21 pm
@arnesami - should we adopt a numbering convention of PART.FRAME like 10.01 - 10.20? This allows for new pFrames to be found without changing all frames after.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/29/2014 04:21 pm
All the digits as drawn by the camera.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/29/2014 04:30 pm
Here's an example of frame 94 with redrawn timestamps. The top part is the isolated pframe, the middle is the output from ffmpeg, and the bottom is repainted.

Only the digits which are known with certainty are modified, so this should be a faithful representation of those pixels. The 1/10 second place is interlaced between 6 and 7. The last digit is not known, so it is left scrambled.

If the first decimal place is interlaced between 6 and 7 one would expect that the second decimal place is interlaced between 9 and 0.  It does look like that.  With the preceding or following frames is that not sufficiently certain?

edit: Forgot to say that I am amazed at the terrific work people are doing here.

The video only runs at ~15 frames per second, so the hundredths digit would be incremented by ~0.067 per (non-interlaced) frame. That makes its value in any particular frame unpredictable. I'm going to put together a table of digit-to-digit transitions that should make it easier to figure out what the p-frame is trying to draw without knowing what exactly was in the previous frame.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Lourens on 05/29/2014 04:50 pm
arnezami, could you post a patch? I was about to start doing the exact same thing :). Hopefully the additional output will allow a kind of guided bit flipping, where we can quickly discard flips that yield obvious nonsense.

@Lourens: I will try but there are still quite a few problems with this version I noticed. For example you can see that bits are missing. Thats not good ;). It's also very hacky atm. Will keep you updated.
Okay, I'll go and have a look at the FFmpeg source myself as well, will post when I have something.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: meadows.st on 05/29/2014 05:00 pm
Coming up on the 150,000 view mark for this thread/effort. Noting it as there's a small active team of people really working hard on this, but also a HUGE amount of people watching that work. Always nice to have a large appreciative audience, both inside and outside the space industry.

You'd be astonished how many views are from IP addresses in a place called Hawthorne, California! ;D

I was curious how this thread ranked overall in the history of SpaceX threads (currently as of this writing, reads are (1) >180,000) so I sorted the other SpaceX dedicated threads and only the 100,000+ and Missions sections have threads that exceed 180,000. Currently this thread is the (2) 27th most read SpaceX thread and I think the NSF server (sorry Chris  ;D) will explode when you all get the video restored "even" to the level of SwissCheese's latest gif (which is awesome BTW). see:


Graphic representation of the historical significance of the work that the video restoration group is doing (see pic).

Added: Updated context with consolidated "conversations" (multiple threads on same topic now rolled-up into a single "conversation").

1) Thread currently at >252,000 reads which puts it at 15th most read SpaceX thread of all time and 13th most read SpaceX conversation[1]. See original link for images of the overall ranking.

2) 27th to 15th in the space of <1 week. Y'all's latest video (oh yeah, and EM's tweet ;) ) really ramped up the hit rate.  I think this is further evidence that the "complete" video is going to rival a live launch hit rate. :D

Phenomenal work everyone!

[1] "conversations" = multiple threads on same topic rolled-up into a single line (e.g. General SpaceX and Falcon Discussion consists of 9 threads for a total of >2.2M views.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: princess on 05/29/2014 05:46 pm
I've spent today going over Shanuson's complete TS file, cross-referencing it with the original raw TS file and fixing up the non-data packet types. All packets of type 0x0000 and 0x0020 are now present with the correct contents and no continuity errors. All padding packets have been identified and set as valid. All packets in the file have a PID of either 0x0000, 0x0020, 0x03e8 or 0x1fff.

I'm using a deliberately old version of VLC (from Ubuntu 12.10) to test with as it was far less error-tolerant back then - although the old VLC can't get a picture from the TS file yet, the file now reports no TS-level errors, only MPEG4 errors. It plays in newer VLC (Ubuntu 14.04) as well as it plays in mplayer or ffmpeg.

Please note that this fix won't affect the data recovery effort (as no data packets have been changed), but it does help if you're trying to get the TS file to play in a video player program.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: princess on 05/29/2014 06:30 pm
On the subject of recovering the images of the clock in the bottom left corner, I think the TS recovery people might be able to help with that!

To understand why, I need to explain a little about how TS data packets work. Each packet is 188 bytes long, and they have a short header at the start that can define a variable-length adaptation field - basically, it says "reserve space at the start of this packet for extra stuff". Then the actual MPEG4 data comes after the adaptation field.

In the last data packet of a video frame. the encoder will use this feature to ensure that the packet only contains the right amount of data. For example, if there was only 88 bytes of data left to encode in the last packet, the encoder would specify that the last data packet has an adaptation field of 100 bytes long. It then pads the fields with 0xff bytes, and puts the MPEG4 data at the END of the packet. So the packet looks like this:

Header length | ffffffffffff.....ffffffff | MPEG 4 data

Why am I telling you all of this? Well, you can see that if there are bitflips in the "Header length" part, we might not be able to sync up the last piece of MPEG4 data. If the header is too long the MPEG 4 data gets lost, and if it's too short there is a huge run of 0xff bytes which can confuse the decoder mightily.

As the clock is on the last line of macroblocks, it's very susceptible to this kind of corruption. Also, these kind of problems don't register in ffmpeg or VLC as TS errors, so we don't know they're happening.

So what can we do? Well, if we're having problems decoding the last line of macroblocks in an I-frame, it might be possible that it's actually a TS-level problem. We should examine the I-frames where they seem fine but the clock is being stubbornly corrupt, and look to see what the correct TS header length should be. As the packets are only 188 bytes long it should be fairly trivial to brute-force the search and see which makes the best MPEG4 data for the clock.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/29/2014 06:50 pm
The latest TS files have been added to the online editor
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: michaelni on 05/29/2014 06:54 pm
1) There are a lot of possible combination to try. If an error correction code was used then that information could possibly allow us to discard some of them (e.g. if a single-error correcting code was used at the bit level, then any single-bit flips will have already been fixed and don't need to be tried, and there may be a more limited number of combinations of flipped bits to check).

You miss that a FEC code capable to fix n bit errors, does fix them for the transmitted packet/message, while the transmitted packet is made up of data and parity symbols in a systematic code or at least more symbols than there are data symbols in the more generic case. Thus for example a 1-3 bit correcting FEC code could fix 3 bits in the transmitted packet but there might be just 1 or eve none of these in the data part. We only have the data part of the packets, not the parity part. Also if in that same example there are 4 bit flips, the FEC decoder might actualy (depending on what type of code and implementation it is) add more errors.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/29/2014 06:55 pm
arnezami, could you post a patch? I was about to start doing the exact same thing :). Hopefully the additional output will allow a kind of guided bit flipping, where we can quickly discard flips that yield obvious nonsense.

@Lourens: I will try but there are still quite a few problems with this version I noticed. For example you can see that bits are missing. Thats not good ;). It's also very hacky atm. Will keep you updated.
Okay, I'll go and have a look at the FFmpeg source myself as well, will post when I have something.
Hi Lourens,

In that case this will probably help. This is my hacky version based on an older version of h263dec.c Mostly mpeg4videodec.c is changed though. Watch for "BITLOG".

For everyone: this is not meant to be merged directly with any repository! It's just experimental stuff.

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: michaelni on 05/29/2014 06:58 pm
Maybe. Currently the mmb-command work in the "scope" after the VOP-quant etc have been decoded. Maybe/probably we are too late there. So I'm not sure if it's possible using the mmb-commands. But maybe using a different command. @michaelni: do you know how to do this?

mmb should be able to flip bits in  mpeg4-es headers, but i didnt try
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/29/2014 07:32 pm
@arnesami - should we adopt a numbering convention of PART.FRAME like 10.01 - 10.20? This allows for new pFrames to be found without changing all frames after.
Sure. That could work. It's a bit more safe I guess.

Unless we know for sure each I-frame has no more than 19 P-frames. Because from what I see it really looks like that is the case.

I don't have a strong preference. I like simpler numbers if possible.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mhenderson on 05/29/2014 07:40 pm
The latest TS files have been added to the online editor

If possible, could you add the new .ts to the older spacex.slapbet.org version of the editor?

The "iFrame only" online tool is MUCH faster to work with. I believe we are back to square one on the iFrame mmps.

Thank you.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: MTom on 05/29/2014 07:46 pm
This is why we needed a youtube version ;D

Elon Musk ‏@elonmusk  4m 
 More
 Amazing repair job of Falcon 9 ocean landing vid by @NASASpaceflight forum. Now shows leg deploy http://www.youtube

Totally distracted me during Soyuz docking. I hope he doesn't do that again! (;D Joke!)

Again, that's all deserved for what you guys are doing. It's direct praise from one of the most important people on the planet right now. You're a motivated bunch, but I'm sure that helps while you dig through all these bytes of data :)


... and there is the update on the SpaceX website showing the video from yesterday:  :D

Quote
UPDATE (5/28/2014)

Members of NASASpaceFlight.com's forum are continuing to make great progress with the frame by frame repair of the Falcon 9 first stage soft splashdown in the Atlantic Ocean.

http://www.spacex.com/news/2014/04/29/first-stage-landing-video
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/29/2014 08:09 pm
The latest TS files have been added to the online editor

If possible, could you add the new .ts to the older spacex.slapbet.org version of the editor?

The "iFrame only" online tool is MUCH faster to work with. I believe we are back to square one on the iFrame mmps.

Thank you.

If someone who knows how to do it can generate the mp4-img files for the iframes then I can add them to the old editor.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/29/2014 08:49 pm
The latest TS files have been added to the online editor

If possible, could you add the new .ts to the older spacex.slapbet.org version of the editor?

The "iFrame only" online tool is MUCH faster to work with. I believe we are back to square one on the iFrame mmps.

Thank you.

Many frames are the same as with previous version, and the ones that changed are quite easily fixed by finding a few bit offsets between the previous and the current version.

I will try to prepare the wiki tomorrow, today neither Moralec nor I can do it.

In any case if we want to have a full video soon, I think we should concentrate on parts 7, 8, 13 and 15 (maybe also 3, 4 and 5, and finish part 9, see my previous post), for the other ones we can always use what was done before.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/29/2014 08:50 pm
The latest TS files have been added to the online editor

If possible, could you add the new .ts to the older spacex.slapbet.org version of the editor?

The "iFrame only" online tool is MUCH faster to work with. I believe we are back to square one on the iFrame mmps.

Thank you.

If someone who knows how to do it can generate the mp4-img files for the iframes then I can add them to the old editor.

You can do this:

Quote
./ffmpeg -i fixed_edit8_part_169.ts -vcodec copy -an -f image2 fixed_edit_part_169_%d.mpg4-img

This assumes you have fixed_edit8_part_169.ts and produces .mp4-img files. In this case you want the first one "fixed_edit_part_169_1.mpg4-img" since it's the I-frame. You can throw away the other mpg4-img files, since these are the P-frames.

And you have to do this for each of the 15 ts-parts.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mhenderson on 05/29/2014 09:13 pm
The latest TS files have been added to the online editor

If possible, could you add the new .ts to the older spacex.slapbet.org version of the editor?

The "iFrame only" online tool is MUCH faster to work with. I believe we are back to square one on the iFrame mmps.

Thank you.

If someone who knows how to do it can generate the mp4-img files for the iframes then I can add them to the old editor.

You can do this:

Quote
./ffmpeg -i fixed_edit8_part_169.ts -vcodec copy -an -f image2 fixed_edit_part_169_%d.mpg4-img

This assumes you have fixed_edit8_part_169.ts and produces .mp4-img files. In this case you want the first one "fixed_edit_part_169_1.mpg4-img" since it's the I-frame. You can throw away the other mpg4-img files, since these are the P-frames.

And you have to do this for each of the 15 ts-parts.

You may ignore my request.

I found a quick fix that makes the newer online editor much more responsive ... null out the pFrames by adding this to the mmb for the full sequence. It quickly redraws the iFrame.

=FRAME1:0:0:-1=FRAME2:0:0:-1=FRAME3:0:0:-1=FRAME4:0:0:-1
=FRAME5:0:0:-1=FRAME6:0:0:-1=FRAME7:0:0:-1=FRAME8:0:0:-1
=FRAME9:0:0:-1=FRAME10:0:0:-1=FRAME11:0:0:-1=FRAME12:0:0:-1
=FRAME13:0:0:-1=FRAME14:0:0:-1=FRAME15:0:0:-1=FRAME16:0:0:-1
=FRAME17:0:0:-1=FRAME18:0:0:-1=FRAME19:0:0:-1

Sorry, I was thinking inside the box. ;)

Mike
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/29/2014 09:52 pm
The latest TS files have been added to the online editor

If possible, could you add the new .ts to the older spacex.slapbet.org version of the editor?

The "iFrame only" online tool is MUCH faster to work with. I believe we are back to square one on the iFrame mmps.

Thank you.

If someone who knows how to do it can generate the mp4-img files for the iframes then I can add them to the old editor.

You can do this:

Quote
./ffmpeg -i fixed_edit8_part_169.ts -vcodec copy -an -f image2 fixed_edit_part_169_%d.mpg4-img

This assumes you have fixed_edit8_part_169.ts and produces .mp4-img files. In this case you want the first one "fixed_edit_part_169_1.mpg4-img" since it's the I-frame. You can throw away the other mpg4-img files, since these are the P-frames.

And you have to do this for each of the 15 ts-parts.

You may ignore my request.

I found a quick fix that makes the newer online editor much more responsive ... null out the pFrames by adding this to the mmb for the full sequence. It quickly redraws the iFrame.

=FRAME1:0:0:-1=FRAME2:0:0:-1=FRAME3:0:0:-1=FRAME4:0:0:-1
=FRAME5:0:0:-1=FRAME6:0:0:-1=FRAME7:0:0:-1=FRAME8:0:0:-1
=FRAME9:0:0:-1=FRAME10:0:0:-1=FRAME11:0:0:-1=FRAME12:0:0:-1
=FRAME13:0:0:-1=FRAME14:0:0:-1=FRAME15:0:0:-1=FRAME16:0:0:-1
=FRAME17:0:0:-1=FRAME18:0:0:-1=FRAME19:0:0:-1

Sorry, I was thinking inside the box. ;)

Mike

I have added them anyway, thanks arnezami for the instructions :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/29/2014 10:32 pm
arnezami, could you post a patch? I was about to start doing the exact same thing :). Hopefully the additional output will allow a kind of guided bit flipping, where we can quickly discard flips that yield obvious nonsense.

Yes we can do that. I'll put the current pages on archive and create new empty ones. :)

@moralec: I think you meant to reply on this post by SwissCheese (quoted below) right?  (instead of Laurens post)
@moralec/SwissCheese: would it be a good idea to number the I-frames to 1,21,41,61 etc? And the P-frames in between? Since we expect each I-frame to contain exactly 19 P-frames right?

Yes. My Bad.... I´m struggling from a mobile device.

I think thats a good idea. I'm going to build the new pages to upload results based on that numbering. :)
However, we need to keep in mind the correspondences in order to import the previous improvements.

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Lourens on 05/29/2014 10:46 pm
1) There are a lot of possible combination to try. If an error correction code was used then that information could possibly allow us to discard some of them (e.g. if a single-error correcting code was used at the bit level, then any single-bit flips will have already been fixed and don't need to be tried, and there may be a more limited number of combinations of flipped bits to check).

You miss that a FEC code capable to fix n bit errors, does fix them for the transmitted packet/message, while the transmitted packet is made up of data and parity symbols in a systematic code or at least more symbols than there are data symbols in the more generic case. Thus for example a 1-3 bit correcting FEC code could fix 3 bits in the transmitted packet but there might be just 1 or eve none of these in the data part. We only have the data part of the packets, not the parity part. Also if in that same example there are 4 bit flips, the FEC decoder might actualy (depending on what type of code and implementation it is) add more errors.

I'm not sure I understand completely what you wrote here, but I think you're right that it doesn't help as much as I thought it would.

arnezami, could you post a patch? I was about to start doing the exact same thing :). Hopefully the additional output will allow a kind of guided bit flipping, where we can quickly discard flips that yield obvious nonsense.

@Lourens: I will try but there are still quite a few problems with this version I noticed. For example you can see that bits are missing. Thats not good ;). It's also very hacky atm. Will keep you updated.
Okay, I'll go and have a look at the FFmpeg source myself as well, will post when I have something.
Hi Lourens,

In that case this will probably help. This is my hacky version based on an older version of h263dec.c Mostly mpeg4videodec.c is changed though. Watch for "BITLOG".

For everyone: this is not meant to be merged directly with any repository! It's just experimental stuff.

Regards,

arnezami

That does help, thanks! I've modded the MB pos/size line a bit in my version to show, it now looks like this:

[mpeg4 @ 0x1c2bba0] MB pos/size: 0 000:40:03:8960 39 ac_pred: 1 aic: 0 aic_dir: 0 dquant: 0 qscale: 4 dc: 142 143 140 141 - 135 124

but that doesn't actually help much. I've put your additions into my version as well, and they are much more useful. Now to link it up to my automated bit flipper and see if it yields something. That's for tomorrow though.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Oersted on 05/29/2014 11:37 pm
1) Thread currently at >252,000 reads which puts it at 15th most read SpaceX thread of all time and 13th most read SpaceX conversation[1]. See original link for images of the overall ranking.

Never in the history of space flight was a thread read by so many who understood so little...  ;-)

- I'm one of them, btw! - But I think we all share the joy of seeing consummate professionals cooperating in an amateur endeavour, coaxing intelligible images from a seemingly hopelessly mangled blur of bits and bytes. To me it replicates the fascination of listening to ground control during a Saturn-Apollo lift-off; obscure abbreviations tersely announced...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/30/2014 12:38 am
The latest TS files have been added to the online editor

Following this update the wiki now has 16 new (empty) pages for you all to  upload your progress on each frame.
I followed the same structure as last time (one main page, and 15 secondary pages). Please bookmark http://spacexlanding.wikispaces.com/raw_final_fixedMMB :)

The old pages are still available *look under OLD/ARCHIVED in the homepage*. They have all the MMB codes we were able to obtain from the previous ts files. Many of them should work (specially from those sections that already had 19 p-frames in the previous version).  However, please take into account that frame numbers have changed as new p-frames where discovered. A good way to orient yourself is using the 15 I-frames that remain the same.

Good luck! And for those anxiously waiting for the Dragon II event tonight, remember that working on p-frames is a fantastic way to kill time. 
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/30/2014 03:06 am
The latest TS files have been added to the online editor

Following this update the wiki now has 16 new (empty) pages for you all to  upload your progress on each frame.
I followed the same structure as last time (one main page, and 15 secondary pages). Please bookmark http://spacexlanding.wikispaces.com/raw_final_fixedMMB :)

The old pages are still available *look under OLD/ARCHIVED in the homepage*. They have all the MMB codes we were able to obtain from the previous ts files. Many of them should work (specially from those sections that already had 19 p-frames in the previous version).  However, please take into account that frame numbers have changed as new p-frames where discovered. A good way to orient yourself is using the 15 I-frames that remain the same.

Good luck! And for those anxiously waiting for the Dragon II event tonight, remember that working on p-frames is a fantastic way to kill time.

I filled in this page: http://spacexlanding.wikispaces.com/Frames+Part+10

I created all new pictures for them. Most of them were the same as before but the following were (clearly) different and/or I had to change:  (these are the new numbers)

- 187: changed mmb
- 191: looks different, I did not change mmb, not sure if needed
- 193: changed mmb
- 194: needs work, quite different

I haven't tried these in a video, so please check!

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/30/2014 03:28 am
I filled in this page: http://spacexlanding.wikispaces.com/Frames+Part+10

I've been meaning to ask... what's the story behind the big grey rectangles in frames 186-188? Can you tell if it's an MPEG error or an NTSC video camera error?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Mariusuiram on 05/30/2014 03:33 am

Never in the history of space flight was a thread read by so many who understood so little...  ;-)


Its like magic. I check it every morning and skim through. My eyes glaze over for 90% as I look for a new gif or image. Then I feel guilty just looking for the pictures and I go back to give a concerted effort to understand the conversation. And then I just feel dense, old, and confused so I go back to looking at the pictures  ;)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/30/2014 04:29 am
I filled in this page: http://spacexlanding.wikispaces.com/Frames+Part+10

I've been meaning to ask... what's the story behind the big grey rectangles in frames 186-188? Can you tell if it's an MPEG error or an NTSC video camera error?

MPEG error. I was able to fix some of them in 185 by random bit flipping.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: beancounter on 05/30/2014 04:36 am
So we are moving to a society where the line is becoming significantly blurred between technology that appears to be magic to the majority leading to a downplaying of the importance of STEM educcation and eventually society regresses back to the discovery of the wheel.   :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/30/2014 04:56 am
Digit deltas. They are split into two tables because the p-frames often only show half of the rows. By comparing the pixels to the p-frame, you can determine what digits are being displayed even if the previous frame is missing.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/30/2014 05:06 am
On the subject of recovering the images of the clock in the bottom left corner, I think the TS recovery people might be able to help with that!

Thanks for looking into this. I still think that we will have a lot of illegible clock digits, but this would definitely reduce the number. What tools are you using to inspect the packets?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/30/2014 06:48 am

Ok, the first part of this is done. Unfortunately in PNG format the zip file is 85 MB, so trying to auto-upload it to the wiki is probably pushing it. No pun intended. Instead, I put it on Google Drive at this url:

https://drive.google.com/file/d/0B_wTAwfYg03FQ3p2UVlkZE9MckU/edit?usp=sharing

Also on BTSync at B76OWSNRL5HWE47N7SHT4ZIX2B3B63IXT

This is ~250 frames in PNG format generated by scraping the wiki and then running it through ffmpeg. Why is this useful? Well, you can turn these into a movie file with this command:

ffmpeg -y -start_number 40 -i landing_base_set/frame-%03d.png -c:v libx264 -r 15 -pix_fmt yuv420p out.mp4

Also, you can run a batch process on these images to do post-processing or add annotations.

The zip archive will be regenerated every hour if the wiki has seen any updates.

I updated the auto-build script to renumber the frames to follow the wiki. Frames 181-200 are now taken from rawsplit_final.ts.

Also the BTSync share includes a video version with the frame-rate fixed as SwissCheese suggested.

SwissCheese, can you share a PNG of the title card?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/30/2014 07:49 am
Hi guys,

I've been adding pictures to the wiki where no pictures were uploaded and noticed that if you downsized them the original was gone (you couldnt upscale them anymore).

I've now re-uploaded all images to 100% size in:

http://spacexlanding.wikispaces.com/Frames+Part+10
http://spacexlanding.wikispaces.com/Frames+Part+11
http://spacexlanding.wikispaces.com/Frames+Part+12
http://spacexlanding.wikispaces.com/Frames+Part+13
http://spacexlanding.wikispaces.com/Frames+Part+14
http://spacexlanding.wikispaces.com/Frames+Part+15

While you get less of an overview that way it is way easier for somebody else to check the quality of a frame. And quality is what we need most.

So from now on I think it's a good idea just to keep the pictures their original size.

@IainCole: we could sure use a "Save Image"-link ;)

Regards,

arnezami

[edit] Added Frames Part 11, 12, 14 and 15
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Helodriver on 05/30/2014 08:09 am
To all you dedicated video archaeologists digging up flipped bits, another by name shoutout by the man himself during the Q&A period after Dragon V2 reveal. Video of this to be posted tomorrow. Your efforts are being appreciated all the way at the highest levels.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/30/2014 10:57 am
So we are moving to a society where the line is becoming significantly blurred between technology that appears to be magic to the majority leading to a downplaying of the importance of STEM educcation and eventually society regresses back to the discovery of the wheel.   :)
(http://indylp.org/wp-content/uploads/2013/06/what-did-the-socialists-use-before-candles-electricity.jpg)

MPEG error. I was able to fix some of them in 185 by random bit flipping.

The uniform character and well-defined scope of the problem may well make it possible to identify a pattern leading to the root cause of the issue and an easy correction of the other pframes short of just ignoring them.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/30/2014 11:09 am
One p-frame looks very strange, with "too intense" colors, could anyone come with an explanation and maybe a solution to this?

It's frame 227 / part 12-19

mmb: 0:0:-1,30:1:10681,10:3:-1,31:3:19630

I checked the VOP quants in this section and that frame does look like a bit of an outlier. Here are the VOP quants in fixed_edit8_part_209.ts:
1  00100
2  00010
3  00001
4  00001
5  00010
6  00010
7  00010
8  00001
9  00001
10 00000
11 00001
12 00001
13 00001
14 00001
15 00001
16 00001
17 00001
18 00001
19 00111
20 00001

My first guess is that the correct VOP quant is 00001, and there's a double flip, but other values are possible. I attach a ts file where that VOP quant is set to 00001. It's at offset x43A9B so other values can be tried. I can't check it myself as I'm yet to establish a proper setup for multiframe work. Let me know how it looks.

Thanks to Quialiss (from the wiki) this issue is now fixed with the mmb command:

Quote
X:59:C0

:)

Look at frame 239 over here: http://spacexlanding.wikispaces.com/Frames+Part+12
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/30/2014 12:07 pm
Quote
X:59:C0

That was me actually ;) My first edit to the wiki.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/30/2014 12:24 pm
Thank you all for the hard work! Plenty of updates in the wiki on the past 8 hours. At this rate we will soon have a new version ready for distribution. 
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/30/2014 12:53 pm
Quote
X:59:C0

That was me actually ;) My first edit to the wiki.

Ah Cool! That was you! Nicely done :).
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: CuddlyRocket on 05/30/2014 01:27 pm
Thank you all for the hard work! Plenty of updates in the wiki on the past 8 hours. At this rate we will soon have a new version ready for distribution. 

I have only one criticism: Please be more quantitative than 'soon' so as to save me from the temptation of updating the thread every 5 minutes! :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/30/2014 01:52 pm
Thank you all for the hard work! Plenty of updates in the wiki on the past 8 hours. At this rate we will soon have a new version ready for distribution. 

I have only one criticism: Please be more quantitative than 'soon' so as to save me from the temptation of updating the thread every 5 minutes! :)

Soon as in a few days I would say.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 05/30/2014 01:58 pm
Ah Cool! That was you! Nicely done :).

It did not occur to me, but michaelni suggested this could work here: http://forum.nasaspaceflight.com/index.php?topic=34597.msg1206072#msg1206072

All I had to do was find the bits :)

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/30/2014 02:29 pm
Hi guys,

Important note to everyone using the wiki/online editor: the online editor has been updated lately. This means you have to use -3s instead of -1s for P-frames. Just keep that in mind.

I am currently replacing all -1s with -3s for all P-frames on the wiki and re-uploading all those images. Again. :o

Regards,

arnezami

PS. The -1s are actually causing the gray "bands" in the picture...
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/30/2014 03:12 pm
All I had to do was find the bits :)

Perhaps we can prevail upon @Shanuson to document the VOP header bits on the Wiki, if he hasn't already.  ;)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Comga on 05/30/2014 03:14 pm
To all you dedicated video archaeologists digging up flipped bits, another by name shoutout by the man himself during the Q&A period after Dragon V2 reveal. Video of this to be posted tomorrow. Your efforts are being appreciated all the way at the highest levels.

helodriver also posted this (http://forum.nasaspaceflight.com/index.php?topic=34841.msg1206773#msg1206773) in the event thread. 
"The telemetry from the CRS-3 booster splashdown was collected by an antenna made from a pizza dish stuck in his plane's window."

You are making an exquisite silk purse out of a sow's ear.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mhenderson on 05/30/2014 04:24 pm
iFrame 12 mmb and png


Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/30/2014 07:03 pm
Hi guys,

I've been adding pictures to the wiki where no pictures were uploaded and noticed that if you downsized them the original was gone (you couldnt upscale them anymore).

I've now re-uploaded all images to 100% size in:

http://spacexlanding.wikispaces.com/Frames+Part+10
http://spacexlanding.wikispaces.com/Frames+Part+11
http://spacexlanding.wikispaces.com/Frames+Part+12
http://spacexlanding.wikispaces.com/Frames+Part+13
http://spacexlanding.wikispaces.com/Frames+Part+14
http://spacexlanding.wikispaces.com/Frames+Part+15

While you get less of an overview that way it is way easier for somebody else to check the quality of a frame. And quality is what we need most.

So from now on I think it's a good idea just to keep the pictures their original size.

@IainCole: we could sure use a "Save Image"-link ;)

Regards,

arnezami

[edit] Added Frames Part 11, 12, 14 and 15

Added the save link. Tested in FF and Chrome
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/30/2014 07:09 pm

Ok, the first part of this is done. Unfortunately in PNG format the zip file is 85 MB, so trying to auto-upload it to the wiki is probably pushing it. No pun intended. Instead, I put it on Google Drive at this url:

https://drive.google.com/file/d/0B_wTAwfYg03FQ3p2UVlkZE9MckU/edit?usp=sharing

Also on BTSync at B76OWSNRL5HWE47N7SHT4ZIX2B3B63IXT

This is ~250 frames in PNG format generated by scraping the wiki and then running it through ffmpeg. Why is this useful? Well, you can turn these into a movie file with this command:

ffmpeg -y -start_number 40 -i landing_base_set/frame-%03d.png -c:v libx264 -r 15 -pix_fmt yuv420p out.mp4

Also, you can run a batch process on these images to do post-processing or add annotations.

The zip archive will be regenerated every hour if the wiki has seen any updates.

I updated the auto-build script to renumber the frames to follow the wiki. Frames 181-200 are now taken from rawsplit_final.ts.

Also the BTSync share includes a video version with the frame-rate fixed as SwissCheese suggested.

SwissCheese, can you share a PNG of the title card?

Here you have! In the movie, I replaced frames 1 to 40 by the title image.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/30/2014 07:26 pm

My current thinking on this is that rather than putting a duplicate timestamp in a matte, I'll redraw the known timestamps using the bitmap numerals already displayed. No interpolated timestamps added.

When I get a chance I also want to put together a system that automatically generates png sequences from the mmbs, zips them up, and uploads them to the wiki. That way anyone can download the frames and do their own batch processing on them.

Ok, the first part of this is done. Unfortunately in PNG format the zip file is 85 MB, so trying to auto-upload it to the wiki is probably pushing it. No pun intended. Instead, I put it on Google Drive at this url:

https://drive.google.com/file/d/0B_wTAwfYg03FQ3p2UVlkZE9MckU/edit?usp=sharing

Also on BTSync at B76OWSNRL5HWE47N7SHT4ZIX2B3B63IXT

This is ~250 frames in PNG format generated by scraping the wiki and then running it through ffmpeg. Why is this useful? Well, you can turn these into a movie file with this command:

ffmpeg -y -start_number 40 -i landing_base_set/frame-%03d.png -c:v libx264 -r 15 -pix_fmt yuv420p out.mp4

Also, you can run a batch process on these images to do post-processing or add annotations.

The zip archive will be regenerated every hour if the wiki has seen any updates.

Don't suppose you can also put a video somewhere on google drive too? If so I can probably automate pushing it to youtube or something.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/30/2014 08:00 pm
Here you have! In the movie, I replaced frames 1 to 40 by the title image.

Thank you!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/30/2014 08:01 pm
Don't suppose you can also put a video somewhere on google drive too? If so I can probably automate pushing it to youtube or something.

I can take care of this tonight.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 05/30/2014 08:07 pm
Sorry if someone already posted this, but your efforts are now linked on the SpaceX website :)

http://www.spacex.com/news/2014/04/29/first-stage-landing-video

And we hear Elon specifically mentioned you guys at the post Dragon V2 reveal Q&A!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Cinder on 05/30/2014 09:20 pm
I'd put the SpaceX logo above.

Ok, the first part of this is done. Unfortunately in PNG format the zip file is 85 MB, so trying to auto-upload it to the wiki is probably pushing it. No pun intended. Instead, I put it on Google Drive at this url:

https://drive.google.com/file/d/0B_wTAwfYg03FQ3p2UVlkZE9MckU/edit?usp=sharing

Also on BTSync at B76OWSNRL5HWE47N7SHT4ZIX2B3B63IXT

This is ~250 frames in PNG format generated by scraping the wiki and then running it through ffmpeg. Why is this useful? Well, you can turn these into a movie file with this command:

ffmpeg -y -start_number 40 -i landing_base_set/frame-%03d.png -c:v libx264 -r 15 -pix_fmt yuv420p out.mp4

Also, you can run a batch process on these images to do post-processing or add annotations.

The zip archive will be regenerated every hour if the wiki has seen any updates.

I updated the auto-build script to renumber the frames to follow the wiki. Frames 181-200 are now taken from rawsplit_final.ts.

Also the BTSync share includes a video version with the frame-rate fixed as SwissCheese suggested.

SwissCheese, can you share a PNG of the title card?

Here you have! In the movie, I replaced frames 1 to 40 by the title image.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/30/2014 09:27 pm
Don't suppose you can also put a video somewhere on google drive too? If so I can probably automate pushing it to youtube or something.

I can take care of this tonight.

Cool, I think I pretty much have everything set up. All I need is a public accessible URL to hit that has the latest video. Also you need to send a POST to a url when it's uploaded so I know when it's changed, that do-able?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: catdlr on 05/30/2014 09:32 pm
Sorry if someone already posted this, but your efforts are now linked on the SpaceX website :)

http://www.spacex.com/news/2014/04/29/first-stage-landing-video

And we hear Elon specifically mentioned you guys at the post Dragon V2 reveal Q&A!

YES!!  Congratulations to you Chris and the great efforts of the NSF team.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/30/2014 09:36 pm
Hi guys,

Important note to everyone using the wiki/online editor: the online editor has been updated lately. This means you have to use -3s instead of -1s for P-frames. Just keep that in mind.

I am currently replacing all -1s with -3s for all P-frames on the wiki and re-uploading all those images. Again. :o

Regards,

arnezami

PS. The -1s are actually causing the gray "bands" in the picture...

The online editor gives a correct image with the -3 but does not seem to update the macroblock information and errors. Is it on purpose?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/30/2014 09:38 pm
Hi guys,

Important note to everyone using the wiki/online editor: the online editor has been updated lately. This means you have to use -3s instead of -1s for P-frames. Just keep that in mind.

I am currently replacing all -1s with -3s for all P-frames on the wiki and re-uploading all those images. Again. :o

Regards,

arnezami

PS. The -1s are actually causing the gray "bands" in the picture...

The online editor gives a correct image with the -3 but does not seem to update the macroblock information and errors. Is it on purpose?

I think I know what this might be, I'll take a look in a bit.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/30/2014 09:43 pm
Don't suppose you can also put a video somewhere on google drive too? If so I can probably automate pushing it to youtube or something.

I can take care of this tonight.

Cool, I think I pretty much have everything set up. All I need is a public accessible URL to hit that has the latest video. Also you need to send a POST to a url when it's uploaded so I know when it's changed, that do-able?

Yep. URL?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/30/2014 09:44 pm
Hi guys,

Important note to everyone using the wiki/online editor: the online editor has been updated lately. This means you have to use -3s instead of -1s for P-frames. Just keep that in mind.

I am currently replacing all -1s with -3s for all P-frames on the wiki and re-uploading all those images. Again. :o

Regards,

arnezami

PS. The -1s are actually causing the gray "bands" in the picture...

The online editor gives a correct image with the -3 but does not seem to update the macroblock information and errors. Is it on purpose?

I think I know what this might be, I'll take a look in a bit.

Oh and while you are here, would it be possible to add a slider to enhance the image contrast? It helps a lot for repairing the p-frames but I have to use an image viewer to do this currently.

Thank you again for your great work!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/30/2014 09:45 pm
Don't suppose you can also put a video somewhere on google drive too? If so I can probably automate pushing it to youtube or something.

I can take care of this tonight.

Cool, I think I pretty much have everything set up. All I need is a public accessible URL to hit that has the latest video. Also you need to send a POST to a url when it's uploaded so I know when it's changed, that do-able?

Yep. URL?

Sent in PM as it contains an API key =)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/30/2014 09:45 pm
Hi guys,

Important note to everyone using the wiki/online editor: the online editor has been updated lately. This means you have to use -3s instead of -1s for P-frames. Just keep that in mind.

I am currently replacing all -1s with -3s for all P-frames on the wiki and re-uploading all those images. Again. :o

Regards,

arnezami

PS. The -1s are actually causing the gray "bands" in the picture...

The online editor gives a correct image with the -3 but does not seem to update the macroblock information and errors. Is it on purpose?

I think I know what this might be, I'll take a look in a bit.

Oh and while you are here, would it be possible to add a slider to enhance the image contrast? It helps a lot for repairing the p-frames but I have to use an image viewer to do this currently.

Thank you again for your great work!

If I even had the slightest idea how to do that I'd be all over it =)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/30/2014 09:52 pm
Hi guys,

Important note to everyone using the wiki/online editor: the online editor has been updated lately. This means you have to use -3s instead of -1s for P-frames. Just keep that in mind.

I am currently replacing all -1s with -3s for all P-frames on the wiki and re-uploading all those images. Again. :o

Regards,

arnezami

PS. The -1s are actually causing the gray "bands" in the picture...

The online editor gives a correct image with the -3 but does not seem to update the macroblock information and errors. Is it on purpose?

I think I know what this might be, I'll take a look in a bit.

Oh and while you are here, would it be possible to add a slider to enhance the image contrast? It helps a lot for repairing the p-frames but I have to use an image viewer to do this currently.

Thank you again for your great work!

If I even had the slightest idea how to do that I'd be all over it =)

hmmmm this might do it http://camanjs.com/
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/30/2014 09:57 pm
Hi guys,

Important note to everyone using the wiki/online editor: the online editor has been updated lately. This means you have to use -3s instead of -1s for P-frames. Just keep that in mind.

I am currently replacing all -1s with -3s for all P-frames on the wiki and re-uploading all those images. Again. :o

Regards,

arnezami

PS. The -1s are actually causing the gray "bands" in the picture...

The online editor gives a correct image with the -3 but does not seem to update the macroblock information and errors. Is it on purpose?

Can you give me a specific example of what isn't working and what you expect to see?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/30/2014 10:13 pm
Hi guys,

Important note to everyone using the wiki/online editor: the online editor has been updated lately. This means you have to use -3s instead of -1s for P-frames. Just keep that in mind.

I am currently replacing all -1s with -3s for all P-frames on the wiki and re-uploading all those images. Again. :o

Regards,

arnezami

PS. The -1s are actually causing the gray "bands" in the picture...

The online editor gives a correct image with the -3 but does not seem to update the macroblock information and errors. Is it on purpose?

Can you give me a specific example of what isn't working and what you expect to see?

Just take any p-frame with errors, and write 0:0:-3 (should discard the whole frame) the yellow squares stay and the information on top of the frame still shows the discarded information.

(difficult to write from a mobile phone)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/30/2014 10:47 pm
Hi guys,

Important note to everyone using the wiki/online editor: the online editor has been updated lately. This means you have to use -3s instead of -1s for P-frames. Just keep that in mind.

I am currently replacing all -1s with -3s for all P-frames on the wiki and re-uploading all those images. Again. :o

Regards,

arnezami

PS. The -1s are actually causing the gray "bands" in the picture...

The online editor gives a correct image with the -3 but does not seem to update the macroblock information and errors. Is it on purpose?

Can you give me a specific example of what isn't working and what you expect to see?

Just take any p-frame with errors, and write 0:0:-3 (should discard the whole frame) the yellow squares stay and the information on top of the frame still shows the discarded information.

(difficult to write from a mobile phone)

That seems to be what's coming back from the log :/ Dunno if it's specific to the online editor?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mlindner on 05/31/2014 01:26 am
Just want to say this is an amazing job you guys have done! I had to drop out from doing it because real life intervened in my ability to have time to assist. Hopefully I'll have time to catch up on the some 30 pages that have been posted since I last helped.

Just as a quick question, is there still need to repair I-frames? I notice that many of the I-frames have not advanced much in repair state from when I was last involved.
Hi mlindner! Glad you're back :)

There is definitely still a need for work on the I-frames. I think many moved their focus to the P-frames because making video is so tempting and rewarding ;)

But improving the I-frames clearly makes the 19 P-frames that follow much better!

Your excellent work on frame 169 shows that!

Regards,

arnezami

Where's the most recent version of the source code people have been using? Presumably people are no longer using my out-of-date repo anymore?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/31/2014 02:16 am
Just want to say this is an amazing job you guys have done! I had to drop out from doing it because real life intervened in my ability to have time to assist. Hopefully I'll have time to catch up on the some 30 pages that have been posted since I last helped.

Just as a quick question, is there still need to repair I-frames? I notice that many of the I-frames have not advanced much in repair state from when I was last involved.
Hi mlindner! Glad you're back :)

There is definitely still a need for work on the I-frames. I think many moved their focus to the P-frames because making video is so tempting and rewarding ;)

But improving the I-frames clearly makes the 19 P-frames that follow much better!

Your excellent work on frame 169 shows that!

Regards,

arnezami

Where's the most recent version of the source code people have been using? Presumably people are no longer using my out-of-date repo anymore?

Is good to see you back mlindner! Let me try yo give you a fast update.
- we have are still using MMB codes to recover I and (most recently) p frames. We now have many partially fixed, but none as good as your original iframe.
- There has been several rounds of improvements on the .ts files since you left. After coerced it came try1, edit5 and edit8 (all with more frames recovered).  The latest one is called raw_final_fixed.ts. From that one we were finally able to extract all I and P frames from the video. The zip file was posted here some pages back.
- There have been also improvements on the FFmpeg utility. The most important is that it now supports multi-frame MMB codes to fix both frames and Iframes. It's still hosted in GitHub. (http://spacexlanding.wikispaces.com/Custom+FFmpeg)
- We also have a terrific online tool to fix the frames online, that is capable of identifying errors, showing previews and logs -among other things. You can find it here: http://spacex2.slapbet.org/

You should be able to update yourself pretty fast by reading the wiki home page.
http://spacexlanding.wikispaces.com
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 05/31/2014 03:32 am
I wonder if we could get one of these for the next flight, to replace the pizza pan...

The Moon Gets Broadband Wireless Connection (http://www.iflscience.com/space/moon-gets-broadband-wireless-connection)
Quote
The Lunar Laser Communication Demonstration (LLCD) transmitted over 384,633 kilometers between here and the moon at a download rate of 622 megabits per second. They also sent data from Earth to the moon at 19.44 megabits per second. That’s 4,800 times faster than the best radio frequency uplink ever used.

(http://www.iflscience.com/sites/www.iflscience.com/files/styles/ifls_large/public/blog/%5Bnid%5D/optical%20module%20drawing.png)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: MarsInMyLifetime on 05/31/2014 05:02 am
Nah, he got exactly the branding he was hoping for by using his low-cost, MacGuyverish patch antenna to get proof of this landing's success. That corrupted stream with its "can do" back story is turning into a company legacy that will pay off for years down the road. They'll improve their game as they go, but on that stormy day, a million dollar cat toy would not have been the right tool for the job.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/31/2014 08:49 am
Ok, the first part of this is done. Unfortunately in PNG format the zip file is 85 MB, so trying to auto-upload it to the wiki is probably pushing it. No pun intended. Instead, I put it on Google Drive at this url:

https://drive.google.com/file/d/0B_wTAwfYg03FQ3p2UVlkZE9MckU/edit?usp=sharing

Also on BTSync at B76OWSNRL5HWE47N7SHT4ZIX2B3B63IXT

This is ~250 frames in PNG format generated by scraping the wiki and then running it through ffmpeg. Why is this useful? Well, you can turn these into a movie file with this command:

ffmpeg -y -start_number 40 -i landing_base_set/frame-%03d.png -c:v libx264 -r 15 -pix_fmt yuv420p out.mp4

Also, you can run a batch process on these images to do post-processing or add annotations.

The zip archive will be regenerated every hour if the wiki has seen any updates.

Don't suppose you can also put a video somewhere on google drive too? If so I can probably automate pushing it to youtube or something.

This is done. The video is here:

http://adama.nocdirect.com/~wronkiew/spx_crs3/landing_base_set.mp4

Whenever it is regenerated your URL will get a POST.

I added SwissCheese's title card. Frames 161-200 and 281-283 now come from the fixed .ts files.

I have noticed some block bugs that aren't showing on the wiki, which might be because my copy of the split .ts files is out of date. Do you have a newer set of split (i + 19p) files?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 05/31/2014 09:11 am
I think he is using the one i send him, which is the same as the last one i uploaded  here  (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1205739#msg1205739)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/31/2014 09:26 am
Ok, the first part of this is done. Unfortunately in PNG format the zip file is 85 MB, so trying to auto-upload it to the wiki is probably pushing it. No pun intended. Instead, I put it on Google Drive at this url:

https://drive.google.com/file/d/0B_wTAwfYg03FQ3p2UVlkZE9MckU/edit?usp=sharing

Also on BTSync at B76OWSNRL5HWE47N7SHT4ZIX2B3B63IXT

This is ~250 frames in PNG format generated by scraping the wiki and then running it through ffmpeg. Why is this useful? Well, you can turn these into a movie file with this command:

ffmpeg -y -start_number 40 -i landing_base_set/frame-%03d.png -c:v libx264 -r 15 -pix_fmt yuv420p out.mp4

Also, you can run a batch process on these images to do post-processing or add annotations.

The zip archive will be regenerated every hour if the wiki has seen any updates.

Don't suppose you can also put a video somewhere on google drive too? If so I can probably automate pushing it to youtube or something.

This is done. The video is here:

http://adama.nocdirect.com/~wronkiew/spx_crs3/landing_base_set.mp4

Whenever it is regenerated your URL will get a POST.

I added SwissCheese's title card. Frames 161-200 and 281-283 now come from the fixed .ts files.

I have noticed some block bugs that aren't showing on the wiki, which might be because my copy of the split .ts files is out of date. Do you have a newer set of split (i + 19p) files?

Right the videos end up here: https://www.youtube.com/spacexlandingrestore (https://www.youtube.com/spacexlandingrestore)

Hopefully it all works :D

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/31/2014 10:16 am
This is done. The video is here:

http://adama.nocdirect.com/~wronkiew/spx_crs3/landing_base_set.mp4

Whenever it is regenerated your URL will get a POST.

I added SwissCheese's title card. Frames 161-200 and 281-283 now come from the fixed .ts files.

I have noticed some block bugs that aren't showing on the wiki, which might be because my copy of the split .ts files is out of date. Do you have a newer set of split (i + 19p) files?
Cool  8)

I was just wondering though. Sometimes we put real comments combined with actuall mmbs in the wiki like this: (see: frame 63 (http://spacexlanding.wikispaces.com/Frames+Part+4))

Quote
25:1:44861,41:1:-3,
38:4:9585,16:16:-3,
0:18:45141,18:27:-3,
43:27:68567,11:28:-3,
17:28:70673,7:29:-3,
8:29:71924,43:29:-3
unsure about the bottom part (btw
43:29:-3 is needed for next frame)

How do you make sure you extract the correct mmb from something like this? Should we adhere to some kind of format? Or should we not post comments at the place we post mmbs?

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/31/2014 10:46 am
This is done. The video is here:

http://adama.nocdirect.com/~wronkiew/spx_crs3/landing_base_set.mp4

Whenever it is regenerated your URL will get a POST.

I added SwissCheese's title card. Frames 161-200 and 281-283 now come from the fixed .ts files.

I have noticed some block bugs that aren't showing on the wiki, which might be because my copy of the split .ts files is out of date. Do you have a newer set of split (i + 19p) files?
Cool  8)

I was just wondering though. Sometimes we put real comments combined with actuall mmbs in the wiki like this: (see: frame 63 (http://spacexlanding.wikispaces.com/Frames+Part+4))

Quote
25:1:44861,41:1:-3,
38:4:9585,16:16:-3,
0:18:45141,18:27:-3,
43:27:68567,11:28:-3,
17:28:70673,7:29:-3,
8:29:71924,43:29:-3
unsure about the bottom part (btw
43:29:-3 is needed for next frame)

How do you make sure you extract the correct mmb from something like this? Should we adhere to some kind of format? Or should we not post comments at the place we post mmbs?

Regards,

arnezami

Perhaps keeping this file https://github.com/IainCole/SpaceXVideoApp2/blob/master/data.json up to date based off the wiki might be useful, for 2 reasons, it gives us a solid data structure for MMBs so that people can use it to generate videos without having to parse the wiki. Also the online editor will take those values as defaults so people don't have to copy everything across from the wiki to work on one part of a segment.


Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/31/2014 12:14 pm
Hi guys,

I've been busy trying to let the decoder output the bits and their interpretation of all the (macro)block parts. And I think I have succeeded. I still need to clean it up and make it compatible/configurable with the existing codebase. But I wanted to share some details so others (mainly @IainCole) can already start thinking/working on an extention of the online tool. Also for others to get used to the idea.

I think it will bring us to the next level.  8)

Here is the output of the extra log: (the stream of bits was a bit too wide for the forum so I put a newline in it)

Quote
BITLOG:000:00:00:cbpc:550:551:0
BITLOG:000:00:00:ac_pred:551:552:1
BITLOG:000:00:00:cbpy:552:554:15
BITLOG:000:00:00:dc_lum0_size:554:557:4
BITLOG:000:00:00:dc_lum0_level:557:561:-13
BITLOG:000:00:00:dct561:578:0:3:-6
BITLOG:000:00:00:dct578:585:1:1:-1
BITLOG:000:00:00:dc_lum1_size:585:588:4
BITLOG:000:00:00:dc_lum1_level:588:592:8
BITLOG:000:00:00:dct592:602:1:12:-1
BITLOG:000:00:00:dc_lum2_size:602:604:1
BITLOG:000:00:00:dc_lum2_level:604:605:-1
BITLOG:000:00:00:dct605:608:0:0:1
BITLOG:000:00:00:dct608:614:0:2:-1
BITLOG:000:00:00:dct614:619:0:1:-1
BITLOG:000:00:00:dct619:649:0:29:1
BITLOG:000:00:00:dct649:659:1:14:-1
BITLOG:000:00:00:dc_lum3_size:659:661:1
BITLOG:000:00:00:dc_lum3_level:661:662:-1
BITLOG:000:00:00:dct662:667:1:0:-1
BITLOG:000:00:00:dc_chrom4_size:667:671:4
BITLOG:000:00:00:dc_chrom4_level:671:675:13
BITLOG:000:00:00:dc_chrom5_size:675:678:3
BITLOG:000:00:00:dc_chrom5_level:678:681:-7

BITLOG:000:00:00:bits:550:681:11110010010000001100001101110011111001100000001001111101
000101111110100000111100111101000000000001100001000111100111100011101001000

What you can see here is the bit-by-bit decoding of the first macroblock of frame 181.

Red: These are the macroblock header parts: mainly cbpc, ac_pred and cbpy.
Blue: These are the DC-values: the size (in bits) of the level and the level itself. Bitflips here are very damaging and easily fixed! This is on a block-level: 16x16px or 8x8px.
Green: These are the DCT values. This is basically sub-block information. So sub-16x16 or sub-8x8 data.

Purple: these are the actual bits in the macroblock. The red/blue/green parts above it are encoded by these bits. Each log-line contains it's start and end position. So they can easily be matched with these purple bits (by the editor I mean ;)).

I've been thinking of the easiest way to incorporate this into the online editor. What I'm thinking of is the following:

* When you double-click on a macro-block an additional (large) area becomes visible to the right side of the editor. If you have a wide monitor there should be plenty of room there.
* In the backround the application calls the server/ffmpeg with an additional command with the double-clicked macroblock x,y as argument (exact syntax still to-be-determined). This gives back the normal log plus the log shown above (if needed you could also omit the usual pos/size log). However it only gives BITLOG lines for the macroblock clicked and a few following it. So it logs only what you want to know.
* The extra log gives all bits for a macroblock (purple here). The application recognizes the BITLOG and for each part (red/blue/green) it creates a <span> with the corresponding bits (from purple) in it. It adds a "tittle" containing the corresponding BITLOG-line (without the "BITLOG:000:00:00:" part). So if you hover your mouse over it it will show you the meaning of the bits. It also colors the spans according to their type (red/blue/green). It puts these spans next to each other. And for every macroblock there is a break.

So you would get something like this:

Quote
        <div>
            <span class="red" title="cbpc:550:551:0">1</span>
            <span class="red" title="ac_pred:551:552:1">1</span>
            <span class="red" title="cbpy:552:554:15">11</span>
            <span class="blue" title="dc_lum0_size:554:557:4">001</span>
            <span class="blue" title="dc_lum0_level:557:561:-13">0010</span>
            <span class="green" title="dct561:578:0:3:-6">00000110000110111</span>
            <span class="green" title="dct578:585:1:1:-1">0011111</span>
            <span class="blue" title="dc_lum1_size:585:588:4">001</span>
            <span class="blue" title="dc_lum1_level:588:592:8">1000</span>
        <div>
   

Resulting in this:

(http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=586548;image)

You can then hover those strings of colored bits and see what they represent. If something is off (for example a DC is way too high) you can now see which bits are responsible for that. Ideally you want to be able to click on those bits to flip them (that's not for a first version I think). And it would also be nice  to see the macroblock (2x-4x larger than original) left next to the string of bits. So you can see the effect. But just having this information is already very revealing I think. It will help bitflipping a lot already.

I'm still working on this but let me know if you have ideas/suggestions.

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/31/2014 12:30 pm
Looks interesting, is there still a restriction on the number of bitflip operations? I seem to remember there was a max of 8 at some point
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/31/2014 01:23 pm
Perhaps keeping this file https://github.com/IainCole/SpaceXVideoApp2/blob/master/data.json up to date based off the wiki might be useful, for 2 reasons, it gives us a solid data structure for MMBs so that people can use it to generate videos without having to parse the wiki. Also the online editor will take those values as defaults so people don't have to copy everything across from the wiki to work on one part of a segment.
I think keeping an up-to-date file in a central place is a good idea.

But I'm not sure how to keep up with all the updates on the wiki. If you could somehow show the differences between the .json and the wiki that would be great. Then you can edit the .json if needed.

Alternatively you could have a centralized database with a little web-app around it where you can enter/edit mmbs.

Not sure what would work best.

[edit] As to your question about the possible restriction of bitflips: I dont know, will have to look into that.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 05/31/2014 04:00 pm
Hi guys,

Important note to everyone using the wiki/online editor: the online editor has been updated lately. This means you have to use -3s instead of -1s for P-frames. Just keep that in mind.

I am currently replacing all -1s with -3s for all P-frames on the wiki and re-uploading all those images. Again. :o

Regards,

arnezami

PS. The -1s are actually causing the gray "bands" in the picture...

The online editor gives a correct image with the -3 but does not seem to update the macroblock information and errors. Is it on purpose?

Can you give me a specific example of what isn't working and what you expect to see?

Just take any p-frame with errors, and write 0:0:-3 (should discard the whole frame) the yellow squares stay and the information on top of the frame still shows the discarded information.

(difficult to write from a mobile phone)

That seems to be what's coming back from the log :/ Dunno if it's specific to the online editor?

Quialiss found out that explicitly writing 0:0:66 (first macroblock) solved this issue. Does someone have an idea about what's causing this?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 05/31/2014 04:25 pm

Alternatively you could have a centralized database with a little web-app around it where you can enter/edit mmbs.

Wow, I like this idea. The way we are sharing codes is really suboptimal. However,  we should really avoid building yet another system. What if:
- we start by creating a database in a server to store all current MMB codes.
- we then could use the existing online tool to read and write to the dataset. I'm thinking about save and load MMB code buttons.  8)
- In order to simplify the wiki update process, we could add  small widgets in each table/row to display the latest MMB code from this server.
- we could even go beyond, and have an export image to wiki button in the online tool, that saves the file as framenumber.jpg in the wiki site, overwriting previous image files and thus directly updating the table

If we go in this direction, the wiki will become a place to monitor progress (to see what frames are fixed, partially fixed or untouched), and only a few of us would require to do edits. All the real interaction of the people working with frames, in turn, will happen via the online tool :).

I doesn't seem like rocket science, but it does seem like a lot of work specially on the online tool side (we don't want you to hate us Iaincole). Ideas? Any of our silent readers is up to the challenge of contributing on this task?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/31/2014 05:21 pm
Hi guys,

Here is the source code for logging macroblock data on a bit-level. :)

The new command is:

Quote
-mb_bitlog <framenr>:<mb_x>:<mb_y>:<nr_of_mbs_to_log>

So for example the following command will log the macroblock data from frame 0 (first frame in file), from macroblock 3,10 to macroblock 23,10:

Quote
-mb_bitlog 0:3:10:20

These files are contained in the zip file that is attached:

libavcodec/options_table.h
libavcodec/avcodec.h
libavcodec/h263dec.c
libavcodec/mpeg4videodec.c

I used the latest version and changed those files so if you have the latest version (with the "bruteforcing"-addition) then you should be fine.

For further explanation of what this does go here (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1205842#msg1205842) and here (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1207391#msg1207391).

I might write a more thorough explanation of what all the parts in the macroblocks are (as far as I understand them now) if there is a need for that. And which tables you probably need (from the MPEG4 spec) in order to put this to better use. But you should already be able to get quite a bit information out of this.

Good luck and have fun!

Regards,

arnezami

@IainCole: if you have any trouble please let me know.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: cartman on 05/31/2014 05:58 pm
Ok, so i'm a linux guy with web servers and the works, and would like to help. First of all, i would like to help with recovering the video. Is there an updated guide of where to start? should i use arnezami's latest build?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 05/31/2014 06:10 pm
Ok, so i'm a linux guy with web servers and the works, and would like to help. First of all, i would like to help with recovering the video. Is there an updated guide of where to start? should i use arnezami's latest build?
On the wiki (http://spacexlanding.wikispaces.com/) there is a section calls "Work in Progress " which has an "ACTIVE" part which contains two links: one to a tutorial one to where we post our results.

There is also a "Reconstruction Tools" section in the wiki. The two top links (one for P-frames one for I-frames) go to the online editors. There you can try to fix the frames.

You don't need my latest addition to the ffmpeg-tool to help for now. What we are developing is for making the online tools more powerful.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/31/2014 07:25 pm
Cool  8)

I was just wondering though. Sometimes we put real comments combined with actuall mmbs in the wiki like this: (see: frame 63 (http://spacexlanding.wikispaces.com/Frames+Part+4))

Quote
25:1:44861,41:1:-3,
38:4:9585,16:16:-3,
0:18:45141,18:27:-3,
43:27:68567,11:28:-3,
17:28:70673,7:29:-3,
8:29:71924,43:29:-3
unsure about the bottom part (btw
43:29:-3 is needed for next frame)

How do you make sure you extract the correct mmb from something like this? Should we adhere to some kind of format? Or should we not post comments at the place we post mmbs?

Regards,

arnezami

Yes, that line would have messed up the scraper, which is why I had not transitioned it yet. I think I will simplify the parsing so it panics and sends me an email when it sees something like that instead of trying to send it to ffmpeg. Part 4 is fixed now. I moved frame 63's instructions to a text file, and the auto-build uses rawfinal for that segment.

Would it help if I had it upload the generated segment mmbs to the wiki?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/31/2014 07:52 pm
Youtube video exports seem to be running along nicely :) https://www.youtube.com/user/spacexlandingrestore (https://www.youtube.com/user/spacexlandingrestore)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: cscott on 05/31/2014 08:01 pm
Yes, that line would have messed up the scraper, which is why I had not transitioned it yet. I think I will

Can you just use a regexp to ignore any lines that don't match the expected format?  Then it should be straightforward to add whatever sort of comment you like, so long as it's obvious isn't not a valid option.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: maschnitz on 05/31/2014 08:04 pm
I doesn't seem like rocket science, but it does seem like a lot of work specially on the online tool side (we don't want you to hate us Iaincole). Ideas?

VCSes - version control systems - are really good at moralec's database problem. One idea is to shoehorn this into a VCS by using a VCS as storage.

Here's one way to do that: force MMB lists into a VCS friendly format, by
a) breaking up the MMBs into one per line [VCSes tend to work best on whole lines at a time] and
b) sort the MMBs into a standard sort order, most likely the raster order most folks are already used to.  [EDIT: I guess this isn't strictly necessary, but may make IainCole's life easier]
c) Then you need to always check in and out of the VCS via something that understands the format. IainCole's website, or some other website, for example, would be the only entity with access to the VCS. This website would be responsible for parsing raw MMBs, and providing the MMBs back in a ffmpeg-friendly format.
d) Ideally this file format would have comments to allow for notes and simple alternatives. The VCS would be responsible for full 'branches' [git/mercurial is good for this].

And then a flat file format might come in handy in other ways.

There are other ways: you could use raw RCS diffs to manage things; you could find a diff program that's really good at big long lines (most aren't); you can code up the delta checking by hand; you can simply store all the MMBs ever in groups, normalized in the database, for users to sort out; etc.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/31/2014 08:04 pm
Youtube video exports seem to be running along nicely :) https://www.youtube.com/user/spacexlandingrestore (https://www.youtube.com/user/spacexlandingrestore)

Oh, right, in the new version 0:0:-3 skips a pframe, not 0:0:-1.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 05/31/2014 08:09 pm
Yes, that line would have messed up the scraper, which is why I had not transitioned it yet. I think I will

Can you just use a regexp to ignore any lines that don't match the expected format?  Then it should be straightforward to add whatever sort of comment you like, so long as it's obvious isn't not a valid option.

I think I'd rather have it error out on unexpected input rather than risk losing mmbs. If it makes any mistakes it's going to be really difficult to figure out where they are.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 05/31/2014 08:18 pm
I doesn't seem like rocket science, but it does seem like a lot of work specially on the online tool side (we don't want you to hate us Iaincole). Ideas?

VCSes - version control systems - are really good at moralec's database problem. One idea is to shoehorn this into a VCS by using a VCS as storage.

Here's one way to do that: force MMB lists into a VCS friendly format, by
a) breaking up the MMBs into one per line [VCSes tend to work best on whole lines at a time] and
b) sort the MMBs into a standard sort order, most likely the raster order most folks are already used to.  [EDIT: I guess this isn't strictly necessary, but may make IainCole's life easier]
c) Then you need to always check in and out of the VCS via something that understands the format. IainCole's website, or some other website, for example, would be the only entity with access to the VCS. This website would be responsible for parsing raw MMBs, and providing the MMBs back in a ffmpeg-friendly format.
d) Ideally this file format would have comments to allow for notes and simple alternatives. The VCS would be responsible for full 'branches' [git/mercurial is good for this].

And then a flat file format might come in handy in other ways.

There are other ways: you could use raw RCS diffs to manage things; you could find a diff program that's really good at big long lines (most aren't); you can code up the delta checking by hand; you can simply store all the MMBs ever in groups, normalized in the database, for users to sort out; etc.

This was pretty much exactly my suggestion here: http://forum.nasaspaceflight.com/index.php?topic=34597.msg1207365#msg1207365

The added bonus being that the online editor is already using that file to load MMBs.

People can either submit MMBs by pull request, or a few key people can be made collaborators on that git repo to add them in based off submissions to the wiki.

Edit: There's obviously some administrative overhead with making sure that the wiki / json file are in sync, but short of writing a new system to do the job of both things, I think it's the simplest option.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: cscott on 05/31/2014 08:29 pm
VCSes - version control systems - are really good at moralec's database problem. One idea is to shoehorn this into a VCS by using a VCS as storage.

Suggestion: store the data in github, and use https://github.com/creationix/js-git to access it/commit new versions.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Lourens on 05/31/2014 10:02 pm
Hi guys,

I've been busy trying to let the decoder output the bits and their interpretation of all the (macro)block parts. And I think I have succeeded. I still need to clean it up and make it compatible/configurable with the existing codebase. But I wanted to share some details so others (mainly @IainCole) can already start thinking/working on an extention of the online tool. Also for others to get used to the idea.

I think it will bring us to the next level.  8)

Thanks, arnezami, great work! I've been using your previous version with my automated bit flipper, and it still doesn't work. But that's due to the objective function not yet being good enough, and maybe (hopefully not) due to the search algorithm being too greedy. I'll keep working on it, and I'll integrate the new version.

In the mean time, I've seen a few posts by people saying that they have no idea what's going on. That's usually my position (especially with ISS stuff, acronym soup alert!), but now I'm coming from the other side, so I'll make an attempt at an explanation of all the stuff that's going on here. Hopefully, that will allow more people to help fix things. I'll probably make a mistake here and there and/or oversimplify something, so if you spot an error, let me know and I'll fix the text.

MPEG4

The damaged video stream that SpaceX made available is in the MPEG4 format. MPEG4 is a standard that says how you should convert a video into a list of ones and zeros and back, in such a way as to make that list nice and short so you can download it quickly. A video here is a sequence of pictures, each of which consists of a rectangle of pixels. Each of the pixels has a colour, expressed as the amount of Red light, the amount of Green light, and the amount of Blue light. That's what we're trying to pack into the MPEG4 file. What I'm going to describe here is the procedure for converting video from the Falcon 9 camera into the MPEG4 bitstream that we have. To turn this back into a picture, you just do everything in reverse.

Downsampling and colour space conversion

So, we take the first picture, and we split it up into a set of 16x16 pixel squares, called macroblocks (MBs), which we can process separately. The SpaceX video has 44 MBs horizontally and 30 MBs vertically. We convert each pixel in the macroblock from Red-Green-Blue (RGB) into a brightness value (Luma or Y) and two additional values that describe the colour (Chroma or C). Since the human eye doesn't see as much detail in colour as it does in brightness, we the average each 2x2 pixel block of chroma values, halving the resolution of the two sets of chroma values to 8x8. So, of the original 16x16x3 = 768 numbers in the macroblock, we now have 16x16 + 8x8 + 8x8 = 384 numbers. That's a 50% reduction in space already! If we split the 16x16 block of Luma numbers into 4 8x8 blocks, then we have a total of 6 8x8 blocks per macroblock.

Discrete Cosine Transform

Next, we apply to each 8x8 block a mathematical trick called the Discrete Cosine Transform (DCT). The outcome of the DCT is another 8x8 block of numbers, one of which is simply the average of all 64 numbers, and the other 63 describe the differences (e.g. how much brighter the left half is than the right half, and 62 other patterns (http://en.wikipedia.org/wiki/File:Dctjpeg.png)). The average is called the DC value, the differences are the AC values. Another way of interpreting this is to say that the DC value describes the overall brightness and colour, while the AC values describe the texture of the block.

If you look at arnezami's BITLOG output (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1207391#msg1207391), then you see dc_lum0, dc_lum1, dc_lum2, dc_lum3, dc_chrom4 and dc_chrom5. These are the DC values of the six blocks in the macroblock.

Quantisation

In itself, the DCT does not save any space. The reason we do this DCT, is that it turns out that for most blocks in your typical image, most of the resulting 64 values are 0, or close to 0. Also, small errors in these numbers do not make the image look much worse. So, the next step is to quantise these values, which basically means dividing them by a given number, and rounding to a whole number again. Smaller values use less digits (bits, in this case), so that saves space, and if we store the value that we divided by, then the player can multiply by it and get almost the correct values to put into its inverse DCT.

If you quantise too much, then you get those typical artifacts around sharp edges that you sometimes see in JPEG images. Do it right, and you can save quite a lot of space with little degradation in image quality. In this post (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1205415#msg1205415), another problem was found: transmission damage resulted in the decoder reading a quantisation factor of 4, rather than 1. Thus, it multiplied in this case the chroma (colour) values by 4 instead of by 1, resulting in overly intense colours. Changing the 4 back to a 1 fixed the image.

DC and AC prediction

The basic idea of the DCT is that all those pixels in a block usually have quite a lot in common (DC value), so it helps to store what they have in common once, and then add only a short description of the few differences (the quantised AC values) here and there. After we've done the above for all the macroblocks in an image, we can try to apply the same trick across the macroblocks. For example, adjacent macroblocks often have roughly the same colour, and thus similar DC values for their six blocks. If we order the macroblocks in reading order (left-to-right and top-to-bottom), then we can store the difference of the DC values of the six blocks relative to the corresponding ones for the left or top neighbour macroblock. This again results in shorter numbers and space savings. Actually, since texture is also often similar between adjacent macroblocks, we can do the same with the AC values (if ac_pred equals 1, that's what's happening). And with the quantisation multiplier: if you see a dquant value then that's a difference as well.

It gets even better. Sometimes, the differences between a current block and the corresponding block in a neighbouring MB that we're using for reference, are so small that we can just drop them altogether. In that case, we don't need to store anything about this block at all. The cbpc (for chroma blocks) and cbpy (luma blocks) values specify for which blocks in this macroblock information is stored (see tables B-6 through B-11 in the standard document linked from the wiki).

This does introduce a dependency between macroblocks. If one macroblock gets damaged and is decoded incorrectly, all the other macroblocks that are encoded as differences relative to this one will also be wrong. Overwriting the damaged macroblock with one that looks about right gives the dependent macroblocks a good reference again, and if there are no other errors will allow them to be decoded (more or less) correctly. In this way, a single -mmb command, which disables and/or fixes broken macroblocks, can fix a whole chunk of the image. The -mmb -1:x:y and similar commands do different kinds of resetting broken blocks.

Variable length coding

As you've probably noticed, we're moving a lot of numbers around here. We've used various tricks already to make those numbers smaller, and thus shorter. There's another trick we can apply if some numbers occur more often than others, and that is to translate each number into a code. A code is simply another number, but the trick is that commonly occurring numbers have a shorter code, while uncommon numbers have a longer code. Thus, we lose a bit of space on uncommon numbers, but gain a lot on common numbers. MPEG4 refers to this as variable length coding (VLC).

As a result of this, most numbers that you see in arnezami's debug output do not actually occur in the file as the corresponding binary number, and a single flipped bit can result in a very different value. In fact, it can completely mess up decoding if the flipped bit results in a shorter or longer code being read. The decoder will then start reading the next value too early or too late, causing another misread, lots of error messages, and colourful junk for output. Sometimes, this can be fixed by manually restarting the decoding process of the next (or a later subsequent) macroblock at the correct position. That's what the -mmb x:y:position commands do. Usually, you want to combine this with a reset (see above) just before the restart, to give the differential values something to reference.

I-frames and P-frames

So, if you've done the above for a whole picture, then you have an I-frame. I-frames are frames that stand alone, they contain all the information to decode the whole picture. As far as we currently know (and we're pretty sure I think) the SpaceX video contains 15 such I-frames, every 20th frame is one. The other 269 frames are P-frames. A P-frame uses the store-only-differences trick between frames. So, if there are parts of the picture that are the same as in the previous frame (e.g. the rocket body at the bottom of the image), then a P-frame will not store them at all. If there are parts that still look the same, but have moved, then the P-frame will contain a motion vector, which tells the decoder where to find it in the previous frame. So, to decode a P-frame, you need to fix the last I-frame before it, plus all the P-frames in between that I-frame and the P-frame, to get a correct reference image for the P-frame to use.

Transport Stream

Doing all the above yields a list of ones and zeros that describe a video. However, it's only the video. In this case that's all we have, but usually you also have audio, and sometimes multiple sound and video tracks, subtitles, station information, etc., that all have to be combined into a single file or stream. This is where the Transport Stream (TS) comes in. It works by splitting the bits for each substream into 188-byte chunks called packets. The packets are numbered sequentially, and contain a description of which stream they belong to and what kind of content it contains. The decoder unravels this, picks out the video data, and decodes it as described above.

Thanks mainly to Shanuson and Princess, we now have the TS packets all fixed up, and I think we've also got the starting points for (almost) all the MPEG4 frames. So what remains is fixing those up. It's one epic jigsaw puzzle, but then I hear it takes several months to get to Mars, and this particular jigsaw puzzle is zero-g compatible :).


There is much much much more to be said about MPEG4 (the standard document linked from the wiki is 536 pages!), but I think I've covered most of the stuff that's been discussed in this thread (which is the limit of on-topicness I guess) and we've also pretty much reached the end of my current knowledge. So I'll leave it at this. If anything is unclear or wrong, please post or PM me and I'll try to fix it, or if you have questions, ask!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: princess on 06/01/2014 12:05 am
Thanks mainly to Shanuson and Princess, we now have the TS packets all fixed up

Thank you for that Lourens! It's great to get a mention like that, it makes the effort all worthwhile.

I don't think the TS work is quite over yet - we've definitely found the start points for all the I- and P-frames, but as I mentioned in an earlier post we still have one or two issues aligning the ends properly.

The second byte of a TS packet's adaptation field is a set of flags, and none of these should be set on the last data packet in an MPEG4 frame (i.e. this byte should be zero). There are four frames where this byte is not zero, indicating definite corruption in the adaptation field:

P-frame 13: Corrupt in packet 982, the header is too short and padding bytes have been inserted into the MPEG4 stream.
P-frame 14: Corrupt in packet 1059, the header is too short and padding bytes have been inserted into the MPEG4 stream.
P-frame 87: Corrupt in packet 6869, however the header length seems correct and the MPEG4 data should be aligned.
P-frame 95: Corrupt in packet 7507, the header is too long, and valid MPEG4 data will be skipped by the decoder.

There are other places where visually the boundary between padding and MPEG4 data in the last packet of a frame seems wrong:

P-frame 4: AF marked as absent incorrectly, padding bytes in MPEG4 stream
P-frame 12: AF looks too long (MPEG4 data lost)
P-frame 15: AF marked as absent incorrectly, padding bytes in MPEG4 stream
P-frame 16: AF looks too short (padding bytes inserted into MPEG4 stream)
I-frame 21: Corruption around the existing boundary between AF and data (many 0xff data bytes indicate AF may be too short?)
P-frame 31: AF looks too long (MPEG4 data lost)
P-frame 43: AF marked as absent incorrectly, padding bytes in MPEG4 stream
P-frame 138: AF marked as absent incorrectly, padding bytes in MPEG4 stream
P-frame 155: AF marked as absent incorrectly, padding bytes in MPEG4 stream
P-frame 188: AF marked as length zero, padding bytes in MPEG4 stream
P-frame 194: AF marked as absent incorrectly, padding bytes in MPEG4 stream
P-frame 218: AF marked as absent incorrectly, padding bytes in MPEG4 stream

The following P-frames are marked as having no AF, which (although valid) seems to occur too often. These frames are worth a closer look, but may be already valid:

7, 24, 133, 185, 231, 254

So people who are working on MMBs for any of these frames, please be aware that there may be corruption on the last few macroblocks, but that we might be able to fix it at the TS level. It would be great to get feedback from the MMB guys as I'm no expert on the MPEG4-level stuff!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Quialiss on 06/01/2014 01:15 am
Hello, I'm not from around here... I've been working on restoring frames though. 

I frames 6, 7, 8, and 12(though mhenderson has already mostly restored 12) all have new data in them that invalidates the old MMBs for them. 

If someone could find out for each of these frames

1) where new data starts
2) how many bits are in the new piece of data

We can apply those offsets to the old MMBs, and they'll look as good as they did before. I can do this, I built a script to test the concept and it works really well, I just need the offsets, as there are multiple places data is added in the frames and finding them manually is a PITA. 

After that's done, we can improve the frames in the areas where there's new data.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: QuantumG on 06/01/2014 01:54 am
Hello, I'm not from around here... I've been working on restoring frames though. 

Welcome to the forum!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Quialiss on 06/01/2014 02:40 am

Would it be possible to add a slider to enhance the image contrast? It helps a lot for repairing the p-frames but I have to use an image viewer to do this currently.

Thank you again for your great work!

I came up with a very crude way to do this in the editor this morning.  Flip bits in the VOP quant to make it intentionally the wrong value, like what was wrong with frame 239.  Flipping the 1st/2nd/3rd bit at X:59 generally gets the desired high contrast.  It definitely makes lining up those very subtle movement vectors much easier, just remember to get rid of the flip when you're done. 
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 06/01/2014 05:37 am
In the mean time, I've seen a few posts by people saying that they have no idea what's going on. That's usually my position (especially with ISS stuff, acronym soup alert!), but now I'm coming from the other side, so I'll make an attempt at an explanation of all the stuff that's going on here. Hopefully, that will allow more people to help fix things. I'll probably make a mistake here and there and/or oversimplify something, so if you spot an error, let me know and I'll fix the text.

<snip>

Wauw!! Great explanation man! :D

This needs to be put on a prominent place on the wiki. For sure.

Thanks for that. A lot.

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 06/01/2014 05:40 am
Thanks mainly to Shanuson and Princess, we now have the TS packets all fixed up

Thank you for that Lourens! It's great to get a mention like that, it makes the effort all worthwhile.

Well princess you definitely deserve the credit. You and Shanuson did the majority of the extraction work. Which is huge. And you did very well together! This is vital to our effort. So very big thanks to you two.

Keep up the good work!

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 06/01/2014 05:50 am
Hello, I'm not from around here... I've been working on restoring frames though. 

I frames 6, 7, 8, and 12(though mhenderson has already mostly restored 12) all have new data in them that invalidates the old MMBs for them. 

If someone could find out for each of these frames

1) where new data starts
2) how many bits are in the new piece of data

We can apply those offsets to the old MMBs, and they'll look as good as they did before. I can do this, I built a script to test the concept and it works really well, I just need the offsets, as there are multiple places data is added in the frames and finding them manually is a PITA. 

After that's done, we can improve the frames in the areas where there's new data.
Welcome Quialiss!  8)

I've seen you on the wiki a lot. You did a lot of work on the P-frames there!

You are very welcome to this forum.

Regards,

arnezami

PS. Also @SwissCheese: you are beyond belief when it comes to fixing frames: incredibly good. You might actually be the most dedicated of the bunch. So precise.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Req on 06/01/2014 06:05 am
If you guys have use for an extremely robust MySQL instance that listens on a public port(ideally restricted to the slapbet/wiki webservers IPs, though) on a fast connection that is ridiculously well-routed/peered and won't be going anywhere, I can set that up for you.  It would be on a(redundant) system(s) with PCIe flash storage for the tablespace, 40 i7 xeon cores, and 128GB ram per member.

Would be happy to provide phpmyadmin access to whoever needs it as well if desired.  And I always retain binary logs and automated systems create binary-log-position-stamped backups from the slave, so there's a very high degree of data integrity and also flexibility if you wanted to "re-write history", as it were, by modifying the binary logs and replaying them from an arbitrary position.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 06/01/2014 06:05 am
Clock demo! Frames 101-120, running at half speed.

(http://adama.nocdirect.com/~wronkiew/spx_crs3/clock_anim.gif)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 06/01/2014 07:20 am
Clock demo! Frames 101-120, running at half speed.

I've put a page up on the Wiki detailing the process and listing the current set of clockfix files. I'll put up an online editor as soon as I can, but I warn you that it will be nowhere near as fancy as IainCole's.

http://spacexlanding.wikispaces.com/Fixing+the+clock
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Quialiss on 06/01/2014 09:17 am
Thanks for the welcome guys.  :)

I've been playing a bit more with the concept of shifting the mmb corrections for the new ts files so we don't  have to completely redo the i frames.  I haven't been able to find a good way to FIND what offsets are needed, but they sure do clean up the file in a hurry. 

This is a proof of concept of frame 101, using the previous MMB sequence. I offset all MMB corrections past 27000 by 1824, and past 53631 by 2480, with the top half blanked out because I haven't found the correct offset(s) for it.  Almost as good as it was before.

Also attached is the python script I'm using to apply offsets.  Not attached are my fruitless attempts to automate finding the values for those offsets.   :-\ 

The best manual way to find them is to look at the new, raw iframe, find a recognizable feature on it, note the position in the bitstream, look at the old, corrected iframe, find the same feature, subtract the two locations, apply that offset to the MMB using the script, and then see how the modified MMB looks on the new frame. 
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 06/01/2014 09:49 am

Would it be possible to add a slider to enhance the image contrast? It helps a lot for repairing the p-frames but I have to use an image viewer to do this currently.

Thank you again for your great work!

I came up with a very crude way to do this in the editor this morning.  Flip bits in the VOP quant to make it intentionally the wrong value, like what was wrong with frame 239.  Flipping the 1st/2nd/3rd bit at X:59 generally gets the desired high contrast.  It definitely makes lining up those very subtle movement vectors much easier, just remember to get rid of the flip when you're done.

Oo, that's a very good idea, works well!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 06/01/2014 09:54 am
Thanks for the welcome guys.  :)

I've been playing a bit more with the concept of shifting the mmb corrections for the new ts files so we don't  have to completely redo the i frames.  I haven't been able to find a good way to FIND what offsets are needed, but they sure do clean up the file in a hurry. 

This is a proof of concept of frame 101, using the previous MMB sequence. I offset all MMB corrections past 27000 by 1824, and past 53631 by 2480, with the top half blanked out because I haven't found the correct offset(s) for it.  Almost as good as it was before.

Also attached is the python script I'm using to apply offsets.  Not attached are my fruitless attempts to automate finding the values for those offsets.   :-\ 

The best manual way to find them is to look at the new, raw iframe, find a recognizable feature on it, note the position in the bitstream, look at the old, corrected iframe, find the same feature, subtract the two locations, apply that offset to the MMB using the script, and then see how the modified MMB looks on the new frame.

Yes, that's the way to do it quickly, but we should also check the new data that are causing these offsets, part of it might be good.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Quialiss on 06/01/2014 10:06 am
Yes, that's the way to do it quickly, but we should also check the new data that are causing these offsets, part of it might be good.

Yes, we should definitely be looking at the new data once the old work is all aligned, the offsets will show where the new data is.

There are just over 100 changes to frame 101, redoing all that work manually for is not my idea of a good time!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: princess on 06/01/2014 10:31 am
Yes, we should definitely be looking at the new data once the old work is all aligned, the offsets will show where the new data is.

There are just over 100 changes to frame 101, redoing all that work manually for is not my idea of a good time!

Quialiss, I could come up with a program that could list the MPEG4 decoding offset for each data packet in the TS files, then we could go through and compare the differences between the old and new TS files. It would be a long list, there are 17103 data packets in the current TS file! However, it would give you a definitive list of MPEG4 changes for all the I- and P- frames.

Would this be useful for you?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Quialiss on 06/01/2014 10:54 am
Yes, we should definitely be looking at the new data once the old work is all aligned, the offsets will show where the new data is.

There are just over 100 changes to frame 101, redoing all that work manually for is not my idea of a good time!

Quialiss, I could come up with a program that could list the MPEG4 decoding offset for each data packet in the TS files, then we could go through and compare the differences between the old and new TS files. It would be a long list, there are 17103 data packets in the current TS file! However, it would give you a definitive list of MPEG4 changes for all the I- and P- frames.

Would this be useful for you?

Yes, this is the information I'd need to do it automatically for the rest of the frames!  I have zero familiarity with mpeg4 encoding, which is why I asked for some help. 

These are the offsets I've found in frame 101, perhaps you can use them to see if you can get the values needed?  There may be some intermediate values between the ones I've found, but these should stay the same. 

(3405,272), (13493,1744), (27000,1824), (53655,2480)

The first values are approximate value, they just have to fall between two MMB's from the previous work, the second is the exact, cumulative number of bits added in the new TS file compared to the old one.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: princess on 06/01/2014 11:30 am
(3405,272), (13493,1744), (27000,1824), (53655,2480)

The first values are approximate value, they just have to fall between two MMB's from the previous work, the second is the exact, cumulative number of bits added in the new TS file compared to the old one.

At the TS level everything is byte aligned, so the number of bits inserted and the insertion point will always be a multiple of 8 bits.

Just to clarify - in your example above, are the first numbers bit positions in the old TS file? I can make the output in this format for you, listing everything in bit positions instead of bytes. I'll let you know when I have something, but I'm fairly busy today so I can get to work on it more seriously in the evening.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Quialiss on 06/01/2014 11:50 am
At the TS level everything is byte aligned, so the number of bits inserted and the insertion point will always be a multiple of 8 bits.
I was mostly just clarifying that in my small set, the number of bits inserted is accurate, but the insertion point is not precisely accurate, so you shouldn't expect to match it. 

Just to clarify - in your example above, are the first numbers bit positions in the old TS file? I can make the output in this format for you, listing everything in bit positions instead of bytes. I'll let you know when I have something, but I'm fairly busy today so I can get to work on it more seriously in the evening.

Yes, the bit positions are from the old TS file.  It doesn't matter all that much so long as I know which one it is, it just changes the sign of the offset. 


Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: princess on 06/01/2014 12:30 pm
Hi Quialiss,

I made a quick edit to my TS program so that for each data packet it dumps the MP4 frame it's assigned to (identified by its PCR timestamp) and the byte offset in the MP4 stream. I've dumped I-frame 101 (PCR=6262246) for raw_edit8.ts and raw_fixed_final.ts. If you open up the two fixes side by side (either in two editors or with a diff tool like WinDiff or Meld), then you can look at the old file to see the old byte offsets, and then look across to the new file to see what the new byte offsets are.

As an example, the first few lines of the old file are:

PID 0x03e8 PUSI AF[7] Pay13:000001e000008180 MP4 frame@6262246 bytes 0-160
PID 0x03e8 Pay14:a74b71c963a5402e MP4 frame@6262246 bytes 160-344
PID 0x03e8 Pay15:955c594e1a752f33 MP4 frame@6262246 bytes 344-528
Invalid PID 0x0be2 AF[33]
PID 0x03e8 Pay1:e6d894f02f3ce3ef MP4 frame@6262246 bytes 528-712


The corresponding lines in the new file are:

PID 0x03e8 PUSI AF[7] Pay13:000001e000008180 MP4 frame@6262246 bytes 0-160
PID 0x03e8 Pay14:a74b71c963a5402e MP4 frame@6262246 bytes 160-344
PID 0x03e8 Pay15:955c594e1a752f33 MP4 frame@6262246 bytes 344-528
PID 0x03e8 Pay0:21623132ae67d9e0 MP4 frame@6262246 bytes 528-712
PID 0x03e8 Pay1:e6d894f02f3ce3ef MP4 frame@6262246 bytes 712-896


This shows the first change is an insertion of 184 bytes at offset 528 bytes.

I've attached the two text files as a ZIP file, I did them in Linux so they'll have Unix line endings. Sorry it's in byte offsets instead of bits, I can fix that later when I have some time. I'm not sure the offsets that you already found match up to these, so let me know if I've made a mistake anywhere.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: cartman on 06/01/2014 01:59 pm
Ok, so i'm a linux guy with web servers and the works, and would like to help. First of all, i would like to help with recovering the video. Is there an updated guide of where to start? should i use arnezami's latest build?
On the wiki (http://spacexlanding.wikispaces.com/) there is a section calls "Work in Progress " which has an "ACTIVE" part which contains two links: one to a tutorial one to where we post our results.

There is also a "Reconstruction Tools" section in the wiki. The two top links (one for P-frames one for I-frames) go to the online editors. There you can try to fix the frames.

You don't need my latest addition to the ffmpeg-tool to help for now. What we are developing is for making the online tools more powerful.
Thanks for your answer, somehow i missed it before.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Quialiss on 06/01/2014 02:02 pm
Hi princess,

The insertions shown by the log file are vastly larger than the ones I figured out, I think we may have to go deeper!  For reference, the four manually found values for bytes of data inserted are 34, 218, 228, 310.  10 bytes for the smallest change, that might not even be a single macroblock added.  It doesn't look like the level of detail is there in that log.

I don't know nearly enough about mpeg4 encoding, is it possible that the data packets that show as errors were partially decoded, and thus the actual amount of new data inserted is less than shown? 

I'm heading out for the day, I will get back to this later this evening! 
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: princess on 06/01/2014 03:36 pm
Hi Quialiss,

An offset of 34 bytes is still within the first TS packet of frame 101 (which contains 160 bytes of MPEG4), and as far as I can see that packet hasn't changed in the two files. You can check it with a hex editor, the offsets you need are:

raw_edit8.ts: Frame 101 starts at TS packet 9279, offset 0x001a9e44 into the file
raw_final_fixed.ts: Frame 101 starts at TS packet 7923, offset 0x0016ba74 into the file

In both cases, the first packet contains the following data:

00000000  47 43 e8 3d 07 10 00 2f  c6 f3 7e 00 00 00 01 e0  |GC.=.../..~.....|
00000010  00 00 81 80 07 21 01 7d  cd ad ff ff 00 00 01 b0  |.....!.}........|
00000020  03 00 00 01 b5 0d 0f 00  00 01 00 00 00 01 20 00  |.............. .|
00000030  c4 8d c0 00 46 56 c0 1e  c0 01 f0 04 9a fc 7c 2e  |....FV........|.|
00000040  ee 2c 08 78 28 c7 00 00  01 b2 65 6d 34 76 20 34  |.,.x(.....em4v 4|
00000050  2e 33 2e 30 2e 30 00 c3  ff 00 00 01 b6 14 70 dc  |.3.0.0........p.|
00000060  0e 61 78 5e 61 fd 94 64  0c 78 aa b4 74 f4 40 e8  |.ax^a..d.x..t.@.|
00000070  eb 3f 21 5f f5 ec 17 fd  f6 9f ab ba 5b a4 f4 90  |.?!_........[...|
00000080  de 31 dd b1 c5 3d 38 b9  fd 1d 62 0d c8 39 00 4b  |.1...=8...b..9.K|
00000090  2f 3d c9 bb 41 71 84 ff  4e 1f 93 ff de ac 47 39  |/=..Aq..N.....G9|
000000a0  d8 66 0e 21 9d 73 43 20  71 03 0b 22 18 96 5f 69  |.f.!.sC q..".._i|
000000b0  ee f7 26 9a 0a ab 8c fb  1d 73 fa 98              |..&......s..|


If you're on Linux, the following commands show you the TS frame in each case:

dd if=raw_edit8.ts bs=1 skip=1744452 count=188 | hexdump -C
dd if=raw_final_fixed.ts bs=1 skip=1489524 count=188 | hexdump -C

In each case I've just converted the offset into decimal as the dd command doesn't like hex numbers. Please check I'm correct and let me know if I've missed anything, but when I try that on my machine I get identical output.

If this is correct then the numbers you're using don't correspond exactly to the bit offset from the start of the frame - we would then need to look into what exactly the numbers mean in ffmpeg, which unfortunately I don't have much experience with! If we can work out the correspondence between ffmpeg's numbers and the actual byte position in the raw file, we can then calculate the proper offsets.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: MTom on 06/01/2014 04:03 pm
In the mean time, I've seen a few posts by people saying that they have no idea what's going on. That's usually my position (especially with ISS stuff, acronym soup alert!), but now I'm coming from the other side, so I'll make an attempt at an explanation of all the stuff that's going on here. Hopefully, that will allow more people to help fix things. I'll probably make a mistake here and there and/or oversimplify something, so if you spot an error, let me know and I'll fix the text.

<snip>

Wauw!! Great explanation man! :D

This needs to be put on a prominent place on the wiki. For sure.

Thanks for that. A lot.

Regards,

arnezami

Not only the post but the whole thread have to placed on the wiki after finishing: the learning cycles from the beginning has also worth to read if someone really wants to learn about MPEG4.

Btw, this "project" have an other very interesting aspect: the way how it is managed without really managing it.

Explanation: My profession is how to organize companies (actual working at an automobile factory, over 10t employee). The way how here self-organizing works, the dynamic of evolving the roles, aligning and sharing the work between participants, the continuous improvements in the working processes and rapidly integrating into the project is incredible.
As mentioned from others before, the key is the motivation of the participants. But there are many places (and companies) overall where motivated people come together. Such an effective working group comes not automatically as a result. And the good examples remain often hidden. We hear about them (companies, projects, etc) but there aren't insider information to reach how it really works (and built up from the beginning).

I read this thread from the beginning and I enjoyed it very well: it is a great learning effect also for me in my profession. Thanks for the opportunity being here.

And of course: Respect for your huge work (if I have to guess, it could be between 1000 and 2000 working hours until now, maybe more...), and for these results!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 06/01/2014 04:30 pm
we would then need to look into what exactly the numbers mean in ffmpeg

Bitpositions in mmb command are applied after all TS and PES level data is stripped. They are basically bit positions in the mpeg4-img files. For iframes they start with the Visual object layer start code and for P frames the VOP start code. Counting starts from 1 (there is no bitposition zero). Inclusion of a standard TS (no AF) packet in the middle of the frame should then I believe theoretically introduce a 184*8=1472 bit shift in subsequent positions. If there's anything going on involving adaptation fields, then a different number of bytes may be included (or excluded) I believe.

Looks like only one of the four insertions that Quialiss has identified corresponds to such an event. Looks to me there's quite a lot of AF jiggerypokery going on in the new TS update.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 06/01/2014 05:10 pm
Hey guys, here's another shoutout for your work, from Elon! :)

http://youtu.be/uBaLYDbk4fY

Go to 23 mins 27 seconds! :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Oersted on 06/01/2014 07:38 pm
Hehe, you owe that interviewer a beer, Chris...  :-)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 06/01/2014 07:42 pm
I made a quick edit to my TS program so that for each data packet it dumps the MP4 frame it's assigned to (identified by its PCR timestamp) and the byte offset in the MP4 stream. I've dumped I-frame 101 (PCR=6262246) for raw_edit8.ts and raw_fixed_final.ts. If you open up the two fixes side by side (either in two editors or with a diff tool like WinDiff or Meld), then you can look at the old file to see the old byte offsets, and then look across to the new file to see what the new byte offsets are.

I looked at the logs and I think I see what's going on. Surprisingly ffmpeg does not ignore packets with wrong PIDs but includes them into the stream. The difference between the new and old ts file interpretation comes form adaptation fields that are disabled in the new .ts file. The bytes in them were previously discarded, but are now interpreted as mpeg4 data. Here are the first 4 packets with bad PID and AF from the old TS file from your log:
Invalid PID 0x0be2 AF[33]
Invalid PID 0x035e SCR AF[242] Pay4: PCR: 5405681769 (gap 5399419523)
Invalid PID 0x03b9 PUSI SCR AF[9] PCR: 5586719654 (gap 181037885)
Invalid PID 0x00bf TEI AF[81]

The second AF size is bigger than TS packet and the whole 184 bytes gets discarded in the old version and included in the new one. The AF sizes if we add the AF size byte correspond to the insertions identified by Quialiss (34, 184, 10 and 82 bytes).
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Helodriver on 06/01/2014 07:48 pm
Hehe, you owe that interviewer a beer, Chris...  :-)

He already gave me lifetime L2 membership, so I think were all good.  ;)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mhenderson on 06/01/2014 08:09 pm
Transcript of Musk Q&A -

Interviewer: Can you talk about the splashdown of the CRS-3 first stage out off in
the Atlantic? What you've learned from the telemetry and the video you might have
gotten back?

Musk: Oh uh, (head cocks back, eyes roll skyward) Yes, as far as the soft landing of
the boost stage.

Interviewer: Yes

Musk: It was interesting when we got the corrupted video back because we
really actually had a really difficult time getting the telemetry back. I'll say a
funny thing. Normally we get the telemetry off of a boat. And we also have a
backup a Navy P3 that was going to go up. And the P3 got iced up and the boats
couldn't go out so actually I sent my plane up. And we had to design and build and
fabricate an antenna that exactly fit in the window of the plane. So we started off
with a pizza dish and were able to <unintelligible> an antenna with a pizza dish and
point it out the window to get the thing.

Interviewer: How about the decoding of that corrupted file? How is that going?

Musk: Yeah, so we got ... the data came through really well but the video was corrupted
because unfortunately when you compress video it's .... it's hard to uncorrupt a
compressed video because you need to figure out the compression on the thing and all
sorts of things.

Interviewer: Right, right, right

Musk: We weren't able to get very far but we put the video online and we sort of
crowdsourced the cleanup of the video. And umm people did a really good job of fixing
it. And I actually created a link to their latest thing. And it was a bunch of people
on the NASASpaceFlight forum that were able to fix the video.

Interviewer: That's really cool

<Next question is asked>

Interviewer: Chris, that's for you.


Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: hutchel on 06/01/2014 09:25 pm
OK so I used to consider myself good with coding.  I am a complete newbie when I comes to MPEG4, so I tried the online editor and it's still beyond mortal man w/o lots of explanation.

What I am proposing to get more eyes working on this is to focus on getting correct data packets and then the rest of the crowd can start flipping bits in the payload.  What I am hoping for (it may not be possible due to the compression) is to be able to break the payload down to specific parts and display them as such.  Then if the tool can allow us to concentrate on the payload part by part, mortal eyes may have a better chance of flipping bits to start fixing the rest of the picture.  But the key is to get us to the point where we can focus on the parts of the payload as opposed to try to guess at the whole payload as appears to be the case right now.

My other comment is as the online editor gets used by more people - you probably need to moderate inputs before committing them to a master database so we don't inadvertently screw something up and 2. we guard against some malicious event (not that I am expecting that, but stranger things have happened).

Standing by to help, but we have to get this a bit less technical.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 06/01/2014 09:52 pm
I just finished cleaning up the remaining frames in part 11 (201-220). There is an issue with this part: when ffmpeg does not find the end of a p-frame, it just does not use the frame. Usually we can find the end of a frame and paste it at the end, but for frame 218 I can't manage to find it.

Can someone help? Would it be possible to "force" an end of frame MB?

Edit: Could find the end of that frame. But it's not very practical...

Anyway so part 11 is complete!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: princess on 06/01/2014 11:05 pm
Surprisingly ffmpeg does not ignore packets with wrong PIDs but includes them into the stream. The difference between the new and old ts file interpretation comes form adaptation fields that are disabled in the new .ts file. The bytes in them were previously discarded, but are now interpreted as mpeg4 data.

Firstly, thank you for catching that and working it out. The bit offsets from Quialiss make sense now.

Secondly - OMG, ffmpeg puts invalid PIDs into the MPEG4 stream? Seriously? If that's true it means we will probably need to shift to using the raw_final_fixed_allpids.ts file that I posted earlier in the thread, because the existing raw_final_fixed.ts file has invalid PIDs in the padding sections! These could be corrupting the MPEG4 data!

Link to my post with raw_final_fixed_allpids.ts: http://forum.nasaspaceflight.com/index.php?topic=34597.msg1206016#msg1206016 (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1206016#msg1206016)

Calling Shanuson: what's your opinion on this?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: princess on 06/01/2014 11:12 pm
I just finished cleaning up the remaining frames in part 11 (201-220). There is an issue with this part: when ffmpeg does not find the end of a p-frame, it just does not use the frame. Usually we can find the end of a frame and paste it at the end, but for frame 218 I can't manage to find it.

Can someone help?

Frame 218 has a known TS-level corruption at the end, see my post here: http://forum.nasaspaceflight.com/index.php?topic=34597.msg1207719#msg1207719 (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1207719#msg1207719)

If ffmpeg is treating invalid packets as data we might need to move to new TS files anyway, so we could use your help identifying the correct bytes in P-frame 218 so we can fix that at the same time too.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 06/01/2014 11:29 pm
I just finished cleaning up the remaining frames in part 11 (201-220). There is an issue with this part: when ffmpeg does not find the end of a p-frame, it just does not use the frame. Usually we can find the end of a frame and paste it at the end, but for frame 218 I can't manage to find it.

Can someone help?

Frame 218 has a known TS-level corruption at the end, see my post here: http://forum.nasaspaceflight.com/index.php?topic=34597.msg1207719#msg1207719 (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1207719#msg1207719)

If ffmpeg is treating invalid packets as data we might need to move to new TS files anyway, so we could use your help identifying the correct bytes in P-frame 218 so we can fix that at the same time too.

I have to say I would prefer not to move to another ts file again... :P I prefer to skip bad data with mmb commands than to do everything again.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: princess on 06/01/2014 11:36 pm
I have to say I would prefer not to move to another ts file again... :P I prefer to skip bad data with mmb commands than to do everything again.

I can quite understand that! I've just checked through raw_final_fixed.ts and it seems that all the invalid packets are between frames. My (limited) understanding of MPEG4 is that once one frame ends it will look for a specific bit pattern to start the next frame, so corruption between frames shouldn't be a problem.

That all means that we probably don't need to shift to new TS files, you'll be happy to know ;)

But yes, some frames (listed in my post) will need specific tweaks at the ends. If the -mmb command can do that easily then that's fine.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Shanuson on 06/01/2014 11:40 pm
Surprisingly ffmpeg does not ignore packets with wrong PIDs but includes them into the stream. The difference between the new and old ts file interpretation comes form adaptation fields that are disabled in the new .ts file. The bytes in them were previously discarded, but are now interpreted as mpeg4 data.

Firstly, thank you for catching that and working it out. The bit offsets from Quialiss make sense now.

Secondly - OMG, ffmpeg puts invalid PIDs into the MPEG4 stream? Seriously? If that's true it means we will probably need to shift to using the raw_final_fixed_allpids.ts file that I posted earlier in the thread, because the existing raw_final_fixed.ts file has invalid PIDs in the padding sections! These could be corrupting the MPEG4 data!

Link to my post with raw_final_fixed_allpids.ts: http://forum.nasaspaceflight.com/index.php?topic=34597.msg1206016#msg1206016 (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1206016#msg1206016)

Calling Shanuson: what's your opinion on this?

I never expected that ffmpeg could do something like that. If you have more than one stream in there, how should it decide to which it belongs? I would understand that it needs the right PID but the AF flag has to be set right too to get all of the data.
I think i also tested that a TS-packet with the wrong PID would get neglected. But I could remember it wrong. It should easy to test though. We could compare the length of the frame files between your version and mine to decide it for example. Or change one header to a wrong PID and see what ffmpeg does with the data.

Also I was maybe a bit to eager to put final in that file name.  Like you wrote, we should at least correct the AF-length where we can or reduce it to 1 if we cant decide what the length is at the end of a frame. That will probably be the last things done on the TS-level side since you fixed the rest of the Ts-headers I were to lazy to fix.

But if the wrong PIDs do not get included into a frame (and i think they don't) this edit would only add maybe a few bytes at the end and thus only be really minor changes to a frame and the mmb - code if at all.

Attached you find the python script I used to split the raw-file into parts.

I have to get back to my real work so I can not really help much from here on out.

Cheers
Shanuson
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Quialiss on 06/02/2014 12:46 am
Surprisingly ffmpeg does not ignore packets with wrong PIDs but includes them into the stream. The difference between the new and old ts file interpretation comes form adaptation fields that are disabled in the new .ts file. The bytes in them were previously discarded, but are now interpreted as mpeg4 data.

Firstly, thank you for catching that and working it out. The bit offsets from Quialiss make sense now.

Wizards, both of you! 

Princess, can you provide the logs with the informative errors for Part 7, Part 8 and Part 12 so I can fix up those iframes too?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 06/02/2014 01:14 am
I never expected that ffmpeg could do something like that. If you have more than one stream in there, how should it decide to which it belongs? I would understand that it needs the right PID but the AF flag has to be set right too to get all of the data.

While I was going through spme of the parts I started to get a feeling that it wasn't quite ignoring everything that it should have, considering that my before- and after-cleaning images were coming out the same length when I knew that I had corrected TS headers that were supposed to be part of 0x3e8 but had had a bit or two flipped to change the PID. Hopefully one of us can make some time to test that - my money's on Princess. :D At the time I just thought I'd made a late-night mistake in extracting the images, but maybe not. This ill-at-ease sensation is probably why I started correcting every byte of padding packets to 0xff, even though their contents are supposed to be ignored. Maybe overkill, but it does restore the transport stream more closely to its original state in any case, even if it has no bearing on the frames.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: lgjy98d on 06/02/2014 02:25 am
I doesn't seem like rocket science, but it does seem like a lot of work specially on the online tool side (we don't want you to hate us Iaincole). Ideas?

VCSes - version control systems - are really good at moralec's database problem. One idea is to shoehorn this into a VCS by using a VCS as storage.

Here's one way to do that: force MMB lists into a VCS friendly format, by
a) breaking up the MMBs into one per line [VCSes tend to work best on whole lines at a time] and
b) sort the MMBs into a standard sort order, most likely the raster order most folks are already used to.  [EDIT: I guess this isn't strictly necessary, but may make IainCole's life easier]
c) Then you need to always check in and out of the VCS via something that understands the format. IainCole's website, or some other website, for example, would be the only entity with access to the VCS. This website would be responsible for parsing raw MMBs, and providing the MMBs back in a ffmpeg-friendly format.
d) Ideally this file format would have comments to allow for notes and simple alternatives. The VCS would be responsible for full 'branches' [git/mercurial is good for this].

And then a flat file format might come in handy in other ways.

There are other ways: you could use raw RCS diffs to manage things; you could find a diff program that's really good at big long lines (most aren't); you can code up the delta checking by hand; you can simply store all the MMBs ever in groups, normalized in the database, for users to sort out; etc.

This was pretty much exactly my suggestion here: http://forum.nasaspaceflight.com/index.php?topic=34597.msg1207365#msg1207365

The added bonus being that the online editor is already using that file to load MMBs.

People can either submit MMBs by pull request, or a few key people can be made collaborators on that git repo to add them in based off submissions to the wiki.

Edit: There's obviously some administrative overhead with making sure that the wiki / json file are in sync, but short of writing a new system to do the job of both things, I think it's the simplest option.

I consider pull-request-for-best-mmbs approach to be too complex for majority or video recovery effort participants. Wiki was simple but not simple enough. I've gone 1 step further, introduced Google Spreadsheets. You can compare the original and spreadsheet version of the page about 4th part:
 * http://spacexlanding.wikispaces.com/Frames+Part+4
 * http://spacexlanding.wikispaces.com/Frames+Part+4+Spreadsheet

To unify best MMB storage and facilitate structural data retrieval I'd setup 15 Google Spreadsheets (one per part) and for convenience linked them in Spreadsheet column to Progress table of http://spacexlanding.wikispaces.com/raw_final_fixedMMB.

There is an idea to make Online Editor to pull "Best MMBs" data direcly from Spreadsheet tables, and idea is laid out in more details at: https://github.com/IainCole/SpaceXVideoApp2/pull/1.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 06/02/2014 03:05 am
OK so I used to consider myself good with coding.  I am a complete newbie when I comes to MPEG4, so I tried the online editor and it's still beyond mortal man w/o lots of explanation.

What I am proposing to get more eyes working on this is to focus on getting correct data packets and then the rest of the crowd can start flipping bits in the payload.  What I am hoping for (it may not be possible due to the compression) is to be able to break the payload down to specific parts and display them as such.  Then if the tool can allow us to concentrate on the payload part by part, mortal eyes may have a better chance of flipping bits to start fixing the rest of the picture.  But the key is to get us to the point where we can focus on the parts of the payload as opposed to try to guess at the whole payload as appears to be the case right now.

My other comment is as the online editor gets used by more people - you probably need to moderate inputs before committing them to a master database so we don't inadvertently screw something up and 2. we guard against some malicious event (not that I am expecting that, but stranger things have happened).

Standing by to help, but we have to get this a bit less technical.
Hi hutchel,

I am assuming you are not the only one who is feeling this "steep learning curve". So I really want to ask you about your opinion (I might be too "deep into this" and not see it). @all_potential_helpers: If you had a similar feeling like hutchel did then the following is also meant for you.

1 - Do you think we need better tutorials for using the editor? What would you consider better?
2 - Do you think the online editor should be more intuitive to use? What is not intuitive right now?
3 - Do you need more feedback flipping single bits or changing bitpositions? Do you need to see the actual bits? What kind of feedback apart from the resulting frame?
4 - Do you need to have a better explanation about MPEG4? Do you want to see a concrete example based on the actual video data?
5 - Do you want simpler and more concrete tasks to do? It's now divided into roughly 300 parts. Should we divide it more? Or should it be divided differently?

Can you please at least sort the above numbers in order of priority? And then explain why you think some are more important than others? And maybe then try to answer (some of) the questions? Thanks!

Could you also tell what you read (tutorial/wiki page wise) before using the online editor? Because maybe we can do something about the (more prominent) placement of the explanations already available on the wiki/forum.

I really like to know. And hopefully you can help us here.

Thanks in advance.

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Quialiss on 06/02/2014 04:51 am
1 - Do you think we need better tutorials for using the editor? What would you consider better?
2 - Do you think the online editor should be more intuitive to use? What is not intuitive right now?
3 - Do you need more feedback flipping single bits or changing bitpositions? Do you need to see the actual bits? What kind of feedback apart from the resulting frame?
4 - Do you need to have a better explanation about MPEG4? Do you want to see a concrete example based on the actual video data?
5 - Do you want simpler and more concrete tasks to do? It's now divided into roughly 300 parts. Should we divide it more? Or should it be divided differently?

Short version: 3 and to a lesser extent 1.  2 4 5 are completely off my radar for concerns, but I'm not your ideal person because I haven't had a problem with the learning curve.


Lets see... I started contributing about one week ago with exactly zero knowledge of video recovery except what I'd picked up by following the efforts here, as I'd been following what was going on in the recovery effort since it started.  I couldn't see a way to contribute until the web editor went online.  That dropped the barrier to entry monumentally, and I've been fine with the learning curve from that point, so I'm not exactly the perfect person to answer the questions, but it's a POV from someone new to the recovery effort with no relevant prior experience. 

I started by looking at the work done by other people and trying to figure out how they'd done it... moving the blocks around and nulling out others was self explanatory, but I couldn't figure out how in the world people found the correct start positions for blocks in the i frames when they're wrong, and honestly I'm still not sure about that part and if it's more than trial and error.  So this is the one thing that I definitely would like some more information on.  If I recall correctly I fiddled around for a day or three trying things out before I started getting results that I thought were good enough to share. 

The p frame tutorial is great!  The first thing I did was start with p frames, and they were surprisingly easy to figure out, much easier than repairing i frames... of course, by pure chance I chose a set to start on that was only slightly damaged, but I still find it easier to work on p frames.

I went back to the i frames after that and still couldn't figure out how to get more data out of them... so I figured I'd work on all the luminance and chroma errors that plague most of the iframes.  I tried to start by just looking at the corrections people had made, didn't get terribly good results, so I went digging into the posts about macroblock structure and sub blocks, and how values propagate through the frame.  I had to go back several times and reread and pick up on new things as I was working, but the existing information on the wiki was sufficient to understand how to improve frames that way. 

These would be the two resources I kept going back to for reference.  I STILL don't understand why the propagation of values changes so dramatically sometimes, but I've learned to work with it.  (If I understand correctly the direction a block inherits from is dynamic.. I just don't quite understand why it switches seemingly arbitrarily due to luminance changes)
http://spacexlanding.wikispaces.com/MMB+Syntax 
http://forum.nasaspaceflight.com/index.php?topic=34597.msg1194911#msg1194911 - macroblock and subblock structure

And I didn't exactly reference this, but I have a scratchpad where I keep things like 26:10  dust spot, 21:14 left leg base, 15-16:11 left leg tip, which help in aligning features.  Knowing where the static features in the frames should be helps a lot when working with p frames, and the coordinates help much more than inexact circles on an image. 
http://forum.nasaspaceflight.com/index.php?topic=34597.msg1196634#msg1196634 - locations of the dust on the camera. 
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Quialiss on 06/02/2014 05:11 am
And to follow the wall of text, a bit of eye candy and showing some of the repair process.  I saved the image I was working on periodically to check chroma values at a higher saturation and to make sure I'm not missing too much.  I decided to keep the intermediary images and animate the process. 

There's no new data in the frame, just a lot of luminance and chroma adjustments.  You can see the left-right, top-down progression and then a final pass on chroma issues, there's around ~10 changes in each frame.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 06/02/2014 07:22 am
I started by looking at the work done by other people and trying to figure out how they'd done it... moving the blocks around and nulling out others was self explanatory, but I couldn't figure out how in the world people found the correct start positions for blocks in the i frames when they're wrong, and honestly I'm still not sure about that part and if it's more than trial and error.  So this is the one thing that I definitely would like some more information on.

For me it's trial and error really. But you really need ffmpeg to be set up locally to do it. What I've been doing is  running a script that tries out a range of starting positions (usually you can eyeball a region where there's a reasonably good chance to hit upon good data, e.g. lots of green blocks usually means that the search will come up empty). All the resulting images are dumped to a folder and can later be quickly flipped through in an image viewer to see if any positions worked. Same goes for flipping bits, flip a range of bits and look at all results(I try both single flips and the triple flip pattern identified previously in this thread 1000000000000011), but you need to be confident in MB start positions, so it really only works immediately after good data.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 06/02/2014 08:00 am
And to follow the wall of text, a bit of eye candy and showing some of the repair process.  I saved the image I was working on periodically to check chroma values at a higher saturation and to make sure I'm not missing too much.  I decided to keep the intermediary images and animate the process. 

There's no new data in the frame, just a lot of luminance and chroma adjustments.  You can see the left-right, top-down progression and then a final pass on chroma issues, there's around ~10 changes in each frame.
I love your visualization of the luma/chroma fixing! It shows its power :)

It's very cool that after doing the extracting, positioning and now the repairing, the incredible result of the work of all people who worked on it, finally hits you.  8)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Geron on 06/02/2014 08:26 am
You guys are awesome, strong work!!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 06/02/2014 09:11 am
Stunning!

(http://spacexlanding.wikispaces.com/file/view/part_9.gif/512412160/part_9.gif)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 06/02/2014 09:26 am
Amazing work guys! Both the recent frame fixes and the redraw of the clock look amazing.

Regarding, the questions arenzami posted, I would like to invite all to join  the discussion. There are probably a lot of  newcomers, interested in helping out, and the better we incorporate them to our work the best final video we will get.

I assume that some of the people that still visit this forum actually tried to help, but failed and quit trying due to the technical difficulties associated with it. If any of you are reading this, your input will be greatly appreciated.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: hutchel on 06/02/2014 11:37 am


1 - Do you think we need better tutorials for using the editor? What would you consider better?
2 - Do you think the online editor should be more intuitive to use? What is not intuitive right now?
3 - Do you need more feedback flipping single bits or changing bitpositions? Do you need to see the actual bits? What kind of feedback apart from the resulting frame?
4 - Do you need to have a better explanation about MPEG4? Do you want to see a concrete example based on the actual video data?
5 - Do you want simpler and more concrete tasks to do? It's now divided into roughly 300 parts. Should we divide it more? Or should it be divided differently?

I guess I was impatient and only experimented a little bit.  So let me see if I can address 1-5
1/2/3.   The online editor seems to be straight forward - although a bit level display might help when we get to the payload work.  I suspect the issue is I haven't really taken the time to fully understand the embedded format - that may be the crux of the issue - I think I was asking for the editor to completely decode the format and display the decoded parts for us to address individually.  I think that way helpers can focus on the values of the fields as opposed to having to decode on the fly.
4.  This may help also - How do the good blocks look and an explanation of how one impacts others.
5.  Discrete tasks are good, but again, I think the biggest barrier to entry is the steep learning curve .... I suspect most potential helpers have an hour or 2 here and there they can give to the effort.  If it takes 10 or more hours of investigation and trial or error to get anything, then most will remain on the sidelines.

Maybe a concise enough tutorial for each set of tasks.  i.e. fix abc in the following blocks.  Here's the mechanics (and maybe the theory) of how to fix abc.....

I recognize the data is compressed, what I keep struggling with is why all the padding that you are finding in the stream?  That doesn't make sense.   If I was compressing, the padding is the easiest to get rid of.

This is going to sound like a no nothing, but are we to the point in the effort to make a non-compressed stream?  It might be easier to work on.  It appears you are doing that a little at a time to tweak it, but the results appear to be recompressed and inserted back into the compressed stream.

Just some idle thoughts from someone who would like to help, but can only give you an hour here or there....

I will say I am VERY impressed with those that have been willing to give so much more to this effort than I.

edited to fix quotes
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 06/02/2014 12:10 pm
I recognize the data is compressed, what I keep struggling with is why all the padding that you are finding in the stream?

The uplink has a constant bit rate, so it will send a fixed number of bits in one second regardless. But the video stream produces about 15 frames per second and no more, and if all those frames are compressed well enough to wind up using less than the number of bits that are available in the uplink in that one second... it's a "stream," so it can't just stop transmitting or else the receiver would think something broke - every available transport stream packet has to be filled with something, even if all the video data has already been sent. That's where the padding comes in.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mhenderson on 06/02/2014 12:10 pm
Feature request: Excel has an "audit..., trace dependents" feature. Choose a cell and see which cells are dependent upon it (a chain of formula references). There is also a "trace precedents" feature to see which cells contribute to the current cell.

Excel shows related cells visually with arrows. Nice, but a simple list would suffice.

Similar functions would help here.

Typing on mobile, hope I am clear.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 06/02/2014 12:16 pm
Stunning!

I think there's going to be some SpaceX people weeping for joy this morning when they see that. Just... wow.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 06/02/2014 12:32 pm
Stunning!

(http://spacexlanding.wikispaces.com/file/view/part_9.gif/512412160/part_9.gif)

Amazing!

I'd note that you guys need to think about the next milestone point, then we should think about another updated video and we'll get that out there like the previous.

We're not going to have Elon tweeting every time, but when we get to the point we've done all we can, we can show the milestones.

When we do get to that "we've done all we can" point, then we should consider an article on site. Heck, we have Elon quotes now! :) I'll need a LOT of help with to translate the technical to something readable by Joe Public, quote some of the team here and use the Elon quotes, etc.

Also, it was suggested to me we could also have a highly technical overview as a press release style write up. I'm cool with all of that, obviously.

Something to keep in mind.

PS Per the next milestone video. What we absolutely should do is "where we are" before the ORBCOMM launch, as that launch is probably going to have a good video beamed back. Regardless, this is the historic first, so there's no value lost on this effort if ORBCOMM sends a good video back.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: RanulfC on 06/02/2014 01:09 pm
Amazing!

I'd note that you guys need to think about the next milestone point, then we should think about another updated video and we'll get that out there like the previous.

We're not going to have Elon tweeting every time, but when we get to the point we've done all we can, we can show the milestones.

When we do get to that "we've done all we can" point, then we should consider an article on site. Heck, we have Elon quotes now! :) I'll need a LOT of help with to translate the technical to something readable by Joe Public, quote some of the team here and use the Elon quotes, etc.

Also, it was suggested to me we could also have a highly technical overview as a press release style write up. I'm cool with all of that, obviously.

Something to keep in mind.

PS Per the next milestone video. What we absolutely should do is "where we are" before the ORBCOMM launch, as that launch is probably going to have a good video beamed back. Regardless, this is the historic first, so there's no value lost on this effort if ORBCOMM sends a good video back.

Just another little "tidbit" to keep in the back of your minds is this:

If the history of "space" related aerospace is anything like the "aero" side then at some point "space-ports" and "space-bases" will take on names derived from supporters and pioneers who helped things along.

So, the "question" then becomes do we push for "NASASpaceflightForums" Spaceport or just name it "Chris Bergin" Space Force Base and name all the streets after the L2 members?

Remember, it's never to Early to start thinking about the Future! And how to mess with it's inhabitants :)

Randy
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Lee Jay on 06/02/2014 01:27 pm
PS Per the next milestone video. What we absolutely should do is "where we are" before the ORBCOMM launch, as that launch is probably going to have a good video beamed back. Regardless, this is the historic first, so there's no value lost on this effort if ORBCOMM sends a good video back.

I think the Orbcomm video, even if it's solid, will be in darkness.  So, this video will remain unique even if the Orbcomm video is good.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: rocketguy101 on 06/02/2014 01:44 pm
Stunning!

I think there's going to be some SpaceX people weeping for joy this morning when they see that. Just... wow.
Trust me, there are non SpaceX people jumping for joy when they log on in the mornings to see advances made overnight!!! 8)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Galactic Penguin SST on 06/02/2014 02:00 pm
PS Per the next milestone video. What we absolutely should do is "where we are" before the ORBCOMM launch, as that launch is probably going to have a good video beamed back. Regardless, this is the historic first, so there's no value lost on this effort if ORBCOMM sends a good video back.

I think the Orbcomm video, even if it's solid, will be in darkness.  So, this video will remain unique even if the Orbcomm video is good.

It probably won't if the launch slips by "only" 2 days, which I think might be possible...... ;)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Untribium on 06/02/2014 02:17 pm
1 - Do you think we need better tutorials for using the editor? What would you consider better?
2 - Do you think the online editor should be more intuitive to use? What is not intuitive right now?
3 - Do you need more feedback flipping single bits or changing bitpositions? Do you need to see the actual bits? What kind of feedback apart from the resulting frame?
4 - Do you need to have a better explanation about MPEG4? Do you want to see a concrete example based on the actual video data?
5 - Do you want simpler and more concrete tasks to do? It's now divided into roughly 300 parts. Should we divide it

1 - Maybe...As some people have mentioned before, it's pretty straightforward to use, but I think we need to make it clearer to people that even after we figured out how to fix the frames, there's a ton of trial and error. When I first decided to join the effort, I just took a frame and started working on it. After some time I got some semi-good-looking results and headed back here to post them. However, in the meantime, someone had fixed something like 3 entire frames which had me wondering...Must be black magic, right? :D But it turns out, it's really mostly just about practice. You start to understand what works and what doesn't. At some point you may realize, that to fix a chroma error, you better start from the address of the next block and go backwards, because the chroma is at the end of the block...lots of stuff like this. So I guess we could use some sort of a "tips and tricks" section, maybe even FAQ-style:
Q: "Chroma is off, how do I fix it?"
A: Try this, and that and this...
Just an idea, don't know how well that would actually work :)

3 - Yes, absolutely! Actually seeing the bits could be enormously helpful.
However: When I started trying to correct chroma errors by flipping bits, I got confused by how often flips would change the length of the block, because I was expecting some kind of prefix based encoding i.e. 2 size bits encoding n and then n payload bits. Turns out, they're using variable length coding, which makes a ton of sense, but it also makes it more difficult to actually interpret the bits you're seeing (because they essentially don't have a meaning anymore). So what we need in order to be able to find the flipped bit, is the encoding table (I'm guessing it's some kind of tree...huffman comes to mind :) ). I'm guessing there's one table per frame (maybe only iframe?...), so it would be easy do decode them all once (they shouldn't change, right?) and post them on the wiki, maybe even make them available on the online editor. Again, just an idea :)

I'd note that you guys need to think about the next milestone point, then we should think about another updated video and we'll get that out there like the previous.

We're not going to have Elon tweeting every time, but when we get to the point we've done all we can, we can show the milestones.

When we do get to that "we've done all we can" point, then we should consider an article on site. Heck, we have Elon quotes now! :) I'll need a LOT of help with to translate the technical to something readable by Joe Public, quote some of the team here and use the Elon quotes, etc.

Also, it was suggested to me we could also have a highly technical overview as a press release style write up. I'm cool with all of that, obviously.

PS Per the next milestone video. What we absolutely should do is "where we are" before the ORBCOMM launch, as that launch is probably going to have a good video beamed back. Regardless, this is the historic first, so there's no value lost on this effort if ORBCOMM sends a good video back.

I agree, we need some kind of game plan :) It might be really helpful to declare certain parts as "done", so that people can start fine tuning them (e.g. wronkiew's timestamp fix (looks great btw!)), while the others can focus on the parts that still need a lot of work.
As for the article, we'll definitely need to do a lot of "translating". Maybe a new thread, where people can work on some of the parts could help.

One thing to consider as well is the future of "The Tool". If you're looking for a program to fix your broken video file, a forum that discusses space topics is probably not the place you'd be looking first ;) So I think it would definitely make sense to clean up the code, maybe add some more functionality (I'm thinking about the various scripts posted) and then add the changes to the official FFmpeg release (or a fork) in order to create a really powerful mpeg fixing tool. But that's still pretty far away I guess :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: JohnKiel on 06/02/2014 03:55 pm
I consider pull-request-for-best-mmbs approach to be too complex for majority or video recovery effort participants. Wiki was simple but not simple enough. I've gone 1 step further, introduced Google Spreadsheets. You can compare the original and spreadsheet version of the page about 4th part:
 * http://spacexlanding.wikispaces.com/Frames+Part+4
 * http://spacexlanding.wikispaces.com/Frames+Part+4+Spreadsheet

To unify best MMB storage and facilitate structural data retrieval I'd setup 15 Google Spreadsheets (one per part) and for convenience linked them in Spreadsheet column to Progress table of http://spacexlanding.wikispaces.com/raw_final_fixedMMB.

There is an idea to make Online Editor to pull "Best MMBs" data direcly from Spreadsheet tables, and idea is laid out in more details at: https://github.com/IainCole/SpaceXVideoApp2/pull/1.

Seeing the spreadsheets go up this morning, I decided to take a hack at pulling data into online editor:
http://pastebin.com/raw.php?i=iiQxTySY

You can test the script in Chrome if you open a script console in your browser when on http://spacex2.slapbet.org/ and enter:
$.getScript('http://pastebin.com/raw.php?i=iiQxTySY');

Once it loads, it should create two buttons at the top of the page that will either load an individual MMB, or all MMBs for the specified frame/part.

Currently, it depends on the index of the row on the sheet, not the frame number in the "Frame Number" column.  Not sure if this will cause issues later.

EDIT:  Changed script to trim MMB results from spreadsheet.  Script now has new URL on pastebin.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 06/02/2014 03:59 pm
PS Per the next milestone video. What we absolutely should do is "where we are" before the ORBCOMM launch, as that launch is probably going to have a good video beamed back. Regardless, this is the historic first, so there's no value lost on this effort if ORBCOMM sends a good video back.

Speaking of which, someone should make sure to remind SpaceX to keep the pizza pan antenna somewhere safe, since they're going to want to display that at the Smithsonian too, decades hence. :D
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 06/02/2014 05:16 pm
Stunning!

(http://spacexlanding.wikispaces.com/file/view/part_9.gif/512412160/part_9.gif)

Amazing!

I'd note that you guys need to think about the next milestone point, then we should think about another updated video and we'll get that out there like the previous.

We're not going to have Elon tweeting every time, but when we get to the point we've done all we can, we can show the milestones.

When we do get to that "we've done all we can" point, then we should consider an article on site. Heck, we have Elon quotes now! :) I'll need a LOT of help with to translate the technical to something readable by Joe Public, quote some of the team here and use the Elon quotes, etc.

Also, it was suggested to me we could also have a highly technical overview as a press release style write up. I'm cool with all of that, obviously.

Something to keep in mind.

PS Per the next milestone video. What we absolutely should do is "where we are" before the ORBCOMM launch, as that launch is probably going to have a good video beamed back. Regardless, this is the historic first, so there's no value lost on this effort if ORBCOMM sends a good video back.

Latest updates applied to the wiki are automatically added to youtube here https://www.youtube.com/user/spacexlandingrestore (https://www.youtube.com/user/spacexlandingrestore)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 06/02/2014 05:22 pm
I consider pull-request-for-best-mmbs approach to be too complex for majority or video recovery effort participants. Wiki was simple but not simple enough. I've gone 1 step further, introduced Google Spreadsheets. You can compare the original and spreadsheet version of the page about 4th part:
 * http://spacexlanding.wikispaces.com/Frames+Part+4
 * http://spacexlanding.wikispaces.com/Frames+Part+4+Spreadsheet

To unify best MMB storage and facilitate structural data retrieval I'd setup 15 Google Spreadsheets (one per part) and for convenience linked them in Spreadsheet column to Progress table of http://spacexlanding.wikispaces.com/raw_final_fixedMMB.

There is an idea to make Online Editor to pull "Best MMBs" data direcly from Spreadsheet tables, and idea is laid out in more details at: https://github.com/IainCole/SpaceXVideoApp2/pull/1.

Seeing the spreadsheets go up this morning, I decided to take a hack at pulling data into online editor:
http://pastebin.com/raw.php?i=nCG9ZpPp

You can test the script in Chrome if you open a script console in your browser when on http://spacex2.slapbet.org/ and enter:
$.getScript('http://pastebin.com/raw.php?i=nCG9ZpPp');

Once it loads, it should create two buttons at the top of the page that will either load an individual MMB, or all MMBs for the specified frame/part.

Currently, it depends on the index of the row on the sheet, not the frame number in the "Frame Number" column.  Not sure if this will cause issues later.

Looks good :)

I'm hoping that a pull request with all this stuff magically appears some point soon ;) If not I'll have a look at integrating this into the editor. I think myroslav might be working on something like this.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: JohnKiel on 06/02/2014 05:34 pm
I consider pull-request-for-best-mmbs approach to be too complex for majority or video recovery effort participants. Wiki was simple but not simple enough. I've gone 1 step further, introduced Google Spreadsheets. You can compare the original and spreadsheet version of the page about 4th part:
 * http://spacexlanding.wikispaces.com/Frames+Part+4
 * http://spacexlanding.wikispaces.com/Frames+Part+4+Spreadsheet

To unify best MMB storage and facilitate structural data retrieval I'd setup 15 Google Spreadsheets (one per part) and for convenience linked them in Spreadsheet column to Progress table of http://spacexlanding.wikispaces.com/raw_final_fixedMMB.

There is an idea to make Online Editor to pull "Best MMBs" data direcly from Spreadsheet tables, and idea is laid out in more details at: https://github.com/IainCole/SpaceXVideoApp2/pull/1.

Seeing the spreadsheets go up this morning, I decided to take a hack at pulling data into online editor:
http://pastebin.com/raw.php?i=nCG9ZpPp

You can test the script in Chrome if you open a script console in your browser when on http://spacex2.slapbet.org/ and enter:
$.getScript('http://pastebin.com/raw.php?i=nCG9ZpPp');

Once it loads, it should create two buttons at the top of the page that will either load an individual MMB, or all MMBs for the specified frame/part.

Currently, it depends on the index of the row on the sheet, not the frame number in the "Frame Number" column.  Not sure if this will cause issues later.

Looks good :)

I'm hoping that a pull request with all this stuff magically appears some point soon ;) If not I'll have a look at integrating this into the editor. I think myroslav might be working on something like this.

FYI:  I just updated the script on a new URL:
http://pastebin.com/raw.php?i=iiQxTySY

I'll try to do a pull request at some point, but I was hoping for some feedback from others to see if this script is  workable solution first.

Wasn't sure if myroslav was simply stating an idea that others should implement, or it it was a feature he was planning on implementing -- hopefully I'm not reinventing the wheel here.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 06/02/2014 05:38 pm
I consider pull-request-for-best-mmbs approach to be too complex for majority or video recovery effort participants. Wiki was simple but not simple enough. I've gone 1 step further, introduced Google Spreadsheets. You can compare the original and spreadsheet version of the page about 4th part:
 * http://spacexlanding.wikispaces.com/Frames+Part+4
 * http://spacexlanding.wikispaces.com/Frames+Part+4+Spreadsheet

To unify best MMB storage and facilitate structural data retrieval I'd setup 15 Google Spreadsheets (one per part) and for convenience linked them in Spreadsheet column to Progress table of http://spacexlanding.wikispaces.com/raw_final_fixedMMB.

There is an idea to make Online Editor to pull "Best MMBs" data direcly from Spreadsheet tables, and idea is laid out in more details at: https://github.com/IainCole/SpaceXVideoApp2/pull/1.

Seeing the spreadsheets go up this morning, I decided to take a hack at pulling data into online editor:
http://pastebin.com/raw.php?i=nCG9ZpPp

You can test the script in Chrome if you open a script console in your browser when on http://spacex2.slapbet.org/ and enter:
$.getScript('http://pastebin.com/raw.php?i=nCG9ZpPp');

Once it loads, it should create two buttons at the top of the page that will either load an individual MMB, or all MMBs for the specified frame/part.

Currently, it depends on the index of the row on the sheet, not the frame number in the "Frame Number" column.  Not sure if this will cause issues later.

Looks good :)

I'm hoping that a pull request with all this stuff magically appears some point soon ;) If not I'll have a look at integrating this into the editor. I think myroslav might be working on something like this.

FYI:  I just updated the script on a new URL:
http://pastebin.com/raw.php?i=iiQxTySY

I'll try to do a pull request at some point, but I was hoping for some feedback from others to see if this script is  workable solution first.

Wasn't sure if myroslav was simply stating an idea that others should implement, or it it was a feature he was planning on implementing -- hopefully I'm not reinventing the wheel here.

Yeah I'm not sure either, it should be fairly simple to integrate the concept directly into the editor rather than using a hack to get the data in. If you want to try it and submit a PR then go for it.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 06/02/2014 06:06 pm
I consider pull-request-for-best-mmbs approach to be too complex for majority or video recovery effort participants. Wiki was simple but not simple enough. I've gone 1 step further, introduced Google Spreadsheets. You can compare the original and spreadsheet version of the page about 4th part:
 * http://spacexlanding.wikispaces.com/Frames+Part+4
 * http://spacexlanding.wikispaces.com/Frames+Part+4+Spreadsheet

To unify best MMB storage and facilitate structural data retrieval I'd setup 15 Google Spreadsheets (one per part) and for convenience linked them in Spreadsheet column to Progress table of http://spacexlanding.wikispaces.com/raw_final_fixedMMB.

There is an idea to make Online Editor to pull "Best MMBs" data direcly from Spreadsheet tables, and idea is laid out in more details at: https://github.com/IainCole/SpaceXVideoApp2/pull/1.

Seeing the spreadsheets go up this morning, I decided to take a hack at pulling data into online editor:
http://pastebin.com/raw.php?i=nCG9ZpPp

You can test the script in Chrome if you open a script console in your browser when on http://spacex2.slapbet.org/ and enter:
$.getScript('http://pastebin.com/raw.php?i=nCG9ZpPp');

Once it loads, it should create two buttons at the top of the page that will either load an individual MMB, or all MMBs for the specified frame/part.

Currently, it depends on the index of the row on the sheet, not the frame number in the "Frame Number" column.  Not sure if this will cause issues later.

Looks good :)

I'm hoping that a pull request with all this stuff magically appears some point soon ;) If not I'll have a look at integrating this into the editor. I think myroslav might be working on something like this.

FYI:  I just updated the script on a new URL:
http://pastebin.com/raw.php?i=iiQxTySY

I'll try to do a pull request at some point, but I was hoping for some feedback from others to see if this script is  workable solution first.

Wasn't sure if myroslav was simply stating an idea that others should implement, or it it was a feature he was planning on implementing -- hopefully I'm not reinventing the wheel here.

Yeah I'm not sure either, it should be fairly simple to integrate the concept directly into the editor rather than using a hack to get the data in. If you want to try it and submit a PR then go for it.

I just spoke to myroslav and he is still intending on working on it, if you want to contribute I'd recommend coming to the IRC channel to talk directly.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: lgjy98d on 06/02/2014 06:14 pm
I consider pull-request-for-best-mmbs approach to be too complex for majority or video recovery effort participants. Wiki was simple but not simple enough. I've gone 1 step further, introduced Google Spreadsheets. You can compare the original and spreadsheet version of the page about 4th part:
 * http://spacexlanding.wikispaces.com/Frames+Part+4
 * http://spacexlanding.wikispaces.com/Frames+Part+4+Spreadsheet

To unify best MMB storage and facilitate structural data retrieval I'd setup 15 Google Spreadsheets (one per part) and for convenience linked them in Spreadsheet column to Progress table of http://spacexlanding.wikispaces.com/raw_final_fixedMMB.

There is an idea to make Online Editor to pull "Best MMBs" data direcly from Spreadsheet tables, and idea is laid out in more details at: https://github.com/IainCole/SpaceXVideoApp2/pull/1.

Seeing the spreadsheets go up this morning, I decided to take a hack at pulling data into online editor:
http://pastebin.com/raw.php?i=nCG9ZpPp

You can test the script in Chrome if you open a script console in your browser when on http://spacex2.slapbet.org/ and enter:
$.getScript('http://pastebin.com/raw.php?i=nCG9ZpPp');

Once it loads, it should create two buttons at the top of the page that will either load an individual MMB, or all MMBs for the specified frame/part.

Currently, it depends on the index of the row on the sheet, not the frame number in the "Frame Number" column.  Not sure if this will cause issues later.

Looks good :)

I'm hoping that a pull request with all this stuff magically appears some point soon ;) If not I'll have a look at integrating this into the editor. I think myroslav might be working on something like this.

FYI:  I just updated the script on a new URL:
http://pastebin.com/raw.php?i=iiQxTySY

I'll try to do a pull request at some point, but I was hoping for some feedback from others to see if this script is  workable solution first.

Wasn't sure if myroslav was simply stating an idea that others should implement, or it it was a feature he was planning on implementing -- hopefully I'm not reinventing the wheel here.

I've just had limited time in a day to do the pull request, thus I've laid out ideas and did foundation of the data storage only, but having your code the effort becomes a "translation" of jquery into angular :) and I expect to do this later today.

Other "missing" parts of the puzzle that are still in my todo list  are:
* automatic generation of tables that has always up to date images, and
* always up-to date gif animations.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 06/02/2014 06:33 pm
Just recovered i-frame 141 (using offsets from previous version). The image still needs a color tuning (Quialiss! Help! ;) you're the best at this)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: princess on 06/02/2014 07:09 pm
For Quialiss and the other people tweaking MMB values, here are the changes needed to map the old bit positions to new ones:

I-frame 1: None needed
I-frame 21: None needed
I-frame 41: None needed
I-frame 61: 92728-95672 have been deleted
I-frame 81: None needed

I-frame 101:
  4224-14255: add 272 bits
  14256-26031: add 1744 bits
  26032-53919: add 1824 bits
  53920-151936: add 2480 bits

I-frame 121:
  26304-46711: add 200 bits
  46712-62479: add 624 bits
  62480-173007: add 1968 bits
  173008-253192: add 2920 bits

I-frame 141:
  26304-33463: add 200 bits
  33464-34936: Deleted
  34936-46711: Remove 1271 bits
  46712-62479: Remove 848 bits
  62480-150928: Add 496 bits
 
I-frame 161: None needed
I-frame 181: 108736-163616: Add 1280 bits
I-frame 201: None needed

I-frame 221:
  16000-78287: Add 1008 bits
  78288-95015: Add 1248 bits
  95016-96488: Deleted

I-frame 241: None needed
I-frame 261: None needed

I-frame 281:
  33664-148992: Add 4416 bits
 
I hope this is all correct, it seems to match up with the existing ones that Quialiss found. Hopefully we can have some sexy new I-frames soon!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Swatch on 06/02/2014 07:46 pm
...

Speaking of which, someone should make sure to remind SpaceX to keep the pizza pan antenna somewhere safe, since they're going to want to display that at the Smithsonian too, decades hence. :D

Fear not, it is well-preserved.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Eer on 06/02/2014 08:17 pm
Actually, it should go into a SpaceX HQ hallway display that consists of the pizza antenna, a video display of the reconstructed video, and a poster copy of the NasaSpaceflight article Chris will publish describing Elon's shout out asking for help, and the forum response, process, success and lessons learned. And yes, this seems to be a notable case of crowd sourcing problem solving, collaborative effort with a very organic leadership that has worked very well.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Untribium on 06/02/2014 08:27 pm
Just recovered i-frame 141 (using offsets from previous version). The image still needs a color tuning (Quialiss! Help! ;) you're the best at this)

Remove 14:21:96672 and add 1:21:94587. Adds some more blocks (around the rocket's Great Black Spot) and fixes some color stuff too. I also have some bitflips color correction:
X:7179:80,X:69235:80
And then there are a ton of flips that seem to work, but I'm suspecting that it's mainly because they make some blocks change their inheritance behavior, so it's not an actual fix.
X:20883:80 seems okay, X:14053:80 is interesting, it fixes the grayish block at 43:3 (by making it 2 bits longer), but messes with some of the stuff at the bottom right...
I'll probably work on it some more, so I won't update the wiki just yet. I'll attach my current version.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Quialiss on 06/02/2014 09:25 pm
For Quialiss and the other people tweaking MMB values, here are the changes needed to map the old bit positions to new ones:

Brilliant, princess!  Oddly enough there was another offset at the end of frame 121, but fortunately the timestamp provides a good reference point so I was able to find the correct value.  The iframe set is now complete again-- Well, except for 1 and 2, but I don't think we're ever going to recover those.



Someone mentioned progress reports... Since the mighty SwissCheese declares me an expert on doing the final pass on the i frames to clean them up, this is the current status of the i frames in relation to what I can do with them if I'm the only one finishing them up.  Definitely wouldn't mind not being the only one finishing them up.  Untribium, have at it! 

1 -
2 -
3 practically perfect (the least satisfying frame ever to say that for)
4 minor issues that should be easy to fix
5 already worked on, still has quite a few issues because this frame is a nightmare.  I might be able to improve it a little further.
6 already worked on, minor issues remaining
7 major issues
8 major issues

9 already worked on, minor issues remaining
10 practically perfect, I could fix up the very bottom a bit.
11 minor issues, and I've tried to work out a few of them with no measurable success, it'll be hard to improve on this one.
12 major issues
13 minor issues remaining, I left this frame halfway cleaned (whoops)
14 major issues
15 major issues


I should be able to fix up at least 7 and 8 tomorrow, which, once all the p frames are polished, will give us the full leg deploy to just before splashdown in amazingly good quality considering what we started with.

12, 13 and 15 all have massive gaps in them where we have no (good) data, which should mean they'll take less time for me to clean up.  And speaking of those frames, is it possible to tell the decoder to completely ignore the blanked out portions of the frame, and show the previous I frame + cumulative effects of the p frames where there isn't data?  Just a thought to improve the final video
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 06/02/2014 09:32 pm
For Quialiss and the other people tweaking MMB values, here are the changes needed to map the old bit positions to new ones:

Brilliant, princess!  Oddly enough there was another offset at the end of frame 121, but fortunately the timestamp provides a good reference point so I was able to find the correct value.  The iframe set is now complete again-- Well, except for 1 and 2, but I don't think we're ever going to recover those.



Someone mentioned progress reports... Since the mighty SwissCheese declares me an expert on doing the final pass on the i frames to clean them up, this is the current status of the i frames in relation to what I can do with them if I'm the only one finishing them up.  Definitely wouldn't mind not being the only one finishing them up.  Untribium, have at it! 

1 -
2 -
3 practically perfect (the least satisfying frame ever to say that for)
4 minor issues that should be easy to fix
5 already worked on, still has quite a few issues because this frame is a nightmare.  I might be able to improve it a little further.
6 already worked on, minor issues remaining
7 major issues
8 major issues

9 already worked on, minor issues remaining
10 practically perfect, I could fix up the very bottom a bit.
11 minor issues, and I've tried to work out a few of them with no measurable success, it'll be hard to improve on this one.
12 major issues
13 minor issues remaining, I left this frame halfway cleaned (whoops)
14 major issues
15 major issues


I should be able to fix up at least 7 and 8 tomorrow, which, once all the p frames are polished, will give us the full leg deploy to just before splashdown in amazingly good quality considering what we started with.

12, 13 and 15 all have massive gaps in them where we have no (good) data, which should mean they'll take less time for me to clean up.  And speaking of those frames, is it possible to tell the decoder to completely ignore the blanked out portions of the frame, and show the previous I frame + cumulative effects of the p frames where there isn't data?  Just a thought to improve the final video

I can try to focus on iframe 14 and 15. Hopefully I will have some time tomorrow.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 06/02/2014 09:46 pm
12, 13 and 15 all have massive gaps in them where we have no (good) data, which should mean they'll take less time for me to clean up.  And speaking of those frames, is it possible to tell the decoder to completely ignore the blanked out portions of the frame, and show the previous I frame + cumulative effects of the p frames where there isn't data?  Just a thought to improve the final video

I have been contemplating this for the past few days. And I'm going to throw this idea out here for feedback. It should be possible to convert I frames to P frames while keeping all data in them. By my reconing what's needed is to add the not_coded bit to every macroblock and convert all mcbpcs from iframe version to pframe version. The rest of macroblock data should convert directly. This would allow using -3 mmb command in iframes. I could probably only do it in the ts file (but it could conceivably be done in ffmpeg, but that's beyond my skill). There are many obvious problems. The iframe would not be tweakable (except dc values) after this and it would throw off existing dc edits. So it would have to be the penultimate pass before final chroma/luma corrections. Just food for thought for you guys.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: princess on 06/02/2014 10:33 pm
Does anyone know of a simple tool that can show what fields and values are present in an MPEG4 stream? It would be great to be able to see the fields in the macroblocks, see what their current values are - that way, we could see what effect various bitflips have, and also what patterns are present in the stream.

We had a lot of success in cleaning the TS level because there were several common patterns to the data. The idea is that the transmitting device in the rocket is going to be optimised for weight and power consumption, so therefore it's likely to be some simple ARM SoC paired with some form of DSP video processor. It has tight timing constraints and can't use lots of electric power, so it's likely to just encode in a simple way.

Because of this, there were lots of patterns and similarities to the data - we could look at good areas to find the patterns, and then apply these patterns to the corrupt areas. After many rounds of this, the structure of the TS file begins to emerge, and you can fix a lot of extra stuff.

So my idea is to see what we can do with the same technique at the MPEG4 level. Reading through the MPEG specification is very challenging because of the myriad of options available to an MPEG4 encoder - however, my bet is that the encoder on the rocket won't have been doing anything bizarre, it would just use a simple subset of MPEG4 that can be blasted through a DSP by an ARM core.

So my questions really are:

1. Is there a freely-available MPEG4 stream examination tool?
2. What simple subset of MPEG4 did the rocket's encoder use?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 06/02/2014 11:04 pm
1. Is there a freely-available MPEG4 stream examination tool?
2. What simple subset of MPEG4 did the rocket's encoder use?

1. I don't think such a tool exists at this time. Arnezami has been workong on this, but the current version is a far cry from simple at this point in time I believe.

2. No B frames are used only I and P frames. On the macroblock level it seems to me that in I frames only type 3 macroblocks are used and in P frames (I think, but have not checked) only not coded, type 0 and type 3 macroblocks are used (maybe type 2 also) i.e. no macroblocks with dquant. I believe these are optimizations for live encoding. The constraints these limits impose on the bitstream are small and not much use for making sense of the data.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 06/02/2014 11:16 pm
Does anyone know of a simple tool that can show what fields and values are present in an MPEG4 stream? It would be great to be able to see the fields in the macroblocks, see what their current values are - that way, we could see what effect various bitflips have, and also what patterns are present in the stream.

We had a lot of success in cleaning the TS level because there were several common patterns to the data. The idea is that the transmitting device in the rocket is going to be optimised for weight and power consumption, so therefore it's likely to be some simple ARM SoC paired with some form of DSP video processor. It has tight timing constraints and can't use lots of electric power, so it's likely to just encode in a simple way.

Because of this, there were lots of patterns and similarities to the data - we could look at good areas to find the patterns, and then apply these patterns to the corrupt areas. After many rounds of this, the structure of the TS file begins to emerge, and you can fix a lot of extra stuff.

So my idea is to see what we can do with the same technique at the MPEG4 level. Reading through the MPEG specification is very challenging because of the myriad of options available to an MPEG4 encoder - however, my bet is that the encoder on the rocket won't have been doing anything bizarre, it would just use a simple subset of MPEG4 that can be blasted through a DSP by an ARM core.

So my questions really are:

1. Is there a freely-available MPEG4 stream examination tool?
2. What simple subset of MPEG4 did the rocket's encoder use?
I was indeed working on this.

This adaptation to ffmpeg is capable of logging (sub)macroblock data: http://forum.nasaspaceflight.com/index.php?topic=34597.msg1207538#msg1207538

Here you can see the output (bit-by-bit) what the decoder is interpreting: http://forum.nasaspaceflight.com/index.php?topic=34597.msg1207391#msg1207391

There is indeed quite a lot of things in MPEG4 that is not being used by the encoder on the rocket. No B-frames, no slices, no short header, no changes in q-values (I think), only minor ac-prediction. It does use what are called "escape"-codes for the DCT. Most (if not all of this) is logged by this addition to ffmpeg. Only for I-frames though.

If you're interested I can make a list/explanation of the tables from MPEG-4 are used. But maybe first try to play with it so you get a sense of what is going on. Just let me know if you need more info.

I have to say though (I have been looking at this too) that due to the VLC MPEG-4 data is far less predictable than TS-data. It really looks completely random until you see it interpreted as a picture/macroblock.

Regards,

arnezami

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: JohnKiel on 06/02/2014 11:17 pm
The online editor, http://spacex2.slapbet.org/ (http://spacex2.slapbet.org/), now has buttons to allow loading MMBs directly from spreadsheets that are now listed in the wiki (http://spacexlanding.wikispaces.com/raw_final_fixedMMB).

These spreadsheets probably still need some cleaning up and updating to sync them up with the wiki tables.  I believe myroslav is working on embedding images in the sheet.

Make sure that MMBs entered in the sheets don't have any notes or the like in them.

Thanks to myroslav, a.k.a. lgjy98d, for setting the sheets up, and IainCole for the online editor and his help in getting these changes to the editor live.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 06/02/2014 11:18 pm
The online editor, http://spacex2.slapbet.org/ (http://spacex2.slapbet.org/), now has buttons to allow loading MMBs directly from spreadsheets that are now listed in the wiki (http://spacexlanding.wikispaces.com/raw_final_fixedMMB).

These spreadsheets probably still need some cleaning up and updating to sync them up with the wiki tables.  I believe myroslav is working on embedding images in the sheet.

Make sure that MMBs entered in the sheets don't have any notes or the like in them.

Thanks to myroslav, a.k.a. lgjy98d, for setting the sheets up, and IainCole for the online editor and his help in getting these changes to the editor live.

The spreadsheet for iframe 8 is the one that we have been working on as a template. It has the images in etc.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: DecoLV on 06/03/2014 12:52 am
1 - Do you think we need better tutorials for using the editor? What would you consider better?
2 - Do you think the online editor should be more intuitive to use? What is not intuitive right now?
3 - Do you need more feedback flipping single bits or changing bitpositions? Do you need to see the actual bits? What kind of feedback apart from the resulting frame?
4 - Do you need to have a better explanation about MPEG4? Do you want to see a concrete example based on the actual video data?
5 - Do you want simpler and more concrete tasks to do? It's now divided into roughly 300 parts. Should we divide it more? Or should it be divided differently?

Quick feedback on this....

No experience. I went through the p-frame tutorial for about 20 minutes, and sort of got it a little, but not enough to be clear what I was doing. I don't think anything is wrong with the tutorial, but even if I invested  a couple of hours, I doubt I could make a meaningful contribution. I would just feel in the way and too scared to commit anything to the wiki. I appreciate you folks inviting us in, but I truly don't think this is a job for noobs....you have to know what you are doing, a little. Sorry. You folks are awesome and your work is appreciated. Good luck!

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Untribium on 06/03/2014 01:16 am
1 - Do you think we need better tutorials for using the editor? What would you consider better?
2 - Do you think the online editor should be more intuitive to use? What is not intuitive right now?
3 - Do you need more feedback flipping single bits or changing bitpositions? Do you need to see the actual bits? What kind of feedback apart from the resulting frame?
4 - Do you need to have a better explanation about MPEG4? Do you want to see a concrete example based on the actual video data?
5 - Do you want simpler and more concrete tasks to do? It's now divided into roughly 300 parts. Should we divide it more? Or should it be divided differently?

Quick feedback on this....

No experience. I went through the p-frame tutorial for about 20 minutes, and sort of got it a little, but not enough to be clear what I was doing. I don't think anything is wrong with the tutorial, but even if I invested  a couple of hours, I doubt I could make a meaningful contribution. I would just feel in the way and too scared to commit anything to the wiki. I appreciate you folks inviting us in, but I truly don't think this is a job for noobs....you have to know what you are doing, a little. Sorry. You folks are awesome and your work is appreciated. Good luck!

First of all, thanks for sharing, this really helps! The thing is, I'm a noob myself :D I mean, I study computer science, but I've never done anything even remotely close to what we're doing here. Also. one of my least favorite topics was visual computing :)
One problem that I can see, is that it's incredibly difficult if the first thing you work on is a badly corrupted p-frame. Most of us started on frames like i-frame 8 (back then, now 10), which was almost in perfect shape, and you could quickly make progress. Then there was i-frame 2 (now 4), which was a little bit tougher, but still doable, then 9 (now 11)...you get the point :) Starting on a bad p-frame is like trying to teach integral calculus in first grade - you wouldn't understand anything cause you need to practice simpler stuff first. What might help is a step by step tutorial how to recover e.g. i-frame 4, so people could go through it one part at a time and learn the different techniques required to do other stuff.
Another "problem" is that some people here have gotten incredibly efficient, so one might argue that it's better to leave it to them. I don't think so, though. I find that I spend the most time trying a range of maybe 100 to 400 bit positions for a specific block. So I guess one way to make it easier for people to join would be to post the range and then people can post the most promising results back here.
It all comes back to this website where we generate thousands of images and have people vote on them :D It sounds like a lot of fun, but I'm not sure how well it would work...Anyways, gotta work on 101 some more :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Robotbeat on 06/03/2014 01:53 am
I have a suggestion: a basic tutorial video where you fix something and explain everything you're doing and why. Just an idea.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: meekGee on 06/03/2014 03:40 am
I have a suggestion: a basic tutorial video where you fix something and explain everything you're doing and why. Just an idea.

Yeah, but scramble it so you can only make out a facial expression twice throughout, and lots of green please.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 06/03/2014 05:05 am
I finished putting together a clock fix editor, so anyone who wants to help out can give it a try. Find it here:

http://adama.nocdirect.com/~wronkiew/spx_crs3/clockfix_edit.php

It's quite limited compared to IainCole's amazing mmb editor, but it does let you write clockfix instructions and test them out. Also it shows any potential parsing errors. It has no image inspection tools, so you will need to download the displayed frame and open it in a program like GraphicConverter or Photoshop whatever the Windows equivalent is. It will also be very helpful to download the isolated p-frame images from the wiki.

If you want to try it out, select frame 101 and copy in these instructions:

# erase (row,begin,end)
464,32,40
465,32,39
466,32,39
467,32,39
468,32,39
469,32,39
470,32,39
471,32,39
472,32,39
473,32,39
474,32,39
475,32,39
476,32,39
477,32,39
478,32,39
479,32,39
# digit (char,x,1even/2odd/3both)
9,32,3
:,46,3

Then press the Submit button to see the fixed frame.

The editor/tester does not save the clockfix instructions for you. Once you have cleaned up the gibberish and repainted the digits, save the text to a file clockfix-nnn.csv with the frame number, for example clockfix-047.csv. Then upload it to the wiki.

As I mentioned on the wiki page, only repaint digits if you are 99.9% confident of what they are. The intent is to make numbers readable, not to interpolate between "good" timestamps.

I would welcome any feedback on making the editor more usable.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: deruch on 06/03/2014 06:49 am
I finished putting together a clock fix editor, so anyone who wants to help out can give it a try. Find it here:

http://adama.nocdirect.com/~wronkiew/spx_crs3/clockfix_edit.php

It's quite limited compared to IainCole's amazing mmb editor, but it does let you write clockfix instructions and test them out. Also it shows any potential parsing errors. It has no image inspection tools, so you will need to download the displayed frame and open it in a program like GraphicConverter or Photoshop whatever the Windows equivalent is. It will also be very helpful to download the isolated p-frame images from the wiki.

/snip/

As I mentioned on the wiki page, only repaint digits if you are 99.9% confident of what they are. The intent is to make numbers readable, not to interpolate between "good" timestamps.

I would welcome any feedback on making the editor more usable.

Do you need to delete?  For many of the frames, the numbers are simply missing.  Can we just edit in the correct number without deleting the background?  e.g. for frame #201 giving it this code:

# digit (char,x,even/odd/both)
1,18,3
9,32,3
:,46,3
3,58,3
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 06/03/2014 07:13 am
I finished putting together a clock fix editor, so anyone who wants to help out can give it a try. Find it here:

http://adama.nocdirect.com/~wronkiew/spx_crs3/clockfix_edit.php

It's quite limited compared to IainCole's amazing mmb editor, but it does let you write clockfix instructions and test them out. Also it shows any potential parsing errors. It has no image inspection tools, so you will need to download the displayed frame and open it in a program like GraphicConverter or Photoshop whatever the Windows equivalent is. It will also be very helpful to download the isolated p-frame images from the wiki.

/snip/

As I mentioned on the wiki page, only repaint digits if you are 99.9% confident of what they are. The intent is to make numbers readable, not to interpolate between "good" timestamps.

I would welcome any feedback on making the editor more usable.

Do you need to delete?  For many of the frames, the numbers are simply missing.  Can we just edit in the correct number without deleting the background?  e.g. for frame #201 giving it this code:

# digit (char,x,even/odd/both)
1,18,3
9,32,3
:,46,3
3,58,3

It'd be best to get rid of the squiggly line between the 3 and the 5.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Quialiss on 06/03/2014 10:10 am
Quick note to our MB position finding wizards:  It is definitely worthwhile to go looking at the places data was added in frames 6, 7, 8, and 12.  The relevant bitpositions and number of added bits are on their respective wiki pages. 

I just found a block of 12 macroblocks in frame 7.  I think that's actually the first i frame data I've recovered.  :D 
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 06/03/2014 10:15 am

1 - Do you think we need better tutorials for using the editor? What would you consider better?
2 - Do you think the online editor should be more intuitive to use? What is not intuitive right now?
3 - Do you need more feedback flipping single bits or changing bitpositions? Do you need to see the actual bits? What kind of feedback apart from the resulting frame?
4 - Do you need to have a better explanation about MPEG4? Do you want to see a concrete example based on the actual video data?
5 - Do you want simpler and more concrete tasks to do? It's now divided into roughly 300 parts. Should we divide it more? Or should it be divided differently?

Can you please at least sort the above numbers in order of priority? And then explain why you think some are more important than others? And maybe then try to answer (some of) the questions? Thanks!

Could you also tell what you read (tutorial/wiki page wise) before using the online editor? Because maybe we can do something about the (more prominent) placement of the explanations already available on the wiki/forum.

I really like to know. And hopefully you can help us here.

Thanks in advance.

Regards,

arnezami

Just a few remarks not exactly related to this, but that should help people understanding how to clean p-frames:
- Enhance the contrast with X:59:C0 / X:59:80 / X:59:60 / X:59:40 (usually C0 is the best, but sometimes another is better), it helps a lot seeing errors and good sequences. Remove this when you're finished.
- Correct MB look somehow "natural", they usually don't have crosses in them, have information in the full MB, don't have funny colors.... Try with an already corrected p-frame to understand better what was removed.
- Align features with the previous and following p-frame (surf on water works very well).
- Bitflips work quite well for i-frames; they usually don't work for p-frames: even when it seems to solve your issue, when you look carefully at the MB around your correction you find some "unnatural" features. It is really rare that it really corrects an error.

Otherwise, the more you do it, the better you get, and when you get back to earlier work, you see that you made many mistakes :)

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 06/03/2014 11:02 am
The online editor, http://spacex2.slapbet.org/ (http://spacex2.slapbet.org/), now has buttons to allow loading MMBs directly from spreadsheets that are now listed in the wiki (http://spacexlanding.wikispaces.com/raw_final_fixedMMB).

These spreadsheets probably still need some cleaning up and updating to sync them up with the wiki tables.  I believe myroslav is working on embedding images in the sheet.

Make sure that MMBs entered in the sheets don't have any notes or the like in them.

Thanks to myroslav, a.k.a. lgjy98d, for setting the sheets up, and IainCole for the online editor and his help in getting these changes to the editor live.

The spreadsheet for iframe 8 is the one that we have been working on as a template. It has the images in etc.

Just updated the mmb's of this part, the images come automatically, really nicely done!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Ford Mustang on 06/03/2014 11:52 am
I must've been living under a rock for the past month and missed this thread.  Just wanted to say, you guys have done an absolutely amazing job pulling all this data out of the raw TS file.  Keep up the great work, it's very impressive to know just how diverse this forum is in everything!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: JohnKiel on 06/03/2014 12:11 pm
The online editor, http://spacex2.slapbet.org/ (http://spacex2.slapbet.org/), now has buttons to allow loading MMBs directly from spreadsheets that are now listed in the wiki (http://spacexlanding.wikispaces.com/raw_final_fixedMMB).

These spreadsheets probably still need some cleaning up and updating to sync them up with the wiki tables.  I believe myroslav is working on embedding images in the sheet.

Make sure that MMBs entered in the sheets don't have any notes or the like in them.

Thanks to myroslav, a.k.a. lgjy98d, for setting the sheets up, and IainCole for the online editor and his help in getting these changes to the editor live.

The spreadsheet for iframe 8 is the one that we have been working on as a template. It has the images in etc.

Just updated the mmb's of this part, the images come automatically, really nicely done!

Unfortunately, it looks like the MMBs can create an image request URL that's larger than the ~2KB Google Sheet's IMAGE function can manage.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 06/03/2014 12:39 pm
- Enhance the contrast with X:59:C0 / X:59:80 / X:59:60 / X:59:40 (usually C0 is the best, but sometimes another is better), it helps a lot seeing errors and good sequences. Remove this when you're finished.
This can usually be cranked even further by X:58:80, X:58:C0 or X:58:E0
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Lourens on 06/03/2014 12:45 pm
These would be the two resources I kept going back to for reference.  I STILL don't understand why the propagation of values changes so dramatically sometimes, but I've learned to work with it.  (If I understand correctly the direction a block inherits from is dynamic.. I just don't quite understand why it switches seemingly arbitrarily due to luminance changes)
Which direction to use is determined by the DC values of the left, top-left and top neighbours (section 7.4.3.1 in the standard). If the DC values of the left and top-left neighbours are closer together than the DC values of the top-left and top neighbours, then MPEG4 concludes that the image changes less vertically than horizontally, and uses the top neighbour as a reference. If the top-left and top neighbours are closer in DC values, then it will use the left neighbour to base the current block on. So you need to take into account changes in those three neighbours. Does that make sense?

3 - Yes, absolutely! Actually seeing the bits could be enormously helpful.
However: When I started trying to correct chroma errors by flipping bits, I got confused by how often flips would change the length of the block, because I was expecting some kind of prefix based encoding i.e. 2 size bits encoding n and then n payload bits. Turns out, they're using variable length coding, which makes a ton of sense, but it also makes it more difficult to actually interpret the bits you're seeing (because they essentially don't have a meaning anymore). So what we need in order to be able to find the flipped bit, is the encoding table (I'm guessing it's some kind of tree...huffman comes to mind :) ). I'm guessing there's one table per frame (maybe only iframe?...), so it would be easy do decode them all once (they shouldn't change, right?) and post them on the wiki, maybe even make them available on the online editor. Again, just an idea :)
It's even easier, the tables are fixed by the standard. The tables in Appendix B contain all the secret codes you'll need :).
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Untribium on 06/03/2014 01:07 pm
-snip-
3 - Yes, absolutely! Actually seeing the bits could be enormously helpful.
However: When I started trying to correct chroma errors by flipping bits, I got confused by how often flips would change the length of the block, because I was expecting some kind of prefix based encoding i.e. 2 size bits encoding n and then n payload bits. Turns out, they're using variable length coding, which makes a ton of sense, but it also makes it more difficult to actually interpret the bits you're seeing (because they essentially don't have a meaning anymore). So what we need in order to be able to find the flipped bit, is the encoding table (I'm guessing it's some kind of tree...huffman comes to mind :) ). I'm guessing there's one table per frame (maybe only iframe?...), so it would be easy do decode them all once (they shouldn't change, right?) and post them on the wiki, maybe even make them available on the online editor. Again, just an idea :)
It's even easier, the tables are fixed by the standard. The tables in Appendix B contain all the secret codes you'll need :).

Oh...that seems kinda silly...I mean, the whole point of VLC is to optimize the tables to assign the shortest possible codes to the most common values in the chunk of data you're encoding, right? Sure, you can assume that small numbers will generally be more common, but I feel like a prefix based encoding should be more efficient for this task (it's probably not, I think these people know more about this than I do ;) ). Oh well, I'm definitely not complaining, should make things easier. Thanks for looking it up :))
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Quialiss on 06/03/2014 01:13 pm
If the DC values of the left and top-left neighbours are closer together than the DC values of the top-left and top neighbours, then MPEG4 concludes that the image changes less vertically than horizontally, and uses the top neighbour as a reference. If the top-left and top neighbours are closer in DC values, then it will use the left neighbour to base the current block on.

Aaah, this is the magical part I was missing.  Thank you! 
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Quialiss on 06/03/2014 02:14 pm
Unfortunately, it looks like the MMBs can create an image request URL that's larger than the ~2KB Google Sheet's IMAGE function can manage.

A bit ironic considering the editor had to be modified to deal with the long MMBs.  It still works for all the individual p frames though, which is great! 
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: JohnKiel on 06/03/2014 03:04 pm
Unfortunately, it looks like the MMBs can create an image request URL that's larger than the ~2KB Google Sheet's IMAGE function can manage.

A bit ironic considering the editor had to be modified to deal with the long MMBs.  It still works for all the individual p frames though, which is great!

I'm tinkering with a work-around.  Basically a proxy that allows a much shorter URL to request the image.  Currently sorting out how to cache and respond with 304's to avoid quickly using up the paltry 10GB/month I get on my chosen host. (If anyone with more bandwidth is willing to host the simple php image proxy script once I'm done, let me know.)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mhenderson on 06/03/2014 03:47 pm
Hats off to whoever did the latest P frame work on 12 (221).  Steam plume as hot engines hit water? I spent a lot of time on the I frame (hack as it is / I am) and never thought we'd see this kind of detail.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Chris Bergin on 06/03/2014 03:50 pm
Oh I do like the idea behind this:

https://www.youtube.com/channel/UCyZDgyJBYz3OXD3JbDJzNww (https://www.youtube.com/channel/UCyZDgyJBYz3OXD3JbDJzNww)

That is a great one stop page to see the progress over time.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 06/03/2014 04:05 pm
Correcting the motion vectors would help eliminate that "melting" sensation we see in some of the pframes, particularly on the rocket body. Presumably the MVs for almost all of the MBs in that lower section of the screen should be zeroed out.

I found this piece of code which may help in the identification of where and how the motion vectors are encoded in the pframes - check out the printMVmatrix() function:

http://www.princeton.edu/~jiasic/cos435/motion_vector.c
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 06/03/2014 04:19 pm
Guys, I need some help here. I finally got around to setting up my local system for pframe work, but I'm running into problems. I got the latest ffmpeg from the wiki and the latest ts files from here:
http://forum.nasaspaceflight.com/index.php?topic=34597.msg1205739#msg1205739
Then I made a txt filecontaining: FRAME0:<latest mmb for part 5 iframe>=FRAME1:
Then I ran this command:
FFmpeg-spacexdebug1/ffmpeg -mmb `cat pframe/part5_mmb.txt` -err_detect ignore_err -i final_fixed/sh_rawsplit_part_05_fixed_.ts -f image2 frames/test%2d.png
Got back 20 pngs as expected and the iframe looks correct. But pframes are wrong. I attach a picture of the first pframe I get and what it looks like in online editor. If I set FRAME0:0:0:-1 the first few pframes come out completely blank gray, but later pframes start showing something.

What am I doing wrong?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Quialiss on 06/03/2014 04:32 pm
Correcting the motion vectors would help eliminate that "melting" sensation we see in some of the pframes, particularly on the rocket body. Presumably the MVs for almost all of the MBs in that lower section of the screen should be zeroed out.

I found this piece of code which may help in the identification of where and how the motion vectors are encoded in the pframes - check out the printMVmatrix() function:

http://www.princeton.edu/~jiasic/cos435/motion_vector.c

There is interesting data in the rocket body in some of the pframes.  You can see the black spot of dirt fading away from all the wet air rushing past it, for example. 
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 06/03/2014 04:32 pm
Guys, I need some help here. I finally got around to setting up my local system for pframe work, but I'm running into problems. I got the latest ffmpeg from the wiki and the latest ts files from here:
http://forum.nasaspaceflight.com/index.php?topic=34597.msg1205739#msg1205739
Then I made a txt filecontaining: FRAME0:<latest mmb for part 5 iframe>=FRAME1:
Then I ran this command:
FFmpeg-spacexdebug1/ffmpeg -mmb `cat pframe/part5_mmb.txt` -err_detect ignore_err -i final_fixed/sh_rawsplit_part_05_fixed_.ts -f image2 frames/test%2d.png
Got back 20 pngs as expected and the iframe looks correct. But pframes are wrong. I attach a picture of the first pframe I get and what it looks like in online editor. If I set FRAME0:0:0:-1 the first few pframes come out completely blank gray, but later pframes start showing something.

What am I doing wrong?

Try this:

FFmpeg-spacexdebug1/ffmpeg -threads 1 -mmb `cat pframe/part5_mmb.txt` -err_detect ignore_err -i final_fixed/sh_rawsplit_part_05_fixed_.ts -f image2 frames/test%2d.png
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 06/03/2014 04:43 pm
Try this:

FFmpeg-spacexdebug1/ffmpeg -threads 1 -mmb `cat pframe/part5_mmb.txt` -err_detect ignore_err -i final_fixed/sh_rawsplit_part_05_fixed_.ts -f image2 frames/test%2d.png

:D So many thanks.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mhenderson on 06/03/2014 04:48 pm
Oh I do like the idea behind this:

https://www.youtube.com/channel/UCyZDgyJBYz3OXD3JbDJzNww (https://www.youtube.com/channel/UCyZDgyJBYz3OXD3JbDJzNww)

That is a great one stop page to see the progress over time.

I agree it is awesome. Of course I will beg for more :-) ... It would be sweet to have a slow motion (3 fps?) copy running immediately after the 15 fps video.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Req on 06/03/2014 06:12 pm
Unfortunately, it looks like the MMBs can create an image request URL that's larger than the ~2KB Google Sheet's IMAGE function can manage.

A bit ironic considering the editor had to be modified to deal with the long MMBs.  It still works for all the individual p frames though, which is great!

I'm tinkering with a work-around.  Basically a proxy that allows a much shorter URL to request the image.  Currently sorting out how to cache and respond with 304's to avoid quickly using up the paltry 10GB/month I get on my chosen host. (If anyone with more bandwidth is willing to host the simple php image proxy script once I'm done, let me know.)

I'll host any files this project needs, or even any daemon as long as I don't have to jump through flaming hoops to get it running on a CentOS 5 system(or systems).  Primary circuit is a 90th percentile 10g line that usually sees less than 75Mbps of it's 1Gbps monthly commit, so LOTS of free bandwidth(something like 250TB/mo) is sitting around right now.

Hit me with a PM anytime.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 06/03/2014 07:34 pm
Hi guys,

My attempt at fixing the luma/chroma for iframe 14 (iframe 261). I spent way to much time on this ;).

I find this frame very hard to fix: this mmb is huge. Maybe somebody else has a more elegant fix.

It looks a lot better though.  8)

Regards,

arnezami
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: maximlevitsky on 06/03/2014 08:05 pm
Hi guys,

My attempt at fixing the luma/chroma for iframe 14 (iframe 261). I spent way to much time on this ;).

I find this frame very hard to fix: this mmb is huge. Maybe somebody else has a more elegant fix.

It looks a lot better though.  8)

Regards,

arnezami

WOW! I thought that this frame is a lost case.

You really are advancing state of the art of MPEG video recovery.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Lourens on 06/03/2014 08:23 pm
Hi guys,

My attempt at fixing the luma/chroma for iframe 14 (iframe 261). I spent way to much time on this ;).

I find this frame very hard to fix: this mmb is huge. Maybe somebody else has a more elegant fix.

It looks a lot better though.  8)

Regards,

arnezami
Wow! That's really impressive!

Having a huge mmb does raise the question of whether we can define some kind of measure of how "real" and how "retouched" a certain frame is. Percentage of untouched macroblocks? Percentage of flipped bits? Any ideas?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mvpel on 06/03/2014 08:29 pm
Having a huge mmb does raise the question of whether we can define some kind of measure of how "real" and how "retouched" a certain frame is. Percentage of untouched macroblocks? Percentage of flipped bits? Any ideas?

I'd steer clear of describing any of this as "retouched." The effort here is to restore a highly damaged product as close to its original "real" state as possible. We're not adding anything new or fictional to the video as "retouch" might imply, we're restoring it. Perhaps a better measure would be how corrupted the original frame was as compared to its restored version.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: JohnKiel on 06/03/2014 08:49 pm
Unfortunately, it looks like the MMBs can create an image request URL that's larger than the ~2KB Google Sheet's IMAGE function can manage.

A bit ironic considering the editor had to be modified to deal with the long MMBs.  It still works for all the individual p frames though, which is great!

I'm tinkering with a work-around.  Basically a proxy that allows a much shorter URL to request the image.  Currently sorting out how to cache and respond with 304's to avoid quickly using up the paltry 10GB/month I get on my chosen host. (If anyone with more bandwidth is willing to host the simple php image proxy script once I'm done, let me know.)

I'll host any files this project needs, or even any daemon as long as I don't have to jump through flaming hoops to get it running on a CentOS 5 system(or systems).  Primary circuit is a 90th percentile 10g line that usually sees less than 75Mbps of it's 1Gbps monthly commit, so LOTS of free bandwidth(something like 250TB/mo) is sitting around right now.

Hit me with a PM anytime.

Thanks Req,  I've sent you a PM.

All the spreadsheets should now be pointing to my image proxy script.  Seems to be running alright, but I notice some strange caching issues that  could be related to he script timing out (current host doesn't let me adjust the timeout).

Changes to MMBs may not show up in the images immediately.  Give it a little time, refresh your browser, etc.

After I finish cleaning the script up, I'll upload it somewhere.  Maybe the git for SpaceXVideoApp2 is the best place for it?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Swatch on 06/03/2014 08:54 pm
Keep in mind, there are THOUSANDS of bits in each frame, so a 'long' MMB (in this case 129 commands) is still a very SMALL portion of the data contained there-in.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Swatch on 06/03/2014 09:03 pm
Oh I do like the idea behind this:

https://www.youtube.com/channel/UCyZDgyJBYz3OXD3JbDJzNww (https://www.youtube.com/channel/UCyZDgyJBYz3OXD3JbDJzNww)

That is a great one stop page to see the progress over time.

One thing I've noticed is that the channel is putting out 'updated' videos at a really high rate... it tends to over-saturate the 'cool' effect of watching it develop.

So while a few builds a day is cool up front, perhaps it is worthwhile to delete or archive all but one video per day every couple days.  That way someone who goes there can truly appreciate the evolution of the video from initial to final form without being inundated by small deltas.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 06/03/2014 09:06 pm
Oh I do like the idea behind this:

https://www.youtube.com/channel/UCyZDgyJBYz3OXD3JbDJzNww (https://www.youtube.com/channel/UCyZDgyJBYz3OXD3JbDJzNww)

That is a great one stop page to see the progress over time.

One thing I've noticed is that the channel is putting out 'updated' videos at a really high rate... it tends to over-saturate the 'cool' effect of watching it develop.

So while a few builds a day is cool up front, perhaps it is worthwhile to delete or archive all but one video per day every couple days.  That way someone who goes there can truly appreciate the evolution of the video from initial to final form without being inundated by small deltas.

I think it's triggered every hour and ignored if there are no changes, perhaps we can make this 3 hours or something.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Quialiss on 06/03/2014 09:33 pm
Hi guys,

My attempt at fixing the luma/chroma for iframe 14 (iframe 261). I spent way to much time on this ;).

I find this frame very hard to fix: this mmb is huge. Maybe somebody else has a more elegant fix.

It looks a lot better though.  8)

Regards,

arnezami

That looks about an average length MMB to me.     ;D  Beautiful work. 
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jakusb on 06/03/2014 09:40 pm

Oh I do like the idea behind this:

https://www.youtube.com/channel/UCyZDgyJBYz3OXD3JbDJzNww (https://www.youtube.com/channel/UCyZDgyJBYz3OXD3JbDJzNww)

That is a great one stop page to see the progress over time.

One thing I've noticed is that the channel is putting out 'updated' videos at a really high rate... it tends to over-saturate the 'cool' effect of watching it develop.

So while a few builds a day is cool up front, perhaps it is worthwhile to delete or archive all but one video per day every couple days.  That way someone who goes there can truly appreciate the evolution of the video from initial to final form without being inundated by small deltas.

I think it's triggered every hour and ignored if there are no changes, perhaps we can make this 3 hours or something.

Maybe consider to slow it down to. Something like 3 times or 5 times slower? Or is this or easy to configure?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 06/03/2014 09:41 pm
I dunno wronkiew has to do it =)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Quialiss on 06/03/2014 10:00 pm
I dunno wronkiew has to do it =)

Speaking of, it may now be much easier to pull from the spreadsheets rather than the wiki, seeing as there's a separate column for comments and such.  There's about 30 more p frames and some rather nicer looking i frames that can be pulled into that video.  :) 
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: corrodedNut on 06/03/2014 10:18 pm
Oh I do like the idea behind this:

https://www.youtube.com/channel/UCyZDgyJBYz3OXD3JbDJzNww (https://www.youtube.com/channel/UCyZDgyJBYz3OXD3JbDJzNww)

That is a great one stop page to see the progress over time.

I suggest that you include this link in the OP, thanks!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Quialiss on 06/03/2014 11:04 pm
Revelation for the p frames in part 10 -- the ones with the big gray areas that shouldn't be.

-3 completely ignores a block.  It also clears chroma and luma, like -2, so subsequent blocks don't inherit it.  This isn't terribly noticeable in most p frames because there aren't a lot of non-movement blocks.  Except in part 10.  Part 10, the camera is being overexposed by that big blast of steam and the rest of the frame gets dark, changing the whole frame on a regular basis. 

We stopped using -1 in p frames because it made gray blocks.  It only does that if the blocks that are being blanked with -1 have nothing to inherit from, aka movement blocks. 

Test frame 185, take a look at it in the editor, pick it apart. 

X:19164:80,5:7:-1,23:7:-3,24:7:-1,27:7:-3,29:7:-1,3:8:21583:5:0:0:0:0:0,5:8:21693:0:-5:0:0:0:0, 6:8:21763:-10:0:0:0:0:0,8:8:21872:0:5:0:0:0:0,17:11:-3,35:11:37109,28:12:39566,35:12:40350, 8:13:-1:0:0:-10:0:0:0,9:13:-1,16:13:-3,28:13:43161,32:13:43824,15:14:45251:20:0:0:0:0:0, 6:16:52992,0:19:-3,39:19:70393,38:20:-3,42:20:75770,38:22:-3,17:23:84384,10:28:-3

compared to

X:19164:80,5:7:-3,3:8:21583,17:11:-3,35:11:37109,28:12:39566,35:12:40350,8:13:-3, 28:13:43161,32:13:43824,0:19:-3,39:19:70393,38:20:-3,42:20:75770,38:22:-3,17:23:84384,10:28:-3



I need to sleep now.   :)

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mhenderson on 06/04/2014 12:14 am
Revelation for the p frames in part 10 -- the ones with the big gray areas that shouldn't be.

-3 completely ignores a block.  It also clears chroma and luma, like -2, so subsequent blocks don't inherit it.  This isn't terribly noticeable in most p frames because there aren't a lot of non-movement blocks.  Except in part 10.  Part 10, the camera is being overexposed by that big blast of steam and the rest of the frame gets dark, changing the whole frame on a regular basis. 

We stopped using -1 in p frames because it made gray blocks.  It only does that if the blocks that are being blanked with -1 have nothing to inherit from, aka movement blocks. 

Test frame 185, take a look at it in the editor, pick it apart. 

X:19164:80,5:7:-1,23:7:-3,24:7:-1,27:7:-3,29:7:-1,3:8:21583:5:0:0:0:0:0,5:8:21693:0:-5:0:0:0:0, 6:8:21763:-10:0:0:0:0:0,8:8:21872:0:5:0:0:0:0,17:11:-3,35:11:37109,28:12:39566,35:12:40350, 8:13:-1:0:0:-10:0:0:0,9:13:-1,16:13:-3,28:13:43161,32:13:43824,15:14:45251:20:0:0:0:0:0, 6:16:52992,0:19:-3,39:19:70393,38:20:-3,42:20:75770,38:22:-3,17:23:84384,10:28:-3

compared to

X:19164:80,5:7:-3,3:8:21583,17:11:-3,35:11:37109,28:12:39566,35:12:40350,8:13:-3, 28:13:43161,32:13:43824,0:19:-3,39:19:70393,38:20:-3,42:20:75770,38:22:-3,17:23:84384,10:28:-3



I need to sleep now.   :)

In P frames, the interpreter seems to use the contents of 43:29 as the filler for a good fraction of the scattered blocks. One strategy is to give 43:29 a neutral color, such as the color of water (43:29:0:0:0:0:12:-6.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: SwissCheese on 06/04/2014 12:51 am
Oh I do like the idea behind this:

https://www.youtube.com/channel/UCyZDgyJBYz3OXD3JbDJzNww (https://www.youtube.com/channel/UCyZDgyJBYz3OXD3JbDJzNww)

That is a great one stop page to see the progress over time.

I agree it is awesome. Of course I will beg for more :-) ... It would be sweet to have a slow motion (3 fps?) copy running immediately after the 15 fps video.

It doesn't have all the latest corrections (parts 3, 4, 5, 13, 14 still come from previous version) but here it is with nominal framerate and with 3 frames per second. It's getting really nice!

Edit: it was the wrong quote... changed to the good one :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: JohnKiel on 06/04/2014 01:22 am
I'll host any files this project needs, or even any daemon as long as I don't have to jump through flaming hoops to get it running on a CentOS 5 system(or systems).  Primary circuit is a 90th percentile 10g line that usually sees less than 75Mbps of it's 1Gbps monthly commit, so LOTS of free bandwidth(something like 250TB/mo) is sitting around right now.

Hit me with a PM anytime.

Thanks Req,  I've sent you a PM.

All the spreadsheets should now be pointing to my image proxy script.  Seems to be running alright, but I notice some strange caching issues that  could be related to he script timing out (current host doesn't let me adjust the timeout).

Changes to MMBs may not show up in the images immediately.  Give it a little time, refresh your browser, etc.

After I finish cleaning the script up, I'll upload it somewhere.  Maybe the git for SpaceXVideoApp2 is the best place for it?

Just wanted to publicly acknowledge Req for hosting, and assistance in migrating, the image proxy script used for automatically displaying up-to-date images on the new "Frames" Spreadsheets (http://spacexlanding.wikispaces.com/raw_final_fixedMMB).

Thanks again Req!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: meekGee on 06/04/2014 04:01 am
Keep in mind, there are THOUSANDS of bits in each frame, so a 'long' MMB (in this case 129 commands) is still a very SMALL portion of the data contained there-in.

That's EXACTLY the distinction between a repair and a retouch.

When you repair, a small number of changes result in a large improvements, since the original data that is assumed to still exist in the frame becomes ungarbled.

Once it takes one bit command to "fix" one pixel, then you're 100% retouching it.

Seems to me you're safely in "repair" territory.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 06/04/2014 04:28 am
Hi guys,

I think it would be really cool to include a repair-video of at least one iframe (but probably several) in the final video compilation. This is similar to the one from Quialiss.

(http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=586735;image)

But instead of just showing the luma/chroma fixes, start with the original (corrupted) iframe, then add one corrected position at a time and then show the luma/chroma fixes one at-a-time. Then I think any John Doe will understand it. (i can probably make a script that creates such a repair-video for any iframe). I think this would also work very well for an article (edited by Chris) explaining the way we repaired it and how things went chronologically.

Since the repair of the iframe 14 was so well received I will try frame 15. Its the last part of the video people will see, so I will try to make it look good. Don't get your hopes up though, this one is a toughy. ;)

Btw the reason why the repair mmb's for frame 14 (and probably also 15) are quite large is they have one or several complete "horizontal bars" of -1s" in them. This effectively removes all inherited data from the top, which you have to restore for (almost) every macroblock just below it. It's very tedious work, but in the end it's rewarding :).

Regards,

arnezami

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 06/04/2014 05:20 am
Oh I do like the idea behind this:

https://www.youtube.com/channel/UCyZDgyJBYz3OXD3JbDJzNww (https://www.youtube.com/channel/UCyZDgyJBYz3OXD3JbDJzNww)

That is a great one stop page to see the progress over time.

I agree it is awesome. Of course I will beg for more :-) ... It would be sweet to have a slow motion (3 fps?) copy running immediately after the 15 fps video.

It doesn't have all the latest corrections (parts 3, 4, 5, 13, 14 still come from previous version) but here it is with nominal framerate and with 3 frames per second. It's getting really nice!

Edit: it was the wrong quote... changed to the good one :)
Hi,

Nice videos SwissCheese!  8)

I would love to see the latest corrections in it :).

As for the automated videos: it looks nice but usually a video compiled/released by hand (usually by SwissCheese) looks quite a lot better. When looking at it a little bit more closely I think there are still a few issues:

1) It doesn't remove frames (as in 0:0:-3) that have not been touched yet. For example frames 42-60. They are still unrepaired and should not be put into the video yet.
2) It doesn't seem to make a distinction between "no changes needed" and "lets forget this one". I think.
3) In the video from 19:35:08 through 19:35:10 there seems to be no movement even though frames 121-140 have been repaired.
4) We have no feedback when we haven't filled in an mmb correctly on the wiki (for example added a comment that effectively disables the frame). This would be very useful. Since you can then debug the problem.
5) Is the automated video still based on the wiki or on the spreadsheet now?

This list might not be complete but maybe somebody can take a look at it.

Regards,

arnezami

PS. I think it would be a good idea to add a "comments"-column to the wiki. The mmb-column would strictly be used for making the (automated) video (and might contain 0:0:-3) while the comments-column might contain comments and/or mmbs which aren't good enough yet to put into the video.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 06/04/2014 07:15 am
Hi,

Nice videos SwissCheese!  8)

I would love to see the latest corrections in it :).

As for the automated videos: it looks nice but usually a video compiled/released by hand (usually by SwissCheese) looks quite a lot better. When looking at it a little bit more closely I think there are still a few issues:

1) It doesn't remove frames (as in 0:0:-3) that have not been touched yet. For example frames 42-60. They are still unrepaired and should not be put into the video yet.
2) It doesn't seem to make a distinction between "no changes needed" and "lets forget this one". I think.
3) In the video from 19:35:08 through 19:35:10 there seems to be no movement even though frames 121-140 have been repaired.
4) We have no feedback when we haven't filled in an mmb correctly on the wiki (for example added a comment that effectively disables the frame). This would be very useful. Since you can then debug the problem.
5) Is the automated video still based on the wiki or on the spreadsheet now?

This list might not be complete but maybe somebody can take a look at it.

Regards,

arnezami

PS. I think it would be a good idea to add a "comments"-column to the wiki. The mmb-column would strictly be used for making the (automated) video (and might contain 0:0:-3) while the comments-column might contain comments and/or mmbs which aren't good enough yet to put into the video.

1. A first pass was made on these with the fixed_edit8.ts files. I've been moving all the segments over to the final .ts files as good i-frames are posted. Which, it looks like we have for parts 3-15 now.

2. Yes, this was a bug. Thanks for catching that.

3. 121 did not have an i-frame until recently. I'll move it over to the final.ts file.

4. I will start auto-posting the mmbs to the wiki so they can be compared against the work pages. (Some of this might not get done until tomorrow)

5. Based on the wiki. I took a look at basing it on the spreadsheets, but I noticed problems with some of the frames, for example 266 and 267 are different in the spreadsheet and the wiki, and the fixes on the wiki are better. Which are we considering canonical?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: arnezami on 06/04/2014 07:21 am
Wiki is leading.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 06/04/2014 08:05 am
The autobuilt video is updated with fixes.

Should we look into easing the transitions between segments? For example you can see in pframes 122-140 the block update images are lighter than the iframe and a slightly different color. Then iframe 141 comes along and it matches the preceding pframes but not iframe 121. Is there a global brightness/chroma adjustment we could apply to the entire iframe?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 06/04/2014 08:08 am
Oh I do like the idea behind this:

https://www.youtube.com/channel/UCyZDgyJBYz3OXD3JbDJzNww (https://www.youtube.com/channel/UCyZDgyJBYz3OXD3JbDJzNww)

That is a great one stop page to see the progress over time.

One thing I've noticed is that the channel is putting out 'updated' videos at a really high rate... it tends to over-saturate the 'cool' effect of watching it develop.

So while a few builds a day is cool up front, perhaps it is worthwhile to delete or archive all but one video per day every couple days.  That way someone who goes there can truly appreciate the evolution of the video from initial to final form without being inundated by small deltas.

I'd be happy to add an alternate build, perhaps daily? IainCole, are you able to handle a second video update post?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: lgjy98d on 06/04/2014 08:33 am
Unfortunately, it looks like the MMBs can create an image request URL that's larger than the ~2KB Google Sheet's IMAGE function can manage.

A bit ironic considering the editor had to be modified to deal with the long MMBs.  It still works for all the individual p frames though, which is great!

I'm tinkering with a work-around.  Basically a proxy that allows a much shorter URL to request the image.  Currently sorting out how to cache and respond with 304's to avoid quickly using up the paltry 10GB/month I get on my chosen host. (If anyone with more bandwidth is willing to host the simple php image proxy script once I'm done, let me know.)

I'll host any files this project needs, or even any daemon as long as I don't have to jump through flaming hoops to get it running on a CentOS 5 system(or systems).  Primary circuit is a 90th percentile 10g line that usually sees less than 75Mbps of it's 1Gbps monthly commit, so LOTS of free bandwidth(something like 250TB/mo) is sitting around right now.

Hit me with a PM anytime.

Thanks Req,  I've sent you a PM.

All the spreadsheets should now be pointing to my image proxy script.  Seems to be running alright, but I notice some strange caching issues that  could be related to he script timing out (current host doesn't let me adjust the timeout).

Changes to MMBs may not show up in the images immediately.  Give it a little time, refresh your browser, etc.

After I finish cleaning the script up, I'll upload it somewhere.  Maybe the git for SpaceXVideoApp2 is the best place for it?

Google Spreadsheets are doing all the best to limit the Slashdot DOS effect by extensively caching data. For instance the spreadsheet is published at regular intervals but they can be even up to 5 minutes. I.e. the JSON feed from spreadsheet is often late, thus you cannot immediately get changes you do in spreadsheet via json feed.

Another "fix" Google is employing is caching any external data, i.e. when you pull the image with IMAGE function the request most probably passes the proxy, and unless there is a change in URL parameter to IMAGE function every time, you'll be geting cached image that Google fetched some time ago. This lets google limit its DOS effect on resources lined in the spreadsheet (i.e. it doesn't matter how many visitor spreadsheet gets).

Hope this explains delays in image update frequency if images are being proxied via external system. The cumulative delay here can be up to 10 minutes, that is not too bad, considering the fact that image and all subsequent frames are updated automatically without any extra manual work (frame generation, upload to wiki, updating wiki referencing it, etc.)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Jester on 06/04/2014 01:09 pm
Unfortunately, it looks like the MMBs can create an image request URL that's larger than the ~2KB Google Sheet's IMAGE function can manage.

A bit ironic considering the editor had to be modified to deal with the long MMBs.  It still works for all the individual p frames though, which is great!

I'm tinkering with a work-around.  Basically a proxy that allows a much shorter URL to request the image.  Currently sorting out how to cache and respond with 304's to avoid quickly using up the paltry 10GB/month I get on my chosen host. (If anyone with more bandwidth is willing to host the simple php image proxy script once I'm done, let me know.)

I'll host any files this project needs, or even any daemon as long as I don't have to jump through flaming hoops to get it running on a CentOS 5 system(or systems).  Primary circuit is a 90th percentile 10g line that usually sees less than 75Mbps of it's 1Gbps monthly commit, so LOTS of free bandwidth(something like 250TB/mo) is sitting around right now.

Hit me with a PM anytime.

Thanks Req,  I've sent you a PM.

All the spreadsheets should now be pointing to my image proxy script.  Seems to be running alright, but I notice some strange caching issues that  could be related to he script timing out (current host doesn't let me adjust the timeout).

Changes to MMBs may not show up in the images immediately.  Give it a little time, refresh your browser, etc.

After I finish cleaning the script up, I'll upload it somewhere.  Maybe the git for SpaceXVideoApp2 is the best place for it?

Google Spreadsheets are doing all the best to limit the Slashdot DOS effect by extensively caching data. For instance the spreadsheet is published at regular intervals but they can be even up to 5 minutes. I.e. the JSON feed from spreadsheet is often late, thus you cannot immediately get changes you do in spreadsheet via json feed.

Another "fix" Google is employing is caching any external data, i.e. when you pull the image with IMAGE function the request most probably passes the proxy, and unless there is a change in URL parameter to IMAGE function every time, you'll be geting cached image that Google fetched some time ago. This lets google limit its DOS effect on resources lined in the spreadsheet (i.e. it doesn't matter how many visitor spreadsheet gets).

Hope this explains delays in image update frequency if images are being proxied via external system. The cumulative delay here can be up to 10 minutes, that is not too bad, considering the fact that image and all subsequent frames are updated automatically without any extra manual work (frame generation, upload to wiki, updating wiki referencing it, etc.)

Solution to try and by-pass this (if this is really needed) is to automatically add ?unique-id-here to the URL param, I use this myself for some L2 websites I run to make sure the user doesn't get an outdated cached page when I update something.
Just my 2c.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: JohnKiel on 06/04/2014 02:05 pm
Google Spreadsheets are doing all the best to limit the Slashdot DOS effect by extensively caching data. For instance the spreadsheet is published at regular intervals but they can be even up to 5 minutes. I.e. the JSON feed from spreadsheet is often late, thus you cannot immediately get changes you do in spreadsheet via json feed.

Another "fix" Google is employing is caching any external data, i.e. when you pull the image with IMAGE function the request most probably passes the proxy, and unless there is a change in URL parameter to IMAGE function every time, you'll be geting cached image that Google fetched some time ago. This lets google limit its DOS effect on resources lined in the spreadsheet (i.e. it doesn't matter how many visitor spreadsheet gets).

Hope this explains delays in image update frequency if images are being proxied via external system. The cumulative delay here can be up to 10 minutes, that is not too bad, considering the fact that image and all subsequent frames are updated automatically without any extra manual work (frame generation, upload to wiki, updating wiki referencing it, etc.)

Yes, lots of caching and proxies involved with the image requests -- plenty of places for delays.  I believe the request currently goes through something like: Google Sheets -> Client browser -> Google's proxy -> My "proxy" script (spxi) -> IainCole's image generator (image server).

The "strange caching" issue I was running into appeared to be related to spxi timing out in some cases, and Google's proxy caching that error result, displaying no image in the sheet.  Now that it's on a faster and less restrictive server, that seems to be less of an issue.  Also, now that spxi has more bandwidth headroom, I've adjusted the Cache-control header so hopefully Google's proxy won't cache stale images for too long.

Once Google Sheets saves an MMB the json response seems to follow suit relatively quickly, however spxi will cache requests from the sheet for 10 seconds to avoid having to refresh it for every image request.  In addition, spxi will cache results from the image server, only requesting new image data if it detects that the MMBs have changed.

Unfortunately, changes to MMBs will require refreshing the sheet to see the change in the images.  Even if we append a hash of the MMBs to the sheet's image URL to trigger a new request on image change, the change in MMB likely won't be available to spxi until after it's saved and spxi's sheet cache expires.

In all, I think it's working well enough.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: JohnKiel on 06/04/2014 02:12 pm
Solution to try and by-pass this (if this is really needed) is to automatically add ?unique-id-here to the URL param, I use this myself for some L2 websites I run to make sure the user doesn't get an outdated cached page when I update something.
Just my 2c.

Yes, this would force an image refresh for every sheet load, and could be accomplished by appending NOW() to the URL, but I'd like to avoid it if possible.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: CuddlyRocket on 06/04/2014 02:48 pm
Gwynne Shotwell is giving a speech at the moment to the Atlantic Council in Washington DC. She had a video about the accomplishments of SpaceX and it used a clip of the repaired video!
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: wronkiew on 06/04/2014 02:51 pm
Solution to try and by-pass this (if this is really needed) is to automatically add ?unique-id-here to the URL param, I use this myself for some L2 websites I run to make sure the user doesn't get an outdated cached page when I update something.
Just my 2c.

Yes, this would force an image refresh for every sheet load, and could be accomplished by appending NOW() to the URL, but I'd like to avoid it if possible.

You could add a hash of the mmb to the URL.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: JohnKiel on 06/04/2014 03:13 pm
You could add a hash of the mmb to the URL.

The problem is that the MMBs used to create the hash in the URL may not match the MMBs the proxy script (spxi) sees when it pulls json for the sheet.  If a user changes the MMB, the sheet will request a new image immediately, but the sheet doesn't immediately save the changed MMB to Google's servers, so spxi will return images for the wrong MMB.

I suppose spxi could calculate a hash the same way as the spreadsheet (no simple native functions in google sheets for hashing, so we'd have to get creative), and if they don't match, loop for a bit, pulling new json for the sheet, trying its best to get updated data in the allotted amount of time;  But there's no guarantee it would ever return the correct image;  It could be some time before spxi actually sees the changed MMB.

That said, it's probably better than nothing.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: lgjy98d on 06/04/2014 04:11 pm
You could add a hash of the mmb to the URL.

The problem is that the MMBs used to create the hash in the URL may not match the MMBs the proxy script (spxi) sees when it pulls json for the sheet.  If a user changes the MMB, the sheet will request a new image immediately, but the sheet doesn't immediately save the changed MMB to Google's servers, so spxi will return images for the wrong MMB.

I suppose spxi could calculate a hash the same way as the spreadsheet (no simple native functions in google sheets for hashing, so we'd have to get creative), and if they don't match, loop for a bit, pulling new json for the sheet, trying its best to get updated data in the allotted amount of time;  But there's no guarantee it would ever return the correct image;  It could be some time before spxi actually sees the changed MMB.

That said, it's probably better than nothing.
I'd say that Google Spreadsheet is no way replacement for online editor, but is good enough representation of state of the repair efforts that reduces amount of manual wiki page editing.

I.e. with Spreadsheets the workflow is:
* use data from spreadsheet cell (either manually copying data out and use in standalone ffmpeg, or automatically with online editor (http://spacex2.slapbet.org/) "Load Frame MMBs From Sheet" function)
* improve it until satisfied with the result
* paste updated MMBs back into spreadsheet

Frames images in the Spreadsheet should update automatically after 5-10 minutes, freeing one time to share accomplishments with this forum.

BTW, I've inserted widget embedding "latest" video of spacexlandingrestore channel in http://spacexlanding.wikispaces.com/raw_final_fixedMMB page. Should it be placed on frontpage?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mhenderson on 06/04/2014 04:12 pm
The autobuilt video is updated with fixes.

Oh that is looking sooooooooo sweet. Congrats and thanks.

What are your thoughts on adding a second pass at a slower frame rate (3 fps perhaps?).

I played with the last iFrame -- 15 (281) -- last night and the contrast is easily improved from the grey cast it has now. That would enhance the video by not ending on a washed out image. Even shifting the first pos to 0:0:559 looks better. There may be a more elegant way to handle it. Add a couple chroma / luma touchups in rows 11 and 14 and you have a nice image.

I did not post an update on the wiki because I was trying to get more macroblocks out of the voids. Hopefully tonight unless someone else gets to it first.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Helodriver on 06/04/2014 04:17 pm
Gwynne Shotwell just used a clip of the repaired video in a presentation she gave today.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Quialiss on 06/04/2014 04:56 pm
I've touched up frames 185-190 (part 10) to remove the large gray areas.  186 still looks rather sketchy but it's an improvement. 

The gray areas seem to be screwed up inheritance due to missing blocks, which either require manual matching to the DC values of a good block, or using -1 instead of -3 in select places. 

The darker areas (lots more of these in frames 191+) are corrected with +32 luma, which screams bitflip.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: JohnKiel on 06/04/2014 05:03 pm
Gwynne Shotwell just used a clip of the repaired video in a presentation she gave today.

I believe you're referring to this?
https://www.youtube.com/watch?feature=player_detailpage&v=sYocHwhfFDc

Around 9:40 (https://www.youtube.com/watch?feature=player_detailpage&v=sYocHwhfFDc#t=9m40s)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: moralec on 06/04/2014 06:17 pm
Isnt´ just amazing to see our video there? congratulations everybody!!!!

Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Quialiss on 06/04/2014 06:30 pm
The few seconds before 'our' video are from earlier in the descent, I'd recognize that dust spot anywhere.  No timestamps to tell when that was in relation to the start of the recovered video, though...   
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Alpha Control on 06/04/2014 06:37 pm
Hi guys,

I think it would be really cool to include a repair-video of at least one iframe (but probably several) in the final video compilation. This is similar to the one from Quialiss.

(http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=586735;image)

But instead of just showing the luma/chroma fixes, start with the original (corrupted) iframe, then add one corrected position at a time and then show the luma/chroma fixes one at-a-time. Then I think any John Doe will understand it. (i can probably make a script that creates such a repair-video for any iframe). I think this would also work very well for an article (edited by Chris) explaining the way we repaired it and how things went chronologically.

Since the repair of the iframe 14 was so well received I will try frame 15. Its the last part of the video people will see, so I will try to make it look good. Don't get your hopes up though, this one is a toughy. ;)

Btw the reason why the repair mmb's for frame 14 (and probably also 15) are quite large is they have one or several complete "horizontal bars" of -1s" in them. This effectively removes all inherited data from the top, which you have to restore for (almost) every macroblock just below it. It's very tedious work, but in the end it's rewarding :).

Regards,

arnezami

arnezami,

I think that's a great idea. The video-editing John Does among us (myself included) would find that very illuminating. One suggestion I have would be to label the frames in the upper right corner, to provide sequencing. For example, "Orig" on the raw original frame, then "Repair 1", "Repair 2", etc. Something like that. What do you think?
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: IainCole on 06/04/2014 06:50 pm
Oh I do like the idea behind this:

https://www.youtube.com/channel/UCyZDgyJBYz3OXD3JbDJzNww (https://www.youtube.com/channel/UCyZDgyJBYz3OXD3JbDJzNww)

That is a great one stop page to see the progress over time.

One thing I've noticed is that the channel is putting out 'updated' videos at a really high rate... it tends to over-saturate the 'cool' effect of watching it develop.

So while a few builds a day is cool up front, perhaps it is worthwhile to delete or archive all but one video per day every couple days.  That way someone who goes there can truly appreciate the evolution of the video from initial to final form without being inundated by small deltas.

I'd be happy to add an alternate build, perhaps daily? IainCole, are you able to handle a second video update post?

I could, but having it as an alternative rather than a replacement kinda defeats the object of what Swatch was saying. Unless I upload it to a different channel (which I don't think is a good idea). Unfortunately the tool I'm using to do it doesn't allow you to put it into a specific playlist.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: JohnKiel on 06/04/2014 07:17 pm
Partially because I could, and partially because I thought it would be interesting, I created a read only view of all the frame sheets.  (Warning: it can take some time to load.)

https://docs.google.com/spreadsheets/d/1xEtLQy3tvjv3cyz_wMQssuSFwHYVbMQ5-FdSz-uCsKA/edit?pli=1#gid=0

It's kinda neat to look at the differences between the last p-frame of a part and the following i-frame.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Req on 06/04/2014 07:27 pm
Partially because I could, and partially because I thought it would be interesting, I created a read only view of all the frame sheets.  (Warning: it can take some time to load.)

https://docs.google.com/spreadsheets/d/1xEtLQy3tvjv3cyz_wMQssuSFwHYVbMQ5-FdSz-uCsKA/edit?pli=1#gid=0

It's kinda neat to look at the differences between the last p-frame of a part and the following i-frame.

This view is really cool.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: Quialiss on 06/04/2014 07:35 pm
Partially because I could, and partially because I thought it would be interesting, I created a read only view of all the frame sheets.  (Warning: it can take some time to load.)

https://docs.google.com/spreadsheets/d/1xEtLQy3tvjv3cyz_wMQssuSFwHYVbMQ5-FdSz-uCsKA/edit?pli=1#gid=0

It's kinda neat to look at the differences between the last p-frame of a part and the following i-frame.

This is awesome!  It should also be helpful in doing a final tweaking of some of the i frames to better match up.  To pick a minor issue, 121 looks just a hair too dark.. not surprising considering the data gaps at the top of the frame. 

I also note that there's a surprising amount of valid p frame data in part 2 considering the i frame is complete garbage.  Engine glow in the clouds looks lovely even with only a couple rows of data.  :)
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: mhenderson on 06/04/2014 08:59 pm
I also note that there's a surprising amount of valid p frame data in part 2 considering the i frame is complete garbage.  Engine glow in the clouds looks lovely even with only a couple rows of data.  :)

Which begs the question: Is it possible to use the complete and final pFrame from the prior set as a replacement for the iFrame in the next?  Just throwin' it out there.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: saliva_sweet on 06/04/2014 09:01 pm
It should also be helpful in doing a final tweaking of some of the i frames to better match up.  To pick a minor issue, 121 looks just a hair too dark.. not surprising considering the data gaps at the top of the frame. 

I post an mmb here that, I think, fixes the top row (including the crucial 0:0 MB). But alas, as usual, it throws off existing dc corrections.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: JohnKiel on 06/04/2014 10:03 pm
Which begs the question: Is it possible to use the complete and final pFrame from the prior set as a replacement for the iFrame in the next?  Just throwin' it out there.

Once every bit of original data is teased out, I think the next logical step is using pixels from previous (and future) frames as an approximation for missing blocks in the current frame.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: princess on 06/04/2014 10:56 pm
Is it possible to use the complete and final pFrame from the prior set as a replacement for the iFrame in the next?

I think a cooler thing to do would be to use both the blocks and the motion vectors from the previous P-frame. Just repeat the motion vector again so that you get the same approximate motion, otherwise the duplicated P-frame image would look like an obvious pause or stutter in the stream.
Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
Post by: ajmartin on 06/04/2014 11:43 pm
I've done some more work on error modeling, and I thought I should post an update.  I determined the original values for as much of the "boilerplate" data as I could (null packets, TS headers, PAT and PMT packets, VOL headers, etc.) and located all the bit-flips there in raw.ts, then measured statistics and tested hypotheses.

The postulated 56-byte FEC coding blocks turn out to be composed of alternate 25-byte and 31-byte "sub-blocks", starting at (zero-based) offsets of the form 56n + 29 and 56n + 54.  Single bit-flips are found only in the first and last 15 bits of these blocks, with possible rare exceptions in high-error regions.  All other bit-flip errors are combinations of "triple flips" (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1201383#msg1201383) (i.e., bit shifts of 0x8003) that do not cross the block boundaries.

(That does not include frame-shift errors such as the one originally found by Shanuson (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1196281#msg1196281), which are surprisingly common - I found at least 20 of them just in the "boilerplate" data, by amounts ranging from 1 bit to 12 bits and in both directions.  But these seem to occur only after high-error regions, which may have damaged the framing information that was used in decoding the signal.  They affect entire "sub-blocks" and garbage bits are shifted in at one end.)

I suspect that many of the single-bit flips (X mmb's) that have been found to improve the images are "workarounds" that affect the lengths of VLCs in such a way that the rest of the bit-stream is interpreted closer to its original meaning, but not true "corrections" that restore the original data.

Finding the latter will be more difficult, but may produce even better results.  (It won't be possible everywhere, of course, due to the high error rates in some parts of the stream.)  If anyone would like to make the attempt, then I would recommend a few things to improve the chance of success:
  • Use the error model: Look for "triple flips" and "single flips" according to the sub-block structure described above.  Look for adjacent pairs of "triple flips" (i.e., bit shifts of 0x18005) before more complicated combinations.
  • Work from raw.ts: This is perhaps the hardest change to make from the current approach.  But the original 56-byte alignment and the original values of corrupted TS-headers are needed to properly identify "triple flips".  (Of course, the TS-headers still need to be "virtually" fixed, or the TS-layer decoding bypassed entirely.)
  • Try to identify errors "early": Knowing the first point in the stream where the decoding is clearly erroneous will help to narrow the search.  Perhaps it would help to produce a "coefficient-by-coefficient" visualization of the decoding process for damaged macroblocks, rendering the image after each DCT coefficient is decoded (with undecoded coefficients set to zero)?
  • If it's working and you're feeling ambitious, try to automate this coefficient-by-coefficient evaluation, and use a Viterbi-like algorithm to correct more complicated error patterns.

  • Good luck to all with your continued efforts at recovering the video... the progress so far is impressive!

    Oh, and I noticed something else that seems interesting: the values of two of the header fields, PCR and vop_time_increment, consistently indicate that the video recording started 61 seconds before the start of the file.  (Presumably the rocket was too far away for a useful signal to be received during that time.)

    (Edit: corrected reference to first finding of a frame-shift error.)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: deruch on 06/05/2014 12:24 am
    Gwynne Shotwell just used a clip of the repaired video in a presentation she gave today.

    Around 17:27 (https://www.youtube.com/watch?feature=player_detailpage&v=sYocHwhfFDc#t=1045)

    I think it was at 9:40 (https://www.youtube.com/watch?feature=player_detailpage&v=sYocHwhfFDc#t=9m40s).
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: cscott on 06/05/2014 02:41 am
    Work from raw.ts: This is perhaps the hardest change to make from the current approach.  But the original 56-byte alignment and the original values of corrupted TS-headers are needed to properly identify "triple flips".  (Of course, the TS-headers still need to be "virtually" fixed, or the TS-layer decoding bypassed entirely.)

    I suspect the ship has already sailed and no-ones eager to go back to raw.ts.  But I bet it *would* be helpful if someone would compile a list of the adjusted start/end positions of the FEC coding blocks referenced to the current .ts and including the discovered bit shifts, etc, like princess did:
    For Quialiss and the other people tweaking MMB values, here are the changes needed to map the old bit positions to new ones:
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Quialiss on 06/05/2014 11:22 am
    TS fixers:  How horrified would you be to know that macroblock 0:0 starts at 67 on some of the p frames instead of the normal 66?  I've noticed nothing visually unusual about the frames that start at 67, but...
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: princess on 06/05/2014 12:07 pm
    But I bet it *would* be helpful if someone would compile a list of the adjusted start/end positions of the FEC coding blocks referenced to the current .ts and including the discovered bit shifts, etc, like princess did

    I can calculate a mapping between file offsets in raw.ts and the current TS file if necessary, would this be useful?

    Also, if people want to calculate statistics on the bitflip errors then I have TS files that have more fixes than the current TS file - my current versions have correct contents for all PIDs except the data PID, i.e. all padding is correct, the Program Association Tables and Program Map Tables are all correct and the "padding" AFs in the data are correct too. One of my side projects is trying to make a TS file that's "what the rocket transmitted", so I can make that available to people who are doing statistical anaylsis on the error distribution. Let me know what you need and I'll make it available.

    The current TS file starts at the first I-frame, but the rocket transmitted a few P-frames of data before that. So as part of my "what the rocket transmitted" work I'm also fixing those P-frames up to the same standard - we won't be able to use them in videos (as there's no I-frame to relate them to), but the work could help in statistical analysis of errors.

    Finally, as we're now able to map between TS file offsets and MPEG4 bit positions correctly (thanks to those who discovered ffmpeg's behaviour here), I am planning on making a tool that can apply the XOR parts of -mmb commands directly to the TS file. As far as I can see those should apply fairly easily but the other -mmb commands work at a higher level and can't map directly back to bitflips. However, if we can use even part of the -mmb commands then we can work out some of the bitflips within the MPEG4 stream.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: princess on 06/05/2014 12:12 pm
    TS fixers:  How horrified would you be to know that macroblock 0:0 starts at 67 on some of the p frames instead of the normal 66?  I've noticed nothing visually unusual about the frames that start at 67, but...

    I wouldn't be surprised - in one of the marathon fixing sessions I was missing a Program Map Table packet that should have appeared by a certain point. I eventually found it shifted right by one bit, so I just overwrote the entire thing with the correct PMT. I didn't look into the contents of the data packets, but it wouldn't be a big shock to hear there are further bitshifts in there.

    I believe that a bitshift shouldn't affect the MPEG4 stream much as it's bit-aligned for much of the time. From reading the MPEG4 spec it seems as though sometimes during the headers it aligns to a byte, but once we're into macroblocks then the stream isn't byte aligned and a bitshift won't do much damage. If this is wrong please let me know!
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: mhenderson on 06/05/2014 02:11 pm
    We got the tail end, it sure would be nice to see the rest. I am NOT complaining, this puzzle has enough pieces already.  Getting ten seconds of good transmission might **still be** nice, though.

    Collecting several comments over the last 24 hours:

    @Quialiss said "The few seconds before 'our' video are from earlier in the descent, I'd recognize that dust spot anywhere.  No timestamps to tell when that was in relation to the start of the recovered video, though...   "  (Referring to the video SpaceX showed yesterday at the Atlantic Council conference)

    @ajmartin said "Oh, and I noticed something else that seems interesting: the values of two of the header fields, PCR and vop_time_increment, consistently indicate that the video recording started 61 seconds before the start of the file.  (Presumably the rocket was too far away for a useful signal to be received during that time.)"

    @Princess said "The current TS file starts at the first I-frame, but the rocket transmitted a few P-frames of data before that. So as part of my "what the rocket transmitted" work I'm also fixing those P-frames up to the same standard - we won't be able to use them in videos (as there's no I-frame to relate them to), but the work could help in statistical analysis of errors."

    edit: changed "have been" to "still be" ... @ChrisBergin, can you request 10 seconds of .ts from earlier in the flight?
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: wronkiew on 06/05/2014 02:58 pm
    It's an auto-generated video bonanza!

    http://adama.nocdirect.com/~wronkiew/spx_crs3/landing_base_set_3hr.mp4
    http://adama.nocdirect.com/~wronkiew/spx_crs3/landing_base_set.mp4
    http://adama.nocdirect.com/~wronkiew/spx_crs3/landing_base_set_slow.mp4

    And now with clock fixes applied:

    http://adama.nocdirect.com/~wronkiew/spx_crs3/landing_clockfix_3hr.mp4
    http://adama.nocdirect.com/~wronkiew/spx_crs3/landing_clockfix.mp4
    http://adama.nocdirect.com/~wronkiew/spx_crs3/landing_clockfix_slow.mp4

    The _3hr videos are updated every three hours if anything interesting has happened. The _slow videos run at 3 fps instead of a bit less than 15.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: mvpel on 06/05/2014 03:10 pm
    I'm pretty sure SpaceX has much more raw .ts to supply.  The released portion just contains what they're most interested in recovering.

    Hundreds of frames of looking down towards the cloud deck would be pretty boring to watch, I expect. But that might be an interesting data set to try to apply the ideas that AJMartin proposes with respect to algorithmically identifying bit flips and shifts based on the FEC coding patterns, since that section of the transport stream and the frame data could very well be considerably more uniform than this section.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: jpfulton314 on 06/05/2014 04:09 pm
    This story made the pages of Slashdot!  See: http://beta.slashdot.org/story/202953 (http://beta.slashdot.org/story/202953)

    It makes me proud to be a member here.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: SwissCheese on 06/05/2014 04:17 pm
    My computer just had a major breakdown  :'(, I probably won't be able to help much in the next days...
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Quialiss on 06/05/2014 05:06 pm
    My computer just had a major breakdown  :'(, I probably won't be able to help much in the next days...

    Noooooooooo!

    ... Maybe by the time you get back I'll be able to recover p frames well enough you don't have to fix all my 1-2 off errors in alignment.   ;D
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: cscott on 06/05/2014 05:14 pm
    We got the tail end, it sure would be nice to see the rest. I am NOT complaining, this puzzle has enough pieces already.  Getting ten seconds of good transmission might **still be** nice, though.

    I'm pretty sure SpaceX has much more raw .ts to supply.  The released portion just contains what they're most interested in recovering.

    see:
    http://www.reddit.com/r/spacex/comments/24bsn2/first_stage_landing_video/ch6eqte?context=3

    I'm guessing that telemetry and video were interleaved, and they've given us the bitrange where they could confidently separate the telemetry bits from the video bits.  The in-house repair guy might have been trusted with some more "maybe video maybe telemetry" bits (might even be ITAR restrictions in play).   The more corrupted frames at the start and end of transmission are the ones where separating out the telemetry would be most difficult.

    I'd suppose that Gwynne's earlier video is from the main tracking/telemetry sites before the first stage dipped over the horizon.  I agree that a segment of equivalent .ts might be interesting to us (and boring to the public at large!).
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: mhenderson on 06/05/2014 09:04 pm
    @wronkiew - The slowmo and clock fix are great. An interesting byproduct is how clearly the slowmo helps (me at least) see where the low hanging fruit is -- big improvement with narrowly focused attention. Thank you.

    Chris Bergin has requested .ts data from earlier in the flight. Fingers crossed.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: princess on 06/05/2014 10:32 pm
    Chris Bergin has requested .ts data from earlier in the flight. Fingers crossed.

    I'll jump on that and do fixups as soon as we have it!
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: jdnz82 on 06/06/2014 06:36 am
    @wronkiew - The slowmo and clock fix are great. An interesting byproduct is how clearly the slowmo helps (me at least) see where the low hanging fruit is -- big improvement with narrowly focused attention. Thank you.

    Chris Bergin has requested .ts data from earlier in the flight. Fingers crossed.

    Excellent! good to hear ( i asked this question on the other thread :) )
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Chris Bergin on 06/06/2014 09:34 am
    Going to be difficult with the request as they are all back into launch nerves mode with the upcoming Static Fire of the F9 that had a tantrum on the pad.

    Need the SpaceXers reading this thread, and I know there are a LOT of you, to be proactive and help us with that clip and the requests noted. I'm happy to be the go-between.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: mhenderson on 06/06/2014 11:14 am
    the F9 that had a tantrum on the pad.

    Tell her she gets to go swimming and will be home again soon.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Quialiss on 06/06/2014 01:28 pm
    Request for the web editor:

    Allow switching between frames when the 'Isolate frame' box is checked.  Saving the isolated p frames and flipping between them to get alignment correct is the only p frame related thing I still have to do in an outside editor, getting rid of that step would make finalizing alignments so much less painful. 
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: mhenderson on 06/06/2014 02:50 pm
    Since 281 is the final iFrame and is currently washed out, it would be best to enhance the contrast so the video ends on a nicer image.

    I see the online editor now has a "suggest improvement to MMB" button.  I like that a lot, submissions should be controlled and referreed by a benevolent dictator.

    This is a step in the right direction, but I know my limits ... perhaps someone more skilled at this can take it over the finish line.

    Frame 15(281)
    X:3545:01,X:2003:A0,0:0:559,40:1:-1,0:2:-2:25:25:25:25:0:0,0:10:-2:60:60:60:60:0:0:40,
    1:10:53276,32:10:-1::40,33:10:57352,0:13:69419:0:0:0:1:0:0:63,
    28:15:-2:-70:-70:-70:-70:0:0:63,29:15:83102,8:29:-1

    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Chris Bergin on 06/06/2014 04:17 pm
    I didn't even say anything! :)

    Yet.............just now:
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Chris Bergin on 06/06/2014 04:20 pm
    Oh you know what that means....SpaceXers are following the spacexlandingrestoration channel too. Great work guys!

    (Very nice of SpaceX to add the @nasaspaceflight and thread link! :))
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: spaceboy89 on 06/06/2014 04:24 pm
    SpaceX posted the same on Facebook as well :)

    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: wronkiew on 06/06/2014 04:26 pm
    Apparently the Wikispaces API doesn't allow file uploads.  >:(

    The auto-generated MMBs are uploaded here, in case you need to track down a bug:

    http://adama.nocdirect.com/~wronkiew/spx_crs3/mmbs/

    IainCole, can you change the youtube uploader to grab http://adama.nocdirect.com/~wronkiew/spx_crs3/landing_clockfix_3hr.mp4 ? I can then move the trigger to the 3hr script.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: mhenderson on 06/06/2014 04:29 pm
    Anybody in the Florida area want to get together for the OrbComm launch? I am working a temporary / long term assignment in Miami and considering driving to KSC for the OrbComm OG2 launch next Thursday (backup launch date is Friday).

    PM to me if interested.

    With the shuttle, there were KSC passes that got you past the gate. Is such a thing done now / possible on short notice?
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Quialiss on 06/06/2014 04:42 pm
    I feel like I'm going a bit mad.  Could someone who understands the data structures help out a bit here? The relevant MMBs are at the bottom in case the ones in the wiki change, if you need to see what's going on.   

    Part 13, frame 242.  Used to be: 
    12:12:-3,19:20:43995,43:29:-3

    SwissCheese pulled out a bit more data from it, but for simplicity's sake lets say the new 242 is:
    12:12:-3,19:20:43995

    Just removing the null block at the end of the frame.

    This causes absolutely no changes to frame 243 or 244.

    Then we get to 245.  With the old frame 242, it did not play* without the mmb 43:29:-3 applied to it.
    With the new 242, it doesn't play unless you get rid of that 43:29:-3.

    So we have old version with two frames ending with 43:29:-3, and the new version that is identical except to remove those two commands

    The old version, all the frames play out normally. 
    The new version, 247, 249, 250, 252, 253 and 254 all fail to play. 

    This is problematic, and extremely confusing.  I've noticed an occasional frame that wouldn't display before, but this is the first time I've seen half a sequence go bad because of changes frames earlier in the set.  Something very weird is going on here!  Help! 



    *By 'not play' I mean that when the editor hits it, the whole frame blurs very slightly and none of the changes in the p frame are applied. 


    'old'
    FRAME0:0:0:-1, 29:11:56945, 0:12:58604:0:48:0:0:0:0, 17:12:60055:0:0:-10:5:0:0, 18:12:60234:0:0:5:-5:0:0, 0:13:63030:50:0:0:0:0:0, 18:13:64546:15:0:0:-5:0:0, 19:13:64773:-2:1:0:0:0:0:1, 23:13:65229:0:5:10:0:0:0, 26:13:65548:9:0:0:0:0:0, 29:13:65871:15:0:0:0:0:0, 2:14:67729:0:0:-5:0:0:0, 11:14:68538:0:0:0:0:0:-5, 18:14:69331:0:0:0:20:0:0, 28:15:75458:20:0:0:0:0:0, 28:16:81012:0:0:0:-40:0:0, 31:18:90278:30:0:0:0:0:0, 7:29:-1, 8:29:139153=FRAME1:12:12:-3, 19:20:43995, 43:29:-3=FRAME4:43:29:-3=FRAME5:29:5:-3, 43:9:47279, 24:13:-3, 32:14:77707, 21:22:109507, 1:23:111455=FRAME6:28:10:-3, 42:11:55212, 16:14:66147, 43:29:-3=FRAME7:33:5:25539, 6:7:-3, 4:21:92786, 8:28:116815=FRAME8:9:23:-3=FRAME9:7:5:6416, 41:5:-3, 0:12:13030, 43:29:-3=FRAME10:27:14:-3, 43:15:35848=FRAME11:30:1:3471, 35:11:-3, 38:11:23458, 19:15:-3, 39:19:38646, 10:21:-3, 11:21:40210, 43:29:-3=FRAME12:32:0:3533, 24:4:-3, 25:4:-3, 31:4:21824, 34:8:-3, 2:9:41767, 40:9:-3, 43:11:46658, 32:14:79617, 0:15:-3, 16:15:72454, 33:16:-3, 39:16:79667, 6:17:-3, 24:17:82941, 36:19:92064, 5:23:-3=FRAME13:29:1:-3, 0:12:179561, 39:17:-3=FRAME14:32:0:-3, 23:3:79913, 18:5:-3, 11:7:144389, 28:9:-3, 41:11:179414, 40:17:-3=FRAME15:35:1:-3, 33:8:15768, 11:16:-3, 36:16:31633, 27:17:-3, 0:18:33492, 32:19:35931, 43:29:-3=FRAME16:12:29:-3=FRAME17:19:3:-3=FRAME18:37:2:13241, 3:4:18430, 27:9:-3, 41:21:97555, 6:28:116909, 43:29:-3=FRAME19:39:23:-3, 43:23:39539, 13:29:-3



    'new' 
    FRAME0:0:0:-1, 29:11:56945, 0:12:58604:0:48:0:0:0:0, 17:12:60055:0:0:-10:5:0:0, 18:12:60234:0:0:5:-5:0:0, 0:13:63030:50:0:0:0:0:0, 18:13:64546:15:0:0:-5:0:0, 19:13:64773:-2:1:0:0:0:0:1, 23:13:65229:0:5:10:0:0:0, 26:13:65548:9:0:0:0:0:0, 29:13:65871:15:0:0:0:0:0, 2:14:67729:0:0:-5:0:0:0, 11:14:68538:0:0:0:0:0:-5, 18:14:69331:0:0:0:20:0:0, 28:15:75458:20:0:0:0:0:0, 28:16:81012:0:0:0:-40:0:0, 31:18:90278:30:0:0:0:0:0, 7:29:-1, 8:29:139153=FRAME1:12:12:-3, 19:20:43995=FRAME5:29:5:-3, 43:9:47279, 24:13:-3, 32:14:77707, 21:22:109507, 1:23:111455=FRAME6:28:10:-3, 42:11:55212, 16:14:66147, 43:29:-3=FRAME7:33:5:25539, 6:7:-3, 4:21:92786, 8:28:116815=FRAME8:9:23:-3=FRAME9:7:5:6416, 41:5:-3, 0:12:13030, 43:29:-3=FRAME10:27:14:-3, 43:15:35848=FRAME11:30:1:3471, 35:11:-3, 38:11:23458, 19:15:-3, 39:19:38646, 10:21:-3, 11:21:40210, 43:29:-3=FRAME12:32:0:3533, 24:4:-3, 25:4:-3, 31:4:21824, 34:8:-3, 2:9:41767, 40:9:-3, 43:11:46658, 32:14:79617, 0:15:-3, 16:15:72454, 33:16:-3, 39:16:79667, 6:17:-3, 24:17:82941, 36:19:92064, 5:23:-3=FRAME13:29:1:-3, 0:12:179561, 39:17:-3=FRAME14:32:0:-3, 23:3:79913, 18:5:-3, 11:7:144389, 28:9:-3, 41:11:179414, 40:17:-3=FRAME15:35:1:-3, 33:8:15768, 11:16:-3, 36:16:31633, 27:17:-3, 0:18:33492, 32:19:35931, 43:29:-3=FRAME16:12:29:-3=FRAME17:19:3:-3=FRAME18:37:2:13241, 3:4:18430, 27:9:-3, 41:21:97555, 6:28:116909, 43:29:-3=FRAME19:39:23:-3, 43:23:39539, 13:29:-3
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Chris Bergin on 06/06/2014 05:01 pm
    And there's the 300,000 visit mark for this thread.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: mhenderson on 06/06/2014 05:01 pm
    @Quialiss -

    Does this work better? 43:29:-2:0:0:0:0:12:-6 

    I have noticed that the frame at 43:29 appears to be used as the filler for pFrames ... about a third of those random non-error boxes sprinkled in the image will match 43:29. Perhaps try giving it a color (12, -6 is a gray-blue like water) and code it -2 instead of -3.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Quialiss on 06/06/2014 05:13 pm
    I have noticed that the frame at 43:29 appears to be used as the filler for pFrames

    Do you have a frame/set that this works on?  I've just tried 43:29:-2:0:0:0:0:50:100 which is a rather noticeable shade, and I don't see it used as filler anywhere in part 13.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Helodriver on 06/06/2014 05:15 pm
    I didn't even say anything! :)

    Yet.............just now:

    I have a very good idea of how SpaceX can show their appreciation for all the hard work NSFers have done.  ;)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Chris Bergin on 06/06/2014 05:21 pm
    I didn't even say anything! :)

    Yet.............just now:

    I have a very good idea of how SpaceX can show their appreciation for all the hard work NSFers have done.  ;)

    So do I sir, so do I! ;D

    *Cough* Several SpaceXers known to me need to check their messages! :)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: IainCole on 06/06/2014 06:11 pm
    Apparently the Wikispaces API doesn't allow file uploads.  >:(

    The auto-generated MMBs are uploaded here, in case you need to track down a bug:

    http://adama.nocdirect.com/~wronkiew/spx_crs3/mmbs/

    IainCole, can you change the youtube uploader to grab http://adama.nocdirect.com/~wronkiew/spx_crs3/landing_clockfix_3hr.mp4 ? I can then move the trigger to the 3hr script.

    Done
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: IainCole on 06/06/2014 06:35 pm
    Request for the web editor:

    Allow switching between frames when the 'Isolate frame' box is checked.  Saving the isolated p frames and flipping between them to get alignment correct is the only p frame related thing I still have to do in an outside editor, getting rid of that step would make finalizing alignments so much less painful.

    I can do this but it will probably be prohibitively slow as it stands. I might be able to make a change over the weekend to make this possible.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: mhenderson on 06/06/2014 06:44 pm
    I have noticed that the frame at 43:29 appears to be used as the filler for pFrames

    Do you have a frame/set that this works on?  I've just tried 43:29:-2:0:0:0:0:50:100 which is a rather noticeable shade, and I don't see it used as filler anywhere in part 13.

    I will swear it used to work that way a few weeks ago (using 43:29 as filler in pFrames). I was working on what is now iFrame 221. I checked and it no longer behaves that way. Sorry.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: IainCole on 06/06/2014 07:00 pm
    I need 15 people to perform the unenviable task of receiving a gift from SpaceX.

    Who do we think should be the lucky 15?
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Dudely on 06/06/2014 07:08 pm
    I need 15 people to perform the unenviable task of receiving a gift from SpaceX.

    Who do we think should be the lucky 15?

    That's a tough call.

    Are they T-shirts or what?
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: IainCole on 06/06/2014 07:13 pm
    Yes tshirts
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: arnezami on 06/06/2014 07:23 pm
    Sort the posters by the number of likes received in this thread, then take first 15... ;)

    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: wronkiew on 06/06/2014 07:25 pm
    I need 15 people to perform the unenviable task of receiving a gift from SpaceX.

    Who do we think should be the lucky 15?

    That's a tough call.

    Are they T-shirts or what?

    Pizza pans.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Quialiss on 06/06/2014 07:27 pm
    I need 15 people to perform the unenviable task of receiving a gift from SpaceX.

    Who do we think should be the lucky 15?

    This is everyone who either contributed/was named as a contributor on the wiki.  If I missed anyone I'm  sorry!  33 in all, though I have probably missed out on people who were involved in productive discussion here on the forum.

    arnezami
    Asmegin
    deruch
    dgdpg
    dorkmo
    Exclavion
    Gnonthgol
    grythumn
    IainCole
    jakusb
    John_L
    lewing
    lgjy98d
    Lourens
    MHenderson
    michaelni
    Mlindner
    moralec
    morningdew76
    mvpel
    Noslyl
    princess
    pshrpd
    Quialiss
    saliva_sweet
    seanpg71
    Shanuson
    SwissCheese
    tentonine
    theshadow27
    Turix
    Untribium
    wronkiew


    nobody saw that missing name  :-[
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: arnezami on 06/06/2014 07:29 pm
    We need more T-shirts!!  ;D
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: rocketguy101 on 06/06/2014 07:43 pm
    FYI, I just bought a couple SpaceX tshirts, you may want to order a size larger than you normally do, as they are "snug" (unless you like tight tshirts...)  congrats all on the repair work, wish I understood more of the "naughty bits" of what you are doing!!
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: cscott on 06/06/2014 07:44 pm
    We need more T-shirts!!  ;D

    Clearly the solution is to cut the t-shirts in half.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: mhenderson on 06/06/2014 07:53 pm
    I sent my list to Chris by PM. The top 5 were easy. The next 5 were a little tougher. The third 5 were tough calls. Then, I gave 5 more runners-up.

    No, I won't post my list.  8)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: MTom on 06/06/2014 08:00 pm
    Yes tshirts

    And maybe a sponsoring 1 year L2 membership for the contributors?
    This would be a 2in1 present - also for the big contribution of this site.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Chris Bergin on 06/06/2014 08:07 pm
    Yes tshirts

    And maybe a sponsoring 1 year L2 membership for the contributors?
    This would be a 2in1 present - also for the big contribution of this site.


    I've been gradually going through the main people on here and giving them a L2 membership for their work. A few are already L2 members, so I'll sort out something for them. A few I've yet to sort out, per the list I've just seen.

    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Chris Bergin on 06/06/2014 08:07 pm
    This is the list I've been sent.

    arnezami   Primary coordinator& strategist
    IainCole   Online editor, YouTube feed
    michaelni   ffmpg enhancements
    Shanuson   TS header fix, datasteam analysis
    Wronkiew   YouTube feed, clock
    SwissCheese   MMB
    Princess   TS header fix
    Quialiss   MMB
    mvpel   TS header fix
    Saliva_Sweet   MMB
    Lourens   MMB, strategy / communications
    mhenderson   MMB
    Mlindner   
    ajmartin   Error stats
    theshadow27   error analysis
    untribium   MMB
    Swatch   
    moralec   General commentary
    grythumn   MMB
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: MTom on 06/06/2014 08:22 pm
    Yes tshirts

    And maybe a sponsoring 1 year L2 membership for the contributors?
    This would be a 2in1 present - also for the big contribution of this site.


    I've been gradually going through the main people on here and giving them a L2 membership for their work. A few are already L2 members, so I'll sort out something for them. A few I've yet to sort out, per the list I've just seen.

    Oh, my bad english. I wanted say that NSF should be sponsored so you can give the L2 membership.  ;)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: sittingduck on 06/06/2014 08:35 pm
    I've been following this thread since page one but it updates so fast that I'm not sure if this has been asked before: I notice in one of the two original videos that SpaceX released there is a very clear shot of the cloud deck while the stage rotates.  The video is also ~8 seconds longer.  What's happened to the clear footage of the clouds?
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: arnezami on 06/06/2014 08:40 pm
    This is the list I've been sent.

    arnezami   Primary coordinator& strategist
    IainCole   Online editor, YouTube feed
    michaelni   ffmpg enhancements
    Shanuson   TS header fix, datasteam analysis
    Wronkiew   YouTube feed, clock
    SwissCheese   MMB
    Princess   TS header fix
    Quialiss   MMB
    mvpel   TS header fix
    Saliva_Sweet   MMB
    Lourens   MMB, strategy / communications
    mhenderson   MMB
    Mlindner   
    ajmartin   Error stats
    theshadow27   error analysis
    untribium   MMB
    Swatch   
    moralec   General commentary
    grythumn   MMB

    Pretty good list. :)

    I personally like to see Jakusb (the "dirt-master") on the list too. Because he gave us the list of dirt spots on the camera (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1196618#msg1196618) which allowed others to align blocks in the top part of the frames. But considering having only 15 spots its a good list.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: SwissCheese on 06/06/2014 08:47 pm
    I feel like I'm going a bit mad.  Could someone who understands the data structures help out a bit here? The relevant MMBs are at the bottom in case the ones in the wiki change, if you need to see what's going on.   

    Part 13, frame 242.  Used to be: 
    12:12:-3,19:20:43995,43:29:-3

    SwissCheese pulled out a bit more data from it, but for simplicity's sake lets say the new 242 is:
    12:12:-3,19:20:43995

    Just removing the null block at the end of the frame.

    This causes absolutely no changes to frame 243 or 244.

    Then we get to 245.  With the old frame 242, it did not play* without the mmb 43:29:-3 applied to it.
    With the new 242, it doesn't play unless you get rid of that 43:29:-3.

    So we have old version with two frames ending with 43:29:-3, and the new version that is identical except to remove those two commands

    The old version, all the frames play out normally. 
    The new version, 247, 249, 250, 252, 253 and 254 all fail to play. 

    This is problematic, and extremely confusing.  I've noticed an occasional frame that wouldn't display before, but this is the first time I've seen half a sequence go bad because of changes frames earlier in the set.  Something very weird is going on here!  Help! 



    *By 'not play' I mean that when the editor hits it, the whole frame blurs very slightly and none of the changes in the p frame are applied. 


    'old'
    FRAME0:0:0:-1, 29:11:56945, 0:12:58604:0:48:0:0:0:0, 17:12:60055:0:0:-10:5:0:0, 18:12:60234:0:0:5:-5:0:0, 0:13:63030:50:0:0:0:0:0, 18:13:64546:15:0:0:-5:0:0, 19:13:64773:-2:1:0:0:0:0:1, 23:13:65229:0:5:10:0:0:0, 26:13:65548:9:0:0:0:0:0, 29:13:65871:15:0:0:0:0:0, 2:14:67729:0:0:-5:0:0:0, 11:14:68538:0:0:0:0:0:-5, 18:14:69331:0:0:0:20:0:0, 28:15:75458:20:0:0:0:0:0, 28:16:81012:0:0:0:-40:0:0, 31:18:90278:30:0:0:0:0:0, 7:29:-1, 8:29:139153=FRAME1:12:12:-3, 19:20:43995, 43:29:-3=FRAME4:43:29:-3=FRAME5:29:5:-3, 43:9:47279, 24:13:-3, 32:14:77707, 21:22:109507, 1:23:111455=FRAME6:28:10:-3, 42:11:55212, 16:14:66147, 43:29:-3=FRAME7:33:5:25539, 6:7:-3, 4:21:92786, 8:28:116815=FRAME8:9:23:-3=FRAME9:7:5:6416, 41:5:-3, 0:12:13030, 43:29:-3=FRAME10:27:14:-3, 43:15:35848=FRAME11:30:1:3471, 35:11:-3, 38:11:23458, 19:15:-3, 39:19:38646, 10:21:-3, 11:21:40210, 43:29:-3=FRAME12:32:0:3533, 24:4:-3, 25:4:-3, 31:4:21824, 34:8:-3, 2:9:41767, 40:9:-3, 43:11:46658, 32:14:79617, 0:15:-3, 16:15:72454, 33:16:-3, 39:16:79667, 6:17:-3, 24:17:82941, 36:19:92064, 5:23:-3=FRAME13:29:1:-3, 0:12:179561, 39:17:-3=FRAME14:32:0:-3, 23:3:79913, 18:5:-3, 11:7:144389, 28:9:-3, 41:11:179414, 40:17:-3=FRAME15:35:1:-3, 33:8:15768, 11:16:-3, 36:16:31633, 27:17:-3, 0:18:33492, 32:19:35931, 43:29:-3=FRAME16:12:29:-3=FRAME17:19:3:-3=FRAME18:37:2:13241, 3:4:18430, 27:9:-3, 41:21:97555, 6:28:116909, 43:29:-3=FRAME19:39:23:-3, 43:23:39539, 13:29:-3



    'new' 
    FRAME0:0:0:-1, 29:11:56945, 0:12:58604:0:48:0:0:0:0, 17:12:60055:0:0:-10:5:0:0, 18:12:60234:0:0:5:-5:0:0, 0:13:63030:50:0:0:0:0:0, 18:13:64546:15:0:0:-5:0:0, 19:13:64773:-2:1:0:0:0:0:1, 23:13:65229:0:5:10:0:0:0, 26:13:65548:9:0:0:0:0:0, 29:13:65871:15:0:0:0:0:0, 2:14:67729:0:0:-5:0:0:0, 11:14:68538:0:0:0:0:0:-5, 18:14:69331:0:0:0:20:0:0, 28:15:75458:20:0:0:0:0:0, 28:16:81012:0:0:0:-40:0:0, 31:18:90278:30:0:0:0:0:0, 7:29:-1, 8:29:139153=FRAME1:12:12:-3, 19:20:43995=FRAME5:29:5:-3, 43:9:47279, 24:13:-3, 32:14:77707, 21:22:109507, 1:23:111455=FRAME6:28:10:-3, 42:11:55212, 16:14:66147, 43:29:-3=FRAME7:33:5:25539, 6:7:-3, 4:21:92786, 8:28:116815=FRAME8:9:23:-3=FRAME9:7:5:6416, 41:5:-3, 0:12:13030, 43:29:-3=FRAME10:27:14:-3, 43:15:35848=FRAME11:30:1:3471, 35:11:-3, 38:11:23458, 19:15:-3, 39:19:38646, 10:21:-3, 11:21:40210, 43:29:-3=FRAME12:32:0:3533, 24:4:-3, 25:4:-3, 31:4:21824, 34:8:-3, 2:9:41767, 40:9:-3, 43:11:46658, 32:14:79617, 0:15:-3, 16:15:72454, 33:16:-3, 39:16:79667, 6:17:-3, 24:17:82941, 36:19:92064, 5:23:-3=FRAME13:29:1:-3, 0:12:179561, 39:17:-3=FRAME14:32:0:-3, 23:3:79913, 18:5:-3, 11:7:144389, 28:9:-3, 41:11:179414, 40:17:-3=FRAME15:35:1:-3, 33:8:15768, 11:16:-3, 36:16:31633, 27:17:-3, 0:18:33492, 32:19:35931, 43:29:-3=FRAME16:12:29:-3=FRAME17:19:3:-3=FRAME18:37:2:13241, 3:4:18430, 27:9:-3, 41:21:97555, 6:28:116909, 43:29:-3=FRAME19:39:23:-3, 43:23:39539, 13:29:-3

    You need to look at the log, at the end of the frame: for some reason, sometimes when the end of the frame is not reached at position 43:29 the frame is not used. If this is the case, the log also writes the number of bits remaining (positive or negative) to the end of frame position. Sometimes you can simply put 43:29:-3 to solve this, but in other cases the exact position is needed. The problem is that it is quite difficult to read the log in the online editor (that's the only remaining thing I do in my local build).

    In another news, my laptop "drank" a cup of tea on thursday; after a 24 hour drying it seems to have survived. Had some keyboard issues first but now everything seems OK  ;D
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Quialiss on 06/06/2014 08:48 pm
    I've been following this thread since page one but it updates so fast that I'm not sure if this has been asked before: I notice in one of the two original videos that SpaceX released there is a very clear shot of the cloud deck while the stage rotates.  The video is also ~8 seconds longer.  What's happened to the clear footage of the clouds?

    IIRC, it was mentioned by a member of SpaceX that they only gave out the raw ts for the part they most wanted recovered.  What we have starts just a few seconds before the stage leaves the clouds, to a few seconds after the stage touched down on the water, so it's cropped on both ends.

    Watching the raw video again is interesting.  There are a handful of undamaged p frames in parts 13-14, I recognize them in the raw video, long before the tipping motion. 
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: lgjy98d on 06/06/2014 09:16 pm
    There has been an effort to let people suggest improvements (in case there are doubts, it one is too shy to replace someones work with own variant). This is why we have suggest form (https://docs.google.com/forms/d/1p8-M-Oy5FhBj6Ca9Nn6LNIgpyHYaEzZSGpP0GtyQLX0/viewform) in Online editor (http://spacex2.slapbet.org/) now. When the form is processed the moderators are emailed about the suggestion. See the sample notification e-mail attached.

    We intend to improve the notification logic to notify only frame contributors not all moderators. Thus ones who contributed to frame will be notified only. You are welcome to register as contributors and frame moderators at: https://docs.google.com/forms/d/1HoijPXQJsR8nHkS54UqMJkoNA7h24tF6DCIaeN_PhAE

    Huge cudos to JohnKiel for his spxi.php script that enables these suggestion previews in notification emails and video frames in Google Spreadsheets.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: moralec on 06/06/2014 09:20 pm
    Hahahha thanks for the "General commentary" title Chris hahahha! ;)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Chris Bergin on 06/06/2014 09:42 pm
    Hahahha thanks for the "General commentary" title Chris hahahha! ;)

    Not my list! ;D All I did was copy and paste it into the thread :)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: moralec on 06/06/2014 10:20 pm
    Hahahha thanks for the "General commentary" title Chris hahahha! ;)

    Not my list! ;D All I did was copy and paste it into the thread :)

    Haha I found it hilarious. I'm gonna make a business card with that. 
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: catdlr on 06/06/2014 10:26 pm
    Hahahha thanks for the "General commentary" title Chris hahahha! ;)

    Not my list! ;D All I did was copy and paste it into the thread :)

    Haha I found it hilarious. I'm gonna make a business card with that. 

    and this would be your updated title on NSF...  ;)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: moralec on 06/06/2014 10:31 pm
    Hahahha thanks for the "General commentary" title Chris hahahha! ;)

    Not my list! ;D All I did was copy and paste it into the thread :)

    Haha I found it hilarious. I'm gonna make a business card with that. 

    and this would be your updated title on NSF...  ;)


    OMG! Hahahaha
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: deruch on 06/06/2014 11:10 pm
    Hahahha thanks for the "General commentary" title Chris hahahha! ;)

    Not my list! ;D All I did was copy and paste it into the thread :)

    Haha I found it hilarious. I'm gonna make a business card with that.

    "General Commentary" is a job description.  But, on business cards, you're supposed to list your job title.  Commentary General, (Res.) is a job title.  :)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: deruch on 06/06/2014 11:24 pm
    This is the list I've been sent.

    arnezami   Primary coordinator& strategist
    IainCole   Online editor, YouTube feed
    michaelni   ffmpg enhancements
    Shanuson   TS header fix, datasteam analysis
    Wronkiew   YouTube feed, clock
    SwissCheese   MMB
    Princess   TS header fix
    Quialiss   MMB
    mvpel   TS header fix
    Saliva_Sweet   MMB
    Lourens   MMB, strategy / communications
    mhenderson   MMB
    Mlindner   
    ajmartin   Error stats
    theshadow27   error analysis
    untribium   MMB
    Swatch   
    moralec   General commentary
    grythumn   MMB

    That looks like a pretty good list.  I recently went back through the thread to write up something that acknowledged each of the people that contributed to the TS repair, which I then planned to upload to the wiki.  There were only 3 people that made what I thought were considerable contributions that didn't make the quoted list.  Jared (aka reddit user /u/sharebrained, I don't know if he's a NSF member) for the original coerced.ts, @Adaptation for .ts alignment and search tool, and @gospacex for packet fixing tools and analysis/meta-analysis.  With a list limited to 15 people there will, based solely on the size of this project, be deserving people left off.  I don't feel qualified to assert that one of these three is worthy of bumping someone off Chris' list.  Plus I think it got a very good mix of people involved with the various sub-parts of the project, and I wouldn't bump any of the TS superstars that made the list.   But if the list were expanded at all, these three deserve consideration to be the first added. 

    edited for clarity
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Shanuson on 06/06/2014 11:57 pm
    <snip>

    You need to look at the log, at the end of the frame: for some reason, sometimes when the end of the frame is not reached at position 43:29 the frame is not used. If this is the case, the log also writes the number of bits remaining (positive or negative) to the end of frame position. Sometimes you can simply put 43:29:-3 to solve this, but in other cases the exact position is needed. The problem is that it is quite difficult to read the log in the online editor (that's the only remaining thing I do in my local build).
    <snip>

    It could be that this is one if the effects of a wrong AF length of the last packet. I think more and more that we should have fixed that before releasing/adding the "final" ts file.

    Well as long as there is a way to work around that problem, it's OK I think.

    Cheers,
    Shanuson
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: lgjy98d on 06/07/2014 12:08 am
    Apparently the Wikispaces API doesn't allow file uploads.  >:(

    The auto-generated MMBs are uploaded here, in case you need to track down a bug:

    http://adama.nocdirect.com/~wronkiew/spx_crs3/mmbs/

    IainCole, can you change the youtube uploader to grab http://adama.nocdirect.com/~wronkiew/spx_crs3/landing_clockfix_3hr.mp4 ? I can then move the trigger to the 3hr script.

    Alternatively you can just grab the CSV from https://docs.google.com/spreadsheets/d/1xEtLQy3tvjv3cyz_wMQssuSFwHYVbMQ5-FdSz-uCsKA/export?gid=0&format=csv - this CSV is being autopublished by Google with something like 10 minute delay of the actual edit of the any of the parts' spreadseets.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: cmobile on 06/07/2014 12:28 am
    This is the list I've been sent.

    arnezami   Primary coordinator& strategist
    IainCole   Online editor, YouTube feed
    michaelni   ffmpg enhancements
    Shanuson   TS header fix, datasteam analysis
    Wronkiew   YouTube feed, clock
    SwissCheese   MMB
    Princess   TS header fix
    Quialiss   MMB
    mvpel   TS header fix
    Saliva_Sweet   MMB
    Lourens   MMB, strategy / communications
    mhenderson   MMB
    Mlindner   
    ajmartin   Error stats
    theshadow27   error analysis
    untribium   MMB
    Swatch   
    moralec   General commentary
    grythumn   MMB

    That looks like a pretty good list.  I recently went back through the thread to write up something that acknowledged each of the people that contributed to the TS repair, which I then planned to upload to the wiki.  There were only 3 people that made what I thought were considerable contributions that didn't make the quoted list.  Jared (aka reddit user /u/sharebrained, I don't know if he's a NSF member) for the original coerced.ts, @Adaptation for .ts alignment and search tool, and @gospacex for packet fixing tools and analysis/meta-analysis.  With a list limited to 15 people there will, based solely on the size of this project, be deserving people left off.  I don't feel qualified to assert that one of these three is worthy of bumping someone off Chris' list.  Plus I think it got a very good mix of people involved with the various sub-parts of the project, and I wouldn't bump any of the TS superstars that made the list.   But if the list were expanded at all, these three deserve consideration to be the first added. 

    edited for clarity

    Just submit everyone who made a significant contribution even if it's more than 15.  I doubt they're going to begrudge a few T-shirts.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Untribium on 06/07/2014 01:22 am
    -snip-
    Just submit everyone who made a significant contribution even if it's more than 15.  I doubt they're going to begrudge a few T-shirts.

    Yeah I agree, mostly because I'm #16 on the list ;D No but seriously, it was (and is) a group effort, so it would be kinda sad if we had to make a cut somewhere...

    Back to the actual job :) I haven't been able to spend much time recently, that may change :) I really want go through a single i-frame and try to give meaning to every single bit using the specification and the VLC codes. May not be feasible, we'll see :))
    Also, I saw someone fixed the dc stuff for i-frame 4 (61). Quialiss, I suspect that was you :) Great work, row 10 was driving nuts :)) There's also one more change that morningdew76 suggested a long time ago, but somehow it wasn't  picked up by anybody...I put it back in, still works nicely, but with all the changes that have happened to the wiki (love the google spreadsheets!), I'm not quite sure what needs to be updated manually and what is done automatically. I'm thinking Wiki frames part 4 and the spreadsheet, is that correct?

    Anybody in the Florida area want to get together for the OrbComm launch? I am working a temporary / long term assignment in Miami and considering driving to KSC for the OrbComm OG2 launch next Thursday (backup launch date is Friday).

    I'd love that, but unfortunately I'm not exactly in the area :'( ;)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: lgjy98d on 06/07/2014 01:37 am
    ...I put it back in, still works nicely, but with all the changes that have happened to the wiki (love the google spreadsheets!), I'm not quite sure what needs to be updated manually and what is done automatically. I'm thinking Wiki frames part 4 and the spreadsheet, is that correct?

    Since I was the one who started intoroducing Spreadsheets as structured databaseto results of the work. I'll explain a bit.

    The spreadsheets were introduced in parralel to wiki pages. Since that time they gained image previews that update automagically, and the resulting MMBs of the whole video can be grabbed as CSV (https://docs.google.com/spreadsheets/d/1xEtLQy3tvjv3cyz_wMQssuSFwHYVbMQ5-FdSz-uCsKA/export?gid=0&format=csv) that allows all automations like video generation and contributor parsing, i.e. behaves like a real database.

    At the moment one would have to update both the spreadsheet and wiki. If you are not sure what to do there is new "Suggest (https://docs.google.com/forms/d/1p8-M-Oy5FhBj6Ca9Nn6LNIgpyHYaEzZSGpP0GtyQLX0/viewform)" feature (linked from online editor (http://spacex2.slapbet.org/)). that will submit the suggestion to Moderators (only Quialiss volunteered so far).

    If you want to volunteer as Moderator or if you have your pet-frame and want to get notifications of suggestions of others, you can register your email (https://docs.google.com/forms/d/1HoijPXQJsR8nHkS54UqMJkoNA7h24tF6DCIaeN_PhAE/viewform) and expect some emails soon like this:
    (http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34597.0;attach=587184)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: mvpel on 06/07/2014 02:02 am
    Anybody in the Florida area want to get together for the OrbComm launch? I am working a temporary / long term assignment in Miami and considering driving to KSC for the OrbComm OG2 launch next Thursday (backup launch date is Friday).

    I'll be in McGregor, Texas the evening of the OrbComm launch, after presenting at a symposium in Garland. If anyone would like to get together there, PM me too. :) I hope it doesn't slip, because Friday I'll be airborne flying home from DFW at that time.  :-\
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: cscott on 06/07/2014 02:06 am
    But I bet it *would* be helpful if someone would compile a list of the adjusted start/end positions of the FEC coding blocks referenced to the current .ts and including the discovered bit shifts, etc, like princess did

    I can calculate a mapping between file offsets in raw.ts and the current TS file if necessary, would this be useful?

    Also, if people want to calculate statistics on the bitflip errors then I have TS files that have more fixes than the current TS file - my current versions have correct contents for all PIDs except the data PID, i.e. all padding is correct, the Program Association Tables and Program Map Tables are all correct and the "padding" AFs in the data are correct too. One of my side projects is trying to make a TS file that's "what the rocket transmitted", so I can make that available to people who are doing statistical anaylsis on the error distribution. Let me know what you need and I'll make it available.

    I'd like to pitch in and follow up on this "triple bit flip" lead.  I *think* what I would need is:
    1) the raw.ts file (no problem there)
    2) princess' "corrected as best as possible" .ts file.

    Are there extra bits inserted into (2)?  If so, then I'd have to correlate (1) and (2).  But otherwise it should be easy to retrace ajmartin's steps and figure precisely what he's talking about here:
    The postulated 56-byte FEC coding blocks turn out to be composed of alternate 25-byte and 31-byte "sub-blocks", starting at (zero-based) offsets of the form 56n + 29 and 56n + 54.  Single bit-flips are found only in the first and last 15 bits of these blocks, with possible rare exceptions in high-error regions.  All other bit-flip errors are combinations of "triple flips" (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1201383#msg1201383) (i.e., bit shifts of 0x8003) that do not cross the block boundaries.

    (That does not include frame-shift errors such as the one originally found by Shanuson (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1196281#msg1196281), which are surprisingly common - I found at least 20 of them just in the "boilerplate" data, by amounts ranging from 1 bit to 12 bits and in both directions.  But these seem to occur only after high-error regions, which may have damaged the framing information that was used in decoding the signal.  They affect entire "sub-blocks" and garbage bits are shifted in at one end.)

    Then (hopefully) by correlating the raw.ts with the  raw_final_fixed.ts I could derive bit positions in raw_final_fixed.ts for all of the "25-byte and 31-byte sub-blocks".  Then for a given frame, for each mmb I could see if the bit flipped was part of a "triple flip" and manually try flipping all three of the triple flip bits, to see if that improves the result.

    In theory this could eventually be added to the online tool to make it easy to test "triple flips" at appropriate bit positions.  But baby steps first.

    Have I understood the issues correctly?  Princess, could you get me a copy of your fixed .ts file?

    (And if anyone really understands what ajmartin was talking about, I'd appreciate some more detail.  Are the triple flips aligned to specific positions in the subblock?  And when he says "combinations of triple flips" it is possible for the triple flips to overlap and flip certain bits "twice"?  I'm assuming I can figure this out by retracing his steps starting from princess' fixed .ts file.)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: ajmartin on 06/07/2014 02:17 am
    I suspect the ship has already sailed and no-ones eager to go back to raw.ts.  But I bet it *would* be helpful if someone would compile a list of the adjusted start/end positions of the FEC coding blocks referenced to the current .ts and including the discovered bit shifts, etc, like princess did:

    That's a good idea.  It's actually a bit easier than that, because the frame-shift errors do not change the coding-block boundaries.  (Bits are not shifted from one block to another, instead "garbage" bits from the framing of each block are shifted into the block - I have confirmed this by observation in the "boilerplate" regions.  Even if the shift were corrected in raw_final_fixed.ts, there would be no need to move the block boundaries, as the only difference it would make is to change which end of the blocks has the "garbage" bits.)

    I did the mapping, and for frames 1-254 the 25-byte blocks start at offsets of the form 56n + 13 (zero-based) and the 31-byte blocks start at offsets 56n + 38 in raw_final_fixed.ts.  For frames 255-284, the 25-byte blocks start at offsets 56n + 5 and the 31-byte blocks start at offsets 56n + 30.  (The reason for the discontinuity was explained by Shanuson here (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1205733#msg1205733).)

    As for the issue of fixes to TS headers interfering with finding "triple flips", the simplest thing to do is to treat the coding blocks as if they did not include the TS headers, splitting them in two as needed.  (That is, do not look for triple flips that overlap TS-headers, but instead look for single bit-flips in the 15 bits on either side of them.)  This loses a bit of "free" information about the errors provided by the damaged TS-headers, but it allows the structure of the errors to be used without referring back to raw.ts.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: cscott on 06/07/2014 02:30 am
    I suspect the ship has already sailed and no-ones eager to go back to raw.ts.  But I bet it *would* be helpful if someone would compile a list of the adjusted start/end positions of the FEC coding blocks referenced to the current .ts and including the discovered bit shifts, etc, like princess did:

    That's a good idea.  It's actually a bit easier than that, because the frame-shift errors do not change the coding-block boundaries.

    OK.  I'm still coming to speed here, bear with me.  Could you give me an example of a triple flip -- maybe the first one in the transport stream, just to pick one arbitrarily.  It would help me a bunch if you could give me a short hex dump of a sequence in raw.ts, then the equivalent sequence in the fixed .ts showing the triple flip.  If you gave me the offsets of the various features, as you label them, it would also sanity-check my understanding of how offsets are counted, etc.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Tea Party Space Czar on 06/07/2014 04:56 am
    This just popped up in my Facebook feed.  You guys kick ass.  The post was sponsored by SpaceX.

    Good Work.

    Respectfully,
    Andrew Gasser
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: ajmartin on 06/07/2014 04:57 am
    OK.  I'm still coming to speed here, bear with me.  Could you give me an example of a triple flip -- maybe the first one in the transport stream, just to pick one arbitrarily.  It would help me a bunch if you could give me a short hex dump of a sequence in raw.ts, then the equivalent sequence in the fixed .ts showing the triple flip.  If you gave me the offsets of the various features, as you label them, it would also sanity-check my understanding of how offsets are counted, etc.

    Sorry, I should have given examples earlier.  The easiest way to see "triple flips" is to look at a hex dump of damaged null packets in raw.ts.  That way there is no need to cross-reference with the fixed .ts, since we know all the bytes should be ff.  For instance:

    00001ac0  ff ff ff ff ff ff 7f fc  ff ff ff ff ff ff ff ff  |................|

    Here we are looking at 16 bytes at offset 0x1ac0 = 6848.  This is in the middle of a null packet, and we see two incorrect bytes.  At the bit level:

    Transmitted: 11111111 11111111
    Received:    01111111 11111100
    Flips:       *              **
    XOR:         10000000 00000011


    That's a "triple flip".  The pattern of bit-flips can be represented as a binary number, by XORing the transmitted and received values.  Written more compactly in hex, it is 0x8003.

    The relative positions of the three flipped bits are always the same, but the pattern need not be byte-aligned.  Here's another example:

    00001980  ff ff ff fe ff f9 ff ff  ff ff ff ff ff ff ff ff  |................|

    At the bit level, we see the same pattern of three flips:

    Transmitted: 11111111 11111111 11111111
    Received:    11111110 11111111 11111001
    Flips:              *               **
    XOR:         00000001 00000000 00000110


    The error pattern in hex is 0x10006; which is just 0x8003 shifted left by 1 bit.  (The other two shifts that look different in hex are 0x2000c and 0x40018.)  Moving on to a more complicated case:

    00001a60  ff ff ff ff ff fc ff f5  ff ff ff ff ff ff ff ff  |................|

    Transmitted:          11111111 11111111 11111111
    Received:             11111100 11111111 11111010
    Error pattern (XOR):  00000011 00000000 00000101


    At first glance this is a "quadruple flip".  But it can be written as a combination (XOR) of two triple flips:

        Triple Flip 1: 00000010 00000000 00000110
    XOR Triple Flip 2: 00000001 00000000 00000011
     =  Error Pattern: 00000011 00000000 00000101


    As you suspected, a bit can be "flipped twice" and end up unchanged; one such bit is highlighted in red.  (In binary linear coding, two wrongs make a right!)

    Then there are the errors that are not triple flips.  Here's one:

    00001a20  ff ff ff ff ff|3f ff ff  ff ff ff ff ff ff ff ff  |.....?..........|

    That's where the coding blocks come in.  This line starts at offset 0x1a20 = 6688, and the 3f is at offset 6693; dividing by 56, we get a quotient of 119 and a remainder of 29.  So it is the first byte of a 25-byte block, whose boundary is marked in blue, and we expect single flips rather than triple flips there.

    (I had a hypothesis that these cases were actually "truncated triple flips", with some of the flipped bits affecting the framing of the coding block, and being removed during decoding.  But there were too many places where that did not fit the data - although it does fit this example.)

    These patterns still apply where the data isn't all ff's.  Here's an example from the headers of an I-frame (number 241) showing five triple flips:

    Received:  00 30 81 20 07 21 01 b1 76 bd ff fe 50 07 e1 b0
    Flip1:        20 00 c0
    After:     00 10 81 e0 07 21 01 b1 76 bd ff fe 50 07 e1 b0
    Flip2:        10 00 60
    After:     00 00 81 80 07 21 01 b1 76 bd ff fe 50 07 e1 b0
    Flip3:                                      01 00 06
    After:     00 00 81 80 07 21 01 b1 76 bd ff ff 50 01 e1 b0
    Flip4:                                         40 01 80
    After:     00 00 81 80 07 21 01 b1 76 bd ff ff 10 00 61 b0
    Flip5:                                         10 00 60
    Corrected: 00 00 81 80 07 21 01 b1 76 bd ff ff 00 00 01 b0

     
    You can find this last example at offset 3840068 (0x3a9844) of raw.ts and offset 3585364 (0x36b554) of raw_final_fixed.ts.  (It has been corrected there but not quite fully.)

    I hope this helps to make things less mysterious.  I have lots of other examples, if there's something specific that you would like to see.

    (edit: typo fixes)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: princess on 06/07/2014 07:28 am
    Princess, could you get me a copy of your fixed .ts file?

    Sure, here's my best effort so far. This is a fixed version of faw_final_fixed.ts so it doesn't correspond bit-for-bit to raw.ts. The problem is that raw.ts loses its alignment from time to time and has to be realigned by removing 56 bytes each time (this may be significant).

    My best guess at the realignments are:

    Remove 56 bytes after packet 1224,
    and then remove 56 bytes after packet 4688,
    and then remove 56 bytes after packet 11637,
    and then remove 56 bytes after packet 18633,
    and then remove 56 bytes after packet 21551.

    Re-generate the file after each single removal for the next packet number to make sense. Each packet is 188 bytes long. Please note that these positions may not be exactly correct but it will give you the places to look for the right ones.

    Once you've done this you need to remove everything before the start of the first I-frame, because the raw_final_fixed files start there.

    I'm busy with other things today but I'll be back in about 12 hours to help out.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: arnezami on 06/07/2014 07:35 am
    Hi guys,

    I'm experimenting with making a "repair video" of sorts. Just for fun and a little bit for education 8)

    Here is a very simple gif that shows the repairing of iframe 3 "in progress"...

    Be careful: since its a gif it's big!

    Regards,

    arnezami

    PS. Maybe its cooler/nicer to first show the positioning, and then show the luma/chroma fixes. Now I show them both at the same time. What do you think? And maybe showing the repairing of all iframes in sequence in a video. Maybe even include all P-frames... That could be cool too. :)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: saliva_sweet on 06/07/2014 08:10 am
    Here is a very simple gif that shows the repairing of iframe 3 "in progress"...

    The concept is very cool and educational, but this version looks like you've taken the final mmb and just apply commands as they come in the sequence. Not quite representative of the actual process. I have saved most of my intermediate mmbs for the frames I worked with. For example for iframe 121 it should show the initial good data search and later alignment. Then there was dc correction done by Quialiss and then final touches by SwissCheese.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: arnezami on 06/07/2014 08:26 am
    Here is a very simple gif that shows the repairing of iframe 3 "in progress"...

    The concept is very cool and educational, but this version looks like you've taken the final mmb and just apply commands as they come in the sequence. Not quite representative of the actual process. I have saved most of my intermediate mmbs for the frames I worked with. For example for iframe 121 it should show the initial good data search and later alignment. Then there was dc correction done by Quialiss and then final touches by SwissCheese.
    Correct. This version is adding a mmb-command each time (sometimes two each time: a -1 command and a new position).

    So it's not a "chronological progress" of what we did. It's more to demonstrate that there are many small adjustments to a single frame in order to fix it. So "progress" is probably not the right word for this. It's more like: showing incremental fixes to the frame up to (full) restoration. Above all it visually demonstrates how correcting MPEG-4 works.

    I think a chronological progress can best be shown by releasing multiple videos from certain dates.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Shanuson on 06/07/2014 08:56 am
    Princess, could you get me a copy of your fixed .ts file?

    Sure, here's my best effort so far. This is a fixed version of faw_final_fixed.ts so it doesn't correspond bit-for-bit to raw.ts. The problem is that raw.ts loses its alignment from time to time and has to be realigned by removing 56 bytes each time (this may be significant).

    My best guess at the realignments are:

    Remove 56 bytes after packet 1224,
    and then remove 56 bytes after packet 4688,
    and then remove 56 bytes after packet 11637,
    and then remove 56 bytes after packet 18633,
    and then remove 56 bytes after packet 21551.

    Re-generate the file after each single removal for the next packet number to make sense. Each packet is 188 bytes long. Please note that these positions may not be exactly correct but it will give you the places to look for the right ones.

    Once you've done this you need to remove everything before the start of the first I-frame, because the raw_final_fixed files start there.

    I'm busy with other things today but I'll be back in about 12 hours to help out.

    Actually when i realigned raw.ts to get to the editX,ts files from which the final.ts file was produced, I added bytes but i think I never removed some. At least never when there was could have been some lost data.
    To bad I was to lazy to write down what I really did.  (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1198705#msg1198705)

    Cheers
    Shanuson
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: saliva_sweet on 06/07/2014 09:38 am
    I made a gif from the mmbs I saved while working on frame 121. I hope it's a bit more illuminating of the process as it happened. Unfortunately I don't have intermediate mmbs for the geat work done by Quialiss and SwissCheese so this is only represented by the last frame. Maybe they have these steps, or they can be recreated.

    Edit: And I don't know why it's not animating. It works fine when I open it locally :(
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: arnezami on 06/07/2014 10:16 am
    Edit: And I don't know why it's not animating. It works fine when I open it locally :(
    Maybe zip it and then upload. It seems the forum is doing something with the gif.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Quialiss on 06/07/2014 10:26 am
    Apparently the Wikispaces API doesn't allow file uploads.  >:(

    The auto-generated MMBs are uploaded here, in case you need to track down a bug:

    http://adama.nocdirect.com/~wronkiew/spx_crs3/mmbs/

    IainCole, can you change the youtube uploader to grab http://adama.nocdirect.com/~wronkiew/spx_crs3/landing_clockfix_3hr.mp4 ? I can then move the trigger to the 3hr script.

    Parts 3 and 4 are auto-generated correctly, but I think the video is still using the old fix8 files for those two parts, part 4 is currently cleaner than what's in the video.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: saliva_sweet on 06/07/2014 10:34 am
    Maybe zip it and then upload. It seems the forum is doing something with the gif.

    Here's the zip.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Quialiss on 06/07/2014 10:42 am
    PS. Maybe its cooler/nicer to first show the positioning, and then show the luma/chroma fixes. Now I show them both at the same time. What do you think? And maybe showing the repairing of all iframes in sequence in a video. Maybe even include all P-frames... That could be cool too. :)

    I think you could do this approximately by separating out all mmb's that contain DC information.. that are not after a -1 or -2 block?  Just trying to think of search terms that could work.  Then you'd have a 'base' and an 'additional modifications' file.  If you wanted to go further you could take the base and remove the DC information from them, apply the raw positioning data first, and then the DC corrections on the positioning, and then apply the additional modifications.... or, thinking along those lines...

    A simpler solution may be to strip all the DC information from the MMB, apply that one step at a time, which should give all the positioning(with a bunch of redundant steps.. hrrm), and then add back the DC information a step at a time. 


    Reconstructing the reconstruction.  :D  I can help out with making said split MMBs if needed, but I don't have the setup to make the actual images. 
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: deruch on 06/07/2014 10:56 am
    Maybe zip it and then upload. It seems the forum is doing something with the gif.

    Here's the zip.

    Wow that's great!  If you change this in the future can you add some frames to the end?  Ideally, repeats of the last frame, so that you get a chance to really see the end product would be awesome.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Jakusb on 06/07/2014 11:39 am

    Pretty good list. :)

    I personally like to see Jakusb (the "dirt-master") on the list too. Because he gave us the list of dirt spots on the camera (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1196618#msg1196618) which allowed others to align blocks in the top part of the frames. But considering having only 15 spots its a good list.

    Haha, the dirt-master. Love it!
    Thanks a lot for the appreciation!
    However I am sure plenty of others have made much bigger contributions. That is the trouble with a list like this in a group effort like ours. There will always be people that should be on the list, but somehow are forgotten or not properly acknowledged.

    I am just happy and very proud to be (a tiny) part of this amazing group effort on the best space forum that I know of. :)
    Although I would never refuse a SpaceX t-shirt if I ever got offered one. ;) I would wear it proudly. :)

    Keep up the good work everyone! Maybe some more detail on the end frames? I would love to see the stage really topple over as many suggest to see in the few frame parts sofar.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: mhenderson on 06/07/2014 12:12 pm
    Since 281 is the final iFrame and is currently washed out, it would be best to enhance the contrast so the video ends on a nicer image.

    I see the online editor now has a "suggest improvement to MMB" button.  I like that a lot, submissions should be controlled and referreed by a benevolent dictator.

    This is a step in the right direction, but I know my limits ... perhaps someone more skilled at this can take it over the finish line.

    Frame 15(281)
    X:3545:01,X:2003:A0,0:0:559,40:1:-1,0:2:-2:25:25:25:25:0:0,0:10:-2:60:60:60:60:0:0:40,
    1:10:53276,32:10:-1::40,33:10:57352,0:13:69419:0:0:0:1:0:0:63,
    28:15:-2:-70:-70:-70:-70:0:0:63,29:15:83102,8:29:-1

    This change was entered in the wiki page and the spreadsheet yesterday, but does not appear in the YouTube channel. Am I missing a step?
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: CraigLieb on 06/07/2014 12:23 pm
    We need more T-shirts!!  ;D

    Clearly the solution is to cut the t-shirts in half.
    I suggest taking all the tshirts, running them through a shredder, and putting all the scraps in
    A box in a public park. Give The 15 people on the list the address.   ;D
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: morningdew76 on 06/07/2014 01:58 pm
    Enhancing the contrast in P-frames with X:59:C0 / X:59:80 / X:59:60 / X:59:40 works great.

    Is there a way to do this for the I-frames too?
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: cscott on 06/07/2014 01:58 pm
    Sure, here's my best effort so far. This is a fixed version of faw_final_fixed.ts so it doesn't correspond bit-for-bit to raw.ts. The problem is that raw.ts loses its alignment from time to time and has to be realigned by removing 56 bytes each time (this may be significant).

    My working theory is that these 56 byte chunks are part of the telemetry stream, and that packet corruption means they weren't filtered out properly.

    Actually when i realigned raw.ts to get to the editX,ts files from which the final.ts file was produced, I added bytes but i think I never removed some. At least never when there was could have been some lost data.

    Well, another equally-plausible theory is that 56-byte chunks of "real data" were removed because they were mistakenly thought to be part of the telemetry stream. ;)

    I guess a close examination of these regions might be able to distinguish the two theories -- do there seem to be macroblocks missing?
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: saliva_sweet on 06/07/2014 05:01 pm
    Wow that's great!  If you change this in the future can you add some frames to the end?  Ideally, repeats of the last frame, so that you get a chance to really see the end product would be awesome.

    Made a new gif that I think more accurately represents the evolution of this frame from total garbage to what we have now. I split the dc corrections into luma and chroma passes and applied them sequentially (I think this is Qualiss's method). Jumped over a lot of minute changes to reduce the size (it's still large) and some important changes (mostly in aligment) due to lack of data. Quialiss and Swisscheese, would you consider this a faithful representation of what you saw while working on this frame? I'm fairly happy with my part. Also, I extended the showing time of the final result.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Quialiss on 06/07/2014 05:44 pm
    Made a new gif that I think more accurately represents the evolution of this frame from total garbage to what we have now. I split the dc corrections into luma and chroma passes and applied them sequentially (I think this is Qualiss's method). Jumped over a lot of minute changes to reduce the size (it's still large) and some important changes (mostly in aligment) due to lack of data. Quialiss and Swisscheese, would you consider this a faithful representation of what you saw while working on this frame? I'm fairly happy with my part. Also, I extended the showing time of the final result.

    Correct on my method, though there are always tweaks that aren't strictly sequential.  It's pretty close to what actually happened.  It does look like you've put some of the luma changes at the end, starting at frame 113 in the animation.  I'm guessing that these are blocks that have both luma and chroma changes?  I'd put the 'hybrid' DC adjustments in with the rest of the luma pass so they fit in.  I'd fix chroma issues whenever they were making it hard to see if I had luma right, so a few chroma changes before the final pass isn't inaccurate. 

    SwissCheese may have an intermediary MMB for some data on the alignment.  I realigned the one leg tip and found a bit more data in the bottom, the work on the ocean was not mine. 


    ..And it case it hasn't been said enough:  WOW. 
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: wronkiew on 06/07/2014 05:50 pm
    Alternatively you can just grab the CSV from https://docs.google.com/spreadsheets/d/1xEtLQy3tvjv3cyz_wMQssuSFwHYVbMQ5-FdSz-uCsKA/export?gid=0&format=csv - this CSV is being autopublished by Google with something like 10 minute delay of the actual edit of the any of the parts' spreadseets.

    I can take another stab at pulling the data from the spreadsheets, probably tomorrow.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: wronkiew on 06/07/2014 05:52 pm
    Apparently the Wikispaces API doesn't allow file uploads.  >:(

    The auto-generated MMBs are uploaded here, in case you need to track down a bug:

    http://adama.nocdirect.com/~wronkiew/spx_crs3/mmbs/

    IainCole, can you change the youtube uploader to grab http://adama.nocdirect.com/~wronkiew/spx_crs3/landing_clockfix_3hr.mp4 ? I can then move the trigger to the 3hr script.

    Parts 3 and 4 are auto-generated correctly, but I think the video is still using the old fix8 files for those two parts, part 4 is currently cleaner than what's in the video.

    It is using sh_rawsplit_part_03_fixed_.ts and sh_rawsplit_part_04_fixed_.ts. I'm pretty sure that is the most recent segmented ts file. Does anyone know of a newer version?
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: SwissCheese on 06/07/2014 06:18 pm
    Wow that's great!  If you change this in the future can you add some frames to the end?  Ideally, repeats of the last frame, so that you get a chance to really see the end product would be awesome.

    Made a new gif that I think more accurately represents the evolution of this frame from total garbage to what we have now. I split the dc corrections into luma and chroma passes and applied them sequentially (I think this is Qualiss's method). Jumped over a lot of minute changes to reduce the size (it's still large) and some important changes (mostly in aligment) due to lack of data. Quialiss and Swisscheese, would you consider this a faithful representation of what you saw while working on this frame? I'm fairly happy with my part. Also, I extended the showing time of the final result.

    For me it shows nicely the reconstruction process, even if it was probably more chaotic in reality ;)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: saliva_sweet on 06/07/2014 06:35 pm
    Correct on my method, though there are always tweaks that aren't strictly sequential.  It's pretty close to what actually happened.  It does look like you've put some of the luma changes at the end, starting at frame 113 in the animation.  I'm guessing that these are blocks that have both luma and chroma changes?  I'd put the 'hybrid' DC adjustments in with the rest of the luma pass so they fit in.  I'd fix chroma issues whenever they were making it hard to see if I had luma right, so a few chroma changes before the final pass isn't inaccurate. 
    It's probably not due to blocks with both changes. More likely the dc inherit direction command is confusing my algorithm (the mmb is way too big to parse manually at this point). This mysterious (for me) command appears to be used on some occasions, at least some mmb commands contain 10 fields. I didn't know what to do with them so I mostly ignored their presence. When I saw my result was close to your style based on what you've shown of your work I was pretty happy.

    The original result containing all mmb changes was 315 frames this is without counting the changes made by Swisscheese, I left out the tiny changes to leave 129 frames. I think this illustrates the magnitude of the work done, because (at least for me) each change was a result of lots of deliberation, trial and error and scanning through hundreds of bitpositions. But I was smiling every time I got a few macroblocks out so It was all worth it. And that's just one frame out of the 300 odd frames that we got to work with.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Quialiss on 06/07/2014 07:20 pm
    Correct on my method, though there are always tweaks that aren't strictly sequential.  It's pretty close to what actually happened.  It does look like you've put some of the luma changes at the end, starting at frame 113 in the animation.  I'm guessing that these are blocks that have both luma and chroma changes?  I'd put the 'hybrid' DC adjustments in with the rest of the luma pass so they fit in.  I'd fix chroma issues whenever they were making it hard to see if I had luma right, so a few chroma changes before the final pass isn't inaccurate. 
    It's probably not due to blocks with both changes. More likely the dc inherit direction command is confusing my algorithm (the mmb is way too big to parse manually at this point). This mysterious (for me) command appears to be used on some occasions, at least some mmb commands contain 10 fields. I didn't know what to do with them so I mostly ignored their presence. When I saw my result was close to your style based on what you've shown of your work I was pretty happy.

    The original result containing all mmb changes was 315 frames this is without incorporating the changes made by Swisscheese, I left out the tiny changes to leave 129 frames. I think this illustrates the magnitude of the work done, because (at least for me) each change was a result of lots of deliberation, trial and error and scanning through hundreds of bitpositions. But I was smiling every time I got a few macroblocks out so It was all worth it. And that's just one frame out of the 300 odd frames that we got to work with.

    Ah, that makes sense. You can treat the 10 field commands that have direction to be part of the luma correction part and order them accordingly if you end up doing another pass.  Not a big concern, it's a tiny fraction of the frames, I normally just use it around gaps in the data. 

    I agree about it being a wonderful illustration of how much work went into each frame.  It's incredible to watch the i frame come together from the jumbled mess of data... and for me puts into context why so many luma fixes were required.  I remember thinking when I started trying to fix up the luma issues that I must be introducing new errors with not bit-perfect corrections* that I had to correct later in the frame, but when you lay out all the work required to get the macroblock structure correct, having so many errors in the subblocks starts seeming a little more reasonable.


    *Still likely, but I believe that there should statistically be as many errors resulting in 1 or 2 off errors as there are 4 and 8 off errors, bitflips being impartial about where in the subblock they're located, and I can't visually see the 1-2 errors, so they're bound to build up at least as much as my mistakes.  So I tell myself.    ;)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Quialiss on 06/07/2014 10:06 pm
    I'm going to have a very busy week starting Sunday and won't be able to work on this as much as I'd like to, so I figure I'd leave my notes here to give people ideas on what could still use some love still.  :) 

    I've also got a work in progress MMB for part 3 that I'm attaching that has a few alignment fixes from the version currently on the wiki, and increased contrast on all the p frames to help in alignment. 



    Notes!



    Part 3 and 4 need to be double checked for alignment issues
    Part 5  83-88 not done yet

    faulty luma/chroma in p frames
    46, 47, 49, 52, 55, 56, 59 (needs alignment checks first)
    65, 71, 72 (needs alignment checks first)
    95, 96
    104, 108
    139
    163, 167, 173, 175, 176
    186, 192, 194
    224

    Verify alignment around these points
    Two waves doing sharp turns at the start of a new p frame set at 102 and 122. top left, and between legs.

    201 - Rocket body at the bottom of the frame is too light in the i frame given all the p frames that agree on the luma?
    241 - this i frame could still use some attention
    281 - this i frame could still use some attention

    Match i frames in luma/chroma -- the rocket frequently changes colour between i frames.  Match to frames 10-11-12. 
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: arnezami on 06/08/2014 04:40 am
    Apparently the Wikispaces API doesn't allow file uploads.  >:(

    The auto-generated MMBs are uploaded here, in case you need to track down a bug:

    http://adama.nocdirect.com/~wronkiew/spx_crs3/mmbs/

    IainCole, can you change the youtube uploader to grab http://adama.nocdirect.com/~wronkiew/spx_crs3/landing_clockfix_3hr.mp4 ? I can then move the trigger to the 3hr script.
    When looking at the latest videos there seems to be something wrong with the automated video creation.

    For example. The frames 42-60 have been repaired lately. But they don't show up in the video. However I do see the mmb's in wronkiew's link (http://adama.nocdirect.com/~wronkiew/spx_crs3/mmbs/) to the mmbs.

    Also frame 281 has been improved lately but the old one still show up in the video, but it does show up in wronkiew's link (http://adama.nocdirect.com/~wronkiew/spx_crs3/mmbs/) to the mmbs. I haven't checked further. But there is clearly something not working quite right.

    I also don't understand this: the video does sometimes get updated now and then, yet it somehow still uses old mmb's. How can it use older AND newer mmbs? Does it keep it's own database of previous mmb's or something? I don't understand.

    Regards,

    arnezami
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: PaulNY on 06/08/2014 05:13 am
    Not 100% sure I did this right but I may of found 2 bit flips in I frame 7 (121) in the P frame editor.

    X:583:01,X:5386:01

    They seam to clear up the top row quite well and replace 4 MB edits. There does appear to still be errors in the top row though, less then the original frame but more then the current fix.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: mhenderson on 06/08/2014 08:00 am
    Apparently the Wikispaces API doesn't allow file uploads.  >:(

    The auto-generated MMBs are uploaded here, in case you need to track down a bug:

    http://adama.nocdirect.com/~wronkiew/spx_crs3/mmbs/

    IainCole, can you change the youtube uploader to grab http://adama.nocdirect.com/~wronkiew/spx_crs3/landing_clockfix_3hr.mp4 ? I can then move the trigger to the 3hr script.
    When looking at the latest videos there seems to be something wrong with the automated video creation.

    For example. The frames 42-60 have been repaired lately. But they don't show up in the video. However I do see the mmb's in wronkiew's link (http://adama.nocdirect.com/~wronkiew/spx_crs3/mmbs/) to the mmbs.

    Also frame 281 has been improved lately but the old one still show up in the video, but it does show up in wronkiew's link (http://adama.nocdirect.com/~wronkiew/spx_crs3/mmbs/) to the mmbs. I haven't checked further. But there is clearly something not working quite right.

    I also don't understand this: the video does sometimes get updated now and then, yet it somehow still uses old mmb's. How can it use older AND newer mmbs? Does it keep it's own database of previous mmb's or something? I don't understand.

    Regards,

    arnezami

    I concur.

    The latest changes are not in the .mp4 video that IainCole's process is picking up (at http://adama.nocdirect.com/~wronkiew/spx_crs3/landing_clockfix_3hr.mp4).
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: morningdew76 on 06/08/2014 02:20 pm
    Enhancing the contrast in P-frames with X:59:C0 / X:59:80 / X:59:60 / X:59:40 works great.

    Is there a way to do this for the I-frames too?

    It looks like this works for I-frames with X:545:C0 / X:545:80 / X:545:60 / X:545:40
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: cscott on 06/08/2014 03:40 pm
    *Still likely, but I believe that there should statistically be as many errors resulting in 1 or 2 off errors as there are 4 and 8 off errors, bitflips being impartial about where in the subblock they're located, and I can't visually see the 1-2 errors, so they're bound to build up at least as much as my mistakes.  So I tell myself.    ;)

    This is what I'm hoping the "triple bit flip" investigation helps with: it might allow us to insert "invisible but correct" lsb flips based on how the obvious msb flips fit into the triple flip pattern.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Klawd on 06/08/2014 06:32 pm
    Made a new gif that I think more accurately represents the evolution of this frame from total garbage to what we have now. ...

    The patience you guys must have is enviable.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: wronkiew on 06/08/2014 07:47 pm
    Apparently the Wikispaces API doesn't allow file uploads.  >:(

    The auto-generated MMBs are uploaded here, in case you need to track down a bug:

    http://adama.nocdirect.com/~wronkiew/spx_crs3/mmbs/

    IainCole, can you change the youtube uploader to grab http://adama.nocdirect.com/~wronkiew/spx_crs3/landing_clockfix_3hr.mp4 ? I can then move the trigger to the 3hr script.
    When looking at the latest videos there seems to be something wrong with the automated video creation.

    For example. The frames 42-60 have been repaired lately. But they don't show up in the video. However I do see the mmb's in wronkiew's link (http://adama.nocdirect.com/~wronkiew/spx_crs3/mmbs/) to the mmbs.

    Also frame 281 has been improved lately but the old one still show up in the video, but it does show up in wronkiew's link (http://adama.nocdirect.com/~wronkiew/spx_crs3/mmbs/) to the mmbs. I haven't checked further. But there is clearly something not working quite right.

    I also don't understand this: the video does sometimes get updated now and then, yet it somehow still uses old mmb's. How can it use older AND newer mmbs? Does it keep it's own database of previous mmb's or something? I don't understand.

    Frame 42 had some creative formatting that was confusing the scraper. I improved the scraper and it is fixed now.

    The generated mmbs always match the video here:

    http://adama.nocdirect.com/~wronkiew/spx_crs3/landing_base_set.mp4

    landing_clockfix_3hr.mp4 is updated on a different schedule from the previously generated files.

    I do not see any problems with frame 281, and it is being scraped correctly.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: arnezami on 06/08/2014 08:17 pm
    Frame 42 had some creative formatting that was confusing the scraper. I improved the scraper and it is fixed now.

    The generated mmbs always match the video here:

    http://adama.nocdirect.com/~wronkiew/spx_crs3/landing_base_set.mp4

    landing_clockfix_3hr.mp4 is updated on a different schedule from the previously generated files.

    I do not see any problems with frame 281, and it is being scraped correctly.
    Ok. Great! That video now seems to be correct and up-to-date.

    I was looking at the youtube videos (https://www.youtube.com/channel/UCyZDgyJBYz3OXD3JbDJzNww) which are not in sync with your latest video. I thought they were somehow synced.

    Should we change the link/widget on the wiki page (http://spacexlanding.wikispaces.com/raw_final_fixedMMB)?
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: IainCole on 06/08/2014 08:20 pm
    Frame 42 had some creative formatting that was confusing the scraper. I improved the scraper and it is fixed now.

    The generated mmbs always match the video here:

    http://adama.nocdirect.com/~wronkiew/spx_crs3/landing_base_set.mp4

    landing_clockfix_3hr.mp4 is updated on a different schedule from the previously generated files.

    I do not see any problems with frame 281, and it is being scraped correctly.
    Ok. Great! That video now seems to be correct and up-to-date.

    I was looking at the youtube videos (https://www.youtube.com/channel/UCyZDgyJBYz3OXD3JbDJzNww) which are not in sync with your latest video. I thought they were somehow synced.

    Should we change the link/widget on the wiki page (http://spacexlanding.wikispaces.com/raw_final_fixedMMB)?

    The youtube videos are now on the 3hr schedule to prevent so much spam :)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: arnezami on 06/08/2014 08:25 pm
    The youtube videos are now on the 3hr schedule to prevent so much spam :)
    I don't think you understand. Among the other things I mentioned frame 281 has been fixed for days. Yet you still can't see it on the youtube videos.

    So this difference cannot be due to a 3 hour delay.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: IainCole on 06/08/2014 08:28 pm
    The youtube videos are now on the 3hr schedule to prevent so much spam :)
    I don't think you understand. Among the other things I mentioned frame 281 has been fixed for days. Yet you still can't see it on the youtube videos.

    So this difference cannot be due to a 3 hour delay.

    Hmm... youtube has rejected a few recent uploads due to them being "duplicates", are we sure the video here: http://adama.nocdirect.com/~wronkiew/spx_crs3/landing_clockfix_3hr.mp4 is being updated?

    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: arnezami on 06/08/2014 08:35 pm
    Hmm... youtube has rejected a few recent uploads due to them being "duplicates", are we sure the video here: http://adama.nocdirect.com/~wronkiew/spx_crs3/landing_clockfix_3hr.mp4 is being updated?

    landing_clockfix_3hr.mp4 is updated on a different schedule from the previously generated files.

    Aha. I see that landing_clockfix_3hr.mp4 has not been updates for quite some time. ;)  That's the problem.

    So you might wanna use landing_base_set.mp4 for now as the source. Until landing_clockfix_3hr.mp4 is updated again.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: IainCole on 06/08/2014 08:35 pm
    Hmm... youtube has rejected a few recent uploads due to them being "duplicates", are we sure the video here: http://adama.nocdirect.com/~wronkiew/spx_crs3/landing_clockfix_3hr.mp4 is being updated?

    landing_clockfix_3hr.mp4 is updated on a different schedule from the previously generated files.

    Aha. I see that landing_clockfix_3hr.mp4 has not been updates for quite some time. ;)  That's the problem.

    So you might wanna use landing_base_set.mp4 for now as the source. Until landing_clockfix_3hr.mp4 is updated again.

    OK I will change it and trigger it manually now, then we can work out what's going on :)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: arnezami on 06/08/2014 08:47 pm
    OK I will change it and trigger it manually now, then we can work out what's going on :)

    Yes. The new youtube video contains the latest repaired frames now.

    https://www.youtube.com/watch?v=-oU3knnvj6M

    Thank you.

    arnezami
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: saliva_sweet on 06/08/2014 08:58 pm
    Looking at the video I'm not sure I'm personally a big fan of the latest contrast boost to 281. I think the previous version is more correct.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: wronkiew on 06/08/2014 10:08 pm
    Hmm... youtube has rejected a few recent uploads due to them being "duplicates", are we sure the video here: http://adama.nocdirect.com/~wronkiew/spx_crs3/landing_clockfix_3hr.mp4 is being updated?

    landing_clockfix_3hr.mp4 is updated on a different schedule from the previously generated files.

    Aha. I see that landing_clockfix_3hr.mp4 has not been updates for quite some time. ;)  That's the problem.

    So you might wanna use landing_base_set.mp4 for now as the source. Until landing_clockfix_3hr.mp4 is updated again.

    Yes indeed. There was a communication problem between the two jobs. Now fixed.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Chris Bergin on 06/08/2014 11:18 pm
    It seems to me you can see two stages of leg deploy. One the actual deploy (I assume unlatch/deploy) and two the stiffening. Maybe one or two stages of press to prepare them to take the load of the core upon landing.

    Becoming really clear in later iterations.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: mhenderson on 06/08/2014 11:55 pm
    @Chris Bergin - Uummm. There is a separate thread for interpretation, but you are right. Those legs wobble like a newborn foal's. I have thought of the second deploy as evidence of changes to the aerodynamic load. This baby is falling faster than Dorothy's house.

    @saliva_sweet - I defer to your judgement on frame 281. For some unknown reason, the top two rows no longer show in the video. They both contain mostly good MMBs (with high contrast). I did not remove them. I definitely don't like the blue caste at the bottom third or the awful rectangle at the lower right. I put my efforts in the wiki to get more skilled eyes on it.

    To roll back, I left the prior MMB that I replaced in the comment field of the wiki page. Personally, I thought it was inconsistent with the prior frame ... very washed out. As a result, the video ended on an unsatisfying hazy image. 
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: deruch on 06/09/2014 02:09 am
    @Chris Bergin - Uummm. There is a separate thread for interpretation...
    /snip/

    Burn. ;D
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: deruch on 06/09/2014 02:17 am
    /snip/
    Personally, I thought it was inconsistent with the prior frame ... very washed out. As a result, the video ended on an unsatisfying hazy image. 
    Am I the only one who just realized that the end frames are probably all washed out because the booster just landed in the middle of a steam bath?  The exhaust + the hot bell and legs must have kicked up a lot of steam.  I was thinking it was garbling artifact, but it's probably not.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Chris Bergin on 06/09/2014 02:23 am
    @Chris Bergin - Uummm. There is a separate thread for interpretation...
    /snip/

    Burn. ;D

    Hate it when that happens. I hope someone reports that Chris character to the moderators. ;D
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Lar on 06/09/2014 04:53 am
    @Chris Bergin - Uummm. There is a separate thread for interpretation...
    /snip/

    Burn. ;D

    Hate it when that happens. I hope someone reports that Chris character to the moderators. ;D

    Just like cops having some fun in their patrol cars, Chris gets to speed with the sirens off from time to time....
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: mhenderson on 06/09/2014 10:01 am
    /snip/
    Personally, I thought it was inconsistent with the prior frame ... very washed out. As a result, the video ended on an unsatisfying hazy image. 
    Am I the only one who just realized that the end frames are probably all washed out because the booster just landed in the middle of a steam bath?  The exhaust + the hot bell and legs must have kicked up a lot of steam.  I was thinking it was garbling artifact, but it's probably not.

    Possible, but then how do we explain the fine detail in the recovered frame? If this image was captured in a fog, the details of the distant legs and the rim of the rocket body would never be this crisp.  IMHO.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Ohsin on 06/09/2014 11:28 am
    /snip/
    Personally, I thought it was inconsistent with the prior frame ... very washed out. As a result, the video ended on an unsatisfying hazy image. 
    Am I the only one who just realized that the end frames are probably all washed out because the booster just landed in the middle of a steam bath?  The exhaust + the hot bell and legs must have kicked up a lot of steam.  I was thinking it was garbling artifact, but it's probably not.
    Possible, but then how do we explain the fine detail in the recovered frame? If this image was captured in a fog, the details of the distant legs and the rim of the rocket body would never be this crisp.  IMHO
    I think sudden clarity in the end frames could be due to many little things like change in exposure as camera adjusts to lighting , steam and it effect on lighting(making it more uniform) and as plume hits water
    water reflecting all light onto stage body making fine details clearer.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: wronkiew on 06/09/2014 03:23 pm
    YouTube hasn't been updated in a while. IainCole, can you turn auto-generation back on?
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: IainCole on 06/09/2014 05:57 pm
    YouTube hasn't been updated in a while. IainCole, can you turn auto-generation back on?

    I switched it back to 3hr and it's still saying it's duplicate. I'm going to move it back to 1hr again
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: IainCole on 06/09/2014 06:00 pm
    YouTube hasn't been updated in a while. IainCole, can you turn auto-generation back on?

    I switched it back to 3hr and it's still saying it's duplicate. I'm going to move it back to 1hr again

    Hmm... the current 1hr is saying it's a duplicate of this too https://www.youtube.com/watch?v=-oU3knnvj6M
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: morningdew76 on 06/09/2014 06:30 pm
    YouTube hasn't been updated in a while. IainCole, can you turn auto-generation back on?

    I switched it back to 3hr and it's still saying it's duplicate. I'm going to move it back to 1hr again

    Maybe the video isn't "different enough" for Youtube to allow it.

    One way to increase the difference would be to alter the title sheet to show a time and date, the bigger the font, the better.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: moralec on 06/09/2014 06:42 pm
    According to https://support.google.com/youtube/answer/58139?hl=en, changing the compression could also help.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: wronkiew on 06/09/2014 07:25 pm
    YouTube hasn't been updated in a while. IainCole, can you turn auto-generation back on?

    I switched it back to 3hr and it's still saying it's duplicate. I'm going to move it back to 1hr again

    Ok, never mind. The last MMB change was 33 hours ago. I've made clock fixes since then, but those don't currently trigger the uploader.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: mhenderson on 06/09/2014 08:06 pm
    The last MMB change was 33 hours ago. /snip/

    Hmmm. Does that mean we are
    a) Done?
    b) Losing interest?
    c) Too busy with our lives?
    d) Running out of easy fixes and needing new tools (bitflip automation)?
    e) In a summer slump?
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: princess on 06/09/2014 08:18 pm
    Hmmm. Does that mean we are

    f. Busy on the error detection files!

    Over the weekend I've been changing the raw.ts file so that all the fixes we found in raw_final_fixed.ts are incorporated. It turned out to be a LOT harder than I expected, but I believe I have it now. The raw_final_fixed.ts actually had a run of 30456 bytes which was a duplicate of an earlier section within the file, so it didn't line up with raw.ts. Finding that one and fixing it was a great amount of fun...

    I'm going to post two files, and because of the 5Mb limit I'll have to use two posts to do it. The first file is raw_aligned.ts. This is basically SpaceX's raw.ts with the following changes:

    1. At offset 0x382a8, insert 56 bytes and save the file.
    2. At offset 0xd7250, insert 56 bytes and save the file.
    3. At offset 0x215d4c, insert 56 bytes and save the file.
    4. At offset 0x3571ec, insert 56 bytes and save the file.
    5. At offset 0x3dc0ac, insert 56 bytes and save the file.

    To do the insertion I simply copied the range after the insertion point 56 bytes forward, thus leaving a 56-byte duplicate at each insertion point. The result is a 4481732 byte long file with MD5 checksum 0821a52d195f9566ef71921fd777bc14. The insertions were done to re-align the TS stream with the 188-byte packet offsets.

    (continued in second post)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: princess on 06/09/2014 08:26 pm
    Here is my second file, rocket.ts. As the name suggests it's my best guess at what the rocket actually transmitted. It was made from 3 main parts:

    1. 1355 TS packets from raw_aligned.ts at the start. This is the 16 P-frames that were transmitted before the first I-frame.
    2. The raw_final_fixed.ts file was then appended to these packets.
    3. From offset 0x3e03b8 onwards, the contents of raw_aligned.ts were substituted instead so that the duplicate data is removed.

    The resulting file is 4481732 bytes long and aligns byte-for-byte with raw_aligned.ts. The file was then fixed up further (as far as I could) so that it's as good as I can reasonably get it without making too many blind guesses. The file is good enough to play in recent versions of VLC.

    The aim is that people can now compare raw_aligned.ts with rocket.ts, and any differences are bitflip errors. Please take into account the insertions done in raw_aligned.ts if this affects the error pattern, but as the error block size appears to be 56 bytes anyway then it should hopefully not matter.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: cscott on 06/09/2014 08:32 pm
    The aim is that people can now compare raw_aligned.ts with rocket.ts, and any differences are bitflip errors. Please take into account the insertions done in raw_aligned.ts if this affects the error pattern, but as the error block size appears to be 56 bytes anyway then it should hopefully not matter.

    Fantastic!

    Just checking my understanding -- the MMB fixes are *not* included in rocket.ts, right?  So the bitflip errors found when comparing rocket.ts with raw_aligned.ts are just the transport stream errors.

    What I'd like to do is to take our "best attempt at a fix" for a specific iframe, convert it back into the .ts format, and then start looking at thost bitflips, to see if we've inadvertently "rediscovered" some triple flip patterns.  If we have, this would make me more confident in the "most errors are triple flips" hypothesis.

    (I was hoping to start doing some ts-file-parser hacking this weekend, but family stuff took precedence...)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: princess on 06/09/2014 08:40 pm
    Just checking my understanding -- the MMB fixes are *not* included in rocket.ts, right?  So the bitflip errors found when comparing rocket.ts with raw_aligned.ts are just the transport stream errors.

    That's right, I've not included any MMB fixes in rocket.ts. The aim is that the bitflips present are ones that we're fairly confident of, and also that enough are present that we can discern any patterns. We definitely don't have the entire list - otherwise the resultant video would be perfect! :)

    What I'd like to do is to take our "best attempt at a fix" for a specific iframe, convert it back into the .ts format, and then start looking at thost bitflips, to see if we've inadvertently "rediscovered" some triple flip patterns.  If we have, this would make me more confident in the "most errors are triple flips" hypothesis.

    As far as I know there are a lot of MMB fixes that don't map directly back to the TS file, they're playing tricks in ffmpeg - for example, I think that redirecting a block to inherit from a different one isn't possible in MPEG4.

    Also, the MMB fixes are done visually and not at the bitstream level. While the resulting frames look very good to the human eye, they're not what the rocket transmitted and can't be used for error detection purposes.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: moralec on 06/09/2014 08:42 pm
    Amazing work princess!!

    Ok so how do you think we should proceed?
    Should we  go ahead and subsitute raw_final_fixed.ts with one of your newer versions? If that´s the case, do you think the existing MMB codes will still work?
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: cscott on 06/09/2014 08:48 pm
    As far as I know there are a lot of MMB fixes that don't map directly back to the TS file, they're playing tricks in ffmpeg - for example, I think that redirecting a block to inherit from a different one isn't possible in MPEG4.

    Also, the MMB fixes are done visually and not at the bitstream level. While the resulting frames look very good to the human eye, they're not what the rocket transmitted and can't be used for error detection purposes.

    Yup.  So, concretely what I'd like to begin by doing is taking a "lightly damaged" iframe (one without a lot of MMB fixes) and trying to manually figure out a sequence of triple flips which would recreate the effect of the MMB fixes.

    In theory this could be done semi-automatically -- taking the manual .png output of our work and using it as the "target" of a brute force search that only does triple flips.  Basically looking for the "pure triple flip" sequence which best matches our "by eye" fixes.  Dunno if that search space is reasonable yet.  Like I said, the first step is to try to do a small sample manually to get a feel for the issues.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: princess on 06/09/2014 08:58 pm
    Amazing work princess!!

    Ok so how do you think we should proceed?
    Should we  go ahead and subsitute raw_final_fixed.ts with one of your newer versions? If that´s the case, do you think the existing MMB codes will still work?

    Why thank you moralec! No, I don't think we need to replace the TS file that people are using for the video recovery. It's good enough and the new file only makes a few data-related changes - mainly the extra P-frames at the start (which have no I-frame to reference to), and the duplicate run in P-frames 270/271.

    It looks like this duplicate area was put in because the start of frame 270 is just atrociously corrupted, so in the rocket.ts file I actually don't have the start of P-frame 270 as I honestly can't decide where to put it. I'd advise the MMB people that 270 might not start in the right place, and 269 might have junk at the end. It seems as though the duplicate bytes were inserted in order to use some of 271's data to shore up 270 in raw_final_fixed.ts, so this area of the video might pause slightly.

    In summary, the rocket.ts file is mainly for error detection but also might be of academic interest, or might be useful for people to attempt a more thorough recovery of difficult frames by working from the raw MPEG4 stream using ffmpeg.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: cscott on 06/09/2014 09:01 pm
    Ok so how do you think we should proceed?
    Should we  go ahead and subsitute raw_final_fixed.ts with one of your newer versions? If that´s the case, do you think the existing MMB codes will still work?
    Why thank you moralec! No, I don't think we need to replace the TS file that people are using for the video recovery. [...] In summary, the rocket.ts file is mainly for error detection but also might be of academic interest, or might be useful for people to attempt a more thorough recovery of difficult frames by working from the raw MPEG4 stream using ffmpeg.

    I will presumably have to figure out how to translate MMBs from raw_final_fixed.ts to rocket.ts in order to pursue the triple-flip research I want to do, so if it comes to it I might be able to help translate stuff.  But that's a ways in the future yet.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: princess on 06/09/2014 09:11 pm
    I will presumably have to figure out how to translate MMBs from raw_final_fixed.ts to rocket.ts in order to pursue the triple-flip research I want to do, so if it comes to it I might be able to help translate stuff.  But that's a ways in the future yet.

    Well the "X" parts of the MMB command will map directly back, as that's just an XOR command on a direct offset. There may be other ones that can map back - maybe if you change the luma or chroma enough to not need a new luma_size or chroma_size field? Michaelni would be able to help here.

    As for mappings - I can provide mappings between the MPEG4 bit number and the TS file offset. You would apply the MMBs to rocket.ts, and leave raw_aligned.ts alone. The idea is this:

     - raw_aligned.ts represents what was received in Elon's plane, properly aligned to MPEG TS packet boundaries. It's the equivalent of forensic evidence in the investigation, and should remain as untouched as we possibly can.

     - rocket.ts represents the group's best idea of what was actually transmitted from the rocket's antenna. As we're initially using it for error pattern analysis I've tried to be conservative in the bitflips - if I wasn't sure of one, I've left it as it is in raw_aligned.ts. As we become more confident of other bitflips we would accumulate them in this file.

    I hope that clears things up!

    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Quialiss on 06/09/2014 09:55 pm
    The last MMB change was 33 hours ago. /snip/

    Hmmm. Does that mean we are
    c) Too busy with our lives?

    I've been working on part 10 fixing up some of the odd errors in the p frames, good work to do in ten minute stints.  They're done-ish now and the sequence looks a bit cleaner, so up on the wiki they go. 

    Curious notes:  The issues where many blocks would be off in luma are frequently fixed by making sure to -3 data that LOOKS empty but isn't (sizes of <10).  They frequently don't show up as errors, but they can confuse things a lot.  An error at the top of the frame can start screwing up blocks... at the bottom, leaving the majority of the blocks in the middle looking perfect.  But sometimes there doesn't seem to be an obvious error and I end up fixing all the sections individually. 
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: moralec on 06/09/2014 10:12 pm
    Amazing work princess!!

    Ok so how do you think we should proceed?
    Should we  go ahead and subsitute raw_final_fixed.ts with one of your newer versions? If that´s the case, do you think the existing MMB codes will still work?

    Why thank you moralec! No, I don't think we need to replace the TS file that people are using for the video recovery. It's good enough and the new file only makes a few data-related changes - mainly the extra P-frames at the start (which have no I-frame to reference to), and the duplicate run in P-frames 270/271.

    It looks like this duplicate area was put in because the start of frame 270 is just atrociously corrupted, so in the rocket.ts file I actually don't have the start of P-frame 270 as I honestly can't decide where to put it. I'd advise the MMB people that 270 might not start in the right place, and 269 might have junk at the end. It seems as though the duplicate bytes were inserted in order to use some of 271's data to shore up 270 in raw_final_fixed.ts, so this area of the video might pause slightly.

    In summary, the rocket.ts file is mainly for error detection but also might be of academic interest, or might be useful for people to attempt a more thorough recovery of difficult frames by working from the raw MPEG4 stream using ffmpeg.

    Speaking of academic interest, I think we may want to start posting all these files in the wiki, and work on a detailed description of what we did. We don't need to do it now,  be it may be convenient to do it sooner than later.... what do you think about this?  Should I open a page just for this purpose? Or do you prefer to leave this for the end of the process?

    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: princess on 06/09/2014 10:32 pm
    Speaking of academic interest, I think we may want to start posting all these files in the wiki, and work on a detailed description of what we did. We don't need to do it now,  be it may be convenient to do it sooner than later.... what do you think about this?  Should I open a page just for this purpose? Or do you prefer to leave this for the end of the process?

    A wiki page is always helpful for people to get a summary of an issue, and also to find the files without having to wade through the thread here. I've often put links to files in the wiki just so I can find them again!

    So yes, if in doubt wiki it. We can always edit and tidy things up in the future, and it helps to write things down while they're fresh in people's minds.

    As for the files - raw_aligned.ts should remain pristine and untouched unless someone can prove one of the insertions needs moving, whereas rocket.ts will almost certainly have many revisions as we find bitflips. The thing is that rocket.ts will always give a worse video than using MMB commands, so working on it won't give impressive visual results. However, improving it will give us more confidence in our error pattern analysis.

    Lastly, maybe I'm strange but I like to think that recovering rocket.ts is getting us closer to knowing the "last words" of a dying rocket stage. It did its job perfectly, finally achieving what Elon and his team had hoped for it, and was desperate to tell the world, frantically transmitting the news of its success before bravely succumbing to the stormy seas. I may be anthropomorphising things too much, but I think we owe it to that tenacious machine to reconstruct its last historic message to the world.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Quialiss on 06/09/2014 10:41 pm
    It looks like this duplicate area was put in because the start of frame 270 is just atrociously corrupted, so in the rocket.ts file I actually don't have the start of P-frame 270 as I honestly can't decide where to put it. I'd advise the MMB people that 270 might not start in the right place, and 269 might have junk at the end. It seems as though the duplicate bytes were inserted in order to use some of 271's data to shore up 270 in raw_final_fixed.ts, so this area of the video might pause slightly.

    IIRC, SwissCheese and I worked on those frames.. from our notes on the wiki, 269 and 270 were completely unrecoverable in fix8 version, and some data became recoverable in the raw_final_fixed.ts, though they're still missing large chunks of data.  But they don't appear to pause(outside of what's caused by missing a third of a frame) and the timestamps all look correctly recovered.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: princess on 06/09/2014 10:52 pm
    IIRC, SwissCheese and I worked on those frames.. from our notes on the wiki, 269 and 270 were completely unrecoverable in fix8 version, and some data became recoverable in the raw_final_fixed.ts, though they're still missing large chunks of data.  But they don't appear to pause(outside of what's caused by missing a third of a frame) and the timestamps all look correctly recovered.

    The TS timestamps for 270 were altered to make the frame look correct, but I don't know what happened with the MPEG4 stream.

    Basically at that point in the data the stream was all fine with no errors, and then suddenly from around offset 0x003dbb04 to offset 0x003dd284 there was a huge burst of corruption that knocked out 32 TS packets. If you look at it with a TS analyser it's very bad, just totally mangled.

    Anyway, the boundary between 269 and 270 occurred in the middle of this. Usually the corruption is light enough that you can kind of eyeball the closest packet, or break out the Hamming-distance analysis, but in this case it's useless. Whoever did the byte duplicate in raw_final_fixed.ts was basically just pasting the start of 271 into this and resetting the timestamp to be correct. It's the TS equivalent of saying "this macroblock is shot to pieces, use one that looks similar".

    So yes, this will allow extra data to be recovered - the good macroblocks in 270 after the corruption burst finished. But be aware that the end of 269 and the start of 270 are mangled beyond recovery.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: rocketguy101 on 06/09/2014 11:28 pm
    Lastly, maybe I'm strange but I like to think that recovering rocket.ts is getting us closer to knowing the "last words" of a dying rocket stage. It did its job perfectly, finally achieving what Elon and his team had hoped for it, and was desperate to tell the world, frantically transmitting the news of its success before bravely succumbing to the stormy seas. I may be anthropomorphising things too much, but I think we owe it to that tenacious machine to reconstruct its last historic message to the world.
    I love this description!!!  Around here we refer to Space Shuttles as "she" so I think you hit it perfectly!  BTW I stand in awe of your work here, I don't understand 1/10th of what you guys do, but I appreciate your passion and enjoy seeing the fruits of your labors...
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: moralec on 06/10/2014 02:05 am
    IIRC, SwissCheese and I worked on those frames.. from our notes on the wiki, 269 and 270 were completely unrecoverable in fix8 version, and some data became recoverable in the raw_final_fixed.ts, though they're still missing large chunks of data.  But they don't appear to pause(outside of what's caused by missing a third of a frame) and the timestamps all look correctly recovered.

    The TS timestamps for 270 were altered to make the frame look correct, but I don't know what happened with the MPEG4 stream.

    Basically at that point in the data the stream was all fine with no errors, and then suddenly from around offset 0x003dbb04 to offset 0x003dd284 there was a huge burst of corruption that knocked out 32 TS packets. If you look at it with a TS analyser it's very bad, just totally mangled.

    Anyway, the boundary between 269 and 270 occurred in the middle of this. Usually the corruption is light enough that you can kind of eyeball the closest packet, or break out the Hamming-distance analysis, but in this case it's useless. Whoever did the byte duplicate in raw_final_fixed.ts was basically just pasting the start of 271 into this and resetting the timestamp to be correct. It's the TS equivalent of saying "this macroblock is shot to pieces, use one that looks similar".

    So yes, this will allow extra data to be recovered - the good macroblocks in 270 after the corruption burst finished. But be aware that the end of 269 and the start of 270 are mangled beyond recovery.

    Lighting?
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: aero on 06/10/2014 03:54 am
    Lightning.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: mhenderson on 06/10/2014 07:00 am
    Relevant chatter?

    Lightning or otherwise, I imagine there's a significant buildup of electric potential on / around a falling rocket that is re-igniting engines. Rockets falling from space are enveloped in a ball of plasma. The atmosphere separates ionic charges into layers. Are the igniters piezo-electric like my grill?

    ** Even in good weather ** discharges are possible and perhaps likely to occur again. The techniques developed here may become standard operating procedure.

    Is it valid to cut / paste data from adjacent frames to allow the codec to continue?

    IMHO, absolutely yes. That is a necessary part of the restoration. Let the brave little booster sing. 
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: wronkiew on 06/10/2014 07:10 am
    The clockfix script now tracks changes, so the YouTubifier will be notified when landing_clockfix_3hr.mp4 is updated, whether by mmb or by clock. We currently have 73 frames of clock fixes.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: wronkiew on 06/10/2014 07:22 am
    Sorry, there was a message between 1520 and 1521 that I deleted before I saw mhenderson's response.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: princess on 06/10/2014 07:31 am
    Is it valid to cut / paste data from adjacent frames to allow the codec to continue?

    IMHO, absolutely yes. That is a necessary part of the restoration. Let the brave little booster sing.

    Yes, I agree. For raw_final_fixed.ts we were making the file to attempt a visual recovery of the video, so getting the right number of frames is important, meaning that copy/paste of frame starts is fine.

    For rocket.ts the aim is slightly different in that we're attempting to recover the raw bitstream, not the video - this means that copy/paste like that is absolutely not allowed, which is why it took me so long to get the files created and posted up here. Hopefully now people can start working on error analysis which should have some interesting results!

    My best guess at what will happen is this: the TS file seems to have areas where the corruption is light, and then burst corruption that mangles a couple of packets completely. In the light corruption areas, triple bitflips seem to dominate. So we can use the error analysis firstly to determine which areas of the file have "light" corruption, and then we can search in those areas for triple bitflips that restore more data.

    We'll end up with a TS file that has heavy burst errors, but in between these bursts the file will be pretty much bit-perfect and the video data from those regions should be totally smooth. If we're lucky to hit an I-frame with the triple bitflips then the whole of the P-frame sequence will look a lot better too.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Shanuson on 06/10/2014 07:55 am
    IIRC, SwissCheese and I worked on those frames.. from our notes on the wiki, 269 and 270 were completely unrecoverable in fix8 version, and some data became recoverable in the raw_final_fixed.ts, though they're still missing large chunks of data.  But they don't appear to pause(outside of what's caused by missing a third of a frame) and the timestamps all look correctly recovered.

    The TS timestamps for 270 were altered to make the frame look correct, but I don't know what happened with the MPEG4 stream.

    Basically at that point in the data the stream was all fine with no errors, and then suddenly from around offset 0x003dbb04 to offset 0x003dd284 there was a huge burst of corruption that knocked out 32 TS packets. If you look at it with a TS analyser it's very bad, just totally mangled.

    Anyway, the boundary between 269 and 270 occurred in the middle of this. Usually the corruption is light enough that you can kind of eyeball the closest packet, or break out the Hamming-distance analysis, but in this case it's useless. Whoever did the byte duplicate in raw_final_fixed.ts was basically just pasting the start of 271 into this and resetting the timestamp to be correct. It's the TS equivalent of saying "this macroblock is shot to pieces, use one that looks similar".

    So yes, this will allow extra data to be recovered - the good macroblocks in 270 after the corruption burst finished. But be aware that the end of 269 and the start of 270 are mangled beyond recovery.

    I think i wrote what I did when i published raw_final_fixed.ts (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1205733#msg1205733). The problem was not only that one could not identify the exact Ts-Packet where the new frame started, it was also one of these 56 byte missing parts in there too. My best idea to "repair" this was to duplicate the whole part, and cut away some packets at the end of the first duplicate to make the CC continuous, and change the P-frame header of the 2nd duplicate to the one of the second frame. In that way the first frame has data from the 2nd at the end that should be pushed out by MMBs and the 2nd frame has the right data at the end but one has to find where its good data really starts. So one has to work from the end backwards when mmb-ing it.
    But it should be frames 254 and 255 where this happened.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: princess on 06/10/2014 08:22 am
    But it should be frames 254 and 255 where this happened.

    Ah yes, good point! When I was working out which frames it occurred in, I forgot that raw_final_fixed.ts hasn't got the first 15 P-frames before the I-frame. The corruption is in 269/270 in rocket.ts and 254/255 in raw_final_fixed.ts.

    Thank you for the correction Shanuson!
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: SwissCheese on 06/10/2014 10:25 am
    But it should be frames 254 and 255 where this happened.

    Ah yes, good point! When I was working out which frames it occurred in, I forgot that raw_final_fixed.ts hasn't got the first 15 P-frames before the I-frame. The corruption is in 269/270 in rocket.ts and 254/255 in raw_final_fixed.ts.

    Thank you for the correction Shanuson!

    Yep, frames 254 and 255 are the same, contain both the 2 p-frames, and are quite a big mess ;)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: SwissCheese on 06/10/2014 10:46 am
    I had to correct a little bit i frame 121 to align it to the p-frames. The problem is that the colors are wrong now and I'm not good at correcting them... Could someone recover nicer colors?

    The new mmb is in attachment, I haven't posted it to the wiki yet.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: saliva_sweet on 06/10/2014 11:16 am
    I had to correct a little bit i frame 121 to align it to the p-frames. The problem is that the colors are wrong now and I'm not good at correcting them... Could someone recover nicer colors?

    The new mmb is in attachment, I haven't posted it to the wiki yet.

    If dcs need to be redone may I suggest including the changes I posted here:
    http://forum.nasaspaceflight.com/index.php?topic=34597.msg1209819#msg1209819
    It's a couple of xors to 0:0 macroblock (ajmartins triple flip) that I think restores it to the original, it might give a better foundation to the overall color of the frame and make it fit better into the sequence. but it looks like it breaks the dcs much more than Swisscheeses change so might not be worth the effort.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Shanuson on 06/10/2014 11:37 am
    But it should be frames 254 and 255 where this happened.

    Ah yes, good point! When I was working out which frames it occurred in, I forgot that raw_final_fixed.ts hasn't got the first 15 P-frames before the I-frame. The corruption is in 269/270 in rocket.ts and 254/255 in raw_final_fixed.ts.

    Thank you for the correction Shanuson!

    Yep, frames 254 and 255 are the same, contain both the 2 p-frames, and are quite a big mess ;)

    Yes to disentangle them will be really hard. Good luck with that ;) The best way would be to identify the part where the first timestamp is. if we know this the next frame should start in the next few ts-packets. Everything after this should then be part of the second frame.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Quialiss on 06/10/2014 02:18 pm
    I had to correct a little bit i frame 121 to align it to the p-frames. The problem is that the colors are wrong now and I'm not good at correcting them... Could someone recover nicer colors?

    The new mmb is in attachment, I haven't posted it to the wiki yet.

    If dcs need to be redone may I suggest including the changes I posted here:
    http://forum.nasaspaceflight.com/index.php?topic=34597.msg1209819#msg1209819
    It's a couple of xors to 0:0 macroblock (ajmartins triple flip) that I think restores it to the original, it might give a better foundation to the overall color of the frame and make it fit better into the sequence. but it looks like it breaks the dcs much more than Swisscheeses change so might not be worth the effort.

    Curiously, combined, the changes do less 'damage' to the frame than they do apart.  I'll see what I can whip up this afternoon. 
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: princess on 06/10/2014 05:03 pm
    I've been playing with the error analysis a bit and I have a few mildly interesting new ways to visualise the bitflips.

    The first is a file that is the XOR of raw_aligned.ts and rocket.ts - a 1 bit represents a bitflip, and a zero bit represents an uncorrupted bit. The game is to try to find if there's any predictability in the 1s. The file is attached to this message.

    The second is that I've made a series of images representing the bitflips. Each pixel represents a bit, and each image line represents 56 bytes of the data file. A black pixel means there's no error, a red pixel means a transmission error set a bit to 1 when it should be 0, and a green pixel means a transmission error cleared a bit to 0 when it should be 1. I've attached the first few images directly here so you can see what it looks like, then I've also included a ZIP file with all of the images in.

    Let's see if we can determine any patterns...
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: saliva_sweet on 06/10/2014 05:23 pm
    I've been playing with the error analysis a bit and I have a few mildly interesting new ways to visualise the bitflips.
    Most interesting. Do I understand correctly that the green areas are padding that should be xFFs and the diagonal lines of flips are TS packet headers (that you have data for)? Most of these regions look to be in absolutely abysmal condition (except maybe the last picture). Surely this can't be representative of the whole file. Or is it?

    Edit: Also, would it be possible to make separate pics for every frame?
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Quialiss on 06/10/2014 06:44 pm
    I've been playing with the error analysis a bit and I have a few mildly interesting new ways to visualise the bitflips.

    The first is a file that is the XOR of raw_aligned.ts and rocket.ts - a 1 bit represents a bitflip, and a zero bit represents an uncorrupted bit. The game is to try to find if there's any predictability in the 1s. The file is attached to this message.

    The second is that I've made a series of images representing the bitflips. Each pixel represents a bit, and each image line represents 56 bytes of the data file. A black pixel means there's no error, a red pixel means a transmission error set a bit to 1 when it should be 0, and a green pixel means a transmission error cleared a bit to 0 when it should be 1. I've attached the first few images directly here so you can see what it looks like, then I've also included a ZIP file with all of the images in.

    Let's see if we can determine any patterns...

    Well, there's the one pattern that screams out, the periodic short runs of bitflips in otherwise okay data.  Strong peak at ~184 bits between these corrections, with 5 resonant peaks due to occasional gaps in the pattern. 

    The length and content of those short corrections also probably has a pattern, which might be valuable for filling in those predictable gaps. 

    Data's a csv, but the forum doesn't appreciate those.  txt it is. 
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: princess on 06/10/2014 07:35 pm
    Most interesting. Do I understand correctly that the green areas are padding that should be xFFs and the diagonal lines of flips are TS packet headers (that you have data for)?

    Yes that's right in both cases. You'll see the TS headers as a diagonal pattern as they occur every 188 bytes, whereas the picture is sized to be 56 bytes wide.

    Most of these regions look to be in absolutely abysmal condition (except maybe the last picture). Surely this can't be representative of the whole file. Or is it?

    The example pictures (except the last one) are from the start of the file, which is indeed in abysmal condition. I included the last picture to indicate a calmer area of the file. It's very informative to browse through all of the pics in the ZIP file, to see what a clean area looks like.

    Edit: Also, would it be possible to make separate pics for every frame?

    If you feel that would help I can do it, but the frame starts don't always align with 56-byte offsets. I could do pictures that are 188 bytes wide if you like?
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: princess on 06/10/2014 07:38 pm
    Well, there's the one pattern that screams out, the periodic short runs of bitflips in otherwise okay data.  Strong peak at ~184 bits between these corrections, with 5 resonant peaks due to occasional gaps in the pattern. 

    My guess is that you're using a byte offset and not a bit offset, and these are the TS packets at 188 bytes long. Most of the TS file is MPEG4 data packets - we can repair the headers of these packets but not the data, so you see a burst of errors representing the TS header bytes separated by 188 bytes. As we start applying MMBs to the data, you'll see pixels appearing in between these bursts.

    If that truly is BIT error lengths and not bytes, then we have something to work on...
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: saliva_sweet on 06/10/2014 07:54 pm
    Edit: Also, would it be possible to make separate pics for every frame?

    If you feel that would help I can do it, but the frame starts don't always align with 56-byte offsets. I could do pictures that are 188 bytes wide if you like?
    That's OK, I think a rough guide to tell which parts of frames are totally shot and not worth bothering with would be helpful. Although a bitposition scale at the left would be cool ;) It might be also informative to highlight somehow the subpackets (25 + 31 bytes) ajmartin identified (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1209887#msg1209887) and the 15 bit single flip regions, perhaps with vertical lines through the image or a horizontal bar at the top.

    But what we have so far is already wonderful. Superb work.

    Edit: Maybe my wording wasn't super clear here. It's OK that 56 bype packets don't align perfectly with frames. No need to switch to 188 byte width. I think 56 bytes makes more sense in this context.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: cscott on 06/10/2014 08:39 pm
    Edit: Also, would it be possible to make separate pics for every frame?

    It would indeed be nice to see frame-by-frame pictures; that would guide those working on MMBs to give them a rough heads up of where the frame damage lies.

    Also -- could I ask for a graph or other visualization of correlations between bit flips?  I'm thinking of a simple bar graph (or just a table) that says what's the probability of a flip at n+1, n+2, n+3, n+4, etc given the existence of a flip at n.  That would be a nice quantitative sanity-check on the "triple flip" pattern (which should leap out) as well as maybe see if the triple flip patterns are indeed consistent across all frames, or maybe are useful only in frames with "low corruption".
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: princess on 06/10/2014 09:01 pm
    OK, here are the pictures resized so that they're 188 bytes wide, i.e. each line is 1 TS packet. I've included an example of a corrupted area and a clean area, the whole set of images are in the ZIP file.

    The TS headers all line up down the left hand edge of the images, and you can see intriguing multiple bit flip patterns in the clearer areas. Can anyone see any more patterns?
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: morningdew76 on 06/10/2014 09:29 pm
    princess, would it be possible to generate the 448px images with some guides in them?

    I'm just looking for a visual marker to show the regions of single flips vs triple flips.

    Here's an example that I worked up with mspaint that shows the first and last 15 bits with a blue line.. I didn't attempt to mark the inner region.  It is 450px wide, but would be 452px if it had the other two blue lines.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Quialiss on 06/10/2014 09:51 pm
    Well, there's the one pattern that screams out, the periodic short runs of bitflips in otherwise okay data.  Strong peak at ~184 bits between these corrections, with 5 resonant peaks due to occasional gaps in the pattern. 

    My guess is that you're using a byte offset and not a bit offset, and these are the TS packets at 188 bytes long. Most of the TS file is MPEG4 data packets - we can repair the headers of these packets but not the data, so you see a burst of errors representing the TS header bytes separated by 188 bytes. As we start applying MMBs to the data, you'll see pixels appearing in between these bursts.

    If that truly is BIT error lengths and not bytes, then we have something to work on...

    Ack, you're right on the byte vs bit error.  Actually it was a silly mistake on my part in how I was processing the data, so it's not just bits swapped for bytes... but the general gist of what I said holds true and the corrected graph looks nearly identical except for the scale.

    Now that I've fixed that...

    Also -- could I ask for a graph or other visualization of correlations between bit flips?  I'm thinking of a simple bar graph (or just a table) that says what's the probability of a flip at n+1, n+2, n+3, n+4, etc given the existence of a flip at n.  That would be a nice quantitative sanity-check on the "triple flip" pattern (which should leap out) as well as maybe see if the triple flip patterns are indeed consistent across all frames, or maybe are useful only in frames with "low corruption".

    These are the percentages for the distance in bits between all the bitflips between princess's two files.  The big jump up at 13 good bits between errors is a strong indicator that these are triple flips in otherwise good data, the jump at 12 could be a triple flip overlapping another flip. 

    It does look like we should be trying more consecutive flips.  As to how many in a row there should be, I'll have to try some heavier pattern analysis, as this is only looking at the pattern error--some optional length of good data--error.

    0 41.88
    1 19.61
    2 9.79
    3 5.59
    4 3.57
    5 2.56
    6 1.9
    7 1.5
    8 1.23
    9 1.04
    10 0.95
    11 0.96
    12 1.39
    13 2.34
    14 0.28
    15 0.19
    16 0.15
    17 0.13
    18 0.12
    19 0.12
    20 0.11
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: princess on 06/10/2014 10:37 pm
    One thing I have noticed is that as well as triple flips there are also quadruple flips. A quad flip is two flips right next to each other, a gap, and then two flips separated by one bit. I've included a zoom-in of one of the image files here, it shows a patch with only triple and quad flips.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: S.Paulissen on 06/10/2014 11:00 pm
    One thing I have noticed is that as well as triple flips there are also quadruple flips. A quad flip is two flips right next to each other, a gap, and then two flips separated by one bit. I've included a zoom-in of one of the image files here, it shows a patch with only triple and quad flips.
    The quad flip looks like two overlapping triple flips.  The first two flipped bits in a row are the beginning of two triple flip patterns.  The flip-fine-flip is the overlap between the two triple flips where the outer two flip and the inner one double flips, each triple flip correcting each other for that bit.

    EDIT: wording change
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: princess on 06/10/2014 11:07 pm
    The quad flip is two overlapping triple flips.  The first two flipped bits in a row are the beginning of two triple flip patterns.  The flip-fine-flip is the overlap between the two triple flips where the outer two flip and the inner one double flips, each triple flip correcting each other for that bit.

    That makes so much sense, thanks for pointing that out - I hadn't realised that! So, whatever mechanism is causing that triple flip pattern can make two that overlap, that's interesting. What kind of mechanism could cause that pattern?
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: S.Paulissen on 06/10/2014 11:17 pm
    I was originally doubtful of my interpretation when I noticed that the triple flips were ALL green.  After looking at the full PNG you put up of a clean one, there are certainly quite a few red triple flips too.  It almost looks like 50%> are made up of triple flips but overlapping like crazy.

    For random flips it sure seems unlikely that 75+% of flips are 1 to 0.  I can think of a few mechanisms it would be that way.  If the source is made up of mostly 1s the flip from 1 to 0 becomes proportionally more likely and it's still random. 

    As for the mechanism for creating the triple pattern.... I have NO idea. 
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Quialiss on 06/10/2014 11:26 pm
    One thing I have noticed is that as well as triple flips there are also quadruple flips. A quad flip is two flips right next to each other, a gap, and then two flips separated by one bit. I've included a zoom-in of one of the image files here, it shows a patch with only triple and quad flips.
    The quad flip looks like two overlapping triple flips.  The first two flipped bits in a row are the beginning of two triple flip patterns.  The flip-fine-flip is the overlap between the two triple flips where the outer two flip and the inner one double flips, each triple flip correcting each other for that bit.

    EDIT: wording change

    Quickly looking to see if there are more of the double offsets..

    Pattern                          occurrences
    1000000000000011             7290          triple flip
    11000000000000101           3263          triple +1 offset
    101000000000001111         456           +2 offset
    1001000000000011011       261            +3 offset
    10001000000000110011     148            +4 offset

    ...and then they.. keep.. on.. going.  Continuing to find more of these in the more or less clean data.  Excuse me while I write up something to properly do this and see how many of the errors AREN'T triple flips. 

    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: S.Paulissen on 06/10/2014 11:37 pm
    Unfortunately there are as many iterations of this as there are bits, n-12 times n.  Of course this number can be constrained to make it easier.  I'm sure you already realize this.

    10000000000011000000
    11000000000010100000
    11100000000010010000
    11110000000010001000
    10110000000011101000
    11010000000010111000
    11111000000010000100
    10111000000011100100
    11011000000010110100
    11101000000010011100
    .... so on
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: princess on 06/11/2014 12:10 am
    If the source is made up of mostly 1s the flip from 1 to 0 becomes proportionally more likely and it's still random. 

    Yes, that's what we're seeing here. The MPEG4 encoder compresses each frame to a different size, but the TS stream must fit into a fixed bandwidth. If the MPEG4 frame is too small, the TS layer pads it with 0xff bytes (i.e. all 1s), and this padding nearly always occurs in a block at the end of a frame. So what you're seeing is where we have a huge block that we know should be all 1s, but some are flipped to 0s. That's why they mostly show up as green pixels.

    But, as you noticed, there are other times when we can fix different areas, for example TS headers, timestamps, MPEG4 frame starts, things like that. In those areas we can detect if a 1 should be a 0 instead, and you get red pixels.

    So as far as I can see, the flips are indeed random - it's just that we can detect flips far easier in the 0xff padding bytes, so we will have found proportionally far more of the "green" flips than the "red" ones. Even if they recovered the stage from the bottom of the sea and extracted the clean video so we could know every flip, there would still be more green flips than red because the raw stream contained lots more 1s than 0s (due to the presence of the padding bytes).
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Quialiss on 06/11/2014 01:12 am
    Unfortunately there are as many iterations of this as there are bits, n-12 times n.  Of course this number can be constrained to make it easier.  I'm sure you already realize this.


    Thus the need to pause and write up a quick tool to generate the patterns.   ;D .... which I haven't quite gotten around to writing yet, too many other things to do tonight.  However, the stats on only two overlapping 'triple flips' are of note.  I'll get back to this tomorrow with 3+ overlapping flips and pull out the stats on the probability of each of the flips, but for now:

    bits involved in a known flip pattern, total flips in the chunk, chunks as per princess's last set of images with 188 byte width. 

    668 out of 5404 identified in chunk 0
    1623 out of 16571 identified in chunk 900
    158 out of 2639 identified in chunk 1800
    45 out of 497 identified in chunk 2700
    319 out of 2002 identified in chunk 3600
    62 out of 67 identified in chunk 4500
    11 out of 45 identified in chunk 5400
    28 out of 137 identified in chunk 6300
    93 out of 983 identified in chunk 7200
    178 out of 724 identified in chunk 8100
    35 out of 80 identified in chunk 9000
    192 out of 702 identified in chunk 9900
    9 out of 154 identified in chunk 10800
    169 out of 1363 identified in chunk 11700
    58 out of 277 identified in chunk 12600
    180 out of 663 identified in chunk 13500
    9 out of 150 identified in chunk 14400
    168 out of 2499 identified in chunk 15300
    0 out of 22 identified in chunk 16200
    12 out of 17 identified in chunk 17100
    7 out of 15 identified in chunk 18000
    544 out of 5693 identified in chunk 18900
    0 out of 3314 identified in chunk 19800   ---will double check tomorrow to see if it's a bug in my script
    13 out of 127 identified in chunk 20700
    15 out of 95 identified in chunk 21600
    ------
    10.39 % identified patterns
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Adaptation on 06/11/2014 05:39 am
    I would postulate that the paired sets of disjointed bit flips where originally conjoined as the signal traveled through the air but the endianness (http://en.wikipedia.org/wiki/Endianness) during transmission differed from recording.  You may notice that the distance between the middles of the pairs of bit flip areas is about 16 pixels, that would imply 128 bit registers in the transmission and receiving equipment which is quite reasonable.

    If my hypothesis is correct you should be able divide the stream into 128 bit sections, reverse the order within each section while maintaining the order from section to section.  Once done nearly all the disjointed bit flips should be conjoined and it should make it far easer to visualize meaningful patterns in the noise and predict areas where a bit flip is likely.  There is no reason to assume that the start of the 128 bit chucks will align with the start of a ts packet, but I do think it likely that they are byte aligned.

    Assuming I am correct looking for patterns in the errors without correcting for this is similar to looking for patterns in a deck of cards that have been shuffled. 
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: princess on 06/11/2014 07:17 am
    You may notice that the distance between the middles of the pairs of bit flip areas is about 16 pixels, that would imply 128 bit registers in the transmission and receiving equipment which is quite reasonable.

    I'm not sure where you get the 128 bit figure from - these images are just visualisations of the raw bitstream where 1 pixel corresponds to 1 bit. The triple flips we're finding are a pattern within a 16 bit range, and there appears to be no obvious alignment to them.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Sohl on 06/11/2014 12:49 pm
    I would postulate that the paired sets of disjointed bit flips where originally conjoined as the signal traveled through the air but the endianness (http://en.wikipedia.org/wiki/Endianness) during transmission differed from recording. 

     ???

    How would endianness make any difference here?  I'd assume MPEG-4 is a stream of bits (no fundamental endianness) wrapped in the transport stream (TS) framework.  You either get the bits as they come or you don't.

    I speculate that the receiver front-end was not able to effectively filter out large electro-magnetic noise spikes, and that an extra electrical impulse "rang" in the RF-front end circuit long enough to alter the bit sampling process.  But who knows? :o

    While knowing the mechanism would be nice, if a majority of the corruption follows a predictable pattern, it will be possible to correct much of it without knowing the source.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Okie_Steve on 06/11/2014 01:13 pm
    How would endianness make any difference here?  I'd assume MPEG-4 is a stream of bits (no fundamental endianness) wrapped in the transport stream (TS) framework.  You either get the bits as they come or you don't.
    Communications is frequently done with blocks and shift registers and tend to be "little endian" bit streams in my experience. Think about a Uart set for N81 as an example. Sending a byte of 0x40 (01000000) would present bits on the wire  as 0 (start bit)  0 0 0 0 0 0 1 0 1 (stop bit) in temporal order from left to right of this line. Without knowing anything about the encoder, transmission protocol, packet size and equipment it is all speculation though.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: SwissCheese on 06/11/2014 01:39 pm
    I just corrected most of part 3 (frames 47-60). I won't be able to do anything more until Sunday or Monday, going for a small holiday trip :)

    So on my side frames 42-46 remain to be checked, as well as ~10 p-frames in part 4. I-frames 241 and 281 could also have smoother colors.

    Otherwise it's maybe time to generate the full series of images forming the video (in jpg so it does not take too much space) and ask people to comment them (say where something looks wrong), so we can check again those frames.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: cscott on 06/11/2014 02:04 pm
    How would endianness make any difference here?  I'd assume MPEG-4 is a stream of bits (no fundamental endianness) wrapped in the transport stream (TS) framework.  You either get the bits as they come or you don't.
    Communications is frequently done with blocks and shift registers and tend to be "little endian" bit streams in my experience. Think about a Uart set for N81 as an example. Sending a byte of 0x40 (01000000) would present bits on the wire  as 0 (start bit)  0 0 0 0 0 0 1 0 1 (stop bit) in temporal order from left to right of this line. Without knowing anything about the encoder, transmission protocol, packet size and equipment it is all speculation though.

    The fact that we're seeing the exact same triple flip pattern irregardless of where in the byte the flips start indicates that we've got the endianness right.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: cscott on 06/11/2014 02:24 pm
    I speculate that the receiver front-end was not able to effectively filter out large electro-magnetic noise spikes, and that an extra electrical impulse "rang" in the RF-front end circuit long enough to alter the bit sampling process.  But who knows? :o

    It's most likely *not* an analog effect.  My best guess is that it is an artifact of deliberate interleaving, which is done to spread out the effects of burst noise.  See https://en.wikipedia.org/wiki/Forward_error_correction#Interleaving
    This interacts with the specific error correction code in use.   If there are N+1 errors and the ECC is only able to correct N, the uncorrected error might end up with a certain pattern; see https://en.wikipedia.org/wiki/Burst_error-correcting_code#Convolutional_interleaver for something which could look roughly like our pattern.

    I haven't figured out the exact coding system that would produce this pattern, but here's a strawman: consider an error correcting code which operates on groups of three bits and adds a fourth parity bit.  The three bit input is interleaved as in our triple flip pattern.  If the received parity bit doesn't match the parity of the three received data bits, the three bits are flipped (which also flips the parity, so it now matches and the output is "correct").  Now, as is, this is a pretty poor ECC! But in a hand-wavy way it might help visualize how the ECC fixups might lead to specific patterns of bit flips.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: princess on 06/11/2014 03:03 pm
    The fact that we're seeing the exact same triple flip pattern irregardless of where in the byte the flips start indicates that we've got the endianness right.

    Interestingly I generated the images big-endian, which is the opposite of RS-232. In other words, a byte with value 0x0f would be output as 00001111. So in my bitflip images, the first pixel in a line represents the most significant bit of the first byte.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: saliva_sweet on 06/11/2014 03:24 pm
    For anyone working on the flip patterns the posts by ajmartin should be required reading. He's posted 4 times on this forum, first of them is 3 weeks old by now and I highly recommend reading all of them. Cos I'm getting a sense we're going in circles here.

    http://forum.nasaspaceflight.com/index.php?topic=34597.msg1201383#msg1201383
    http://forum.nasaspaceflight.com/index.php?topic=34597.msg1209887#msg1209887
    http://forum.nasaspaceflight.com/index.php?topic=34597.msg1210893#msg1210893
    http://forum.nasaspaceflight.com/index.php?topic=34597.msg1210927#msg1210927
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: cscott on 06/11/2014 03:31 pm
    Interestingly I generated the images big-endian, which is the opposite of RS-232. In other words, a byte with value 0x0f would be output as 00001111. So in my bitflip images, the first pixel in a line represents the most significant bit of the first byte.

    Yes, the MPEG TS is big-endian.  "Network byte order".  (See http://www.itu.int/rec/T-REC-H.222.0-199507-S/en which states on page 7 that it is msb first.)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Sohl on 06/11/2014 05:16 pm
    It's most likely *not* an analog effect.  My best guess is that it is an artifact of deliberate interleaving, which is done to spread out the effects of burst noise.  [ref. to Forward Error Correction]

    Sure, instances of failed FEC could plausibly do this.  Too bad someone at SpaceX has not been able to help specify any kind of FEC methods used in the RF link.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: cscott on 06/11/2014 05:33 pm
    It's most likely *not* an analog effect.  My best guess is that it is an artifact of deliberate interleaving, which is done to spread out the effects of burst noise.  [ref. to Forward Error Correction]

    Sure, instances of failed FEC could plausibly do this.  Too bad someone at SpaceX has not been able to help specify any kind of FEC methods used in the RF link.

    I'm not sure it's strictly necessary: I think the discovery of the consistent triple flip pattern is the bit that's relevant to us.  I don't think we really need to waste brain cycles reverse engineering it further when the effect is so clear.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: wronkiew on 06/11/2014 07:22 pm
    I just corrected most of part 3 (frames 47-60). I won't be able to do anything more until Sunday or Monday, going for a small holiday trip :)

    So on my side frames 42-46 remain to be checked, as well as ~10 p-frames in part 4. I-frames 241 and 281 could also have smoother colors.

    Otherwise it's maybe time to generate the full series of images forming the video (in jpg so it does not take too much space) and ask people to comment them (say where something looks wrong), so we can check again those frames.

    Done. It's 13 MB.

    http://adama.nocdirect.com/~wronkiew/spx_crs3/landing_base_set_jpg.zip
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Quialiss on 06/11/2014 08:06 pm
    Going to clean up i frame 13, and comparing to frame 12... that moment when you realize that the whole bottom of the frame is off by one.   ???  Somehow that slipped by me despite already having cleaned up the DC values on the frame before. 

    Whoops.  :D  Fixed now. 


    edit... I have no idea where that gif came from, that's something completely different   ;D  It is an example of what we could do for post processing, filling in data gaps with good data from adjacent frames. 
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Jakusb on 06/11/2014 08:16 pm
    I just corrected most of part 3 (frames 47-60). I won't be able to do anything more until Sunday or Monday, going for a small holiday trip :)

    So on my side frames 42-46 remain to be checked, as well as ~10 p-frames in part 4. I-frames 241 and 281 could also have smoother colors.

    Otherwise it's maybe time to generate the full series of images forming the video (in jpg so it does not take too much space) and ask people to comment them (say where something looks wrong), so we can check again those frames.

    Done. It's 13 MB.

    http://adama.nocdirect.com/~wronkiew/spx_crs3/landing_base_set_jpg.zip

    Wow! I just unzipped the file and used Windows Photo Viewer to view the sequence with the arrow keys.
    You really can see a lot of detail and like some misalignments. It is incredibly easy to spot them this way..
    Also perfect to try to figure out what it is we actually are seeing. :)
    Really cool!!
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Jakusb on 06/11/2014 08:29 pm
    Small find, but maybe helps alignment a bit:
    frames 219-220 have some strange movement vector, as dirtspot #3 is on the move in these two p-frames

    edit: obviously the dirtspot should not move... ;)

    edit2: is it me, or is the whole rocket body a few pixels (whole mb) off to the right in frame 221, compared to other frames?
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Lourens on 06/11/2014 08:55 pm
    I've been playing with the error analysis a bit and I have a few mildly interesting new ways to visualise the bitflips.

    <snip>

    The second is that I've made a series of images representing the bitflips. Each pixel represents a bit, and each image line represents 56 bytes of the data file. A black pixel means there's no error, a red pixel means a transmission error set a bit to 1 when it should be 0, and a green pixel means a transmission error cleared a bit to 0 when it should be 1.

    Great work Princess! And very interesting result too, wish I had some time to play with it. Hopefully on the weekend. For now, I'd like to point out that the "no error" black pixels are actually "no difference" bits. Where they are part of the TS packets, VOP headers or the padding, i.e. the parts that you and the others fixed, we know that they're "no error", but inside the frames we don't know if they're right or wrong. Would it be possible to distinguish these two? Maybe make the known "no error" bits blue, and leave the "not corrected" bits black?

    That would also make it easier to calculate the flipping statistics over only the bits where we actually know that they're right or wrong, and skip the ones where we don't know yet, removing some noise and perhaps making any patterns stand out more clearly. For that matter, adding the category (e.g. correct, set, cleared, unknown) to the CSV would be useful for further processing as well.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: saliva_sweet on 06/11/2014 09:32 pm
    Small find, but maybe helps alignment a bit:
    frames 219-220 have some strange movement vector, as dirtspot #3 is on the move in these two p-frames

    edit: obviously the dirtspot should not move... ;)

    I think this is nigh impossible to fix, mvs can do all kinds of wonky things even if data is well aligned as you can see on the legs.

    edit2: is it me, or is the whole rocket body a few pixels (whole mb) off to the right in frame 221, compared to other frames?

    Looks like you're right. Good catch.

    edit: Looks like Quialiss already fixed it :)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Quialiss on 06/11/2014 10:33 pm
    Small find, but maybe helps alignment a bit:
    frames 219-220 have some strange movement vector, as dirtspot #3 is on the move in these two p-frames

    edit: obviously the dirtspot should not move... ;)

    I think this is nigh impossible to fix, mvs can do all kinds of wonky things even if data is well aligned as you can see on the legs.

    These are still worth noting, depending on the frame they may be fixable and are worth a second look.  Hrrrmmm... Calling spreadsheet makers, could we have a different, editable version of the 'all frames' spreadsheet for review purposes?

    Three columns:  frame number, the cumulative frame at full size, not the isolated one, and 'notes.'  Should be an ideal setup for people to flip through and make note of what frames could use attention, as it'll stay up to date with all the fixes. 

    edit2: is it me, or is the whole rocket body a few pixels (whole mb) off to the right in frame 221, compared to other frames?

    Looks like you're right. Good catch.

    edit: Looks like Quialiss already fixed it :)

    Today is I frame fixing day.   ;D  3 down, 2 more to go. 
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Adaptation on 06/12/2014 02:05 am
    I'm not sure where you get the 128 bit figure from - these images are just visualisations of the raw bitstream where 1 pixel corresponds to 1 bit. The triple flips we're finding are a pattern within a 16 bit range, and there appears to be no obvious alignment to them.

    Sorry I was half asleep while writing that, I think I though each pixle was a byte. 

    The idea I think from cscots about having too many errors for forward error correction to handle does sound more plausible.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Quialiss on 06/12/2014 04:05 am
    Good news, the frame should be substantially easier to fix this time around.

    Bad news, a chunk of the top of 261 is off by several blocks, screwing up arnezami's hard work on it.  Some of of the corrections for the bottom of the frame may still be valid, I haven't tried yet. 

    If anyone would care to take a stab at pulling a few more macroblocks from the discarded data in gaps in frame, now's the perfect time. 

    New alignment, notably 33:04:15183, which lines up the wavefront on the right side of the frame. 

    0:0:-1,30:0:2617,33:0:-2,38:0:3236,27:1:-2,29:1:5452,35:2:-2,33:4:15183,28:5:17845:64:0:0:0:0:0,39:8:-2,12:9:29306

    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Sohl on 06/12/2014 01:02 pm

    Sure, instances of failed FEC could plausibly do this.  Too bad someone at SpaceX has not been able to help specify any kind of FEC methods used in the RF link.

    I'm not sure it's strictly necessary: I think the discovery of the consistent triple flip pattern is the bit that's relevant to us.  I don't think we really need to waste brain cycles reverse engineering it further when the effect is so clear.

    Yes, I agree and had considered saying as much in that last post I made... if the pattern is consistent enough, it can be recognized and reversed, no matter the real cause.  But I imagine that there could be additional effects that could be predicted if more details of the FEC (if used?) and other transmission details were available.

    Nonetheless, the ad-hoc group here has done incredible things already just based on what can be seen in the files.  Keep up the great work!

    Edit: slight wording change.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: IainCole on 06/12/2014 06:05 pm
    So.. After going back and forth and trying to work out a way to get a list of 15 people who will receive SpaceX goodies, it has come to this, a vote!

    It sucks to have to whittle it down to 15, but this is the situation we find ourselves in.

    Everyone who wishes to vote should send me a PM on the forums with an ordered list of people (forum usernames please if possible) who they think deserve to get goodies. It doesn't need to be 15 long (the longer the better, so long as it is ordered) because I will collate the results of each list and pick the top 15. Also you can't vote for yourself =) Here is a list of names that has already been put forward in various places, you don't need to pick from this list, it's just here as a guide, if there is someone else you think is deserving then feel free to add them.

    Adaptation
    ajmartin
    arnezami
    dorkmo
    gospacex
    grythumn
    jakusb
    Jared (http://www.reddit.com/user/sharebrained)
    John_L
    lgjy98d (myroslav)
    Lourens
    mhenderson
    michaelni
    Mlindner
    moralec
    morningdew76
    mvpel
    princess
    Quialiss
    Saliva_Sweet
    seanpg71
    Shanuson
    SwissCheese
    theshadow27
    Turix
    Untribium
    wronkiew

    I couldn't find these names in the forum (I think they originated from the Wiki, if you know who these people are on the forum, PM me)
    Asmegin
    dgdpg
    Exclavion
    Gnonthgol
    lewing
    Noslyl
    pshrpd
    tentonine
    Turix

    If there is anyone who would not like to be on the list for whatever reason then drop me a PM and I will take that into account when adding up the numbers, also don't include me in your lists as I was lucky enough to receive something already (which I feel quite guilty about if I'm honest)

    Polls close on Sunday so vote now! I will then publish the anonymised results and ask the lucky 15 to PM me their gender (I'm not going to guess), shirt size, and delivery address. Something to bear in mind, assuming that everyone gets something similar to what I did (I hope this is the case), the package will NOT fit through a letter box (not even close ;)) so make sure the delivery address is somewhere where the package can be taken and signed for.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: maximlevitsky on 06/12/2014 06:35 pm
    Since the launch did slip by 'few days' (as expected  ;) ), do you think the next landing will be at day time?
    (unless it slips again by few months, but that isn't funny anymore  :( )
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: starsilk on 06/12/2014 08:19 pm
    So.. After going back and forth and trying to work out a way to get a list of 15 people who will receive SpaceX goodies, it has come to this, a vote!

    It sucks to have to whittle it down to 15, but this is the situation we find ourselves in.

    seriously?

    just send them the full list. I'm sure they can afford 15 extra T-shirts.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: IainCole on 06/12/2014 08:21 pm
    So.. After going back and forth and trying to work out a way to get a list of 15 people who will receive SpaceX goodies, it has come to this, a vote!

    It sucks to have to whittle it down to 15, but this is the situation we find ourselves in.

    seriously?

    just send them the full list. I'm sure they can afford 15 extra T-shirts.

    They aren't sending out single t-shirts
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: mhenderson on 06/12/2014 09:02 pm
    Adaptation
    ajmartin
    arnezami
    dorkmo
    gospacex
    grythumn
    jakusb
    Jared (http://www.reddit.com/user/sharebrained)
    John_L
    lgjy98d (myroslav)
    Lourens
    mhenderson
    michaelni
    Mlindner
    moralec
    morningdew76
    mvpel
    princess
    Quialiss
    Saliva_Sweet
    seanpg71
    Shanuson
    Swatch
    SwissCheese
    theshadow27
    Turix
    Untribium
    wronkiew

    Doh. I was dismayed that IainCole was not on the list. Then I realized he compiled the list.

    Seriously, do you really want us to exclude you from our rankings?
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: moralec on 06/12/2014 09:03 pm
    I'm including him anyway ahaha
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: IainCole on 06/12/2014 09:03 pm
    Yes please don't include me, I explain why in the post ;)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: mvpel on 06/12/2014 09:38 pm
    So.. After going back and forth and trying to work out a way to get a list of 15 people who will receive SpaceX goodies, it has come to this, a vote!

    You'll be interested in the Condorcet Method. (http://en.wikipedia.org/wiki/Condorcet_method)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: jcat on 06/12/2014 10:54 pm
    Hey, guys,

    I'm a Long Time Lurker, haven't posted since Armadillo Aerospace was first taking off.  I have been following this thread with delight and amazement, and wanted to show/compare just how much you all have done.  so I set this up:

    http://www.youtube.com/watch?v=pSu72NNf7Dk

    Cheers! Keep up the awesome work!
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: QuantumG on 06/12/2014 11:31 pm
    Welcome to the forum! Awesome side-by-side video.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Chris Bergin on 06/12/2014 11:46 pm
    Hey, guys,

    I'm a Long Time Lurker, haven't posted since Armadillo Aerospace was first taking off.  I have been following this thread with delight and amazement, and wanted to show/compare just how much you all have done.  so I set this up:

    http://www.youtube.com/watch?v=pSu72NNf7Dk

    Cheers! Keep up the awesome work!

    That's superb! Getting that tweeted out! ;D

    Welcome to the site's forum and thanks for making the effort with that!
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Quialiss on 06/13/2014 04:33 am
    https://docs.google.com/spreadsheets/d/1RZWykghV6XTuMRjikDleiX_J-ifG0g548Fta5WTRY88/edit?usp=sharing

    I mentioned earlier that a spreadsheet would be really nice for doing the 'final' edits, so I put one together.  It shows all the frames, full size, thanks to the wonderful work done previously to make the auto-updating spreadsheets.  It looks like the images were scaled down after loading so it shouldn't be any worse for bandwidth than the All Frames spreadsheet.

    SwissCheese is the master of divining the correct alignment of the p frames, and is still working on some of part 3-4, so I haven't made notes on them yet, but other than that I've put my notes on them for things to still fix, and anyone and everyone can go add more things to the task list as they're noticed. 

    While on that note, fixing the gray blocks(copy the correct DC values from a nearby block that's correct) and too dark/too intense colours(bitflip in a luma or chroma subblock, or remove a bad block upstream) in the p frames is pretty simple, each frame only takes a few minutes, perfect if you don't have too much time to spend.  ;)

    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: wronkiew on 06/13/2014 05:50 am
    I couldn't find these names in the forum (I think they originated from the Wiki, if you know who these people are on the forum, PM me)
    Asmegin
    dgdpg
    Exclavion
    Gnonthgol
    lewing
    Noslyl
    pshrpd
    tentonine
    Turix

    Exclavion posted a message on the previous page!

    Thank you IainCole for taking on this stinky job.

    And yes, Condorcet is a pain to implement, but it is the fairest voting system known to mathematics.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: wronkiew on 06/13/2014 06:37 am
    Ohsin posted a video of the ascent over in the interpretation thread, and it prompted me to see if I could cross-check the digit 2 bitmap. Two and four never appear unscrambled in the landing video, so I had to piece them together from multiple pframes. It turns out I had two bad pixels, as you can see below. This also confirms that the clock digits have identical pixels on the even and odd rows.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Untribium on 06/13/2014 01:35 pm
    -snip-
    If there is anyone who would not like to be on the list for whatever reason then drop me a PM and I will take that into account when adding up the numbers, also don't include me in your lists as I was lucky enough to receive something already (which I feel quite guilty about if I'm honest)
    -snip-

    Don't feel bad, you deserve every bit of whatever it is you got :)) The online editor was absolutely crucial to the success of this effort, it helped getting a lot of people involved (including myself) :)

    Thank you IainCole for taking on this stinky job.

    And yes, Condorcet is a pain to implement, but it is the fairest voting system known to mathematics.

    Agreed :) The problem is that if you pick the top 15 by number of votes, then one can increase their "chance of winning" by not voting...I think...correct me if I'm wrong :)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: moralec on 06/13/2014 01:51 pm
    -snip-
    If there is anyone who would not like to be on the list for whatever reason then drop me a PM and I will take that into account when adding up the numbers, also don't include me in your lists as I was lucky enough to receive something already (which I feel quite guilty about if I'm honest)
    -snip-

    Don't feel bad, you deserve every bit of whatever it is you got :)) The online editor was absolutely crucial to the success of this effort, it helped getting a lot of people involved (including myself) :)

    Thank you IainCole for taking on this stinky job.

    And yes, Condorcet is a pain to implement, but it is the fairest voting system known to mathematics.

    Agreed :) The problem is that if you pick the top 15 by number of votes, then one can increase their "chance of winning" by not voting...I think...correct me if I'm wrong :)

    Offtopic, but every voting system has issues. Actually Arrow (http://en.wikipedia.org/wiki/Arrow's_impossibility_theorem) proved that, almost by construction, there is no mechanism that is capable of satisfying "fairness" the basic fairness criteria. :)

    Let's just let IainCole take care of this.... we are not here for the prizes anyway.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: maximlevitsky on 06/13/2014 10:38 pm
    Since the launch did slip by 'few days' (as expected  ;) ), do you think the next landing will be at day time?
    (unless it slips again by few months, but that isn't funny anymore  :( )

    And its starts to be not funny indeed :(
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Lourens on 06/13/2014 10:55 pm
    It looked nice and difficult, so I had a go at iframe 1 (from raw_final_fixed.zip):  8)

    -mmb X:553:80,X:668:80,X:692:80,X:706:C0,X:732:80,X:746:C0,
    X:921:80,X:936:C0,X:939:80,X:953:C0,X:988:80,X:1002:C0,X:993:80,X:1007:C0

    Those seem to get the first five MBs in a somewhat working order. Mostly triple-flips, that pattern does seem to work quite well indeed! Nice find! I'm not sure about the earlier single flips though, maybe there's still something wrong there.

    I'm a little confused now with all the spreadsheets, wiki, online editor, moderated submission system and how they're all linked together. Do I put the above on the wiki, on the spreadsheet, both? Where, how?

    I did these using my local FFmpeg, which has arnezami's detailed block dump BITLOG in it. I've noticed that these aren't in the online editor, at least not in version 1. They're very helpful though, I found the above mainly by looking at which blocks had lots of DCT coefficients or large DC values. Is there any way the BITLOG patch could be applied to the online editor as well?

    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Llian Rhydderch on 06/13/2014 10:58 pm
    Since the launch did slip by 'few days' (as expected  ;) ), do you think the next landing will be at day time?
    (unless it slips again by few months, but that isn't funny anymore  :( )

    And its starts to be not funny indeed :(

    Wrong thread.  Launch slips are discussed elsewhere.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Quialiss on 06/13/2014 11:51 pm
    I'm a little confused now with all the spreadsheets, wiki, online editor, moderated submission system and how they're all linked together. Do I put the above on the wiki, on the spreadsheet, both? Where, how?

    Wiki is the original system for keeping track of things, it requires you to format the mmb's so they don't run off the page, and you have to update the images separately.  It's also where the auto updating youtube videos are sourced from.

    The spreadsheets(linked to on the wiki) just require you to paste in the MMB in the appropriate cell, and will update the images next time you load the spreadsheet. 

    The moderated submission system is attached to the spreadsheets, but nobody's submitted anything through it yet... I believe the idea was convenience, and also to encourage people who aren't quite sure they've got it right to submit through it rather than directly editing the spreadsheets. 

    So.. both for now.  Phasing out the wiki by having the videos source from the spreadsheets would be nice, as they're much easier to update, but it's okay as is. 



    Also, you're crazy.   ;D  Of parts 1-2, the p frames of part 2 are probably the most recoverable, but if you're intent on the i frames, you might get some help referencing the raw video that SpaceX posted on youtube that has a couple more i frames on either side, for ideas on what you're looking at in frame 1. 

    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: cmobile on 06/14/2014 08:04 am
    So.. After going back and forth and trying to work out a way to get a list of 15 people who will receive SpaceX goodies, it has come to this, a vote!

    It sucks to have to whittle it down to 15, but this is the situation we find ourselves in.

    ...


    I think we should send a list of everyone who made contributions and highlight the top 15 (I'll bet this number was somewhat arbitrary).  SpaceX can do with the list whatever they wish.  Hopefully the runner-ups will get at least a letter of acknowledgment.

    And someone needs to post an un-boxing video of the loot; we're all on the same team regardless of who gets to keep the trophy!
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: JohnKiel on 06/14/2014 02:40 pm
    The spreadsheets(linked to on the wiki) just require you to paste in the MMB in the appropriate cell, and will update the images next time you load the spreadsheet. 

    Just a slight clarification on this point:

    After changing the MMB, the image might update after reloading the spreadsheet, but it could take anywhere from 5 to 10 minutes for this to happen. 

    The problem is Google's image proxy caching the previous image.  Even though my spxi proxy script (a script that pulls a current MMB for a frame from the appropriate spreadsheet, and then either pulls that image from a local cache, or asks IanCole's server to generate new image data for the MMB) says "must revalidate", Google's proxy seems a little more aggressive about caching.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: JohnKiel on 06/14/2014 03:29 pm
    So.. After going back and forth and trying to work out a way to get a list of 15 people who will receive SpaceX goodies, it has come to this, a vote!

    It sucks to have to whittle it down to 15, but this is the situation we find ourselves in.

    Everyone who wishes to vote should send me a PM on the forums with an ordered list of people (forum usernames please if possible) who they think deserve to get goodies. It doesn't need to be 15 long (the longer the better, so long as it is ordered) because I will collate the results of each list and pick the top 15. Also you can't vote for yourself =) Here is a list of names that has already been put forward in various places, you don't need to pick from this list, it's just here as a guide, if there is someone else you think is deserving then feel free to add them.

    Adaptation
    ajmartin
    arnezami
    dorkmo
    gospacex
    grythumn
    jakusb
    Jared (http://www.reddit.com/user/sharebrained)
    John_L
    lgjy98d (myroslav)
    Lourens
    mhenderson
    michaelni
    Mlindner
    moralec
    morningdew76
    mvpel
    princess
    Quialiss
    Saliva_Sweet
    seanpg71
    Shanuson
    SwissCheese
    theshadow27
    Turix
    Untribium
    wronkiew

    I couldn't find these names in the forum (I think they originated from the Wiki, if you know who these people are on the forum, PM me)
    Asmegin
    dgdpg
    Exclavion
    Gnonthgol
    lewing
    Noslyl
    pshrpd
    tentonine
    Turix

    If there is anyone who would not like to be on the list for whatever reason then drop me a PM and I will take that into account when adding up the numbers, also don't include me in your lists as I was lucky enough to receive something already (which I feel quite guilty about if I'm honest)

    Polls close on Sunday so vote now! I will then publish the anonymised results and ask the lucky 15 to PM me their gender (I'm not going to guess), shirt size, and delivery address. Something to bear in mind, assuming that everyone gets something similar to what I did (I hope this is the case), the package will NOT fit through a letter box (not even close ;)) so make sure the delivery address is somewhere where the package can be taken and signed for.

    Probably should put req (req is hosting the spxi image proxy scripts used for displaying images in the spreadsheets) on that list as well.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Lourens on 06/14/2014 09:30 pm
    I'm a little confused now with all the spreadsheets, wiki, online editor, moderated submission system and how they're all linked together. Do I put the above on the wiki, on the spreadsheet, both? Where, how?

    <snip>

    So.. both for now.  Phasing out the wiki by having the videos source from the spreadsheets would be nice, as they're much easier to update, but it's okay as is. 

    Thanks for the explanation! I've decided to not yet put my results up there until I'm a bit more sure that they make sense. See also below.

    Quote
    Also, you're crazy.   ;D  Of parts 1-2, the p frames of part 2 are probably the most recoverable, but if you're intent on the i frames, you might get some help referencing the raw video that SpaceX posted on youtube that has a couple more i frames on either side, for ideas on what you're looking at in frame 1.

    Well, I'm still working on automated bit flipping, and a frame that no human could or would do sounds like just what we would like to have an automatic system for. I think I'm getting a little bit closer to something workable, although I doubt it will ever be fully automated. But that's okay. And even if we never get I-frame 1 back, it may still be useful for filling in some of the gaps in the more interesting I-frames.

    My current program tries only triple flips, and judges its performance not by looking at the picture, but by analysing arnezami's BITLOG output. I took a frame that we have a pretty good solution for (I-frame 52), and I collected some statistics on the various MB fields, e.g. which values occur for dc_lum, and how often? I use that information to judge whether a macroblock is likely to be correct. If it has a lot of common values, I assume that it's okay, if not, I'll run through the possible triple-flips and see if they help. I add the one that gives the most improvement to the mmb, and repeat, until the MB is good enough. Then I move on to the next one.

    As you can see below in the top-left corner, it does seem to be improving things, but it's not quite there yet. One major omission is that it's not including AC values yet when checking how good a macroblock is, and the "good enough" threshold isn't tuned very well yet. As a result, there's some high frequency noise in the "fixed" blocks that probably shouldn't be there, and the whole thing gets desynchronised enough to give an unfixable green tint from MB 10 on. So even though the first ten MBs look good in this picture, they probably aren't completely. Also, if it does desynchronise somewhere, then the later grey blocks may well be an example of my program creating something that isn't really there. On the other hand, the bit flip rate here is not that big, on the order of one triple flip per 100 bits.

    The code is an ugly hack in Bash, but if anyone wants it, just let me know. Meanwhile I'll keep fiddling with it :).
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Quialiss on 06/14/2014 10:05 pm
    Well, I'm still working on automated bit flipping, and a frame that no human could or would do sounds like just what we would like to have an automatic system for. I think I'm getting a little bit closer to something workable, although I doubt it will ever be fully automated. But that's okay. And even if we never get I-frame 1 back, it may still be useful for filling in some of the gaps in the more interesting I-frames.

    My current program tries only triple flips, and judges its performance not by looking at the picture, but by analysing arnezami's BITLOG output. I took a frame that we have a pretty good solution for (I-frame 52), and I collected some statistics on the various MB fields, e.g. which values occur for dc_lum, and how often? I use that information to judge whether a macroblock is likely to be correct. If it has a lot of common values, I assume that it's okay, if not, I'll run through the possible triple-flips and see if they help. I add the one that gives the most improvement to the mmb, and repeat, until the MB is good enough. Then I move on to the next one.

    As you can see below in the top-left corner, it does seem to be improving things, but it's not quite there yet. One major omission is that it's not including AC values yet when checking how good a macroblock is, and the "good enough" threshold isn't tuned very well yet. As a result, there's some high frequency noise in the "fixed" blocks that probably shouldn't be there, and the whole thing gets desynchronised enough to give an unfixable green tint from MB 10 on. So even though the first ten MBs look good in this picture, they probably aren't completely. Also, if it does desynchronise somewhere, then the later grey blocks may well be an example of my program creating something that isn't really there. On the other hand, the bit flip rate here is not that big, on the order of one triple flip per 100 bits.

    The code is an ugly hack in Bash, but if anyone wants it, just let me know. Meanwhile I'll keep fiddling with it :).

    I didn't realize you were doing them automatically!  I wouldn't mind seeing your script. 

    Have you tested it on any of the more intact frames?  That should help figuring out the 'correctness' thresholds, giving you a reference frame to look at to find any places where your script is making incorrect flips.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: saliva_sweet on 06/14/2014 10:45 pm
    Well, I'm still working on automated bit flipping, and a frame that no human could or would do sounds like just what we would like to have an automatic system for. I think I'm getting a little bit closer to something workable, although I doubt it will ever be fully automated. But that's okay. And even if we never get I-frame 1 back, it may still be useful for filling in some of the gaps in the more interesting I-frames.

    Very interesting approach, that I think has a lot of potential, but also needs a lot of effort to get working. I agree with Quialiss, you should be working on good frames first to see if it works and improve the method, like iframe 181. We don't even know if iframe 1 has anything recoverable at all. There seems to be a fairly sharp threshold where the FEC fails completely (possibly starts introducing even more errors) and data becomes totally unrecoverable by current methods. With enough flips and patience we could turn random noise into the mona lisa, but that's not what we want to accomplish.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: wronkiew on 06/15/2014 04:25 am
    Probably should put req (req is hosting the spxi image proxy scripts used for displaying images in the spreadsheets) on that list as well.

    Also I noticed that deruch is missing from the list. Deruch contributed to the ts cleanup effort.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Req on 06/15/2014 06:35 am
    So.. After going back and forth and trying to work out a way to get a list of 15 people who will receive SpaceX goodies, it has come to this, a vote!

    It sucks to have to whittle it down to 15, but this is the situation we find ourselves in.

    Everyone who wishes to vote should send me a PM on the forums with an ordered list of people (forum usernames please if possible) who they think deserve to get goodies. It doesn't need to be 15 long (the longer the better, so long as it is ordered) because I will collate the results of each list and pick the top 15. Also you can't vote for yourself =) Here is a list of names that has already been put forward in various places, you don't need to pick from this list, it's just here as a guide, if there is someone else you think is deserving then feel free to add them.

    Adaptation
    ajmartin
    arnezami
    dorkmo
    gospacex
    grythumn
    jakusb
    Jared (http://www.reddit.com/user/sharebrained)
    John_L
    lgjy98d (myroslav)
    Lourens
    mhenderson
    michaelni
    Mlindner
    moralec
    morningdew76
    mvpel
    princess
    Quialiss
    Saliva_Sweet
    seanpg71
    Shanuson
    SwissCheese
    theshadow27
    Turix
    Untribium
    wronkiew

    I couldn't find these names in the forum (I think they originated from the Wiki, if you know who these people are on the forum, PM me)
    Asmegin
    dgdpg
    Exclavion
    Gnonthgol
    lewing
    Noslyl
    pshrpd
    tentonine
    Turix

    If there is anyone who would not like to be on the list for whatever reason then drop me a PM and I will take that into account when adding up the numbers, also don't include me in your lists as I was lucky enough to receive something already (which I feel quite guilty about if I'm honest)

    Polls close on Sunday so vote now! I will then publish the anonymised results and ask the lucky 15 to PM me their gender (I'm not going to guess), shirt size, and delivery address. Something to bear in mind, assuming that everyone gets something similar to what I did (I hope this is the case), the package will NOT fit through a letter box (not even close ;)) so make sure the delivery address is somewhere where the package can be taken and signed for.

    Probably should put req (req is hosting the spxi image proxy scripts used for displaying images in the spreadsheets) on that list as well.

    ...Which JohnKiel made and continues to maintain and expand, adding database support and other features. :)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: wronkiew on 06/15/2014 07:31 am
    I am testing a change to the auto-generator script to pull mmbs from the spreadsheets instead of the wiki. This is not enabled for the clockfixer or the YouTubifier yet.

    Standard video:
    http://adama.nocdirect.com/~wronkiew/spx_crs3/landing_base_set_alt.mp4

    Slow-mo:
    http://adama.nocdirect.com/~wronkiew/spx_crs3/landing_base_set_alt_slow.mp4

    JPEG frames:
    http://adama.nocdirect.com/~wronkiew/spx_crs3/landing_base_set_alt_jpg.zip

    Generated mmb files:
    http://adama.nocdirect.com/~wronkiew/spx_crs3/mmbs_alt/

    There are a few minor differences in the generated mmbs between the wiki and the spreadsheets. Please take a look through the files and let me know if you see any problems. Otherwise it will go live tomorrow night.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: deruch on 06/15/2014 10:13 am
    Probably should put req (req is hosting the spxi image proxy scripts used for displaying images in the spreadsheets) on that list as well.

    Also I noticed that deruch is missing from the list. Deruch contributed to the ts cleanup effort.
     

    Thanks.  Iain originally included me on the list, but I PMed him to take me off of it.  I did like two little things, neither of them very well.  Really, I just wanted to be able to say that I helped/was a bit involved.  So, for me, mission accomplished.  Plus, it introduced me to this excellent forum, double bonus (for me anyways, not sure that it's to your benefit).
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: moralec on 06/15/2014 05:15 pm
    Hi guys,
    Just a note to let you know that with the help of Princess, Arnezami and wronkiew, I have started to work on a new "progress report". The idea is to come up with a long write up describing all the different aspects of the video recovery effort (MMB sequences, bitfilps, clock recovery, etc). I'm also planning to include a section for video interpretation.  I'm thinking this could eventually be used as a base for a NSF article. 

    The latest version is the wiki (http://spacexlanding.wikispaces.com/Latest+Progress+Report). Any comments or additions are more than appreciated, please feel free to add new paragraphs or to improve what is already there.



    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Jester on 06/15/2014 06:27 pm
    Hi guys,
    Just a note to let you know that with the help of Princess, Arnezami and wronkiew, I have started to work on a new "progress report". The idea is to come up with a long write up describing all the different aspects of the video recovery effort (MMB sequences, bitfilps, clock recovery, etc). I'm also planning to include a section for video interpretation.  I'm thinking this could eventually be used as a base for a NSF article. 

    The latest version is the wiki (http://spacexlanding.wikispaces.com/Latest+Progress+Report). Any comments or additions are more than appreciated, please feel free to add new paragraphs or to improve what is already there.


    I'm cleaning it up a bit, did we ever get a picture of the famous pizza-dish antenna ? it would be nice to add it .
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Quialiss on 06/15/2014 07:33 pm
    <snip>
    Generated mmb files:
    http://adama.nocdirect.com/~wronkiew/spx_crs3/mmbs_alt/

    There are a few minor differences in the generated mmbs between the wiki and the spreadsheets. Please take a look through the files and let me know if you see any problems. Otherwise it will go live tomorrow night.

    I ran a diff on those and and the ones generated from the wiki.. someone may have gotten to them before me as I couldn't see any differences anymore.  So all good to go :) 
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Lourens on 06/15/2014 08:01 pm
    Well, I'm still working on automated bit flipping, and a frame that no human could or would do sounds like just what we would like to have an automatic system for. I think I'm getting a little bit closer to something workable, although I doubt it will ever be fully automated. But that's okay. And even if we never get I-frame 1 back, it may still be useful for filling in some of the gaps in the more interesting I-frames.

    <snip>

    The code is an ugly hack in Bash, but if anyone wants it, just let me know. Meanwhile I'll keep fiddling with it :).

    I didn't realize you were doing them automatically!  I wouldn't mind seeing your script. 

    Have you tested it on any of the more intact frames?  That should help figuring out the 'correctness' thresholds, giving you a reference frame to look at to find any places where your script is making incorrect flips.

    The flips I posted before were done by hand, and I'm now trying to automate that process. I agree that it would be good to try it on one of the frames that we have some ground truth for. I'll try that next time I have time to work on this, hopefully somewhere in the coming week.

    I've attached the script, the original extension is .sh, but that doesn't fly with the forum software, which is a good thing. Like I said it's a bit of a mess, and it's getting a bit too complex for bash really, but my Python is rusty, C++ (my usual weapon of choice) even less suitable, and Octave isn't really it either, so bash it is.

    Hi guys,
    Just a note to let you know that with the help of Princess, Arnezami and wronkiew, I have started to work on a new "progress report". The idea is to come up with a long write up describing all the different aspects of the video recovery effort (MMB sequences, bitfilps, clock recovery, etc). I'm also planning to include a section for video interpretation.  I'm thinking this could eventually be used as a base for a NSF article.

    Great! I've been working on an article with Chris. It's in the early draft stages, and we think we can make something good out of it. I don't have time right now, but I'll have a look next time I work on the article.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Quialiss on 06/16/2014 06:21 am
    I got around to calculating those statistics on triple flip patterns.  Without further ado, the top 15.  The top 2 were already known, but the top 3-5 may be useful to know, beyond that the statistical significance fades drastically.

    10   100011000000001100101
    10   10111000000000111001
    10   11111000000000100001
    11   10011000000000110101
    13   10001000000000110011
    13   1101000000000010111
    14   10000001000000110000011
    14   100001000000001100011
    18   1011000000000011101
    20   1111000000000010001
    25   1001000000000011011
    54   101000000000001111
    87   111000000000001001
    342 11000000000000101
    754 1000000000000011

    With reasonable completeness, searching for 10 flips within 10 bits of the start of the first flip, and 4 flips within 16 bits.  So the fact that the most common flips are within 4 bits is not a sampling bias, and may be helpful with the automated bit flip searches, reducing the space to search. 

    The other way to go about calculating stats on relative occurrences of the flips would be to take a portion of the padding and attempt to cancel out all the differences between princess's two files with triple flips.  That assumes that all the flips are triple flips, if that's true it would give a more complete picture of the distribution of the flips in the heavily corrupted areas, as simple matching can't find a constantly overlapping sequence of flips.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: wronkiew on 06/16/2014 07:09 am
    <snip>
    Generated mmb files:
    http://adama.nocdirect.com/~wronkiew/spx_crs3/mmbs_alt/

    There are a few minor differences in the generated mmbs between the wiki and the spreadsheets. Please take a look through the files and let me know if you see any problems. Otherwise it will go live tomorrow night.

    I ran a diff on those and and the ones generated from the wiki.. someone may have gotten to them before me as I couldn't see any differences anymore.  So all good to go :) 

    Thanks for checking it over. I have enabled this by default. The newest version on YouTube was generated from the spreadsheets.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: mhenderson on 06/16/2014 01:06 pm
    Is an evolution / linear programming approach to bitflipping workable?

    Some optimization algorithms work by exploring a gradient to find the way down a sloping tableau to a minimum.  Example: Find the deepest part of an ocean by systematically dropping 1000 large lead spheres and determine where they settle. (Assume they roll downhill). From the best candidates, repeat the process in that neighborhood using smaller spheres. This works as long as the deepest part of the ocean isn't a well drilled in a flat plateau. Flat areas would be eliminated early in the search.

    In our case, we are searching for good MBs. Do our spheres roll downhill?  i.e. Are improvements incremental? We have some evidence (triple flips), bands of bad MBs, etc. that these errors came in bunches.

    We know that "good" MBs score well in certain criteria such a "MB has no quant error", "iFrame MB has a size that is comparable to other iFrame MBs in the same position", and "dependent neighbor MBs to the right and below do not have errors." You could even add "this bitflip is part of a triple flip that further improves the MB Score".

    1) Develop a MB scoring metric: Start with good MBs, add random bitflips. If errors accumulate to continuously decrease the "MB score", then that is a good metric that might be applied to bad MBs.

    2) Use the metric on bad MBs ... add random bitflips, keep the ones that improve the "MB score". Lather, rinse, repeat. 
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: cscott on 06/16/2014 03:09 pm
    1) Develop a MB scoring metric: Start with good MBs, add random bitflips. If errors accumulate to continuously decrease the "MB score", then that is a good metric that might be applied to bad MBs.

    I think the best MB scoring metric would be "closeness to hand-tuned MB".  We've already done a lot of work on manual MB fixup, but probably missed some bit flips in LSBs which are invisible (but accumulate).   A triple flip search could use the work we've already done by hand to guide an automated search for triple flip patterns, potentially fixing up some LSBs and finding valid MB corrections where we are currently using alignment or DC level overrides of various kinds.

    That is, use the PNG output for am i frame as the target, and see if we can improve on what we have.  iframe improvements should lead to p-frame improvements, etc.  Hopefully we can use the humans to bootstrap the automated search, by giving it a solid idea of what it should be looking for.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: IainCole on 06/16/2014 05:53 pm
    The results are in.

    In a slight change of plan I want to get information from everyone who received at least 1 vote, this way if SpaceX are feeling extra generous then they have everything they need ;)

    So in no particular order (actually alphabetically) here are the chosen 15:

    arnezami
    Lourens
    mhenderson
    michaelni
    Mlindner
    moralec
    mvpel
    Princess
    Quialiss
    Saliva_Sweet
    Shanuson
    SwissCheese
    theshadow27
    Untribium
    wronkiew

    And here are the other 15 who I still want to get addresses for :)

    Adaptation
    ajmartin
    dgdpg
    dorkmo
    Exclavion
    gospacex
    grythumn
    jakusb 
    Jared
    John_L
    lgjy98d
    morningdew76
    pshrpd
    seanpg71
    Turix

    I need the following information from people:

    Name:
    Gender: (for clothing purposes I'm not a stalker)
    T shirt size: (S/M/L/XL/XXL)
    Address: (this needs to be somewhere that a fairly large package can be taken and signed for e.g. a work address, don't forget to include the country ;))

    I'm going to start chasing people pretty soon :) However I won't chase forever, if I haven't got the information by the end of the week then you will be removed from the list :o (Hopefully it doesn't come to that though)

    Thank you to everyone that voted, and in particular to those who graciously removed themselves from the list.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: IainCole on 06/16/2014 06:16 pm
    Hi guys,
    Just a note to let you know that with the help of Princess, Arnezami and wronkiew, I have started to work on a new "progress report". The idea is to come up with a long write up describing all the different aspects of the video recovery effort (MMB sequences, bitfilps, clock recovery, etc). I'm also planning to include a section for video interpretation.  I'm thinking this could eventually be used as a base for a NSF article. 

    The latest version is the wiki (http://spacexlanding.wikispaces.com/Latest+Progress+Report). Any comments or additions are more than appreciated, please feel free to add new paragraphs or to improve what is already there.


    I'm cleaning it up a bit, did we ever get a picture of the famous pizza-dish antenna ? it would be nice to add it .

    I also corrected some things (spelling/grammar mainly) and left a couple of comments.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: catdlr on 06/16/2014 07:10 pm
    Perhaps a few seconds of credits could be added to the end of the video so that everyone involved has their member name (or real name) noted?
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: moralec on 06/16/2014 09:13 pm
    Great! I've been working on an article with Chris. It's in the early draft stages, and we think we can make something good out of it. I don't have time right now, but I'll have a look next time I work on the article.

    Awesome Lourens! I´m happy to integrate both if that works for you.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: saliva_sweet on 06/16/2014 11:51 pm
    I'm able to convert I frames to P frames now, this enables using the -3 mmb command in iframes. Nobody is using -1 or -2 mbs in p frames because -3 is just much better for the video. I think there are iframes (particularly towards the end of the video) that may greatly benefit. However, mmb conversion isn't working yet and manual conversion is not an option at this point. So results look pretty crappy atm. I'm attaching the leg deploy sequence where iframe 81 is converted to pframe. dcs get thrown, all looks ugly, but I think there's potential.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Quialiss on 06/17/2014 12:16 am
    I'm able to convert I frames to P frames now, this enables using the -3 mmb command in iframes. Nobody is using -1 or -2 mbs in p frames because -3 is just much better for the video. I think there are iframes (particularly towards the end of the video) that may greatly benefit. However, mmb conversion isn't working yet and manual conversion is not an option at this point. So results look pretty crappy atm. I'm attaching the leg deploy sequence where iframe 81 is converted to pframe. dcs get thrown, all looks ugly, but I think there's potential.

    It looks like the issues with the i frame converted to p frame are caused where we're using -1 or -2 with DC adjustments to make the next block look better?  That shouldn't be too much of a headache to work around, and you're right, it could seriously improve the look of parts 12-15, with their large gaps.  Nicely done.   :)

    Also, I do sometimes use -1 in p frames.  Not often, but it has it's uses on frames where it's better to have the DC data on the missing mbs rather than leaving them blank. 
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Quialiss on 06/17/2014 02:35 am
    Progress report!

    Current
    https://www.youtube.com/watch?v=lBgVTWbtgVQ 


    10 days ago   
    https://www.youtube.com/watch?v=-oU3knnvj6M


    I'm reaching the end of what I can do with MMB work.  Parts 3 and 4 are the only ones left that I know need working on, SwissCheese's alignments, and then my work on the DC values.  More eyes on the frames to spot things that have been missed are always appreciated!
    Spreadsheet Review (https://docs.google.com/spreadsheets/d/1RZWykghV6XTuMRjikDleiX_J-ifG0g548Fta5WTRY88/edit#gid=0)
    Download all frames (http://adama.nocdirect.com/~wronkiew/spx_crs3/landing_base_set_jpg.zip) (I think that's the right one.  wronkiew?)

    Promising avenues that I know of are saliva_sweet's work with converting i frames to p frames to effectively fill gaps in the i frames, Lourens's work on automatically finding bitflips in i frames to recover more data, and wronkiew's work restoring the clock.  Is there anything ongoing I've missed? 
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: wronkiew on 06/17/2014 04:58 am
    Download all frames (http://adama.nocdirect.com/~wronkiew/spx_crs3/landing_base_set_jpg.zip) (I think that's the right one.  wronkiew?)

    Yes, that is the hourly image build.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: mhenderson on 06/17/2014 08:22 am
    Side by side comparison in youtubedoubler of the ten day progress videos posted by @Quialiss. The newer video is on the right.

    http://youtubedoubler.com/cID9 (http://youtubedoubler.com/cID9)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: mhenderson on 06/17/2014 08:38 am
    Side by side comparison of current video (right) to original raw (left).

    http://youtubedoubler.com/cIDx

    The videos repeat but have different lengths. Click the Double Up button or reload the page to re-start both.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: zodiacchris on 06/17/2014 09:51 am
    Guys,
    What you have achieved leaves me speechless, brilliant work!! I would never have thought this quality would be possible, well done!  :D
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Paul Adams on 06/17/2014 10:27 am
    Superb work everyone, really well done.

    I would have thought that deserves a SpaceX t-shirt for the main contributors!

    Paul
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: kevinof on 06/17/2014 10:33 am
    Wow well done everyone. Least SpaceX can do is offer you all a free ride in one!  :)

    Fantastic work.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: SwissCheese on 06/17/2014 11:36 am
    I finished corrected part 3, that was difficult and I'm not completely sure of the alignment for the first frames. Now only part 4 remains (also quite difficult...).

    Meanwhile I made a gif animation of the p-frame alignment process (with frame 111) which could maybe be added to the progress report.

    Edit: small corrections
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Lourens on 06/17/2014 11:39 am
    I agree, that is really quite incredible! Has anyone looked yet at the motion vectors in the P-frames? It looks like there's some improvement to be had there still.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: SwissCheese on 06/17/2014 12:31 pm
    I agree, that is really quite incredible! Has anyone looked yet at the motion vectors in the P-frames? It looks like there's some improvement to be had there still.

    Yes, but the best we can do now when we find an error with a motion vector is to discard the full macroblock.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: moralec on 06/17/2014 02:19 pm
    Great progress guys!! Almost all frames have now been fixed...

    Just to check: Is anybody working on Parts 1 and 2?  Those are the only ones that remain untouched....

    It looks like we are getting closer to the limits of the MMB approach. Any new ideas on how to continue moving forward?  I'm guessing that princess and others are still working on the triple bit flips... any help you guys need with that? Or what other things we should begin consider? Traditional Image correction? Interpolation of frames?

    By the way, looking at the latest video I think we are going to need to add the clock fixes for frames 168-180. In the latest you tube version, the clock stays at 19:35:8 for two seconds and then jumps to 19:35:10 (Check from 0:07 to  0:10 in the video)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: SwissCheese on 06/17/2014 04:11 pm
    Great progress guys!! Almost all frames have now been fixed...

    Just to check: Is anybody working on Parts 1 and 2?  Those are the only ones that remain untouched....

    It looks like we are getting closer to the limits of the MMB approach. Any new ideas on how to continue moving forward?  I'm guessing that princess and others are still working on the triple bit flips... any help you guys need with that? Or what other things we should begin consider? Traditional Image correction? Interpolation of frames?

    By the way, looking at the latest video I think we are going to need to add the clock fixes for frames 168-180. In the latest you tube version, the clock stays at 19:35:8 for two seconds and then jumps to 19:35:10 (Check from 0:07 to  0:10 in the video)

    I had a look at parts 1 and 2, there are very few good data sequences until frame 28; from this point on, the data are OK. But it does not make too much sense to spend time on these p-frames when there is no i-frame...
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Chris Bergin on 06/17/2014 04:12 pm
    Great stuff, so are we getting to the point where we call this as best that can be done? I note moralec's post of course, but it seems we're very close.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: mhenderson on 06/17/2014 04:59 pm
    Great progress guys!! Almost all frames have now been fixed...

    Just to check: Is anybody working on Parts 1 and 2?  Those are the only ones that remain untouched....

    It looks like we are getting closer to the limits of the MMB approach. Any new ideas on how to continue moving forward?  I'm guessing that princess and others are still working on the triple bit flips... any help you guys need with that? Or what other things we should begin consider? Traditional Image correction? Interpolation of frames?

    By the way, looking at the latest video I think we are going to need to add the clock fixes for frames 168-180. In the latest you tube version, the clock stays at 19:35:8 for two seconds and then jumps to 19:35:10 (Check from 0:07 to  0:10 in the video)

    I believe that @Wronkiew is (correctly) applying a "99.9% certainty" rule to clock fixes. This is a 'restore vs enhance' issue. I also believe that the clock MBs are badly corrupted in that region. There is no way to visually tell precisely when :08 becomes :09 so he left it alone.

    Another complicating factor - the 15 FPS frame rate does not align cleanly with hundredths of a second. This limits the ability to overlay timestamps with absolute precision, so @Wronkiew is not "enhancing" with a best guess.

    An argument could be made for using the clock in the TS header (@princess mentioned it, I think) to map frames to a timestamp. That TS counter is not an extrapolation; it is hard evidence - just not from visual MB information.

    Yes, the the clock **could** be completely "fixed" to be visually smoothed. Be aware that would introduce the possibility of small errors. When all is said and done, I support @Wronkiew's judgement on this issue.


    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: cscott on 06/17/2014 05:29 pm
    Yes, the the clock **could** be completely "fixed" to be visually smoothed. Be aware that would introduce the possibility of small errors. When all is said and done, I support @Wronkiew's judgement on this issue.

    This is another test case for automatic triple flip detection.  I wish I had more time to work on this myself!  But here we have a fairly good idea of exactly what the clock should look like, so it should be possible to turn loose an automatic tool that looked for the best set of triple flips that gives the expected output.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: princess on 06/17/2014 05:54 pm
    An argument could be made for using the clock in the TS header (@princess mentioned it, I think) to map frames to a timestamp. That TS counter is not an extrapolation; it is hard evidence - just not from visual MB information.

    If anyone needs the TS timestamps for each frame, I've included them here. First column is frame number, second column is time in seconds after the start of the recording (note that we don't have the start):


    1 62.907178
    2 62.973911
    3 63.040644
    4 63.107378
    5 63.174111
    6 63.240844
    7 63.307578
    8 63.374311
    9 63.441044
    10 63.507778
    11 63.574511
    12 63.641244
    13 63.707978
    14 63.774711
    15 63.841444
    16 63.908178
    17 63.974911
    18 64.041644
    19 64.108378
    20 64.175111
    21 64.241844
    22 64.308578
    23 64.375311
    24 64.442044
    25 64.508778
    26 64.575511
    27 64.642244
    28 64.708978
    29 64.775711
    30 64.842444
    31 64.909178
    32 64.975911
    33 65.042644
    34 65.109378
    35 65.176111
    36 65.242844
    37 65.309578
    38 65.376311
    39 65.443044
    40 65.509778
    41 65.576511
    42 65.643244
    43 65.709978
    44 65.776711
    45 65.843444
    46 65.910178
    47 65.976911
    48 66.043644
    49 66.110378
    50 66.177111
    51 66.243844
    52 66.310578
    53 66.377311
    54 66.444044
    55 66.510778
    56 66.577511
    57 66.644244
    58 66.710978
    59 66.777711
    60 66.844444
    61 66.911178
    62 66.977911
    63 67.044644
    64 67.111378
    65 67.178111
    66 67.244844
    67 67.311578
    68 67.378311
    69 67.445044
    70 67.511778
    71 67.578511
    72 67.645244
    73 67.711978
    74 67.778711
    75 67.845444
    76 67.912178
    77 67.978911
    78 68.045644
    79 68.112378
    80 68.179111
    81 68.245844
    82 68.312578
    83 68.379311
    84 68.446044
    85 68.512778
    86 68.579511
    87 68.646244
    88 68.712978
    89 68.779711
    90 68.846444
    91 68.913178
    92 68.979911
    93 69.046644
    94 69.113378
    95 69.180111
    96 69.246844
    97 69.313578
    98 69.380311
    99 69.447044
    100 69.513778
    101 69.580511
    102 69.647244
    103 69.713978
    104 69.780711
    105 69.847444
    106 69.914178
    107 69.980911
    108 70.047644
    109 70.114378
    110 70.181111
    111 70.247844
    112 70.314578
    113 70.381311
    114 70.448044
    115 70.514778
    116 70.581511
    117 70.648244
    118 70.714978
    119 70.781711
    120 70.848444
    121 70.915178
    122 70.981911
    123 71.048644
    124 71.115378
    125 71.182111
    126 71.248844
    127 71.315578
    128 71.382311
    129 71.449044
    130 71.515778
    131 71.582511
    132 71.649244
    133 71.715978
    134 71.782711
    135 71.849444
    136 71.916178
    137 71.982911
    138 72.049644
    139 72.116378
    140 72.183111
    141 72.249844
    142 72.316578
    143 72.383311
    144 72.450044
    145 72.516778
    146 72.583511
    147 72.650244
    148 72.716978
    149 72.783711
    150 72.850444
    151 72.917178
    152 72.983911
    153 73.050644
    154 73.117378
    155 73.184111
    156 73.250844
    157 73.317578
    158 73.384311
    159 73.451044
    160 73.517778
    161 73.584511
    162 73.651244
    163 73.717978
    164 73.784711
    165 73.851444
    166 73.918178
    167 73.984911
    168 74.051644
    169 74.118378
    170 74.185111
    171 74.251844
    172 74.318578
    173 74.385311
    174 74.452044
    175 74.518778
    176 74.585511
    177 74.652244
    178 74.718978
    179 74.785711
    180 74.852444
    181 74.919178
    182 74.985911
    183 75.052644
    184 75.119378
    185 75.186111
    186 75.252844
    187 75.319578
    188 75.386311
    189 75.453044
    190 75.519778
    191 75.586511
    192 75.653244
    193 75.719978
    194 75.786711
    195 75.853444
    196 75.920178
    197 75.986911
    198 76.053644
    199 76.120378
    200 76.187111
    201 76.253844
    202 76.320578
    203 76.387311
    204 76.454044
    205 76.520778
    206 76.587511
    207 76.654244
    208 76.720978
    209 76.787711
    210 76.854444
    211 76.921178
    212 76.987911
    213 77.054644
    214 77.121378
    215 77.188111
    216 77.254844
    217 77.321578
    218 77.388311
    219 77.455044
    220 77.521778
    221 77.588511
    222 77.655244
    223 77.721978
    224 77.788711
    225 77.855444
    226 77.922178
    227 77.988911
    228 78.055644
    229 78.122378
    230 78.189111
    231 78.255844
    232 78.322578
    233 78.389311
    234 78.456044
    235 78.522778
    236 78.589511
    237 78.656244
    238 78.722978
    239 78.789711
    240 78.856444
    241 78.923178
    242 78.989911
    243 79.056644
    244 79.123378
    245 79.190111
    246 79.256844
    247 79.323578
    248 79.390311
    249 79.457044
    250 79.523778
    251 79.590511
    252 79.657244
    253 79.723978
    254 79.790711
    255 79.857444
    256 79.924178
    257 79.990911
    258 80.057644
    259 80.124378
    260 80.191111
    261 80.257844
    262 80.324578
    263 80.391311
    264 80.458044
    265 80.524778
    266 80.591511
    267 80.658244
    268 80.724978
    269 80.791711
    270 80.858444
    271 80.925178
    272 80.991911
    273 81.058644
    274 81.125378
    275 81.192111
    276 81.258844
    277 81.325578
    278 81.392311
    279 81.459044
    280 81.525778
    281 81.592511
    282 81.659244
    283 81.725978
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: wronkiew on 06/17/2014 05:58 pm
    Yes, the the clock **could** be completely "fixed" to be visually smoothed. Be aware that would introduce the possibility of small errors. When all is said and done, I support @Wronkiew's judgement on this issue.

    This is another test case for automatic triple flip detection.  I wish I had more time to work on this myself!  But here we have a fairly good idea of exactly what the clock should look like, so it should be possible to turn loose an automatic tool that looked for the best set of triple flips that gives the expected output.

    As was pointed out earlier by those more familiar with packet issues, a big problem with the clock is that the data is just not there. We would need to fix the packets and update the .ts files.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: wronkiew on 06/17/2014 06:09 pm
    Great progress guys!! Almost all frames have now been fixed...

    Just to check: Is anybody working on Parts 1 and 2?  Those are the only ones that remain untouched....

    It looks like we are getting closer to the limits of the MMB approach. Any new ideas on how to continue moving forward?  I'm guessing that princess and others are still working on the triple bit flips... any help you guys need with that? Or what other things we should begin consider? Traditional Image correction? Interpolation of frames?

    By the way, looking at the latest video I think we are going to need to add the clock fixes for frames 168-180. In the latest you tube version, the clock stays at 19:35:8 for two seconds and then jumps to 19:35:10 (Check from 0:07 to  0:10 in the video)

    It sounds like you're looking for a project to do! I have clock fixes for 161-167, but I haven't gotten to 168-180 yet.

    I recently put together a new script that generates erase commands. This makes fixing the clock a lot easier. You'll need a php interpreter that has the GD functions included to run it. I attached the script and an example mask image. You run "php -f mask_to_erase.php mask-051.png", and it outputs a list of commands that you can copy into the clockfix file.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: moralec on 06/17/2014 06:18 pm
    An argument could be made for using the clock in the TS header (@princess mentioned it, I think) to map frames to a timestamp. That TS counter is not an extrapolation; it is hard evidence - just not from visual MB information.

    Yes, the the clock **could** be completely "fixed" to be visually smoothed. Be aware that would introduce the possibility of small errors. When all is said and done, I support @Wronkiew's judgement on this issue.

    I support  the idea of restrincting the modifications to hard evidence. But we don't need to be confided to visual MB information. If we are sure that the TS header info is fine, why not using it? :)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: wronkiew on 06/17/2014 06:42 pm
    The MPEG timestamps may be useful for recovering some frames. Just keep in mind that each frame displays two different times. You'll also need to synchronize the clocks to millisecond precision at least.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: saliva_sweet on 06/17/2014 08:40 pm
    mmb autoconversion works now for i frames that have been converted to p frames. Conversion can introduce problems occasionally e.g. in iframe 81 a string of blocks looked good in the original, but bad after conversion, my guess is they may contain an imperceptible error and therefore don't convert correctly, this in turn caused dc issues. As proof of concept I submit iframe 281 converted to p frame. The method worked well with little effort for this one because the blank region was already set to -2. Blocks set to -3 will behave like -2 blocks for the subsequent mbs i.e. they provide no inheritance which is both good and bad - they shouldn't cause propagating errors, but the dc's for subsequent blocks would need to be added manually. This would be much easier in the online editor.

    IainCole, would it be possible to put these frames into the online editor so others could take a crack at setting blocks to -3 and filling the blanks in iframes? Lot of dc work is needed for this.

    Edit: gif not animating, so zip it must be.
    Edit2: I'll also add a png of the actual frame 281.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: IainCole on 06/17/2014 08:43 pm
    mmb autoconversion works now for i frames that have been converted to p frames. Conversion can introduce problems occasionally e.g. in iframe 81 a string of blocks looked good in the original, but bad after conversion, my guess is they may contain an imperceptible error and therefore don't convert correctly, this in turn caused dc issues. As proof of concept I submit iframe 281 converted to p frame. The method worked well with little effort for this one because the blank region was already set to -2. Blocks set to -3 will behave like -2 blocks for the subsequent mbs i.e. they provide no inheritance which is both good and bad - they shouldn't cause propagating errors, but the dc's for subsequent blocks would need to be added manually. This would be much easier in the online editor.

    IainCole, would it be possible to put these frames into the online editor so others could take a crack at setting blocks to -3 and filling the blanks in iframes? Lot of dc work is needed for this.

    You want me to do what?
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: saliva_sweet on 06/17/2014 08:50 pm
    You want me to do what?
    Add iframes converted to pframes to the online editor so mmbs for them could be better edited. They are mpg4-img files, which I would provide. Not sure how your setup works currently.

    edit: They should probably be separate from the main final_fixed.ts frames like there used to be several selecable ts versions earlier. Would that be possible?
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: IainCole on 06/17/2014 09:15 pm
    You want me to do what?
    Add iframes converted to pframes to the online editor so mmbs for them could be better edited. They are mpg4-img files, which I would provide. Not sure how your setup works currently.

    edit: They should probably be separate from the main final_fixed.ts frames like there used to be several selecable ts versions earlier. Would that be possible?

    The p-frame editor was build to handle only 1 TS set, but I can add alternative frames into the frameset
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: saliva_sweet on 06/17/2014 09:31 pm
    The p-frame editor was build to handle only 1 TS set, but I can add alternative frames into the frameset

    Would it be possible to clone it and put it to a different url altogether? Alternatively maybe it's possible to add sections to the dropdown menu (like P-15(281))? Are you using mpg4-imgs or ts as the ffmpeg input? Because to converted frames are mpg4-imgs and can't be (easily) put back into the ts container.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: SwissCheese on 06/17/2014 11:35 pm
    mmb autoconversion works now for i frames that have been converted to p frames. Conversion can introduce problems occasionally e.g. in iframe 81 a string of blocks looked good in the original, but bad after conversion, my guess is they may contain an imperceptible error and therefore don't convert correctly, this in turn caused dc issues. As proof of concept I submit iframe 281 converted to p frame. The method worked well with little effort for this one because the blank region was already set to -2. Blocks set to -3 will behave like -2 blocks for the subsequent mbs i.e. they provide no inheritance which is both good and bad - they shouldn't cause propagating errors, but the dc's for subsequent blocks would need to be added manually. This would be much easier in the online editor.

    IainCole, would it be possible to put these frames into the online editor so others could take a crack at setting blocks to -3 and filling the blanks in iframes? Lot of dc work is needed for this.

    Edit: gif not animating, so zip it must be.
    Edit2: I'll also add a png of the actual frame 281.

    Could you explain what you did with more details? Did you modify the program (h263dec.c)? I don't really understand... And what about the frames following the one you converted? Do they also inherit the data?
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: saliva_sweet on 06/17/2014 11:59 pm
    mmb autoconversion works now for i frames that have been converted to p frames. Conversion can introduce problems occasionally e.g. in iframe 81 a string of blocks looked good in the original, but bad after conversion, my guess is they may contain an imperceptible error and therefore don't convert correctly, this in turn caused dc issues. As proof of concept I submit iframe 281 converted to p frame. The method worked well with little effort for this one because the blank region was already set to -2. Blocks set to -3 will behave like -2 blocks for the subsequent mbs i.e. they provide no inheritance which is both good and bad - they shouldn't cause propagating errors, but the dc's for subsequent blocks would need to be added manually. This would be much easier in the online editor.

    IainCole, would it be possible to put these frames into the online editor so others could take a crack at setting blocks to -3 and filling the blanks in iframes? Lot of dc work is needed for this.

    Edit: gif not animating, so zip it must be.
    Edit2: I'll also add a png of the actual frame 281.

    Could you explain what you did with more details? Did you modify the program (h263dec.c)? I don't really understand...

    No, I'm not skilled enough to touch the ffmpeg source. Frames can contain intra and inter macroblocks. I frames contain only inrtras, but p frames have both. I changed the header of the I frame into a p frame header and converted all I frame macroblocks to p frame intra macroblocks so that the new p frame contains only intra mbs. This turned out to be surprisingly easy because all that needed to be done was adding the not_coded bit (with zero value) to each macroblock and the mcbpc had to be changed to pframe equivalent (there are only 4 valid ones in this vid).

    And what about the frames following the one you converted? Do they also inherit the data?

    Yes, data (and errors for that matter, but this could be easily mitigated) can potentially propagate from the first frame to the last in the video with this method.

    Meanwhile, here's another piece of low hanging fruit. Parts 12+13 from the video with iframe 241 in the middle. This time in mp4 format.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Quialiss on 06/18/2014 06:50 am
    Meanwhile, here's another piece of low hanging fruit. Parts 12+13 from the video with iframe 241 in the middle. This time in mp4 format.

    Nice to see part 13 with the top half of the frame as more than just an accumulation of the p frames!  Other than the chroma mismatch that's looking pretty seamless.  :D


    I finished corrected part 3, that was difficult and I'm not completely sure of the alignment for the first frames. Now only part 4 remains (also quite difficult...)

    Parts 3 and 4 are certainly nightmare material, lots of corruption, lots of data to fix up in each p frame because of how much the frames change as the rocket drops through the clouds... I've done a first pass over them now to fix all the major issues, lots of medium-minor ones left. 
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: pippin on 06/18/2014 10:10 am
    @Chris

    could you maybe add the link to the YouTube channel (and maybe the Wiki, too) to the first post of the thread.
    Makes it easier to find; I just realized that a search on YouTube finds all kind of old versions first.

    I'm talking about this one:
    https://www.youtube.com/channel/UCyZDgyJBYz3OXD3JbDJzNww
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Chris Bergin on 06/18/2014 12:15 pm
    @Chris

    could you maybe add the link to the YouTube channel (and maybe the Wiki, too) to the first post of the thread.
    Makes it easier to find; I just realized that a search on YouTube finds all kind of old versions first.

    I'm talking about this one:
    https://www.youtube.com/channel/UCyZDgyJBYz3OXD3JbDJzNww

    Copy that. Done. :)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: SwissCheese on 06/18/2014 01:08 pm
    Perhaps a few seconds of credits could be added to the end of the video so that everyone involved has their member name (or real name) noted?

    I added 3 frames to the video. Nothing very artistic, so if someone makes something nicer it would be very cool. Also tell me if I forgot to mention someone, or if we should put the real names, or if something should be changed (colors, add pictures, ...)

    (P.S.: my video does not have the clock corrections since I generate it locally)

    edit: additional_frames.zip contains the logos, adobe illustrator files and png images
    edit2: added Swatch to the list
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: pippin on 06/18/2014 01:58 pm
    Copy that. Done. :)

    Thank you! :)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: catdlr on 06/18/2014 03:57 pm
    Perhaps a few seconds of credits could be added to the end of the video so that everyone involved has their member name (or real name) noted?

    I added 3 frames to the video. Nothing very artistic, so if someone makes something nicer it would be very cool. Also tell me if I forgot to mention someone, or if we should put the real names, or if something should be changed (colors, add pictures, ...)

    (P.S.: my video does not have the clock corrections since I generate it locally)

    edit: additional_frames.zip contains the logos, adobe illustrator files and png images
    edit2: added Swatch to the list

    excellent SwissCheese and more.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: wronkiew on 06/18/2014 04:49 pm
    I did some experimenting with interlaced output. We've been taking interlaced source material and outputting it as 15p. We should be able to output partial frames at a higher rate. Indeed, the clock difference between partial frames is ~0.04s. However, the motion of the rest of the video does not seem to match the clock spacing, so while the video is smoother it is not the slam dunk I had hoped it would be. I attached both formats for comparison. _clockfix.mp4 is 15p, _clockfix_30.mp4 is 30i.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: IainCole on 06/18/2014 05:29 pm
    The p-frame editor was build to handle only 1 TS set, but I can add alternative frames into the frameset

    Would it be possible to clone it and put it to a different url altogether? Alternatively maybe it's possible to add sections to the dropdown menu (like P-15(281))? Are you using mpg4-imgs or ts as the ffmpeg input? Because to converted frames are mpg4-imgs and can't be (easily) put back into the ts container.

    It uses TS files, 1 TS file per Iframe and subsequent p frames, no idea if / how it would work with mp4-img files
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: saliva_sweet on 06/18/2014 06:02 pm
    It uses TS files, 1 TS file per Iframe and subsequent p frames, no idea if / how it would work with mp4-img files

    It should work identically I believe. ffmpeg -i part_x.ts behaves exactly the same as ffmpeg -i part_x_frame_%2d.mpg4-img. At least in my experience.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: IainCole on 06/18/2014 06:03 pm
    It uses TS files, 1 TS file per Iframe and subsequent p frames, no idea if / how it would work with mp4-img files

    It should work identically I believe. ffmpeg -i part_x.ts behaves exactly the same as ffmpeg -i part_x_frame_%2d.mpg4-img. At least in my experience.

    OK cool, you gonna send me the stuff?
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: saliva_sweet on 06/18/2014 06:09 pm
    I'll put it together in a zip and post here maybe? It might take a little time, all frames aren't converted yet.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: moralec on 06/18/2014 06:12 pm
    I did some experimenting with interlaced output. We've been taking interlaced source material and outputting it as 15p. We should be able to output partial frames at a higher rate. Indeed, the clock difference between partial frames is ~0.04s. However, the motion of the rest of the video does not seem to match the clock spacing, so while the video is smoother it is not the slam dunk I had hoped it would be. I attached both formats for comparison. _clockfix.mp4 is 15p, _clockfix_30.mp4 is 30i.

    I think it looks better! Any idea on how to treat these partial frames? what they contain exactly?
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: mvpel on 06/18/2014 06:14 pm
    I did some experimenting with interlaced output. We've been taking interlaced source material and outputting it as 15p. We should be able to output partial frames at a higher rate. Indeed, the clock difference between partial frames is ~0.04s. However, the motion of the rest of the video does not seem to match the clock spacing, so while the video is smoother it is not the slam dunk I had hoped it would be. I attached both formats for comparison. _clockfix.mp4 is 15p, _clockfix_30.mp4 is 30i.

    I think what we were able to determine with Princess' analysis of the settings of the video is that the MPEG is encoded as ~15fps progressive, and the interlacing artifacts we're seeing in the clock and elsewhere likely arise from the encoder being fed by a ~30fps NTSC interlaced video camera, which was interlacing two ~30fps frames into a single frame which was then encoded as progressive at ~15fps for transmission.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: princess on 06/18/2014 07:33 pm
    I think what we were able to determine with Princess' analysis of the settings of the video is that the MPEG is encoded as ~15fps progressive, and the interlacing artifacts we're seeing in the clock and elsewhere likely arise from the encoder being fed by a ~30fps NTSC interlaced video camera, which was interlacing two ~30fps frames into a single frame which was then encoded as progressive at ~15fps for transmission.

    When talking about interlaced source material it helps to distinguish fields from frames. A "field" in this case is a half-height image consisting of alternate lines from the video camera. Two fields make a frame, where a frame is a full-height image. I'm used to PAL video where a camera captures 50 fields per second and interlaces them into 25 frames per second; for NTSC this is ~60 fields per second, interlaced to ~30 frames per second.

    Without knowing SpaceX's transmission setup it's hard to know what's happening, but it could be that the camera is capturing at ~60 fields per second (standard NTSC). The timestamp might be overlaid onto each field separately, before they get combined into frames. Then the MPEG4 encoder may just ignore every second frame. This would mean that each frame might consist of two fields that are only ~1/60th of a second apart, not ~1/30th.

    To me this seems at least plausible, as the only reason you'd use interlaced video is if you were using an off-the-shelf analogue NTSC camera to capture the video. Anything digital like a webcam would use progressive video.

    If I was to hazard a guess, I would say SpaceX found some space-rated analogue NTSC camera that outputs composite video. This is then run into an off-the-shelf board that has two main components: a capture chip outputting BT656 uncompressed digital video into some form of small ARM SoC with a DSP-assisted MPEG4 encoder. They'd do some software work in the ARM core to overlay a timestamp onto the incoming stream, which at this point is still at ~60 fields per second. Then they'd need to reduce the amount of data being output, so they'd tweak the MPEG4 encoder to only encode at half the framerate. This could give you ~15 frames per second, where each individual frame is made of two fields that are only 1/60th apart.

    Or I may be totally wrong ;)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: ugordan on 06/18/2014 07:39 pm
    Without knowing SpaceX's transmission setup it's hard to know what's happening, but it could be that the camera is capturing at ~60 fields per second (standard NTSC). The timestamp might be overlaid onto each field separately, before they get combined into frames. Then the MPEG4 encoder may just ignore every second frame. This would mean that each frame might consist of two fields that are only ~1/60th of a second apart, not ~1/30th.

    It looks to me they're really transmitting fields taken 1/30th of a second apart. If you look closely at the SES-8 highlights video, they seem to have converted certain portions of the onboard video into 30 fps, albeit at noticeably lower vertical resolution. It looks smooth so it leads me to believe they're capturing video at 30 fields per second, strange as that seems. I think two consecutive, 1/60th-second frames would end up looking more jerky.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: saliva_sweet on 06/18/2014 08:35 pm
    I attach a zip conatining folders each containing 40 mpg4-img files. They start with iframe and are followed by the 19 pframes of the section followed by P frame converted iframe of the next section and its subsequent p frames. Each folder also contains a txt file with the mmb for the converted frame (frame_21.mpg4-img in the folders). There is an issue with conversion for iframe 121 (in part5_6) I still have to investigate, might be bad vop_quant.

    I have modified some mmbs already with -3 commands for the blank regions (part3_4, part12_13 and part14_15). Others remain to be done, but probably require more dc tweaking.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Quialiss on 06/18/2014 09:38 pm
    Is there an easy way to figure out what bits a DC correction changes?

    Example, i frame 4, these are effectively 3 single bitflips that fix the luma down in the clock region.  The clock itself is still damaged, which might be fixable if these flips aren't single flips, but triple flips that just happen to have one part in a luma sub block...

    6:29:86644:64:0:0:0:0:0, 7:29:87881:0:-32:0:-64:0:0
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: mhenderson on 06/18/2014 09:49 pm
    Ours is better.  8)

    https://www.youtube.com/watch?v=DgLBIdVg3EM

    Onboard hi-def camera of F9R test at McGregor yesterday (17-Jun). Now with steerable fins.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: saliva_sweet on 06/18/2014 10:06 pm
    Is there an easy way to figure out what bits a DC correction changes?

    Example, i frame 4, these are effectively 3 single bitflips that fix the luma down in the clock region.  The clock itself is still damaged, which might be fixable if these flips aren't single flips, but triple flips that just happen to have one part in a luma sub block...

    6:29:86644:64:0:0:0:0:0, 7:29:87881:0:-32:0:-64:0:0

    If dcs are off by that much (32 and 64) there are probably errors in dc size vlc codes, but I fear there is no easy way to figure out anything about that ;) Is it a flip in dc size vlc or there's a flip before, which causes it to be read wrong? Go figure.

    edit: But if the clock itself is wrong the errors are obviously aready in the luma blocks so no wonder chromas are broken too. Sorry, you were already talking about luma. Still the error could be in the previous block, but dc_size luminance would be my best guess. I would just xor through the whole block if it hasn't been done already.

    edit2: flipped through the end of 5:29 and the first block of 6:29 (singles, triples and quads) - nothing. This macroblock is huge and covers most of a ts packet. I think the whole ts packet is bad, too many errors for me.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: SwissCheese on 06/18/2014 10:47 pm
    Made a new version of the additional frames. Feel free to comment, modify or simply make another, nicer version ;) all the source data are in additional_frames.zip

    @ Wronkiew: I copy 40 times each frame for the video, if you want to add them to the online video.

    edit: added "soft" to water landing  8)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Quialiss on 06/18/2014 11:32 pm
    @SwissCheese:

    I've finished doing corrections on part 4, I did make two small alignment changes, could you double check them? 
    frame 71 -- 24:7:25925 (and 37:10:36157 to keep the same alignment between the two pieces)
    frame 72 -- 23:18:35367


    Is there an easy way to figure out what bits a DC correction changes?

    Example, i frame 4, these are effectively 3 single bitflips that fix the luma down in the clock region.  The clock itself is still damaged, which might be fixable if these flips aren't single flips, but triple flips that just happen to have one part in a luma sub block...

    6:29:86644:64:0:0:0:0:0, 7:29:87881:0:-32:0:-64:0:0

    If dcs are off by that much (32 and 64) there are probably errors in dc size vlc codes, but I fear there is no easy way to figure out anything about that ;) Is it a flip in dc size vlc or there's a flip before, which causes it to be read wrong? Go figure.

    edit: But if the clock itself is wrong the errors are obviously aready in the luma blocks so no wonder chromas are broken too. Sorry, you were already talking about luma. Still the error could be in the previous block, but dc_size luminance would be my best guess. I would just xor through the whole block if it hasn't been done already.

    All the mb's are 1000+ bits, not my favorite kind to do exhaustive searches through.  ;)  Silly massive clock. 

    I continued to mess with it, and actually got the numbers to show up quite nicely using some direction adjustment.  I really wish the direction field was displayed in binary rather than decimal, it would make it so much easier to tweak.  Not quite sure why it's screwed up considering direction's calculated on the fly, but it does work... I think there are a few more digits in the i frames I can recover this way. 

    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: saliva_sweet on 06/18/2014 11:39 pm
    All the mb's are 1000+ bits, not my favorite kind to do exhaustive searches through.  ;)  Silly massive clock. 

    I continued to mess with it, and actually got the numbers to show up quite nicely using some direction adjustment.  I really wish the direction field was displayed in binary rather than decimal, it would make it so much easier to tweak.  Not quite sure why it's screwed up considering direction's calculated on the fly, but it does work... I think there are a few more digits in the i frames I can recover this way.

    Wow, birilliant. I just made an edit to my previous post to admit my utter failiure. Looks like the root cause of the issue lies in fact in MB 5:28.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: AncientU on 06/18/2014 11:55 pm
    Ours is better.  8)

    https://www.youtube.com/watch?v=DgLBIdVg3EM

    Onboard hi-def camera of F9R test at McGregor yesterday (17-Jun). Now with steerable fins.

    There is only one first soft landing -- and that is yours!
    (And SpaceX's)

    You all are doing an incredible job!!!
    (And so is SpaceX)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Quialiss on 06/19/2014 12:23 am
    All the mb's are 1000+ bits, not my favorite kind to do exhaustive searches through.  ;)  Silly massive clock. 

    I continued to mess with it, and actually got the numbers to show up quite nicely using some direction adjustment.  I really wish the direction field was displayed in binary rather than decimal, it would make it so much easier to tweak.  Not quite sure why it's screwed up considering direction's calculated on the fly, but it does work... I think there are a few more digits in the i frames I can recover this way.

    Wow, brilliant. I just made an edit to my previous post to admit my utter failure. Looks like the root cause of the issue lies in fact in MB 5:28.

    Upon a bit more investigation, if you take any i frame and -1 out the top of the clock and leave the bottom, the bottom becomes garbled, even though all the mb positions are correct.  The clock mbs are MUCH pickier about their inheritance than any other block.  Changing the direction is just mimicking the data it should be getting from the blocks above it.   
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: wronkiew on 06/19/2014 07:04 am
    Made a new version of the additional frames. Feel free to comment, modify or simply make another, nicer version ;) all the source data are in additional_frames.zip

    @ Wronkiew: I copy 40 times each frame for the video, if you want to add them to the online video.

    Looks great! I'll put those into the generated videos. Probably tomorrow.

    How do folks feel about putting real names into the credits list? I wouldn't want to do it unless we had significant buy-in from the contributors.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: SwissCheese on 06/19/2014 09:49 am
    @SwissCheese:

    I've finished doing corrections on part 4, I did make two small alignment changes, could you double check them? 
    frame 71 -- 24:7:25925 (and 37:10:36157 to keep the same alignment between the two pieces)
    frame 72 -- 23:18:35367

    Had another look at them, shifted your correction by one to the left for frame 71, water surf on the right seem to align better with following frame. Frame 72 looks better, just removed the errors at the end of the sequence. Also not very happy with water on frame 70, but it's very difficult to align... Anyway there are still probably tens of errors in the video, don't want to recheck everything though ;)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: gospacex on 06/19/2014 10:29 am
    Ours is better.  8)

    https://www.youtube.com/watch?v=DgLBIdVg3EM

    Onboard hi-def camera of F9R test at McGregor yesterday (17-Jun). Now with steerable fins.

    Ha ha, famous rocket cows of McGregor seem to adapt quite quickly :D
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: CuddlyRocket on 06/19/2014 10:36 am
    Ha ha, famous rocket cows of McGregor seem to adapt quite quickly :D
    There's a (probably Disney) film in there somewhere - The Rocket Cows of McGregor! :)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: mhenderson on 06/19/2014 01:41 pm
    Quick poll -

    How many hours have you put into this since it began almost twelve weeks ago?

    To avoid clutter, PM me.  I will post anonymous stats (hi, low, avg, total)

    (My guess is the core team effort is around 12,000-14,000 person-hours.)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: SwissCheese on 06/19/2014 01:52 pm
    I attach a zip conatining folders each containing 40 mpg4-img files. They start with iframe and are followed by the 19 pframes of the section followed by P frame converted iframe of the next section and its subsequent p frames. Each folder also contains a txt file with the mmb for the converted frame (frame_21.mpg4-img in the folders). There is an issue with conversion for iframe 121 (in part5_6) I still have to investigate, might be bad vop_quant.

    I have modified some mmbs already with -3 commands for the blank regions (part3_4, part12_13 and part14_15). Others remain to be done, but probably require more dc tweaking.

    Really nice! Many i frames could be vastly improved by this.

    I tried a bit for i frame 221, but we really need the data in the online editor to tune the dc values I think, it takes too much time without it.

    Also made a video (spacex_landing_i2p.mp4) with your conversions (part3_4, part12_13 and part14_15).
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: saliva_sweet on 06/19/2014 02:40 pm

    I tried a bit for i frame 221, but we really need the data in the online editor to tune the dc values I think, it takes too much time without it.

    Also made a video (spacex_landing_i2p.mp4) with your conversions (part3_4, part12_13 and part14_15).

    Online editor would be indeed be really helpful. I hope IainCole gets a chance to do it. I'm also contemplating trying to make an automatic dc fixing tool. I believe it should be possible because we already know the (more or less) correct dcs for all blocks, they can be extracted from the fixed iframe ffmpeg logs.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: MTom on 06/19/2014 05:35 pm
    Quick poll -

    How many hours have you put into this since it began almost twelve weeks ago?

    To avoid clutter, PM me.  I will post anonymous stats (hi, low, avg, total)

    (My guess is the core team effort is around 12,000-14,000 person-hours.)

    My guess is about 6000-7000 person-hours. I'm also curious.
    (Note: for the last couple of weeks there remain less people on board: the work became very difficult, only the "bests" could follow it. They made a huge work - RESPEKT for them!!! - but it added less for the total amount of person-hours than in may.)

    My earlier guess see her (it was underestimated, now I would say 4000-5000 person-hours at this time):
    http://forum.nasaspaceflight.com/index.php?topic=34597.msg1208031#msg1208031
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: saliva_sweet on 06/19/2014 06:23 pm
    There is an issue with conversion for iframe 121 I still have to investigate, might be bad vop_quant.

    The attached mpg4-img fixes the vop_quant in frame 121. This file replaces frames6_7/frame_21.mpg4-img in i2p.zip
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: saliva_sweet on 06/19/2014 07:25 pm
    There is a way to use the current online editor to fix dcs in converted frames. -3 mbs appear to have the same effect on dcs as -2 mbs. Open two online editors with the frame you want to fix. In one set the region you want to be -3 to -2. Select the blocks starting the dc issues and see what the dcs are, in the other tab see what they should be. Add/subtract the difference. dcs are now fixed, put them into the converted frame mmb.

    edit: Use iframe only editor if you value you sanity.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Req on 06/19/2014 08:06 pm
    Made a new version of the additional frames. Feel free to comment, modify or simply make another, nicer version ;) all the source data are in additional_frames.zip

    @ Wronkiew: I copy 40 times each frame for the video, if you want to add them to the online video.

    Looks great! I'll put those into the generated videos. Probably tomorrow.

    How do folks feel about putting real names into the credits list? I wouldn't want to do it unless we had significant buy-in from the contributors.

    I'm on board with that.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: IainCole on 06/19/2014 09:48 pm

    I tried a bit for i frame 221, but we really need the data in the online editor to tune the dc values I think, it takes too much time without it.

    Also made a video (spacex_landing_i2p.mp4) with your conversions (part3_4, part12_13 and part14_15).

    Online editor would be indeed be really helpful. I hope IainCole gets a chance to do it. I'm also contemplating trying to make an automatic dc fixing tool. I believe it should be possible because we already know the (more or less) correct dcs for all blocks, they can be extracted from the fixed iframe ffmpeg logs.

    Yeah it's going to require some major surgery so might take a while, I should have some time to tackle it on Saturday maybe.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: saliva_sweet on 06/19/2014 10:56 pm
    Yeah it's going to require some major surgery so might take a while, I should have some time to tackle it on Saturday maybe.

    Cool. It's possible to work with the existing iframe only editor meanwhile. Quialiss, you too can help with this if you feel like it even though you weren't running a local ffmpeg (?). We can make mmbs in the iframe editor that have regions that could use data from the previous frame (and actually have something in the previous frame :)) set to -2 and then dcs fixed. Like the one for frame 81 I attach. This will accept data into the gray hole and everything will be peaches and cream.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: wronkiew on 06/20/2014 06:57 am
    Looks great! I'll put those into the generated videos. Probably tomorrow.

    Scratch that. I have been working on the i2p stuff instead. The generator can build frames3_4 now with the modified frame 21. When all the segments are building with i2p, I can switch the clockfix job over to using it instead.

    My apologies to IainCole. This is going to be a pain in the online editor's neck.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: SwissCheese on 06/20/2014 08:16 am
    There is a way to use the current online editor to fix dcs in converted frames. -3 mbs appear to have the same effect on dcs as -2 mbs. Open two online editors with the frame you want to fix. In one set the region you want to be -3 to -2. Select the blocks starting the dc issues and see what the dcs are, in the other tab see what they should be. Add/subtract the difference. dcs are now fixed, put them into the converted frame mmb.

    edit: Use iframe only editor if you value you sanity.

    OK, I got it. I should be able to convert some of them.

    We might need a new spreadsheet for the converted i frames!
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: SwissCheese on 06/20/2014 11:17 am
    So I just converted frame 221.

    I did as Saliva_Sweet explained:
    - open 2 tabs with iframe editor (spacex.slapbet.org)
    - load the iframe you want to edit in each tab (copy the mmb from the spreadsheets)
    - in one of the editors, replace the -1 by -2 where you want to replace the blank MB by MB from previous frame
    - correct dc values so they are the same as in the "reference" frame (check the values in both frames and add the difference)
    - save your solution in a text file

    Now the bit positions must be adapted, since I don't have a formula to convert them from one type to the other I did it that way:
    - since I had additionnal MB positions, I had to run first the whole sequence (before corrections) to get the bit positions (in the log file) of the converted i-frame

    - open in a notepad the file "P21_mmb.txt" that Saliva_Sweet put in the folders (how did you generate these???)
    - replace the "," by a return / line feed
    - save and close
    - do the same for the text file with your solution
    - open both files in Excel, use ":" as delimiter -> you get each number in a separate cell
    - align the MB positions of both text files in your Excel sheet (add / remove MB positions from "P21_mmb.txt")
    - copy the new dc values to "P21_mmb.txt"
    - save as a text file
    - open the new text file and replace tabs by ":", return / line feed by ","
    - save and that's it!

    I hope it is understandable ;)

    attached files:

    frame_221_i_original.png "reference" i-frame 221
    frame_221_i_minus_2.png i-frame 221 with -1 sequences replaced by -2, and corrected dc values
    frame_221_i2p.png final result with i-frame converted to p-frame, -2 replaced by -3, and adapted bit positions
    command_line_before.txt before changing the -1 to -3
    command_line_before.txt after changing the -1 to -3



    How do you find the result? Should we tune further the dc values so the transition is smoother? Is there a formula to convert the bit positions from i-frame to converted i-frame?

    I hope some will be able to help, even if you only do the first part (from -1 to -2) and post your solution! (it is really not complicated at all, just takes some time)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: mhenderson on 06/20/2014 11:26 am
    Hours poll - second request.

    Please PM me with a guesstimate of the total number of hours you have dedicated to this effort since the effort began in April.

    (Even rough range estimates like <50, 50-100, 100-200, 200-500, 500-1000, 1000-1500, ... would be fine)

    I have received four responses so far.  Individual estimates will be kept anonymous (ahem ... yes I did some at my desk at the office, too ... I won't snitch). My intent is to paint a picture in broad strokes of the overall team effort.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: saliva_sweet on 06/20/2014 12:46 pm

    How do you find the result? Should we tune further the dc values so the transition is smoother? Is there a formula to convert the bit positions from i-frame to converted i-frame?

    There is no formula, I generate the P21_mmb.txt during the conversion process. Each macroblock is longer by a couple of bits (~2-4) compared to the iframe version. I store the start position for each macroblock while converting (but these can also be extracted from the log as you did) and then pass over the original mmb and replace all positions. I will try to make a script to port mmbs over ASAP, not sure if I can make it today though. Anyone else care to make it?

    Should we tune further the dc values so the transition is smoother? Is there a formula to convert the bit positions from i-frame to converted i-frame?

    This I've been pondering myself. I think the proof of the pudding is utlimately in how the p frames respond and what it will look like overall. I'm hesitant to change the hue and brightness of iframes based on previous frame data (it could work though). At least for frames with good 0:0 MB like 221. For the frames that have no 0:0 mb I think this should be done absolutely.

    Also looking at the pngs I think I see another conversion issue. p frame version looks a bit more washed out and blurry than the original. I think another vop_quant issue. I don't properly convert them, will have to try to tweak it manually. This has no effect on mmbs fortunately.

    Please PM me with a guesstimate of the total number of hours you have dedicated to this effort since the effort began in April.

    I'm sorry, wish I could help, but I don't have the vaguest idea.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: SwissCheese on 06/20/2014 03:47 pm
    Did the same for frame 181 (not really necessary since it is already really good).

    Also looking at the pngs I think I see another conversion issue. p frame version looks a bit more washed out and blurry than the original. I think another vop_quant issue. I don't properly convert them, will have to try to tweak it manually. This has no effect on mmbs fortunately.

    I did not remark anything special...
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Lar on 06/20/2014 03:57 pm

    Please PM me with a guesstimate of the total number of hours you have dedicated to this effort since the effort began in April.

    I'm sorry, wish I could help, but I don't have the vaguest idea.

    If you don't have the vaguest idea, it was probably a LOT... say several 10s of hours a week. Use that for your guess. :)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: MTom on 06/20/2014 06:35 pm

    Please PM me with a guesstimate of the total number of hours you have dedicated to this effort since the effort began in April.

    I'm sorry, wish I could help, but I don't have the vaguest idea.

    If you don't have the vaguest idea, it was probably a LOT... say several 10s of hours a week. Use that for your guess. :)

    Yes, the guesses about hours a week is a good starting point.
    Some help how could you better guesstimate:
    - when did you start with the work? The first post in the thread is from 30. of April. ( 7 weeks ago)
    - after that you should divide your work into weeks and remember if all the weeks were about the same or were there big differences between them. In case of big differences you should count how many "heavy, middle and light" weeks you had. For help you can look at some posts for every weeks and you will better remember what kind of week you had at this time.
    - after that you can guesstimate, how many hours you worked in your typical weeks (multiply of how many days you worked and an average of daily working hours.)

    This type of guesstimating is not so bad. And if more people guess, it will be better because the answers are typically a mix of underestimated and overestimated guesses.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Chris Bergin on 06/20/2014 07:52 pm
    Over 350,000 visits to this thread now.

    Obviously, there may be a launch later (weather pending) and they may get a clear video back of propulsive return/splashdown, but nothing will take away from the work done on this thread by so many people.

    And of course, this video is the historic first, so it will always have major value.

    So this thread continues and we have an article in work to bookend the effort.
    Title: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Jakusb on 06/20/2014 09:38 pm
    Whoop whoop!!!
    Our video just was shown in live webcast!
    Unfortunately no mentioning our efforts, but still. :)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: SwissCheese on 06/20/2014 09:39 pm
    Oo did I see our video on the webcast?  8)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Req on 06/20/2014 09:39 pm
    They just showed part of the repaired video on the Orbcomm webcast. :)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: SwissCheese on 06/20/2014 09:57 pm
    So they just decided to give us 1 more hour to finish repairing the video :P
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: QuantumG on 06/20/2014 10:04 pm
    So they just decided to give us 1 more hour to finish repairing the video :P

    More than that.. it means the splashdown video of this flight will be in the dark - if they launch tonight.

    Edit: apparently not! 19:01 launch. 20:24 sunset.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: saliva_sweet on 06/20/2014 10:18 pm
    While the launch is on hold... Here's iframe to pframe mmb conversion tool. Extract to a folder, put you mmb file there and run it as (python is needed):

    python port_mmb.py <frame number> <input filename> > <output filename>

    eg. python port_mmb.py 121 my_121_iframe_mmb.txt > my_121_pframe_mmb.txt

    disclaimer: I was drunk while making this (it's launch day) so report any bugs.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: MTom on 06/20/2014 10:21 pm
    So they just decided to give us 1 more hour to finish repairing the video :P

    Maybe more...  Jumping to the end of the launch window sounds not good.  :(
    But still hoping...
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Quialiss on 06/20/2014 10:28 pm
    Should we tune further the dc values so the transition is smoother? Is there a formula to convert the bit positions from i-frame to converted i-frame?

    This I've been pondering myself. I think the proof of the pudding is utlimately in how the p frames respond and what it will look like overall. I'm hesitant to change the hue and brightness of iframes based on previous frame data (it could work though). At least for frames with good 0:0 MB like 221. For the frames that have no 0:0 mb I think this should be done absolutely.

    Definitely fine tune the DC values more if you can!  On my last pass on the i frames I attempted to match up DC values as much as possible, working back and forwards from frame 181 as it's the best in the set. My process for fixing up the DC values was to match against p frames where there were good mb's to match against, and then match against the best i frame+pframes before/after the one I was working on. 

    For the last 3 i frames, there are nearly no mb's in the p frames to match against, and the 0:0 block is frequently missing or there's a large gap, so the only source of DC info I had for these was to match against the previous frame.  They're all just a bit off.. complicated by the chroma errors in frame.

    I won't have too much time to spend on this until later in the weekend, but.. is this about what's needed for tracking progress on converting the i frame mmbs?

    https://docs.google.com/spreadsheets/d/1YG_-wD7nyqsRopiihGqfTiLCE92kuCm_DOua8_UYW6g/edit#gid=0


    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: SwissCheese on 06/20/2014 11:47 pm
    Should we tune further the dc values so the transition is smoother? Is there a formula to convert the bit positions from i-frame to converted i-frame?

    This I've been pondering myself. I think the proof of the pudding is utlimately in how the p frames respond and what it will look like overall. I'm hesitant to change the hue and brightness of iframes based on previous frame data (it could work though). At least for frames with good 0:0 MB like 221. For the frames that have no 0:0 mb I think this should be done absolutely.

    Definitely fine tune the DC values more if you can!  On my last pass on the i frames I attempted to match up DC values as much as possible, working back and forwards from frame 181 as it's the best in the set. My process for fixing up the DC values was to match against p frames where there were good mb's to match against, and then match against the best i frame+pframes before/after the one I was working on. 

    For the last 3 i frames, there are nearly no mb's in the p frames to match against, and the 0:0 block is frequently missing or there's a large gap, so the only source of DC info I had for these was to match against the previous frame.  They're all just a bit off.. complicated by the chroma errors in frame.

    I won't have too much time to spend on this until later in the weekend, but.. is this about what's needed for tracking progress on converting the i frame mmbs?

    https://docs.google.com/spreadsheets/d/1YG_-wD7nyqsRopiihGqfTiLCE92kuCm_DOua8_UYW6g/edit#gid=0

    That's OK with this spreadsheet, thank you!

    Also did frame 161 (changed a bit the bottom dc values)

    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Quialiss on 06/21/2014 07:23 am
    While the launch is on hold... Here's iframe to pframe mmb conversion tool. Extract to a folder, put you mmb file there and run it as (python is needed):

    python port_mmb.py <frame number> <input filename> > <output filename>

    eg. python port_mmb.py 121 my_121_iframe_mmb.txt > my_121_pframe_mmb.txt

    disclaimer: I was drunk while making this (it's launch day) so report any bugs.

    It's ignoring the bit flip operations.. intentionally it seems?  I have not gotten around to getting your i2p frames working in my local ffmpeg so I can't take a look at what happens, but it seems like this would be an issue. 

    101 is now done, looks like there were a couple wave fronts in the gaps that will now show up properly.  The sketchy quality of most of part 4's p frames do make it a little choppy looking though. 
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: wronkiew on 06/21/2014 08:35 am
    I am testing an update to the autogenerator to handle the i2p frames. The resulting video looks a bit softer to me than the regular build, so i would appreciate some extra eyes on it before I make it the default.

    http://adama.nocdirect.com/~wronkiew/spx_crs3/landing_base_set_i2p.mp4

    http://adama.nocdirect.com/~wronkiew/spx_crs3/landing_base_set_i2p_slow.mp4

    http://adama.nocdirect.com/~wronkiew/spx_crs3/landing_base_set_i2p_jpg.zip

    The i2p frames are using the P21 mmbs included in i2p.zip file by default, and then overridden by the new spreadsheet as applicable.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: SwissCheese on 06/21/2014 09:35 am
    I am testing an update to the autogenerator to handle the i2p frames. The resulting video looks a bit softer to me than the regular build, so i would appreciate some extra eyes on it before I make it the default.

    http://adama.nocdirect.com/~wronkiew/spx_crs3/landing_base_set_i2p.mp4

    http://adama.nocdirect.com/~wronkiew/spx_crs3/landing_base_set_i2p_slow.mp4

    http://adama.nocdirect.com/~wronkiew/spx_crs3/landing_base_set_i2p_jpg.zip

    The i2p frames are using the P21 mmbs included in i2p.zip file by default, and then overridden by the new spreadsheet as applicable.

    The jpgs look good but the video is blurred and the file is very small. Mine is ~3 times larger. I use this command line to generate the video:
    ffmpeg -framerate 44999/3003 -i frame_%03d.png -c:v mpeg4 -q:v 3 spacex_landing_i2p.mp4

    I think -q:v 3 defines the quality, maybe it comes from this.

    Edit: I found this in ffmpeg documentation:
    The parameter ’q’ which is displayed while encoding is the current quantizer. The value 1 indicates that a very good quality could be achieved. The value 31 indicates the worst quality. If q=31 appears too often, it means that the encoder cannot compress enough to meet your bitrate. You must either increase the bitrate, decrease the frame rate or decrease the frame size.

    Since our input quality is rather low  :P we should probably avoid degrading it further and use the best possible encoding quality, thus use option "-q:v 1" (although I found here (http://www.transcoding.org/transcode?Export_Modules/Export_Ffmpeg) that for some codecs q=1 was problematic... maybe put 2 or 3 then)

    Without setting this value I get q=17.4 and my video is also small and blurred
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: saliva_sweet on 06/21/2014 09:41 am
    While the launch is on hold... Here's iframe to pframe mmb conversion tool. Extract to a folder, put you mmb file there and run it as (python is needed):

    python port_mmb.py <frame number> <input filename> > <output filename>

    eg. python port_mmb.py 121 my_121_iframe_mmb.txt > my_121_pframe_mmb.txt

    disclaimer: I was drunk while making this (it's launch day) so report any bugs.

    It's ignoring the bit flip operations.. intentionally it seems?  I have not gotten around to getting your i2p frames working in my local ffmpeg so I can't take a look at what happens, but it seems like this would be an issue. 
    Xors get baked into the bitstream during conversion so they're left out of mmbs.

    101 is now done, looks like there were a couple wave fronts in the gaps that will now show up properly.  The sketchy quality of most of part 4's p frames do make it a little choppy looking though.

    Amazing progress. We're getting this done :)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: mhenderson on 06/21/2014 12:18 pm
    Final request -

    Please PM me with a rough guess at the number of hours you have spent working on this little labor of love since it began in April. 

    I have heard from a decent number of contributors, but more is always better.

    I can handle vague.  Range estimates are fine - 0-25, 25-50, 50-100, etc.  Use your gut ... "five hours a week plus or minus over six weeks".  "Two big weeks of 20 and a few weeks of 2-4s".  Even if you never submitted anything, tell me your story "I played with the online editor for three hours but it is over my head." Anecdotal evidence still adds to the picture.

    The purpose is to paint a picture in broad strokes of the effort. Information will remain anonymous, only aggregate stats will be revealed. 

    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: MTom on 06/21/2014 12:25 pm
    Final request -

    Please PM me with a rough guess at the number of hours you have spent working on this little labor of love since it began in April. 

    I have heard from a decent number of contributors, but more is always better.

    I can handle vague.  Range estimates are fine - 0-25, 25-50, 50-100, etc.  Use your gut ... "five hours a week plus or minus over six weeks".  "Two big weeks of 20 and a few weeks of 2-4s".  Even if you never submitted anything, tell me your story "I played with the online editor for three hours but it is over my head." Anecdotal evidence still adds to the picture.

    The purpose is to paint a picture in broad strokes of the effort. Information will remain anonymous, only aggregate stats will be revealed.

    It could happen that earlier contributors don't read this thread actually. Maybe a PM for them would be a good idea.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Quialiss on 06/21/2014 02:26 pm
    Note for converting i frames:  Don't be surprised if you can/have to get rid of DC changes in macroblocks that are just below a gap.  DC fixes for contiguous data is a matter of finding flipped bits, but correcting for gaps is more a matter of adjusting to compensate for what's missing/smeared out.  Ignoring that data with -2/3 sometimes means less total corrections needed.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: aero on 06/21/2014 03:30 pm
    Hey people, thanks for all the good work.

    I just noticed that the latest You tube video on Twitter is two days old, Thursday 2014/6/19 13:02:58 GMT.

    Is that intentional?
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Lindblue on 06/21/2014 05:37 pm
    When I saw "your" video during yesterdays SpaceX webcast I was really happy!
    Been following this thread for months and have to say that you all do fantastic work!
    Really impressive.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: wronkiew on 06/21/2014 05:46 pm
    Thanks SwissCheese, I made the modifications as you suggested.

    The i2p video has some black streaking after frame 161. This is due to error propagation. Frame 160 has always had a black dot in the ocean. Since this block is carried forward it gets duplicated each frame thereafter.

    The next YouTube upload should have the i2p modifications.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: SwissCheese on 06/21/2014 08:14 pm
    Thanks SwissCheese, I made the modifications as you suggested.

    The i2p video has some black streaking after frame 161. This is due to error propagation. Frame 160 has always had a black dot in the ocean. Since this block is carried forward it gets duplicated each frame thereafter.

    The next YouTube upload should have the i2p modifications.

    Could you also add the additionnal frames (credits...) to the YouTube video?
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: saliva_sweet on 06/21/2014 09:31 pm
    Replacements for frames 201 and 221 in i2p.zip. Fixed vop_quant. Difference is subtle for 221, not so subtle for 201.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Chris Bergin on 06/21/2014 09:34 pm
    Ooo, someone please message me if we are close to a new Youtube video! :)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: SwissCheese on 06/21/2014 09:44 pm
    Hey people, thanks for all the good work.

    I just noticed that the latest You tube video on Twitter is two days old, Thursday 2014/6/19 13:02:58 GMT.

    Is that intentional?

    Just to explain a bit: we more or less finished repairing (aligning) the ("difference") p-frames, that's why nothing happened to the YouTube video.

    What we are trying to do now is filling the holes of the ("reference") i-frames with the information of the previous frames. For this, Saliva_Sweet was able to convert the i-frames into p-frames.

    Now we are trying to fill these holes and tune the color/brightness of the images to have smooth transitions. Most of them are done now, but a few still have issues.

    Wronkiew will soon change the automatic video generation to use these frames.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: saliva_sweet on 06/21/2014 10:00 pm
    Replacement for frame 81 in frames4_5 in i2p zip. Fixes most conversion issues. There's a small problem at the bottom right corner I don't know the cause of.

    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: wronkiew on 06/22/2014 07:09 am
    Could you also add the additionnal frames (credits...) to the YouTube video?

    Done. I also set the generator to output in 30i. Your suggestion to increase the output quality parameter helped there too.

    Frames 261-280 are messed up because the new i2p mmb assumes that the segment starts at 221, but the generator is starting at 241. I think in order to fix this I am going to have to modify it to decode everything at once instead of splitting it up into segments. That's going to be a project for tomorrow.

    SwissCheese, if you get a chance can you split the credits png up into two pages? It'd be easier to read that way.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Quialiss on 06/22/2014 09:08 am
    I've made a few small changes to some p frames in parts 5, 8 and 11.  A couple gray spots, the dark spot on the wave front and a bit more data in one frame that falls within the gap used in the next i frame. 

    The dark spot on the wave is curious.  The macroblocks in all the wavefronts are quite large, 100-500 compared to the <50 average for most of the ocean.  I figured there was an error in one of the blocks and blanked out the first part where it appeared darker than it should, but there are either subsequent errors (3 frames total needed touching up to get rid of it) or I've chosen the wrong place to make corrections.


    I'm going to try and get ffmpeg running locally again tomorrow to continue work on the i2p frames... last time I needed to use it it was throwing no errors, but outputting frames that were slightly different from the online editor.  Less than ideal.   ;D
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: saliva_sweet on 06/22/2014 09:52 am
    Frames 261-280 are messed up because the new i2p mmb assumes that the segment starts at 221, but the generator is starting at 241. I think in order to fix this I am going to have to modify it to decode everything at once instead of splitting it up into segments. That's going to be a project for tomorrow.

    Really liking the latest youtube video. But we do need to build it in a single piece at this point. There are a couple of instances where data gets passed through several i frames. I will go ahead and try to fix the chroma in frame 241 and then, I think, I'm inclined to call it done and my part in this will be played.

    edit: Wait a second.. Has someone done it already?
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: SwissCheese on 06/22/2014 11:11 am
    Frames 261-280 are messed up because the new i2p mmb assumes that the segment starts at 221, but the generator is starting at 241. I think in order to fix this I am going to have to modify it to decode everything at once instead of splitting it up into segments. That's going to be a project for tomorrow.

    Really liking the latest youtube video. But we do need to build it in a single piece at this point. There are a couple of instances where data gets passed through several i frames. I will go ahead and try to fix the chroma in frame 241 and then, I think, I'm inclined to call it done and my part in this will be played.

    edit: Wait a second.. Has someone done it already?

    Could you check again the conversion of frame 61? I put on the spreadsheet a new intermediary set of mmb that looks quite good on the online editor but cannot manage to convert it correctly. Data seem to be missing/wrong at 41:3 and the bottom (18:28).
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: saliva_sweet on 06/22/2014 12:23 pm
    Could you check again the conversion of frame 61? I put on the spreadsheet a new intermediary set of mmb that looks quite good on the online editor but cannot manage to convert it correctly. Data seem to be missing/wrong at 41:3 and the bottom (18:28).

    Thanks for pointing this out. There were major issues. Here's a new mpg4-img, mmb based on your intermediary and mmbgen file for the mmb porting tool.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: SwissCheese on 06/22/2014 03:17 pm
    SwissCheese, if you get a chance can you split the credits png up into two pages? It'd be easier to read that way.

    It's in the attachment! (also made the last frame larger)

    Frames 261-280 are messed up because the new i2p mmb assumes that the segment starts at 221, but the generator is starting at 241. I think in order to fix this I am going to have to modify it to decode everything at once instead of splitting it up into segments. That's going to be a project for tomorrow.

    Yes that's probably the best way to do it, to do everything at once.

    And once it is done, I think it is enough for me ;)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: saliva_sweet on 06/22/2014 04:33 pm
    Do we have a consensus? Once wronkview can build the vid in one piece and uploads to youtube we call it finished and champagnes can be uncorked?
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Roy_H on 06/22/2014 05:43 pm
    Sounds like you guys are almost finished. You are to be well congratulated for your stupendous effort! I trust Chris will post a new article with the finished version to highlight your amazing work.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: CraigLieb on 06/22/2014 05:48 pm
    Small suggestion to have the credit slides stay on the video a little longer so folks have a time to read it. Especially the one with the links.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Roy_H on 06/22/2014 06:00 pm
    Small suggestion to have the credit slides stay on the video a little longer so folks have a time to read it. Especially the one with the links.

    People who actually want to read credits hit the pause button.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Chris Bergin on 06/22/2014 06:56 pm
    Sounds like you guys are almost finished. You are to be well congratulated for your stupendous effort! I trust Chris will post a new article with the finished version to highlight your amazing work.

    One is being written for us by one of the team here. I'll subedit and get it on when it's done :)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: wronkiew on 06/22/2014 07:08 pm
    It is done.

    Edit: Updated with a fix for one segment.

    https://www.youtube.com/watch?v=CjZ33C9JZTM
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: chad1011 on 06/22/2014 07:20 pm
    Positively amazing work!  I would have never guessed so much of the video could be recovered especially when the experts said it was a hopelessly lost cause.  This video repair goes to show what the hard work of dedicated people can achieve.  "It is only impossible when you don't try." - Unknown

    -Chad
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: meekGee on 06/22/2014 07:29 pm
    Wait!  The audio still doesn't work!

    EDIT:  And an obvious mega attaboy to all.   When Elon said "we're going to crowd source it", I don't think he imagined this kind of an effort or result.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: wronkiew on 06/22/2014 07:36 pm
    Just a note, if you like the result of our recovery efforts, please click "Like" on the final posts of the clean-up crew. Quialiss, SwissCheese, saliva_sweet, and IainCole, to name a few.

    Also, we would not have been able to make this happen without Chris Bergin and his forum. Keeping this site running requires a huge amount of resources. You can help defray that cost by subscribing to L2 access today.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: SwissCheese on 06/22/2014 07:38 pm
    It is done.

    https://www.youtube.com/watch?v=3eamDETVF-Y

    Wait wait!

    I think the video did not use the latest version of frame 201, could you check it?
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: wronkiew on 06/22/2014 07:43 pm
    Wait wait!

    I think the video did not use the latest version of frame 201, could you check it?

    Here's the mmb file.

    And the frame 201 mmb:

    19:4:15994:0:-8:8:-31:0:0,20:4:-1,24:4:16097,
    19:5:19135:3:0:0:0:0:0,24:5:19672:0:-8:0:0:0:0,
    29:5:20215:0:0:-16:0:0:0,32:5:20506:0:0:0:0:1:-1,
    18:6:22995:-2:0:0:0:0:0,32:6:24818:0:0:0:0:4:3,
    24:8:33251:0:-8:0:-8:0:0,26:8:33546:-36:0:0:0:0:0,
    37:8:34975:0:0:0:0:-2:-1,30:9:39709:0:0:0:0:-5:-1,
    32:9:40011:0:0:0:0:0:3,30:10:45460,
    32:10:45693:0:0:0:0:-2:-3,27:12:57198:0:20:0:0:0:0,
    28:12:57366:0:0:0:0:-4:2,32:13:65733:0:0:0:0:-3:2,
    32:14:73414:0:0:0:0:5:-3,0:15:74953:0:0:8:-4:0:0,
    1:15:-3,3:16:75354:-7:9:0:0:-2:0,
    5:16:75591:0:-6:0:0:0:0,6:16:75649:-9:0:0:0:0:1,
    7:16:75716:7:6:0:0:0:-1,8:16:75804:12:14:0:0:-1:3,
    9:16:75879:7:0:0:0:0:0,11:16:76049:0:2:0:0:-1:0,
    12:16:76122:9:0:0:0:-5:2,13:16:76286:8:0:0:0:0:0,
    14:16:76504:16:17:0:0:-2:4,15:16:76731:0:21:0:0:-13:3,
    16:16:77025:3:0:0:0:0:-2,17:16:77293:0:0:0:0:0:-12,
    19:16:77765:-21:1:0:4:8:11,20:16:77964:8:0:4:0:-10:3,
    21:16:78108:-4:-10:0:0:0:0,23:16:78409:-8:0:-8:0:2:0,
    24:16:78588:8:64:0:0:-9:8,25:16:78810:1:0:0:0:-1:0,
    29:16:79759:0:5:0:0:6:0,30:16:79935:6:3:0:0:0:-4,
    31:16:80169:-11:-9:0:0:0:2,32:16:80345:-9:0:0:0:1:0,
    33:16:80518:0:13:0:0:0:-1,35:16:80806:4:0:0:0:0:4,
    37:16:80988:0:-20:0:0:0:0,39:16:81143:0:-16:0:0:0:0,
    41:16:81346:-21:-17:0:0:0:0,0:17:81464:-77:1:0:0:3:-2,
    1:17:81510:0:6:0:0:0:0,2:17:81582:30:20:6:0:-3:1,
    13:17:82828:0:0:0:0:0:3,20:17:83496:0:0:0:0:6:-5,
    27:17:84079:0:16:0:0:0:0,30:17:84399:0:32:0:-16:0:0,
    40:17:85485:-16:16:0:0:0:0,35:18:88960:8:0:0:0:0:0,
    37:18:89128:8:8:0:8:0:0,2:19:89827:8:0:0:0:0:0,
    5:19:90333:0:-4:0:0:0:0,13:19:90838:-2:0:0:0:0:0,
    23:19:91517:0:0:-8:0:0:0,28:19:91793:-2:0:0:0:0:0,
    41:19:-1,0:20:-3,9:20:92730:-33:0:-14:0:-4:4,
    12:20:92913:0:-3:0:-4:0:0,13:20:92969:-6:0:0:0:0:0,
    0:21:94674:-55:0:0:0:2:-2,2:21:94793:-4:0:0:0:-3:1,
    5:21:94969:0:0:0:0:-3:-1,8:21:95141,
    14:21:95460:0:-8:0:0:0:0,5:22:97212:0:0:-6:0:0:0,
    17:22:97753:0:0:0:-4:0:0,9:23:99529:0:0:0:0:0:2,
    0:24:101208:0:0:0:0:-1:0,8:24:101591:-6:0:-4:0:0:0,
    19:24:102105:-2:-2:-2:0:0:0,20:24:102147:-5:0:0:0:0:0,
    14:25:103698:0:-3:0:-8:0:0,15:26:105600:-4:0:0:0:0:0,
    21:26:-3,4:28:106301:-77:0:73:0:-3:0,
    11:28:110067:0:0:0:0:-3:2,0:29:-3,3:29:111049,
    4:29:111600,5:29:113070:21:0:0:0:0:0,
    8:29:115929:0:43:0:0:0:0
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: wronkiew on 06/22/2014 07:47 pm
    Wait wait!

    I think the video did not use the latest version of frame 201, could you check it?

    Oh fiddlesticks. I see what happened.

    There are two versions of frame 201 in the spreadsheet. Which one should I use?
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: AJA on 06/22/2014 07:48 pm
    This is fantastic.

    One is being written for us by one of the team here. I'll subedit and get it on when it's done :)

    Small suggestion: Since the guys have all the intermediary code/steps/images, a "tutorial" style screencapture video of how they went from the raw dump provided by SpaceX, to the final video would be very useful too - especially for those of us who, annoyingly, had a very general idea of how they were going about the fix, but didn't have the specific expertise to chip in. A video might also allow them to keep all the details that wouldn't make it to the final edit of an article.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: IainCole on 06/22/2014 07:53 pm
    Right, I've chased people a couple of times and unfortunately I didn't get addresses for everyone :(

    The list has been sent so if I didn't get something from you by now I'm afraid you missed the boat!

    I hope to see some "unboxing" pics soon =)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: IainCole on 06/22/2014 07:54 pm
    The latest videos coming out of the I->P work are awesome! I'm sorry I haven't been able to build this into the editor, I just haven't had the time :(
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: wronkiew on 06/22/2014 07:54 pm
    Wait!  The audio still doesn't work!

    EDIT:  And an obvious mega attaboy to all.   When Elon said "we're going to crowd source it", I don't think he imagined this kind of an effort or result.

    You gave me a scare there.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: SwissCheese on 06/22/2014 07:57 pm
    Wait wait!

    I think the video did not use the latest version of frame 201, could you check it?

    Oh fiddlesticks. I see what happened.

    There are two versions of frame 201 in the spreadsheet. Which one should I use?

    The one in the continuous list (not the backup)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: IainCole on 06/22/2014 08:13 pm
    Wait!  The audio still doesn't work!

    EDIT:  And an obvious mega attaboy to all.   When Elon said "we're going to crowd source it", I don't think he imagined this kind of an effort or result.

    Fixed! I found a way to get the audio out of the data!

    https://www.youtube.com/watch?v=gy66xelPbew
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: wronkiew on 06/22/2014 08:22 pm
    New, and hopefully last version. SwissCheese, can you double-check that frame 201 is fixed?
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: SwissCheese on 06/22/2014 08:40 pm
    New, and hopefully last version. SwissCheese, can you double-check that frame 201 is fixed?

    Yes, now it looks good, thank you!
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: MTom on 06/22/2014 08:46 pm
    Fantastic work!!!
    Also the audio! ;D
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: wronkiew on 06/22/2014 08:55 pm
    New, and hopefully last version. SwissCheese, can you double-check that frame 201 is fixed?

    Yes, now it looks good, thank you!

    Original post updated with new version.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: moralec on 06/22/2014 09:07 pm
    Amazing work!!!! Respect :)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Eer on 06/22/2014 09:36 pm
    I propose that the new version be added to message 1 of this thread in place of the interim cleaned up video there.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Chris Bergin on 06/22/2014 09:46 pm
    HAHA! Love the audio! ;D

    Ok, so this coming week, we tie this all up. Article and all!
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: deruch on 06/22/2014 09:49 pm
    Wait!  The audio still doesn't work!

    EDIT:  And an obvious mega attaboy to all.   When Elon said "we're going to crowd source it", I don't think he imagined this kind of an effort or result.

    Fixed! I found a way to get the audio out of the data!

    https://www.youtube.com/watch?v=gy66xelPbew

    WHAT IS THIS??  NSF isn't 60 Minutes. ;D

    God that sound is hilarious!
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: meekGee on 06/22/2014 10:58 pm
    Wait!  The audio still doesn't work!

    EDIT:  And an obvious mega attaboy to all.   When Elon said "we're going to crowd source it", I don't think he imagined this kind of an effort or result.

    Fixed! I found a way to get the audio out of the data!

    https://www.youtube.com/watch?v=gy66xelPbew


    Before I look, please tell me you're going "woooooooooosh  ploink" into your microphone!

    EDIT:  Yes Yes Yes!    Oh lord please let some news channel pick this up.

    EDIT2:  Am I imagining it or are there two (2) audio tracks in there?!   Hat's off to you, sir!
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Lee Jay on 06/23/2014 01:29 am
    If you compare the current version to the first version, it's literally incredible, as in lacking in credibility.  If it weren't for this thread and the continuing progress it documents, one could be excused for thinking you all had CGI'd the current version, given the original was just this side of nothing.

    Astonishing work all.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Quialiss on 06/23/2014 01:56 am
    Wait!  The audio still doesn't work!

    EDIT:  And an obvious mega attaboy to all.   When Elon said "we're going to crowd source it", I don't think he imagined this kind of an effort or result.

    Fixed! I found a way to get the audio out of the data!


    This made my day.  A glorious capstone to all the work done restoring the visuals.  :D 

    I agree, we've done as much as humanly possible to restore this thing.  I'll always see the little things that could still be improved on with a few hours of work... each, but I am perfectly content to let them be considering they're not notable compared to the unfixable things.  :) 
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: WindyCity on 06/23/2014 01:59 am
    Astounding work. This effort deserves widespread recognition. In the field of A/V digital repair, I reckon it will be seen as a notable achievement. That it was a collaborative, crowd-sourced project, involving people from all over the U.S. and beyond, is another intriguing aspect of the work. I hope that Elon flies you all to Hawthorne for a private tour of the rocket factory and a well-deserved party. Bravo!
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: meekGee on 06/23/2014 05:21 am
    I'll always see the little things that could still be improved on with a few hours of work... each, but I am perfectly content to let them be considering they're not notable compared to the unfixable things.  :)

    Heh, of course you can.  It's like SwissCheese up there going "Wait, frame 201 is wrong"...

    The level of intimacy you have to achieve with a video clip in order for that sentence to even be possible is absolutely mind boggling.

    If this video was somehow lost today, I bet you guys can re-create it from memory.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Jakusb on 06/23/2014 07:34 am
    Wow! Absolutely amazing what has been achieved here!
    And so proud to be a (small) part of this group effort!
    Kudos to all!

    Can someone please tell Elon he can launch OG2 now?
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Bargemanos on 06/23/2014 09:23 am
    Haha! Great audio!
    Fantastic work!
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: AncientU on 06/23/2014 01:50 pm
    Maybe there is some substance to this 'crowd-sourcing' thing...
    Impressive beyond belief!
    You, all.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: dglow on 06/23/2014 09:04 pm
    What everyone has already said, ++.

    This thread has perhaps the greatest signal/noise I've seen on an open, public forum. It's a shining example of the 'Web at its very best.

    Kudos to all.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Chris Bergin on 06/23/2014 09:57 pm
    The article for this "how it was done" is ETA Wednesday.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: DecoLV on 06/24/2014 01:53 am

    Chris and others.... if I may, I'd like to direct attention to a lovely video back at reply #373, which is a very accessible 10-min. explanation of MPEG format, compression, macroblocks, etc.  I think it is good foundation. Helped me understand, anyway.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: mme on 06/24/2014 02:10 am

    Chris and others.... if I may, I'd like to direct attention to a lovely video back at reply #373, which is a very accessible 10-min. explanation of MPEG format, compression, macroblocks, etc.  I think it is good foundation. Helped me understand, anyway.
    Great reference. Hunted it down and quoted it:
    Interesting video about how MPEG works...

    https://www.youtube.com/watch?v=P7abyWT4dss
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Quialiss on 06/24/2014 06:05 am
    The article for this "how it was done" is ETA Wednesday.

    Hopefully you've seen them already, but if not, the gif reconstructions of how the frames were repaired are beautiful.  :)

    (First two are start-finish, second two are partial progress)
    http://forum.nasaspaceflight.com/index.php?topic=34597.msg1214780#msg1214780
    http://forum.nasaspaceflight.com/index.php?topic=34597.msg1211091#msg1211091
    http://forum.nasaspaceflight.com/index.php?topic=34597.msg1208350#msg1208350
    http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195675#msg1195675
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: AncientU on 06/24/2014 10:22 am
    The article for this "how it was done" is ETA Wednesday.

    Hopefully you've seen them already, but if not, the gif reconstructions of how the frames were repaired are beautiful.  :)

    (First two are start-finish, second two are partial progress)
    http://forum.nasaspaceflight.com/index.php?topic=34597.msg1214780#msg1214780
    http://forum.nasaspaceflight.com/index.php?topic=34597.msg1211091#msg1211091
    http://forum.nasaspaceflight.com/index.php?topic=34597.msg1208350#msg1208350
    http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195675#msg1195675

    The second is clearly the more dramatic for one like myself who watch (in amazement) from the sidelines.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Chris Bergin on 06/24/2014 02:09 pm
    The article for this "how it was done" is ETA Wednesday.

    Hopefully you've seen them already, but if not, the gif reconstructions of how the frames were repaired are beautiful.  :)

    (First two are start-finish, second two are partial progress)
    http://forum.nasaspaceflight.com/index.php?topic=34597.msg1214780#msg1214780
    http://forum.nasaspaceflight.com/index.php?topic=34597.msg1211091#msg1211091
    http://forum.nasaspaceflight.com/index.php?topic=34597.msg1208350#msg1208350
    http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195675#msg1195675

    Daaaaaaaaaaaaamn. That second one is epic. ;D

    Tonight I'll probably be looking for some of these in reduced size, 350 pixel width for the article. I'll let you all know.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: SwissCheese on 06/24/2014 02:35 pm
    The YouTube videos released first by SpaceX seem to contain a longer sequence or are not exactly synchronized with the ts files they provided (at least their "raw" video). Here are both ts files provided by SpaceX converted into mp4 videos, so everybody can compare them to what we achieved ;)

    (Our latest video can be downloaded here (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1218063#msg1218063))
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: moralec on 06/24/2014 02:42 pm
    The article for this "how it was done" is ETA Wednesday.

    Hopefully you've seen them already, but if not, the gif reconstructions of how the frames were repaired are beautiful.  :)

    (First two are start-finish, second two are partial progress)
    http://forum.nasaspaceflight.com/index.php?topic=34597.msg1214780#msg1214780
    http://forum.nasaspaceflight.com/index.php?topic=34597.msg1211091#msg1211091
    http://forum.nasaspaceflight.com/index.php?topic=34597.msg1208350#msg1208350
    http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195675#msg1195675

    Daaaaaaaaaaaaamn. That second one is epic. ;D

    Tonight I'll probably be looking for some of these in reduced size, 350 pixel width for the article. I'll let you all know.

    Have you seen the "last progress report" page in the wiki?  Feel free to grab anything you need from there for the article :)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: SwissCheese on 06/24/2014 02:48 pm
    The article for this "how it was done" is ETA Wednesday.

    Hopefully you've seen them already, but if not, the gif reconstructions of how the frames were repaired are beautiful.  :)

    (First two are start-finish, second two are partial progress)
    http://forum.nasaspaceflight.com/index.php?topic=34597.msg1214780#msg1214780
    http://forum.nasaspaceflight.com/index.php?topic=34597.msg1211091#msg1211091
    http://forum.nasaspaceflight.com/index.php?topic=34597.msg1208350#msg1208350
    http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195675#msg1195675

    Daaaaaaaaaaaaamn. That second one is epic. ;D

    Tonight I'll probably be looking for some of these in reduced size, 350 pixel width for the article. I'll let you all know.

    Have you seen the "last progress report" page in the wiki?  Feel free to grab anything you need from there for the article :)

    Here: http://spacexlanding.wikispaces.com/Latest+Progress+Report
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Jakusb on 06/24/2014 03:24 pm

    The article for this "how it was done" is ETA Wednesday.

    Hopefully you've seen them already, but if not, the gif reconstructions of how the frames were repaired are beautiful.  :)

    (First two are start-finish, second two are partial progress)
    http://forum.nasaspaceflight.com/index.php?topic=34597.msg1214780#msg1214780
    http://forum.nasaspaceflight.com/index.php?topic=34597.msg1211091#msg1211091
    http://forum.nasaspaceflight.com/index.php?topic=34597.msg1208350#msg1208350
    http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195675#msg1195675

    Daaaaaaaaaaaaamn. That second one is epic. ;D

    Tonight I'll probably be looking for some of these in reduced size, 350 pixel width for the article. I'll let you all know.

    Have you seen the "last progress report" page in the wiki?  Feel free to grab anything you need from there for the article :)

    Here: http://spacexlanding.wikispaces.com/Latest+Progress+Report

    No mention of the 'Common Dirt Spots'?
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: S.Paulissen on 06/24/2014 06:08 pm
    It's been said before, and I'll say it again, "Woe to the video restoration professional that has a customer that knows that this thread exists."

    I'm proud to have played the tiniest part in this video, if only to understand the true difficulty of the task.  You guys did a fantastic job. 
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Lourens on 06/24/2014 07:27 pm

    The article for this "how it was done" is ETA Wednesday.

    Hopefully you've seen them already, but if not, the gif reconstructions of how the frames were repaired are beautiful.  :)

    (First two are start-finish, second two are partial progress)
    http://forum.nasaspaceflight.com/index.php?topic=34597.msg1214780#msg1214780
    http://forum.nasaspaceflight.com/index.php?topic=34597.msg1211091#msg1211091
    http://forum.nasaspaceflight.com/index.php?topic=34597.msg1208350#msg1208350
    http://forum.nasaspaceflight.com/index.php?topic=34597.msg1195675#msg1195675

    Daaaaaaaaaaaaamn. That second one is epic. ;D

    Tonight I'll probably be looking for some of these in reduced size, 350 pixel width for the article. I'll let you all know.

    Have you seen the "last progress report" page in the wiki?  Feel free to grab anything you need from there for the article :)

    Here: http://spacexlanding.wikispaces.com/Latest+Progress+Report

    No mention of the 'Common Dirt Spots'?

    Yes, I had a look at the wiki, used it as a cross-check mainly after I wrote the article, which is based on the notes I collected before you started on the wiki summary (I read through the entire thread, start to finish :o ). That was a good match, and Jakusb, don't worry about the dirt spots :). My text is now with Chris, who will polish it up to NSF standards ;).

    By the way, for those who prefer text over video or are at work or something and want to know a bit more about MPEG4, here's a link to my earlier explanation of the technical aspects of the repair (http://forum.nasaspaceflight.com/index.php?topic=34597.msg1207666#msg1207666).
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Chris Bergin on 06/24/2014 07:38 pm
    Lourens' article now in first draft complete.

    While we're working this, can I request a best res, best view slide as the lead image for the article. So the best recovered image (Dimensions would be 630x250).

    Also, can we create this:
    http://forum.nasaspaceflight.com/index.php?topic=34597.msg1211091#msg1211091
    Into a 350 width x (whatever works) gif, no more than about 1.8 mb in size, to go into the article?
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Lee Jay on 06/24/2014 07:57 pm
    I had no part in this, but I'm partial to these two frames.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: wronkiew on 06/24/2014 08:06 pm
    Lourens' article now in first draft complete.

    While we're working this, can I request a best res, best view slide as the lead image for the article. So the best recovered image (Dimensions would be 630x250).

    Also, can we create this:
    http://forum.nasaspaceflight.com/index.php?topic=34597.msg1211091#msg1211091
    Into a 350 width x (whatever works) gif, no more than about 1.8 mb in size, to go into the article?

    If anyone wants to take this on, you can find all the frames in the attached archive.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: adrianwyard on 06/24/2014 08:18 pm
    Does anyone on the team have any experience with post-production enhancement/VFX? I'm very much an amateur but just a few minutes with After Effects produced a result that is more pleasing to the untrained eye. I selectively blurred the most obvious macroblock edges for those times they were visible, removed the interlacing stripes, and also played around with adding noise to those sections that are already lacking in detail as the result looks more natural.

    I'm sure anyone familiar with this class of tool could do a really great job with this footage.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Chris Bergin on 06/24/2014 08:18 pm
    I had no part in this, but I'm partial to these two frames.

    I'll take number 2 please. Best res that's available.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: SwissCheese on 06/24/2014 08:19 pm
    Also, can we create this:
    http://forum.nasaspaceflight.com/index.php?topic=34597.msg1211091#msg1211091
    Into a 350 width x (whatever works) gif, no more than about 1.8 mb in size, to go into the article?

    It's probably not possible to keep all the frames composing this gif animation with a size below 1.8 mb, even with the resizing. We (actually more precisely saliva_sweet :P) probably either have to remove some frames or maybe post it as a html5 compatible video? (.mp4 / .ogg)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Chris Bergin on 06/24/2014 08:25 pm
    Only .gif will go into the article per the requirement.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Quialiss on 06/24/2014 08:25 pm
    Lourens' article now in first draft complete.

    While we're working this, can I request a best res, best view slide as the lead image for the article. So the best recovered image (Dimensions would be 630x250).

    Best image...  Well, we have a few more choices than those first two partial i frames that were first released.  I'll throw in a few favorites to narrow it down.  183, just after the exhaust plume hits the ocean.  211, just before touchdown.  280, 'Look ma, I float!'
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Quialiss on 06/24/2014 08:38 pm
    Hacked at version of saliva_sweet's gif of the reconstruction at the requested size and about half the frames removed to make it 1.77MB. 
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Chris Bergin on 06/24/2014 08:47 pm
    Hacked at version of saliva_sweet's gif of the reconstruction at the requested size and about half the frames removed to make it 1.77MB. 

    Wonderful! :)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Oersted on 06/24/2014 09:23 pm
    Congrats on a monumental team effort that will surely get its well-deserved footnote in Space Exploration History! - It has been an amazing thing to follow.

    Now it would be great to see another effort: namely to "doll up" the final vid. Yes, we space enthusiasts and purists will always prefer the real thing, but for everybody else out there a nice smooth hi-res video with sound will be superb.

    For inspiration, the wizardry done on the Curiosity Descent video (side-by-side comparison):
    https://www.youtube.com/watch?v=pjeHZ9poew4&list=UUhd-bsg3V3IXyqTIyNZu0sA&index=3

    The result:
    https://www.youtube.com/watch?v=Esj5juUzhpU&list=UUhd-bsg3V3IXyqTIyNZu0sA&noredirect=1

    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: saliva_sweet on 06/24/2014 09:48 pm
    Hacked at version of saliva_sweet's gif of the reconstruction at the requested size and about half the frames removed to make it 1.77MB.

    Might be worth it to add the latest progress with p frame conversion as the last frame. Unfortunately the video is using an old mpg4-img for that frame not the updated one here http://forum.nasaspaceflight.com/index.php?topic=34597.msg1215938#msg1215938
    It's not noticable in the video,  but looks a bit harsh as a still.

    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Lee Jay on 06/24/2014 10:14 pm
    I had no part in this, but I'm partial to these two frames.

    I'll take number 2 please. Best res that's available.

    That's already the best available.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Chris Bergin on 06/24/2014 11:04 pm
    I had no part in this, but I'm partial to these two frames.

    I'll take number 2 please. Best res that's available.

    That's already the best available.

    Copy that!

    This is going to be Thursday now (waiting for someone to get back on site).
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Swatch on 06/25/2014 10:36 pm
    So amazing what you all have done here.  I'm glad I could have contributed in my small way, but it's nothing compared to the hours and hours of amazing work that was performed by everyone on this project.  The video is truly a historic piece of rocket development and without your work it simply would not be appreciated by the public at large.

    All the effort by those involved here really makes me proud to be a member on this forum and involved.

    ~Swatch

    PS.  Totally dig the audio!
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Chris Bergin on 06/26/2014 01:23 pm
    Making good progress on getting the article read - which will be going on today after the USA vs Germany game.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Chris Bergin on 06/26/2014 07:03 pm
    Here's the technical feature article - by the team's Lourens Veen!

    http://www.nasaspaceflight.com/2014/06/recovering-falcon-9-ocean-landing-video-done/
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Lourens on 06/26/2014 07:09 pm
    Yay! My first NSF article! I hope you all like it :).
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: KEdward5 on 06/26/2014 07:11 pm
    Fascinating article with some amazing images. That was great!
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Lar on 06/26/2014 07:21 pm
    Lourens did a great job writing this up and explaining it. Remember everyone.. share share share!

    Thanks, Lourens!
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: catdlr on 06/26/2014 07:22 pm
    Great article Lourens.  You made it easy to understand the mechanics of this effort.  Great contributions on many that work on this exciting project.  Great that it happen on this Forum.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: IainCole on 06/26/2014 07:24 pm
    Nice article :)

    I noticed this link "managed to extract two still frames, but judged the error rate too high to allow further recovery." is broken, missing the first "h"
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: mheney on 06/26/2014 07:26 pm
    A nicely written, entertaining, and informative article.  Great job!
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Chris Bergin on 06/26/2014 07:26 pm
    Nice article :)

    I noticed this link "managed to extract two still frames, but judged the error rate too high to allow further recovery." is broken, missing the first "h"

    That was my fault. Since fixed ;)

    As Lar said. Get it shared. So many of you on twitter here and sit on your hands. Boooooooooooo!

    (Although this is doing well on twitter already).
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Danderman on 06/26/2014 07:45 pm
    Great effort!

    You guys should work on the russian Mars landing video from 1971 to see what you can recover.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: arnezami on 06/26/2014 07:54 pm
    Good work Lourens!
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Lee Jay on 06/26/2014 08:01 pm
    Yay! My first NSF article! I hope you all like it :).

    Gosh...I was wondering how Chris got up to speed on the ins and outs of MPEG so quickly!

    Great job, on the recovery and the article.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Jeff Lerner on 06/26/2014 08:31 pm
    My first and only post todate on this thread was
    way back on page 3...but I have been following the anazing
    Progress on a daily basis. Congrats to everyone who contributed.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: sugmullun on 06/26/2014 08:31 pm
    Outstanding article, Lourens. For what it's worth, I couldn't imagine it being any better.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Kojak on 06/26/2014 08:36 pm
    What a great article! Good job!
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: SwissCheese on 06/26/2014 09:25 pm
    Very nice article, thank you Lourens and Chris!
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Chris Bergin on 06/26/2014 09:55 pm
    This is flying on twitter.

    Some SpaceXers retweeting, but a good one is Rob Pegoraro @robpegoraro with the mention.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Swoopert on 06/26/2014 11:02 pm
    I am so proud to be a member of this community, if not the recovery! Great job everyone, it's been just as exciting checking this thread every few hours to follow the progress and pour over each newly restored frame as it is to follow the launch threads!

    A sincere thank you for all the effort that everyone has put in that's allowed me to live so vicariously!
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: robertross on 06/27/2014 12:25 am
    Here's the technical feature article - by the team's Lourens Veen!

    http://www.nasaspaceflight.com/2014/06/recovering-falcon-9-ocean-landing-video-done/

    The effort you all put in to bring this video back is truly inspiring. Incredible footage.
    Nothing less than a labour of love to make this achievement possible.

    Bravo!
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: yg1968 on 06/27/2014 01:40 am
    Great article and great work!
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: mhenderson on 06/27/2014 01:40 am
    I have been at an Oracle conference in Seattle all week and have been too busy.  I am on the plane home now and compiling the results of the hours survey, I will post something on the forum tomorrow.

    The keynote speaker at the conference spoke on the power of crowdsourcing from volunteer technical people to do social good ... such as visualizing display of disease or poverty statistics, automating the location of abandoned swimming pools where mosquitoes breed and spread malaria, and such.  I talked with him afterward and told him that this has been a classic data cleansing activity that exemplifies many of the things he advocates.  I sent him links to the forum, wiki, etc.

    Let's stay in touch.  We could all continue to do great stuff together.

    Mike
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: arnezami on 06/27/2014 09:51 am
    Hi guys,

    I just got T-shirts from SpaceX.  8)

    Gave me goosebumps. Really worth all the effort.

    Regards,

    arnezami
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: CuddlyRocket on 06/27/2014 10:40 am
    As Lar said. Get it shared. So many of you on twitter here and sit on your hands. Boooooooooooo!

    (Although this is doing well on twitter already).

    It would have done even better if you'd put #SpaceX instead of SpaceX! (I copied your tweet rather than retweeting it so I could do just that.)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Chris Bergin on 06/27/2014 11:43 am
    As Lar said. Get it shared. So many of you on twitter here and sit on your hands. Boooooooooooo!

    (Although this is doing well on twitter already).

    It would have done even better if you'd put #SpaceX instead of SpaceX! (I copied your tweet rather than retweeting it so I could do just that.)

    I never thought it made a difference. Besides, I don't want to turn into one of "those" people! ;D

    #bollocks

    Doing well anyway. Lots and lots of mentions on twitter. Article has 43,000 reads so far.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: SpaceX_MS on 06/27/2014 01:44 pm
    Here's the technical feature article - by the team's Lourens Veen!

    http://www.nasaspaceflight.com/2014/06/recovering-falcon-9-ocean-landing-video-done/

    Incredible work guys. Everyone here is full of admiration and thanks.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: pagheca on 06/27/2014 03:01 pm
    Hi guys,

    I just got T-shirts from SpaceX.  8)

    Gave me goosebumps. Really worth all the effort.

    Regards,

    arnezami

    So, where is the picture of the SpaceX t-shirt??? :)

    Congratulations everyone.

    Like so many people everywhere, I'm amazed by how much knowledge and passion you all put in this. It's really one of the most "stellar" example of "crowd-science" or whatever you want to call it, ever.

    I would really like to see the same model of operation into other kind of analysis and efforts. There is a huge potential for people to be involved, for example, in the analysis of major scientific databases that can't be done because of lack of resource and time.

    And be careful: apparently you can use SpaceX t-shirts only once. If you wash it in water the picture... fades.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Chris Bergin on 06/27/2014 06:04 pm
    Here we go..... ;D
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: luinil on 06/27/2014 07:05 pm
    SpaceX sharing the love on Google+ too =)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: mme on 06/27/2014 08:13 pm
    SpaceX sharing the love on Google+ too =)
    And Facebook too: https://www.facebook.com/SpaceX
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Chris Bergin on 06/27/2014 08:41 pm
    SpaceX sharing the love on Google+ too =)
    And Facebook too: https://www.facebook.com/SpaceX


    >John Kevin Cardinal I've been lurking on Nasaspaceflight for a couple of years. It's an amazing community - truly impressive. This is one of the best examples of tech crowd sourcing I've ever seen!<

    I like him. ;D
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: mhenderson on 06/27/2014 08:45 pm
    "HOW MUCH EFFORT DID THIS TAKE?"

    Following are the results of a poll. The purpose was to make an estimate of how many hours of effort -- in very rough terms -- went into the SpaceX CRS-3 booster landing video repair effort. The survey documents 2,800 estimated hours of effort by the core contributors to the effort. Beyond that, many more thousands of hours were invested by uncredited users and lurkers.

    I posted three requests on the NasaSpaceFlight.com forum used by the team. In those requests, I asked contributors to privately estimate the number of hours they spent on this activity. I promised information will remain anonymous and that their input would be used to paint a picture in broad strokes of the project effort. To encourage participation, respondents were told it was fine to submit "guesstimates" and broad range estimates. Non contributing users were encouraged to submit their time estimate, too.

    Respondents were given no guidance on what qualified as an hour of work. This may contribute to undercounting as at least one commenter took care to distinguish "12 hours of actual work" from time spent surfing / reading. Another commenter explicitly estimated both types of effort and that total was included in this tally. It is difficult to draw a line when the activity is a labor of love. And besides, "goofing off" (a.k.a. online research and team cohesion) played an important role in the overall effort.

    Twenty one users responded to the request.  Respondents can be independently divided into three cohorts: 12 from the "chosen fifteen", seven from the "next 24 credited users" and two from "uncredited users". Most included a brief comment along with their number or range, those comments are listed below.

    If the respondent reported a range of estimated hours, the midpoint of that range was used as their estimate.

    Cohort         Responses     Avg Hrs        Extrapolated Total Hrs
    ------------------------------------------------------------------------------
    Chosen 15       12 (80%)     109.7        1645 
    Next 24             7 (29%)      49.3          1183
    Uncredited        2 (N/A)        12.5          not estimated, but quite large
    Lurkers             -                     -             unknowable, but extremely large

    CHOSEN 15
    From this data set, we can see that the "80:20" rule is alive and well; a good chunk of the work effort was contributed by a committed core, and they were supported by a broader group of lesser contributors. Fifteen users had previously been identified by a peer vote to determine which contributors should received a thank you gift from SpaceX. Twelve of the "chosen 15", or 80%, responded to the survey. Reported total hours for this cohort were 1316, or 109.7 hours each.  By simple extrapolation, the non reporting users would increase the "chosen 15's" estimate to 1645 hours.

    NEXT 24
    The credits at the end of the video identify 39 userids as contributors - that list includes the "chosen 15" plus 24 others. Among the "next 24" seven users, or 29%, responded with estimates. Those seven users reported working a total of 345 hours, an average of 49.3 each. Again by simple extrapolation, the "24 others" groups estimated total effort is 1020 hours. Self-selection bias is always problematic in this sort of survey, and this relies on an assumption that respondents are representative of the others. If this selection is skewed toward heavy contributors who stuck around to the very end, then the total hours estimate for the "next 24" could be a bit high.

    UNCREDITED USERS
    There is a large body of "uncredited users". Application software and online tools were available. Some offered suggestions and many, many tried their hand at the online tool. And yet they never achieved results they actually submitted, or they submitted results anonymously. The survey only received two responses from this silent cohort. Those individuals reported spending 25 hours, an average of 12.5 hours per respondent.

    Any extrapolation from these few responses to the whole would be purely speculative. And yet it is not unreasonable to conclude that the uncredited effort was very, very large.  I suspect that web server statistics from the online tool could provide a better basis for a meaningful estimate; perhaps information such as the number of ip addresses and the number of images served to each address could begin to approximate that effort.

    LURKERS
    The final category of people are "lurkers" who contributed a level of enthusiasm to the project -- a cheering section for the gladiators. No responses were received (because simply responding would make them an uncredited user.) The project has been a viral, highly open activity. The forum has been visited more than 360,000 times, if each user dwelled on a page for one minute per pageview (a conservative estimate) simply reading the forum accounts for 6,000 hours of "effort". A second discussion thread was created on NasaSpaceFlight.com to provide a separate place for interpretation of the emerging images. The discussion thread has served 22,500 pageviews or another 375 hours using the one minute per pageview estimate. Because the video was regularly pushed to a YouTube channel, various iterations of the short video were viewed repeatedly, I have not compiled any statistics from that site. Discussion threads on competing websites were active with commentary and kudos to the team.

    Comments
    In addition to the hours estimates, many respondents provided comments. A selection of these are provided below.

    * "According to your final request on the number of hours : I played with the online editor, and followed the technical discussions, and rehearsed a bit about MPEG-4 FEC. 1 or 2 hours on the editor. 1 or two hours googling on FEC. hours on the thread ? 5-20, very imprecise!"

    * "Thanks to all for this wonderful work. Thanks to you for accounting. I think this might become a reference for studies of crowdfunding as a method os undirected self-organisation for work. And I'd really love to hear what the results are!"

    * "50-100 hours. Would have been more but I'm trying to finish up grad school."

    * "~12 hours of actual work.  Countless hours of job time wasted following developments, goofing around with the online editors, and playing with ffmpeg."

    * "Reading all posts alone will add up into the 50 or so hours. Pitching in will add about another 15 hours orso. Rough guesstimate. :) "

    * "I think I spent roughly 75 hours on this project, mostly fixing alignment in iframes. Might be more, I kinda lost track of time :) Should be in the 50-100 hours category, though. Thanks for starting this poll, I'm curious to see the results :D "

    * "I guess 200-500 hours for me. But it's hard to say: I spent so many hours (at work) just thinking about certain problems. If you count those its much more. ;) "
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Avron on 06/27/2014 08:50 pm
    SpaceX sharing the love on Google+ too =)
    And Facebook too: https://www.facebook.com/SpaceX


    >John Kevin Cardinal I've been lurking on Nasaspaceflight for a couple of years. It's an amazing community - truly impressive. This is one of the best examples of tech crowd sourcing I've ever seen!<

    I like him. ;D

    congrats..  now to get Elon to RT
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Quialiss on 06/27/2014 09:08 pm
    "HOW MUCH EFFORT DID THIS TAKE?"

    Now there's a post-mortem!  Thanks for collecting this data, it's fascinating to see how much time everyone put into this.  Amusingly, after being so involved in the project with everyone here, I can tell who more than a few of those comments came from.  ;D
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Rocket Science on 06/27/2014 09:25 pm
    The next mission for the gifted team at NSF will be where “all the king’s horses and all the king’s men” failed and put Humpty Dumpty back together again... ;D
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Seattle Dave on 06/27/2014 09:30 pm
    The next mission for the gifted team at NSF will be where “all the king’s horses and all the king’s men” failed and put Humpty Dumpty back together again... ;D

    I'm with the FBI and I think we'll be calling on these guy's services!
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: arnezami on 06/27/2014 10:11 pm
    Hi guys,

    I just got T-shirts from SpaceX.  8)

    Gave me goosebumps. Really worth all the effort.

    Regards,

    arnezami

    So, where is the picture of the SpaceX t-shirt??? :)

    Congratulations everyone.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: mvpel on 06/28/2014 12:43 am
    I wore that T-shirt on June 13 in McGregor, as it happens. :)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: CuddlyRocket on 06/28/2014 08:25 am
    I wore that T-shirt on June 13 in McGregor, as it happens. :)

    I hope they washed it before they sent it to arnezami!
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Jcc on 06/29/2014 01:38 pm
    Here's the technical feature article - by the team's Lourens Veen!

    http://www.nasaspaceflight.com/2014/06/recovering-falcon-9-ocean-landing-video-done/

    Incredible work guys. Everyone here is full of admiration and thanks.

    What I would like to see is how the video is being used, and what has been learned from it.
    For instance, it looks like the legs deployed with a slight bounce or wobble before stiffening, was that expected?
    Can you tell how deeply the stage entered the water before the video ended?

    Oops, just noticed the video interpretation thread, excellent!
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: mhenderson on 06/29/2014 05:50 pm
    2800 hours = 10,080,000 seconds.

    I just realized the 20 second video took over ten million seconds of core team effort to repair.

    That's 504,000 seconds of effort per second of repaired video.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: aero on 06/29/2014 06:23 pm
    2800 hours = 10,080,000 seconds.

    I just realized the 20 second video took over ten million seconds of core team effort to repair.

    That's 504,000 seconds of effort per second of repaired video.

    Yes, but how much time is it per repaired pixel? And if that is still to big a number then how much time is it per repaired bit?
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Swatch on 06/29/2014 07:53 pm
    Or better yet... how many hours per MegaBlock... that seems to have been the defining unit of modification for this project. :)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Quialiss on 06/29/2014 08:00 pm
    Here's a fun estimate:  How long would it take to do this again?  Separating out the time it took for all of us to learn how to fix the video from how long we spent actually fixing it. 

    One of the few things I do know how long it took me to do: 6 hours for 38 p frames (2 sets).  SwissCheese also went over them after me and fixed my mistakes, and then I went over them again after that to fix up DC issues, so lets bump that up to ~10 hours.

    So ~10 hours, ~40 p frames

    1 p frame ~ 15 minutes. 

    13 parts * 19 p frames * 15 minutes = ~60 hours to fix all the p frames

    The i frames are harder to account for because they took much longer to fix, and thus tended to be done in stages.  I can throw an estimate of ~2 hours per frame for fixing up the DC values after all the good macroblocks had been found.  The actual time I spent on this varied wildly per frame and was rather higher because I was learning how to do it, but I don't think redoing even the worst of them would would take me more than 3 hours of work now.

    Anyone have any time estimates for how long it took to find the good data in the i frames? 
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: mhenderson on 06/29/2014 10:34 pm
    Here's a fun estimate:  How long would it take to do this again?   

    Excellent assessment ... as we have come expect from you. This project has moved the art of garbled video restoration a large step forward.  I suspect that three or four key people -- knowing what we know now, and using the tools that were built -- could fix the same data stream in less than a week.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: princess on 06/30/2014 10:05 am
    I suspect that three or four key people -- knowing what we know now, and using the tools that were built -- could fix the same data stream in less than a week.

    With the TS-level tools and experience we have now, I reckon I could probably provide a fixed TS file in under a day. That would leave four days for everyone else to do the MMB fixing.

    I just had a thought - if SpaceX decide to drop the next webcast, I'll provide an equivalent length TS file with correct timestamps and frame headers for everyone to get started on ;)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: S.Paulissen on 06/30/2014 02:46 pm
    I only say this half-jokingly.

    You guys should think about making a company to do this for money.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Quialiss on 07/02/2014 06:28 am
    To celebrate some awesome t-shirts arriving at my doorstep, an update on what I've been working on.  It's tangentially related to my coursework... so I tell myself.  Live problems are always so much more interesting.  :)

    I made a minor breakthrough today in automatically finding good MB start positions, with my script only hitting two false positive matches before finding the manually located start position, down from 10+ false positives.  I'm working with frame 10 for a fairly simple start case to work with, but I've tested it on some other frames(and the end of frame 10) with similar results.

    Current methodology:

    Test cases run on macroblocks from the new start position to end of the row(arbitrary end point but effective enough for the moment)
    lum/chrom levels.bit_length() == size.  The levels should fit exactly into the size, if they're smaller, the size/value pair is wrong.
    lum size < 7 and chrom size < 4.  These are derived from analysis of repaired frames. 

    Require 15 blocks from new start position decoded without errors.  This is largely a crutch to shore up the lack of better tests, as it catches DC clipping from bad lum values and alignment.  I put a high priority on being able to shrink this value without increasing false positives as it means smaller pieces of data will be recoverable.

    I haven't tested how many false positives each of these cases catches, it would be interesting to check.  I know reducing the error free blocks to 10 causes ~70 false positives to show up. 


    The automatically generated MMBs, with my notes on what to make new test cases for.  Cut off point is just before the next correctly found block.  Since it doesn't know which one of the candidates is correct, it just blindly continues on trying to find good start positions.

    08:14:-2,9:14:56667 Texture and lum
    08:14:-2,9:14:56677 Texture, lum and alignment
    08:14:-2,9:14:56683 *Hand picked solution
    08:14:-2,9:14:56699 Texture
    08:14:-2,9:14:56711 Texture, alignment
    08:14:-2,9:14:56727 Texture, alignment
    08:14:-2,9:14:56744 lum, alignment

    I know I can build a test case to rule out the ones marked with lum errors.  There are legitimate times where the lum values change that much in a block, however, the rows/columns of very different lum values are a dead giveaway that there's an error.  It'll just be a much more *complicated* test case than the ones I've already established.

    Errors in the DCT causing the texture of the block to be bad... I would love to hear if anyone has ideas on how to test for those!

    Realigning the chunks of data MIGHT not be too difficult.  At least in this one case, it's notable that if the data is out of alignment, dc clipping errors show up everywhere, and go away when the data is correctly aligned.  Nudging the position back and forth 1-5 blocks would work here, but that assumes it was close to correct in the first place.  When I get to this, I'll check to see if I can use the dc clipping errors to verify alignment in other places or if it's just showing up so strongly here because of the detail in the end of the rocket. 

    I've been toying with the idea of advancing the mb position after testing the mean block size of bits(~100).  This would actually be horrible for the first error in frame 10, as the error is in a block of size ~400, and in any case would require putting the 'wiggle' alignment checks in to get any good results out of it. 


    Amusing complications:  My unit tests for valid block values are now 'better' at detecting errors than the error log from ffmpeg.  This is a problem because I'm using the error log to decide when to start looking for new start positions! I correctly detect that the next error in frame 10 starts at 08:21, with a too high lum size, whereas the error log finds the next error at 13:21. 
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: mvpel on 07/02/2014 10:28 am
    To celebrate some awesome t-shirts arriving at my doorstep, an update on what I've been working on.  It's tangentially related to my coursework... so I tell myself.  Live problems are always so much more interesting.  :)

    I think Dr. Young will be quite interested to discuss this with you. He's been working on cognitive video quality analysis (http://spie.org/Publications/Proceedings/Paper/10.1117/12.2018217), and automated detection of visual artifacts in a decoded video stream, (http://spie.org/Publications/Proceedings/Paper/10.1117/12.2051038) and at first glance it seems like that work might be able to automate the selection of the best fix from the list of candidate MMB positions which your code produces.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: princess on 07/02/2014 04:38 pm
    I just received a box via FedEx containing a SpaceX baseball cap and 5 SpaceX shirts, including a CRS-3 mission patch shirt! Yay!

    Hopefully the SpaceX staff who arranged it are reading this thread - thank you very much for sending them, they're very much appreciated. You guys are the best!
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: mvpel on 07/02/2014 07:59 pm
    I just received a box via FedEx containing a SpaceX baseball cap and 5 SpaceX shirts, including a CRS-3 mission patch shirt! Yay!

    The box made it to England already, but not to Massachusetts yet?  ??? Poor Linda's starting to find me annoying when I ask her if it arrived yet on a semi-daily basis. ;)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: princess on 07/02/2014 08:40 pm
    The box made it to England already, but not to Massachusetts yet?

    Aww I'm sure it will come soon... I guess FedEx UK must just be very efficient ;)

    While you're waiting, I've attached a pic of what I received, with the CRS-3 T-shirt taking pride of place in the middle.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: deruch on 07/02/2014 08:46 pm
    I just received a box via FedEx containing a SpaceX baseball cap and 5 SpaceX shirts, including a CRS-3 mission patch shirt! Yay!

    The box made it to England already, but not to Massachusetts yet?  ??? Poor Linda's starting to find me annoying when I ask her if it arrived yet on a semi-daily basis. ;)

    Yeah, about that......IainCole did a quick recount after some voting anomalies were discovered and you actually got bumped to #16.  No one had the heart to tell you, sorry bro.   :'(

    just kidding  ;D
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: mhenderson on 07/03/2014 10:37 pm
    Selfie. Thank you SpaceX, NSF, and the entire team.
    ** Someone should fix this. I haven't got the first clue how to start.  ;)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Dudely on 07/04/2014 02:25 pm
    Looks like a bitflip to me  ;).
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: mvpel on 07/04/2014 08:02 pm
     
    Yeah, about that......IainCole did a quick recount after some voting anomalies were discovered and you actually got bumped to #16.  No one had the heart to tell you, sorry bro.   :'(

    just kidding  ;D

    Or maybe...

    USPS Worker Caught Throwing Packages Into Ravine (https://www.youtube.com/watch?v=M1RsUbl4QsU)

     :o :D
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Chris Bergin on 07/05/2014 01:03 am
    Selfie. Thank you SpaceX, NSF, and the entire team.
    ** Someone should fix this. I haven't got the first clue how to start.  ;)

    HAHA! I was literally about to stick that into MS Paint and do a rotate - pretty much the limit of my skills - until I finally got the gag! ;D
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: boo on 07/05/2014 11:57 am
    Selfie. Thank you SpaceX, NSF, and the entire team.
    ** Someone should fix this. I haven't got the first clue how to start.  ;)

    Looks like an "I" frame to me ;)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: meekGee on 07/05/2014 03:46 pm
    Selfie. Thank you SpaceX, NSF, and the entire team.
    ** Someone should fix this. I haven't got the first clue how to start.  ;)

    Looks like an "I" frame to me ;)

    Ok, that's just too clever.

    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: mvpel on 07/07/2014 04:07 pm
    Yay!  ;D The box finally arrived! Well, it arrived last Thursday morning on the 3rd, but I got the call from shipping this morning.  ::) The cap is a classy grey one with the SpaceX logo, and the CRS-3 Mission Patch t-shirt of course, and a few nice V-neck logo t-shirts.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Chris Bergin on 07/07/2014 04:38 pm
    Great stuff, and this is now about to go through 390,000 views. Given the way people find or revisit threads, this will easily get over the 400,000 mark.

    Absolutely astonishing how this all progressed and the way it was appreciated by SpaceX/Elon.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: getitdoneinspace on 07/08/2014 08:16 pm
    I heard that Elon was asking when you guys will be done with the restoration of the CRS-3 first stage landing video in High Definition. And the audio wasn't yet up to Dolby surround sound quality, but it was close.  ;)

    Chris - You may want to let SpaceX know that their link to the restoration video is pointing to a private video (just see static).

    Take a look at the following URL: http://www.spacex.com/news/2014/04/29/first-stage-landing-video

    This is the URL they are using for the restoration video on the above page:  http://www.youtube.com/watch?v=XoufUV5oGTo
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: mvpel on 07/10/2014 05:29 pm
    Thanks, SpaceX!
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: bilbo on 07/12/2014 04:31 am
    I heard that Elon was asking when you guys will be done with the restoration of the CRS-3 first stage landing video in High Definition. And the audio wasn't yet up to Dolby surround sound quality, but it was close.  ;)

    Chris - You may want to let SpaceX know that their link to the restoration video is pointing to a private video (just see static).

    Take a look at the following URL: http://www.spacex.com/news/2014/04/29/first-stage-landing-video

    This is the URL they are using for the restoration video on the above page:  http://www.youtube.com/watch?v=XoufUV5oGTo


    BAH, Oh well...  I think we all have seen it enough times to get the picture.
    if any of us really wanna see it, link to landing restorations final video.

    https://www.youtube.com/watch?v=CjZ33C9JZTM

    Very proud of all of you for restoring video. Would Have loved to help, but never knew how to restore video, or deal with computers or code so I lost that battle.

    P.S. Great job, really jealous of you guys!
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Quialiss on 07/13/2014 02:33 pm
    Just thought I'd share what I've been working on recently.  GLCM texture analysis.  By calculating the grey comatrix and properties of each 8x8 pixel subblock, you can see the abnormalities caused by faulty DCT texture values.  A picture's worth more than a thousand technical terms, I've attached examples of i frame 3 and 10, and texture data from frame 3 before thresholding. 

    Unsolved problems:  The highlighted blocks on frames 3 and 10 are thresholded at 60% and 80% respectively, and yet it still picks up valid features in frame 10; waves, legs, edge of the rocket body, and fails to highlight a number of obvious texture errors.  Comparing to a local average should get better results.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: gospacex on 07/17/2014 07:38 am
    The video is still not on wiki here?

    http://en.wikipedia.org/wiki/SpaceX_Falcon_9_booster_post-mission,_controlled-descent,_test_program
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: wronkiew on 07/17/2014 02:45 pm
    The video is still not on wiki here?

    http://en.wikipedia.org/wiki/SpaceX_Falcon_9_booster_post-mission,_controlled-descent,_test_program

    SpaceX would need to license the original with terms compatible with Wikipedia. Chris asked them but I don't think it went anywhere.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: cscott on 07/17/2014 03:12 pm
    The video is still not on wiki here?

    http://en.wikipedia.org/wiki/SpaceX_Falcon_9_booster_post-mission,_controlled-descent,_test_program

    SpaceX would need to license the original with terms compatible with Wikipedia. Chris asked them but I don't think it went anywhere.

    I work for Wikipedia.  I'd be interested in helping this discussion progress, if there's anything I can do.

    Note that images uploaded to English wikipedia must fall under one of four categories:
    Quote
    Own work: You own all rights to the image, usually meaning that you created it entirely yourself.
    Freely licensed: You can prove that the copyright holder has released the image under an acceptable free license. Note that images that are licensed for use only on Wikipedia, or only for non-commercial or educational use, or under a license that doesn't allow for the creation of modified/derived works, are unsuitable.
    Public domain: You can prove that the image is in the public domain, i.e. free of all copyrights.
    Fair use: You believe that the image meets the special conditions for non-free content, which exceptionally allow the use of unlicensed material, and you can provide an explicit non-free use rationale explaining why and how you intend to use it.

    Although I'd love to see SpaceX release the material into the public domain or under a free license (since they did post it on youtube, etc), we may be able to proceed under fair use as well.  The [Wikipedia content policy](https://en.wikipedia.org/wiki/Wikipedia:NFC#Policy_2) appears to permit this; in particular the "previous publication" policy appears to hold.

    Wikipedia in general prefers Ogg Theora and WebM formats for video, and limits uploads to 100M.  Considering the fact that the original MPEG encoding is actually significant to the context of the file, there might be an exception here.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: SwissCheese on 07/17/2014 04:00 pm
    Wikipedia in general prefers Ogg Theora and WebM formats for video, and limits uploads to 100M.  Considering the fact that the original MPEG encoding is actually significant to the context of the file, there might be an exception here.

    I don't think there is a problem changing the output video format, since the repaired video was generated from a stack of images and not directly from the original stream.

    I converted it into WebM (VP8 codec) and OGG (Theora codec) containers (attached zip). Somehow my VLC player cuts the first 2 seconds of the OGG movie, I don't know why (it plays correctly on firefox though). WebM version is playing correctly and looks good. Source file is here:

    http://forum.nasaspaceflight.com/index.php?topic=34597.msg1218099#msg1218099
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: mvpel on 07/17/2014 08:44 pm
    Wikipedia in general prefers Ogg Theora and WebM formats for video, and limits uploads to 100M.  Considering the fact that the original MPEG encoding is actually significant to the context of the file, there might be an exception here.

    In the meantime perhaps just an external link to the YouTube page containing the final release?

     https://www.youtube.com/watch?v=CjZ33C9JZTM (https://www.youtube.com/watch?v=CjZ33C9JZTM)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Chris Bergin on 07/28/2014 10:46 am
    There's your 400,000 views. That's another 8,000-10,000 views since the last post, showing a lot of people are still catching up or re-reading this amazing thread.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: mhenderson on 07/28/2014 11:32 am
    The repaired video is used in this YouTube mashup on the OG2 mission, very nicely done video.

    https://www.youtube.com/watch?v=o-3zYpbw53I
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Chris Bergin on 07/29/2014 09:14 am
    Funny how the restored video is still the best of the two, so that was a good idea to mix the footage for a full profile.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Llian Rhydderch on 07/29/2014 09:53 pm
    The repaired video is used in this YouTube mashup on the OG2 mission, very nicely done video.

    https://www.youtube.com/watch?v=o-3zYpbw53I

    Great job by Greg Alexander, the fellow who did the edit and placed this mashup on YouTube!

    If someone familiar with YouTube media knowledge knows how to contact the guy, we ought to invite him to join NSF.  His kind of work would be appreciated over here.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: IainCole on 07/29/2014 10:07 pm
    The repaired video is used in this YouTube mashup on the OG2 mission, very nicely done video.

    https://www.youtube.com/watch?v=o-3zYpbw53I

    Great job by Greg Alexander, the fellow who did the edit and placed this mashup on YouTube!

    If someone familiar with YouTube media knowledge knows how to contact the guy, we ought to invite him to join NSF.  His kind of work would be appreciated over here.

    Done
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: saliva_sweet on 07/29/2014 10:12 pm
    we ought to invite him to join NSF.  His kind of work would be appreciated over here.

    I guess it's kind of late for that. He's the one that originally posted it to NSF

    http://forum.nasaspaceflight.com/index.php?topic=35243.msg1235074#msg1235074
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: IainCole on 07/29/2014 10:13 pm
    Boom ^
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: GregA on 07/29/2014 11:54 pm
    Great job by Greg Alexander, the fellow who did the edit and placed this mashup on YouTube!

    If someone familiar with YouTube media knowledge knows how to contact the guy, we ought to invite him to join NSF.  His kind of work would be appreciated over here.
    Thanks Llian. As others said above, I posted it here first actually. I just didn't want to cross post to multiple threads. :)

    (Iain, I didn't receive any message via Youtube, did you send one? It's my first uploaded vid so might be missing something)

    I started with a video using the animation launch merged to the orbcomm descent, with side by side of the shorter CRS repaired video...  and then got carried away. Combining footage can really improve the narrative of a video, though obviously all credit has to go to SpaceX who made all these great sources/videos in the first place... plus you all at NSF for the repair which is beyond my skill set!

    I did some small visual clean ups too, mainly
    * dropped the loud crackle of the Orbcomm launch,
    * removed the transmission glitch in the middle of the internal cam of stage1 separation - there actually hadn't been any lost frames, just a pause with the error message, which surprised me
    * slight zoom towards the attitude jets (?) on stage 2 to draw attention to the first manoeuvring of the returning stage
    * cropped out the bottom of the repaired and iced videos. This made them 16:9, but also since the restored video's colouring changes slightly at each I-frame, the larger rocket body's colouring drew the viewers eyes away from the more interesting aspects (water and legs) which were the objective of this version.

    BTW I agree the repaired video done by everyone here is clearer than the perfectly transmitted icey video!

    edit: changed restore/repair naming
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: mvpel on 07/30/2014 12:34 am
    Is it still on YouTube? I can't find it with a search and the links here are broken. Something about "WMG content."
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: rocketguy101 on 07/30/2014 12:59 am
    Is it still on YouTube? I can't find it with a search and the links here are broken. Something about "WMG content."
    I was going to share the link on facebook, and youtube warns "This video is unlisted. Be considerate and think twice before sharing." so perhaps Greg needs to change a setting??
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: GregA on 07/30/2014 01:07 am
    I got a note on the Youtube saying the audio was unlicensed. The Youtube information said the owner might just put ads on, and then I noticed ads on it so figured it was fine.

    Hmmm.. and it's still working for me. Might be removed in US. I'll make one without the audio...
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: GregA on 07/30/2014 01:14 am
    "This video is unlisted. Be considerate and think twice before sharing." so perhaps Greg needs to change a setting??
    Ahh I just listed it that way. It's a choice of Public, Private, or Unlisted - and I was just doing something for interest here :)

    Will still make one without the audio... and if it's been removed in the US then I do need to. Can anyone else confirm (since it works in Australia).
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: mvpel on 07/30/2014 01:59 am
    Ok, I just tried it and it works via the embedded screen in the forum on my Linux box, but wasn't working on my iPhone through Tapatalk - I thought that something else was going on since I couldn't find it with a search either. Sorry for the false alarm.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: rocketguy101 on 07/30/2014 02:26 am
    Ok, I just tried it and it works via the embedded screen in the forum on my Linux box, but wasn't working on my iPhone through Tapatalk - I thought that something else was going on since I couldn't find it with a search either. Sorry for the false alarm.
    It works for me [in the US], I just wondered if it was OK to post the link elsewhere...it is very cool!
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: GregA on 07/30/2014 01:46 pm

    It works for me [in the US], I just wondered if it was OK to post the link elsewhere...it is very cool!
    Yeah that's okay. :)
    I may end up redoing with different audio though!
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Jarnis on 08/04/2014 10:39 am
    Protip on YouTube: Never use third party copyrighted audio. You'll get totally random results - some regions see it fine, some see it with ads, some get "blocked by XXXX" - rights to various music pieces are a massive global mess where single piece of music may have dozens of "owners" depending on where you are...

    Either completely original music, or music that is specifically released as royalty-free. Otherwise YouTube will make a mess, one way or the other. Worst thing; You may yourself be completely unaware that, say, 80% of the world can't view the video at all because of copyright claims.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Lars_J on 08/04/2014 09:05 pm
    Protip on YouTube: Never use third party copyrighted audio. You'll get totally random results - some regions see it fine, some see it with ads, some get "blocked by XXXX" - rights to various music pieces are a massive global mess where single piece of music may have dozens of "owners" depending on where you are...

    Either completely original music, or music that is specifically released as royalty-free. Otherwise YouTube will make a mess, one way or the other. Worst thing; You may yourself be completely unaware that, say, 80% of the world can't view the video at all because of copyright claims.


    Another alternative is to upload another version of it without music, and then link to it in the description of the original video with a "in case you can't watch this, go here" message.
    Title: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: GregA on 08/05/2014 12:30 pm
    Yeah I'll make a version without music. Wonder if I should integrate the latest landing video...

    Edit. But I assume there's a good reason that vid keeps disappearing from NSF, which I'd rather blindly respect.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: cscott on 08/06/2014 01:47 am
    Yeah I'll make a version without music. Wonder if I should integrate the latest landing video...

    Edit. But I assume there's a good reason that vid keeps disappearing from NSF, which I'd rather blindly respect.

    The video is now public, full story:
    3) Thankfully, we now have permission to post this openly, without having to use the links of those who clearly have absolutely no regard for sourcing or accreditation. I appreciate NSF members' patience in waiting for a sanctioned upload, which has been uploaded to one of the CRS-3 Propulsive Landing Video youtube accounts.

    http://youtu.be/IBcj38ilNn4
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Chris Bergin on 09/17/2014 01:18 pm
    Coming up on 420,000 views! And bumping as I saw yet more twitter references to it last night.

    It's the gift that just keeps on giving.

    And I have to say, SpaceX continue to appreciate it - it's mentioned almost every time a SpaceXer talks to me for the first time. We're working on building a very strong relationship (long term effort, but one that could be amazing for both parties) that you'll start to see more and more over the future. A lot of that will be thanks to this NSF crowd sourcing effort.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: deruch on 09/18/2014 07:52 am
    Coming up on 420,000 views! And bumping as I saw yet more twitter references to it last night.

    It's the gift that just keeps on giving.

    And I have to say, SpaceX continue to appreciate it - it's mentioned almost every time a SpaceXer talks to me for the first time. We're working on building a very strong relationship (long term effort, but one that could be amazing for both parties) that you'll start to see more and more over the future. A lot of that will be thanks to this NSF crowd sourcing effort.

    I'm sure I'm not the only reader of these fora that gets a smile on his face when there's a "new" badge next to this thread. :)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: mlindner on 11/22/2014 04:25 pm
    https://www.youtube.com/watch?v=XoufUV5oGTo

    The video is private. This is the one linked from Elon's twitter, SpaceX's twitter and SpaceX's website. Why is this video private now???
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: moralec on 12/03/2014 05:32 pm
    Any ideas?
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: mvpel on 12/13/2014 07:39 pm
    That's an earlier version, it would appear. The channel is here:

    www.youtube.com/channel/UCyZDgyJBYz3OXD3JbDJzNww (http://www.youtube.com/channel/UCyZDgyJBYz3OXD3JbDJzNww)

    ... and the final version posted in the channel is here:

    www.youtube.com/watch?v=CjZ33C9JZTM (http://www.youtube.com/watch?v=CjZ33C9JZTM)

    I can't tell who owns the XoufUV5oGTo version...
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Llian Rhydderch on 12/31/2014 10:36 am
    The successful controlled-descent test on CRS-3 has made the 2014 in science (https://en.wikipedia.org/wiki/2014_in_science#April) list on Wikipedia.    8)

    Look at 18 April.

    Congratulations to all who were involved in documenting the video return of this historic flight!
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Chris Bergin on 06/09/2015 08:46 am
    Placing you guys on Standby One just in case Emily Lakdawalla needs some help!

    I've made the offer and if accepted we'll set up a thread. Lightsail is cool, but Emily is also a friend of the site.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: cscott on 06/10/2015 02:14 pm
    They've asked for help decoding a JPEG:
    https://twitter.com/jasonrdavis/status/608625405449261057
    Quote
    CROWDSOURCE CALL: Can you help compile a #LightSail image? This JPG is from other camera, but loads as blank image.  http://t.co/YZ3w0JwS5b
    https://twitter.com/jasonrdavis/status/608630375745593344
    Quote
    The #LightSail image I posted has first 24 bytes removed (metadata), and this header attached: http://t.co/TEDRVQlgXy
    https://twitter.com/jasonrdavis/status/608630904894803968
    Quote
    Finally, here's the original-original #LightSail image, with metadata intact and no headers attached: http://t.co/COTkDCgcYr Thanks!
    https://twitter.com/jasonrdavis/status/608647966170054656
    Quote
    For those having trouble with LightSail links, here's a Google doc: https://docs.google.com/document/d/1DiPleK4f0VEK2joqYXFrceWBS-CWV25kTOYo0UC3AUA/edit?usp=sharing …
    Quote
    LightSail image processing help

    Thanks for helping us try to compile LightSail's second sails-out camera image!

    Original image from spacecraft:
    http://planetary.s3.amazonaws.com/assets/z-misc/2015/1.jpg

    The first 24 bytes store metadata like image resolution (1600 x 1200) and timestamp. This is NOT part of the JPG format and is instead removed and used to generate file names.

    Our process for creating a viewable image is to:

    Remove the first 24 bytes of the file.
    Append a header to the beginning of the file.

    The header can be found here:
    http://planetary.s3.amazonaws.com/assets/z-misc/2015/header.txt

    For this particular image, this is the result:
    http://planetary.s3.amazonaws.com/assets/z-misc/2015/P01_15878x16388_176-1_T47_3_26_D2_20_0.out.jpg

    However, as you can see, the image does not load.

    Here's how things look when the process works correctly.

    Yesterday's raw sails-out JPG:
    http://planetary.s3.amazonaws.com/assets/z-misc/2015/full-sail1.jpg

    After removing the first 24 bytes and appending the header, this was the result:
    http://planetary.s3.amazonaws.com/assets/z-misc/2015/P02_1600x1200_30-1_T0_47_33_D20_0_70.out.jpg
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: saliva_sweet on 06/18/2015 06:53 pm
    They've asked for help decoding a JPEG:
    https://twitter.com/jasonrdavis/status/608625405449261057

    Well. I looked today, and that does not look like a jpg at all. It's some sort of repetitive message. Looks to be 1024 byte packets. May be telemetry or something.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: cscott on 06/18/2015 07:06 pm
    Within a day after receiving this, the satellite's transmitter got stuck in a CW tone of some sort.  I wonder if this is an early indicator of that problem?
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: moralec on 06/18/2015 07:09 pm
    Should we start a new thread for this? Repair Task 2.0 :)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Lee Jay on 06/18/2015 07:46 pm
    They got that picture down:

    http://www.planetary.org/blogs/jason-davis/2015/20160615-lightsail-test-mission-ends.html
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: cscott on 06/18/2015 08:35 pm
    They got that picture down:

    http://www.planetary.org/blogs/jason-davis/2015/20160615-lightsail-test-mission-ends.html

    No, the undecoded data I posted above is the picture from the "opposite side", described in your cited article as, "But before engineers could get a picture from the opposite-side cameras, LightSail’s radio began transmitting a continuous, nonsensical signal, and the spacecraft stopped responding to commands."

    As I speculated above, the repeated patterns found by @saliva_sweet may in fact be the first indicators of the "continuous, nonsensical signal" that LightSail eventually settled into.

    The mystery of this lockup beacon is still unsolved.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: princess on 07/16/2015 12:41 pm
    I had a look at the Lightsail data and I can confirm what saliva_sweet found - it's not JPG data, it's some other data in 1024-byte packets. Other things I found are:

    1. The packets seem to be repeating data with bitflips between each packet. I can't see any values that change numerically, and the bitflips seem fairly randomly distributed through the packet, so I don't think it's telemetry. It seems more like the computer is attempting to transmit the same packet again and again, with corruption occurring.

    2. I tried splitting the packets at different boundaries - the lowest difference between consecutive packets occurs if you start at 928 bytes into the file.

    3. The packets go out of alignment in two different places, by two bytes each time (both are a two byte deletion). If you take the first line of each packet's hexdump and display them one after another, the misalignment places are easy to see.

    4. There seem to be two different types of packet. The file initially only contains the first type of packet, and then the second type starts appearing interleaved with the first. It's not a perfect interleave, you sometimes get multiple instances of one type before it flips to the other type.

    In summary, it looks more like the repeated transmission of two different 1K packets of data, with random corruption. I'm not sure if a radio fault or a CPU fault would do this, but it's certainly intriguing!
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: meetsitaram on 01/14/2016 01:33 pm
    I was looking back at the video that has been improved here, at http://www.spacex.com/news/2014/04/29/first-stage-landing-video (http://www.spacex.com/news/2014/04/29/first-stage-landing-video) but that video no longer works. Had that been removed?
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: moralec on 04/14/2016 07:01 pm
    A little bit of our video made it here.
    Very happy to know that NSF played a role in this amazing project.

    https://www.youtube.com/watch?v=tU1b1H2EWU4
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: princess on 10/19/2016 04:25 pm
    In case anyone is interested, I've posted my MPEG-TS repair software to Github. The repository contains the latest code that I used to make the final version of the TS file - the one that went into the main MPEG4 macroblock repair effort and the fixed TS file that I used to make the pretty error pictures.

    Link is here: https://github.com/pmillerchip/spacex-tsrepair

    Over the next few days I'll try to get together all my fix commands, which will allow someone to download the raw.ts file from SpaceX's page and produce a fixed TS file in one go. Pull requests are always welcome!
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Req on 09/14/2017 07:34 am
    Our video made it into the crashes video  ;D

    https://twitter.com/elonmusk/status/908229340139323392
    @elonmusk
    Just posted a video https://www.instagram.com/p/BZA0s7EAmF1/

    Edit:
    https://twitter.com/elonmusk/status/908254739825025025
    @elonmusk
    High res version at
    https://www.youtube.com/watch?v=bvim4rsNHkQ
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Lar on 09/14/2017 10:45 pm
    Our video made it into the crashes video  ;D
    Awesome, even if uncredited (I tweeted Elon about it, I'm sure he'll get right on fixing that)
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: Chris Bergin on 04/30/2019 07:03 pm
    Five year anniversary!

    https://twitter.com/NASASpaceflight/status/1123300237647519744

    And a video, don't laugh at my editing skills, but it gets it all in there :)

    https://youtu.be/r4anK4RGVQ0
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: meekGee on 05/01/2019 04:45 am
    How Brilliant!
    My eyes are tearing but it's the allergies, I tell you.
    Title: Re: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread
    Post by: HVM on 03/14/2021 02:16 pm
    This dude ask help:
    https://twitter.com/r2x0t/status/1371089254772801543