I found the missing p-frame between i-frames 150 and 169, it is just following the i-frame. Here the debug log at the end of the i-frame, the next frame is a p-frame but keeps the number 000 when it should have 001. Should we try to find the position in the ts file and hand-edit the "ghost" p-frame header?[mpeg4 @ 03d9f020] MB pos/size: 0 000:38:29:0 22 dc: 83 84 83 84 - 129 108[mpeg4 @ 03d9f020] MB pos/size: 0 000:39:29:0 22 dc: 81 78 81 78 - 129 108[mpeg4 @ 03d9f020] MB pos/size: 0 000:40:29:0 22 dc: 68 65 68 65 - 129 109[mpeg4 @ 03d9f020] MB pos/size: 0 000:41:29:0 22 dc: 54 55 54 55 - 130 107[mpeg4 @ 03d9f020] MB pos/size: 0 000:42:29:0 22 dc: 58 73 58 73 - 131 104[mpeg4 @ 03d9f020] MB pos/size: 0 000:43:29:0 22 dc: 79 72 79 72 - 130 104[mpegts @ 0178f6a0] Continuity check failed for pid 1000 expected 10 got 9[mpegts @ 0178f6a0] Continuity check failed for pid 1000 expected 10 got 11[mpegts @ 0178f6a0] Continuity check failed for pid 1000 expected 6 got 12(20 such lines in total)[mpegts @ 0178f6a0] Continuity check failed for pid 1000 expected 5 got 6[mpeg4 @ 03d9f020] MB pos/size: 0 000:00:00:66 96 dc: -15 0 0 0 - -9 -9, MB_type: 12296, MV: 1 -1[mpeg4 @ 03d9f020] MB pos/size: 0 000:01:00:162 151 dc: 0 39 0 0 - 0 0, MB_type: 12296, MV: 0 0[mpeg4 @ 03d9f020] MB pos/size: 0 000:02:00:313 109 dc: 27 0 -9 0 - 0 0, MB_type: 12296, MV: 16 5[mpeg4 @ 03d9f020] MB pos/size: 0 000:03:00:422 159 dc: 0 33 -9 -21 - 0 0, MB_type: 12296, MV: -6 2[mpeg4 @ 03d9f020] MB pos/size: 0 000:04:00:581 239 dc: 123 116 111 112 - 144 122, MB_type: 1, MV: 0 0[mpeg4 @ 03d9f020] MB pos/size: 0 000:05:00:820 47 dc: 0 0 0 0 - 0 0, MB_type: 12296, MV: 22 9

You guyz are doing amazing job!From above gif I can tell that stage was landing with wild angle (tipped to right from viewer perspective) and doing very nice hoover slam.

I'm trying to find flipped bits.The problem I'm having is, it's almost almost impossible to tell if I've found a flipped bit. Just because the next few blocks or rows improve that doesn't mean you found a flipped one. It's very well possible you just found by luck a way to get the alignment of the next macroblocks right, but there's still a subtle propagating error.The advantage of finding a flipped bit is, you preserve the most information possible in the frame. But it's only possible in regions where the error rate is less then perhaps 1 flipped bit per 1000.

EDIT: I concur that this discussion should go in a parallel thread.

I found the missing p-frame between i-frames 150 and 169, it is just following the i-frame. Here the debug log at the end of the i-frame, the next frame is a p-frame but keeps the number 000 when it should have 001. Should we try to find the position in the ts file and hand-edit the "ghost" p-frame header?

Looking at the null packets, I noticed some patterns in the bit errors that might be useful for this kind of effort.Specifically, many (but not all) of the bit-flips come in groups of 3, with one bit flipped, then 13 unaffected, then the next two also flipped. This is most obvious in a hex-dump of the null packets, which are supposed to be filled with 0xff bytes: patterns such as "7f fc" or "fb ff e7" are common. But the same bit-flip patterns apply where errors overlay the TS headers, and presumably also when they occur in image data.Combinations (with XOR) of these 3-bit errors are also seen; mathematically, these error patterns are multiples of the polynomial x^15 + x + 1 over the binary field. This could be the result of FEC coding applied to a signal with too many errors for successful correction. I haven't figured it out completely but there are indications that the data may have been coded in 28-byte or 56-byte blocks (not respecting the TS packet boundaries); in particular, errors that don't fit this pattern seem to be found near offsets that are multiples of 28 bytes.There is still the problem of figuring out if the change helped or not, but this should help to reduce the search space. In low-error regions, there's a good chance that some of the flaws can be fixed with a single instance of this pattern.

Coming up on the 150,000 view mark for this thread/effort. Noting it as there's a small active team of people really working hard on this, but also a HUGE amount of people watching that work. Always nice to have a large appreciative audience, both inside and outside the space industry.You'd be astonished how many views are from IP addresses in a place called Hawthorne, California!

I updated http://spacexlanding.wikispaces.com/Frames+169-188

Quote from: Marams on 05/20/2014 09:25 AMI'm trying to find flipped bits.The problem I'm having is, it's almost almost impossible to tell if I've found a flipped bit. Just because the next few blocks or rows improve that doesn't mean you found a flipped one. It's very well possible you just found by luck a way to get the alignment of the next macroblocks right, but there's still a subtle propagating error.The advantage of finding a flipped bit is, you preserve the most information possible in the frame. But it's only possible in regions where the error rate is less then perhaps 1 flipped bit per 1000.Looking at the null packets, I noticed some patterns in the bit errors that might be useful for this kind of effort.Specifically, many (but not all) of the bit-flips come in groups of 3, with one bit flipped, then 13 unaffected, then the next two also flipped. This is most obvious in a hex-dump of the null packets, which are supposed to be filled with 0xff bytes: patterns such as "7f fc" or "fb ff e7" are common. But the same bit-flip patterns apply where errors overlay the TS headers, and presumably also when they occur in image data.Combinations (with XOR) of these 3-bit errors are also seen; mathematically, these error patterns are multiples of the polynomial x^15 + x + 1 over the binary field. This could be the result of FEC coding applied to a signal with too many errors for successful correction. I haven't figured it out completely but there are indications that the data may have been coded in 28-byte or 56-byte blocks (not respecting the TS packet boundaries); in particular, errors that don't fit this pattern seem to be found near offsets that are multiples of 28 bytes.There is still the problem of figuring out if the change helped or not, but this should help to reduce the search space. In low-error regions, there's a good chance that some of the flaws can be fixed with a single instance of this pattern.

There's a new Online editor up for doing p-frame stuff here: http://spacex2.slapbet.org/It should be fairly self explanatory for those doing p-frame work, it's also very new and is missing some stuff (like log / error info) which I will get to when I can.Have a play.

A 300% like-to-post ratio - is that record? I wonder how high it'll get when the Europeans sign on? I went through rawsplit-9 with Wireshark and bvi, and found a few dozen instances where Shanuson's fixing tools were too conservative to pound a particularly mangled transport stream header into conformance, as he had mentioned. I did this by hand in the attached file, and it now shows no protocol warnings in Wireshark.If someone more conversant with ffmpeg could take a look at it and compare it to rawsplit-9 to see what comes out the other end of it, I'd appreciate it. I'm curious if that might help the ghost pframe reported earlier, or if it will make any noticeable difference in the frames that come out or change any offsets. In any case, a clean transport stream is better than dirty, one would assume.If it's worth the effort, we can come up with a more aggressive transport stream header solver.

My tools are more or less my hands and a few python scripts. and yes, i was conservative or not concerned about that problem yet. Did you only fix some 4byte TS-headers? or more? And only for some part or the whole stream? If only for one part. could you apply those changes to the Part_X.ts file so we can more easily merge your and my changes together?