The last MMB change was 33 hours ago. /snip/
Hmmm. Does that mean we are
The aim is that people can now compare raw_aligned.ts with rocket.ts, and any differences are bitflip errors. Please take into account the insertions done in raw_aligned.ts if this affects the error pattern, but as the error block size appears to be 56 bytes anyway then it should hopefully not matter.
Just checking my understanding -- the MMB fixes are *not* included in rocket.ts, right? So the bitflip errors found when comparing rocket.ts with raw_aligned.ts are just the transport stream errors.
What I'd like to do is to take our "best attempt at a fix" for a specific iframe, convert it back into the .ts format, and then start looking at thost bitflips, to see if we've inadvertently "rediscovered" some triple flip patterns. If we have, this would make me more confident in the "most errors are triple flips" hypothesis.
As far as I know there are a lot of MMB fixes that don't map directly back to the TS file, they're playing tricks in ffmpeg - for example, I think that redirecting a block to inherit from a different one isn't possible in MPEG4.Also, the MMB fixes are done visually and not at the bitstream level. While the resulting frames look very good to the human eye, they're not what the rocket transmitted and can't be used for error detection purposes.
Amazing work princess!! Ok so how do you think we should proceed?Should we go ahead and subsitute raw_final_fixed.ts with one of your newer versions? If thatīs the case, do you think the existing MMB codes will still work?
Quote from: moralec on 06/09/2014 08:42 pmOk so how do you think we should proceed?Should we go ahead and subsitute raw_final_fixed.ts with one of your newer versions? If thatīs the case, do you think the existing MMB codes will still work?Why thank you moralec! No, I don't think we need to replace the TS file that people are using for the video recovery. [...] In summary, the rocket.ts file is mainly for error detection but also might be of academic interest, or might be useful for people to attempt a more thorough recovery of difficult frames by working from the raw MPEG4 stream using ffmpeg.
Ok so how do you think we should proceed?Should we go ahead and subsitute raw_final_fixed.ts with one of your newer versions? If thatīs the case, do you think the existing MMB codes will still work?
I will presumably have to figure out how to translate MMBs from raw_final_fixed.ts to rocket.ts in order to pursue the triple-flip research I want to do, so if it comes to it I might be able to help translate stuff. But that's a ways in the future yet.
Quote from: wronkiew on 06/09/2014 07:25 pmThe last MMB change was 33 hours ago. /snip/Hmmm. Does that mean we are c) Too busy with our lives?
Quote from: moralec on 06/09/2014 08:42 pmAmazing work princess!! Ok so how do you think we should proceed?Should we go ahead and subsitute raw_final_fixed.ts with one of your newer versions? If thatīs the case, do you think the existing MMB codes will still work?Why thank you moralec! No, I don't think we need to replace the TS file that people are using for the video recovery. It's good enough and the new file only makes a few data-related changes - mainly the extra P-frames at the start (which have no I-frame to reference to), and the duplicate run in P-frames 270/271.It looks like this duplicate area was put in because the start of frame 270 is just atrociously corrupted, so in the rocket.ts file I actually don't have the start of P-frame 270 as I honestly can't decide where to put it. I'd advise the MMB people that 270 might not start in the right place, and 269 might have junk at the end. It seems as though the duplicate bytes were inserted in order to use some of 271's data to shore up 270 in raw_final_fixed.ts, so this area of the video might pause slightly.In summary, the rocket.ts file is mainly for error detection but also might be of academic interest, or might be useful for people to attempt a more thorough recovery of difficult frames by working from the raw MPEG4 stream using ffmpeg.
Speaking of academic interest, I think we may want to start posting all these files in the wiki, and work on a detailed description of what we did. We don't need to do it now, be it may be convenient to do it sooner than later.... what do you think about this? Should I open a page just for this purpose? Or do you prefer to leave this for the end of the process?
It looks like this duplicate area was put in because the start of frame 270 is just atrociously corrupted, so in the rocket.ts file I actually don't have the start of P-frame 270 as I honestly can't decide where to put it. I'd advise the MMB people that 270 might not start in the right place, and 269 might have junk at the end. It seems as though the duplicate bytes were inserted in order to use some of 271's data to shore up 270 in raw_final_fixed.ts, so this area of the video might pause slightly.
IIRC, SwissCheese and I worked on those frames.. from our notes on the wiki, 269 and 270 were completely unrecoverable in fix8 version, and some data became recoverable in the raw_final_fixed.ts, though they're still missing large chunks of data. But they don't appear to pause(outside of what's caused by missing a third of a frame) and the timestamps all look correctly recovered.
Lastly, maybe I'm strange but I like to think that recovering rocket.ts is getting us closer to knowing the "last words" of a dying rocket stage. It did its job perfectly, finally achieving what Elon and his team had hoped for it, and was desperate to tell the world, frantically transmitting the news of its success before bravely succumbing to the stormy seas. I may be anthropomorphising things too much, but I think we owe it to that tenacious machine to reconstruct its last historic message to the world.
Quote from: Quialiss on 06/09/2014 10:41 pmIIRC, SwissCheese and I worked on those frames.. from our notes on the wiki, 269 and 270 were completely unrecoverable in fix8 version, and some data became recoverable in the raw_final_fixed.ts, though they're still missing large chunks of data. But they don't appear to pause(outside of what's caused by missing a third of a frame) and the timestamps all look correctly recovered.The TS timestamps for 270 were altered to make the frame look correct, but I don't know what happened with the MPEG4 stream.Basically at that point in the data the stream was all fine with no errors, and then suddenly from around offset 0x003dbb04 to offset 0x003dd284 there was a huge burst of corruption that knocked out 32 TS packets. If you look at it with a TS analyser it's very bad, just totally mangled.Anyway, the boundary between 269 and 270 occurred in the middle of this. Usually the corruption is light enough that you can kind of eyeball the closest packet, or break out the Hamming-distance analysis, but in this case it's useless. Whoever did the byte duplicate in raw_final_fixed.ts was basically just pasting the start of 271 into this and resetting the timestamp to be correct. It's the TS equivalent of saying "this macroblock is shot to pieces, use one that looks similar".So yes, this will allow extra data to be recovered - the good macroblocks in 270 after the corruption burst finished. But be aware that the end of 269 and the start of 270 are mangled beyond recovery.