Quote from: wronkiew on 05/31/2014 07:25 PMYes, that line would have messed up the scraper, which is why I had not transitioned it yet. I think I will Can you just use a regexp to ignore any lines that don't match the expected format? Then it should be straightforward to add whatever sort of comment you like, so long as it's obvious isn't not a valid option.
Yes, that line would have messed up the scraper, which is why I had not transitioned it yet. I think I will
Quote from: moralec on 05/31/2014 04:25 PMI doesn't seem like rocket science, but it does seem like a lot of work specially on the online tool side (we don't want you to hate us Iaincole). Ideas?VCSes - version control systems - are really good at moralec's database problem. One idea is to shoehorn this into a VCS by using a VCS as storage. Here's one way to do that: force MMB lists into a VCS friendly format, by a) breaking up the MMBs into one per line [VCSes tend to work best on whole lines at a time] andb) sort the MMBs into a standard sort order, most likely the raster order most folks are already used to. [EDIT: I guess this isn't strictly necessary, but may make IainCole's life easier]c) Then you need to always check in and out of the VCS via something that understands the format. IainCole's website, or some other website, for example, would be the only entity with access to the VCS. This website would be responsible for parsing raw MMBs, and providing the MMBs back in a ffmpeg-friendly format.d) Ideally this file format would have comments to allow for notes and simple alternatives. The VCS would be responsible for full 'branches' [git/mercurial is good for this].And then a flat file format might come in handy in other ways.There are other ways: you could use raw RCS diffs to manage things; you could find a diff program that's really good at big long lines (most aren't); you can code up the delta checking by hand; you can simply store all the MMBs ever in groups, normalized in the database, for users to sort out; etc.
I doesn't seem like rocket science, but it does seem like a lot of work specially on the online tool side (we don't want you to hate us Iaincole). Ideas?
VCSes - version control systems - are really good at moralec's database problem. One idea is to shoehorn this into a VCS by using a VCS as storage.
Hi guys,I've been busy trying to let the decoder output the bits and their interpretation of all the (macro)block parts. And I think I have succeeded. I still need to clean it up and make it compatible/configurable with the existing codebase. But I wanted to share some details so others (mainly @IainCole) can already start thinking/working on an extention of the online tool. Also for others to get used to the idea.I think it will bring us to the next level.
Thanks mainly to Shanuson and Princess, we now have the TS packets all fixed up
Hello, I'm not from around here... I've been working on restoring frames though.
Would it be possible to add a slider to enhance the image contrast? It helps a lot for repairing the p-frames but I have to use an image viewer to do this currently. Thank you again for your great work!
In the mean time, I've seen a few posts by people saying that they have no idea what's going on. That's usually my position (especially with ISS stuff, acronym soup alert!), but now I'm coming from the other side, so I'll make an attempt at an explanation of all the stuff that's going on here. Hopefully, that will allow more people to help fix things. I'll probably make a mistake here and there and/or oversimplify something, so if you spot an error, let me know and I'll fix the text.<snip>
Quote from: Lourens on 05/31/2014 10:02 PMThanks mainly to Shanuson and Princess, we now have the TS packets all fixed upThank you for that Lourens! It's great to get a mention like that, it makes the effort all worthwhile.
Hello, I'm not from around here... I've been working on restoring frames though. I frames 6, 7, 8, and 12(though mhenderson has already mostly restored 12) all have new data in them that invalidates the old MMBs for them. If someone could find out for each of these frames1) where new data starts2) how many bits are in the new piece of dataWe can apply those offsets to the old MMBs, and they'll look as good as they did before. I can do this, I built a script to test the concept and it works really well, I just need the offsets, as there are multiple places data is added in the frames and finding them manually is a PITA. After that's done, we can improve the frames in the areas where there's new data.
Clock demo! Frames 101-120, running at half speed.
Quote from: SwissCheese on 05/30/2014 09:44 PMWould it be possible to add a slider to enhance the image contrast? It helps a lot for repairing the p-frames but I have to use an image viewer to do this currently. Thank you again for your great work!I came up with a very crude way to do this in the editor this morning. Flip bits in the VOP quant to make it intentionally the wrong value, like what was wrong with frame 239. Flipping the 1st/2nd/3rd bit at X:59 generally gets the desired high contrast. It definitely makes lining up those very subtle movement vectors much easier, just remember to get rid of the flip when you're done.
Thanks for the welcome guys. I've been playing a bit more with the concept of shifting the mmb corrections for the new ts files so we don't have to completely redo the i frames. I haven't been able to find a good way to FIND what offsets are needed, but they sure do clean up the file in a hurry. This is a proof of concept of frame 101, using the previous MMB sequence. I offset all MMB corrections past 27000 by 1824, and past 53631 by 2480, with the top half blanked out because I haven't found the correct offset(s) for it. Almost as good as it was before.Also attached is the python script I'm using to apply offsets. Not attached are my fruitless attempts to automate finding the values for those offsets. The best manual way to find them is to look at the new, raw iframe, find a recognizable feature on it, note the position in the bitstream, look at the old, corrected iframe, find the same feature, subtract the two locations, apply that offset to the MMB using the script, and then see how the modified MMB looks on the new frame.
Yes, that's the way to do it quickly, but we should also check the new data that are causing these offsets, part of it might be good.
Yes, we should definitely be looking at the new data once the old work is all aligned, the offsets will show where the new data is.There are just over 100 changes to frame 101, redoing all that work manually for is not my idea of a good time!
Quote from: Quialiss on 06/01/2014 10:06 AMYes, we should definitely be looking at the new data once the old work is all aligned, the offsets will show where the new data is.There are just over 100 changes to frame 101, redoing all that work manually for is not my idea of a good time!Quialiss, I could come up with a program that could list the MPEG4 decoding offset for each data packet in the TS files, then we could go through and compare the differences between the old and new TS files. It would be a long list, there are 17103 data packets in the current TS file! However, it would give you a definitive list of MPEG4 changes for all the I- and P- frames.Would this be useful for you?