Author Topic: SpaceX Falcon 9 v1.1 CRS-3 Splashdown Video Repair Task Thread  (Read 964534 times)

Offline KEdward5

  • Full Member
  • ****
  • Posts: 809
  • Dallas, TX
  • Liked: 29
  • Likes Given: 107

He only gets the space of a tweet, why not just have him link to this thread or the wiki page?

If you go back one short page you can see it's clearly being dealt with.

http://forum.nasaspaceflight.com/index.php?topic=34597.msg1201779#msg1201779

And yes, this thread is much better, as it's a journey. Really is impressive.

Offline Req

  • Full Member
  • ****
  • Posts: 402
  • Liked: 413
  • Likes Given: 2580

He only gets the space of a tweet, why not just have him link to this thread or the wiki page?

If you go back one short page you can see it's clearly being dealt with.

http://forum.nasaspaceflight.com/index.php?topic=34597.msg1201779#msg1201779

And yes, this thread is much better, as it's a journey. Really is impressive.

In fact you missed several posts which are now gone that made it clear that there was still confusion, at least among several posters here.  But who cares, OT.
« Last Edit: 05/21/2014 08:06 PM by Req »

Online saliva_sweet

  • Full Member
  • ****
  • Posts: 573
  • Liked: 446
  • Likes Given: 1430
X:11807:80,X:11822:80 -> X:11807:80,11815:03
X:11505:80 -> X:11493:80,X:11501:03
Cool. I have already done the same for the first of the two flips. I'll investigate the second one. I have since added more flips, some using ajmartins triple flip pattern. I'm using a bit different convention than you, but it's equivalent. Here's my latest flip string for this frame:
X:4238:80,X:10711:80,X:11505:80,X:11807:80,X:11821:C0,X:11947:80,X:11961:C0,X:12158:80

edit: The second one looks like a tiny improvement too. Thanks.
« Last Edit: 05/21/2014 08:29 PM by saliva_sweet »

Offline arnezami

  • Full Member
  • **
  • Posts: 284
  • Liked: 262
  • Likes Given: 346
Elon asks who's leading the effort?

He only gets the space of a tweet, why not just have him link to this thread or the wiki page?

Also, having kept up with the thread since the beginning, I think that Arnezami would be on the short list as well.  In fact, if anybody is playing a leadership role, it's probably him.  At least in my opinion.

If Elon really is asking that question I think Chris (as an observer) can make the best assessment of that.

Offline Chris Bergin

Elon asks who's leading the effort?

He only gets the space of a tweet, why not just have him link to this thread or the wiki page?

Also, having kept up with the thread since the beginning, I think that Arnezami would be on the short list as well.  In fact, if anybody is playing a leadership role, it's probably him.  At least in my opinion.

If Elon really is asking that question I think Chris (as an observer) can make the best assessment of that.

It was already in work by the time I noted I was tweeting it out. :) That was supposed to be the surprise for the people doing the amazing work here....and still will be if what I was told will happen :)

Oh and a big welcome to morningdew76 and Req!
« Last Edit: 05/21/2014 08:29 PM by Chris Bergin »

Offline laika_fr

  • Full Member
  • *
  • Posts: 194
  • Liked: 79
  • Likes Given: 42
I'am totally useless at this but this hexa editing stuff, reminds me good ol' cheats for DOS games, keep up the good work guys, George Lucas would be proud.
a shrubbery on Mars

p-frame editor has various bugs fixed and now has log parsing / error showing info http://spacex2.slapbet.org/

Offline rsnellenberger

  • Amateur wood butcher
  • Full Member
  • ****
  • Posts: 595
  • Harbor Springs, Michigan
  • Liked: 71
  • Likes Given: 19
Speaking as yet another member of the admiring bleacher team, this is like watching a group of grandmaster-level jigsaw puzzlists work.  Starting with the i-frames as the edges, and now filling in the field with the p-frames...

It couples nicely with Tapatalk's tendency to display so many of the mmb edit strings as smileys X:X:X:

At some point, it would be nice to show the repaired video side-by-side with the raw.ts original...

Thanks!

Online saliva_sweet

  • Full Member
  • ****
  • Posts: 573
  • Liked: 446
  • Likes Given: 1430
Latest mmb for frame 72. Just some more sea macroblocks.

Offline SwissCheese

  • Full Member
  • *
  • Posts: 165
  • Liked: 250
  • Likes Given: 97
p-frame editor has various bugs fixed and now has log parsing / error showing info http://spacex2.slapbet.org/

Just to be clear, for people who want to help cleaning up the p-frames, you have to put 0:0:-1 to all previous frames to edit the p-frames. The field "MMB for Entire Segment" will look like this for example for frame 172: FRAME0:0:0:-1=FRAME1:0:0:-1=FRAME2:0:0:-1=FRAME3:6:20:-1,7:20:84967
« Last Edit: 05/21/2014 09:57 PM by SwissCheese »

p-frame editor has various bugs fixed and now has log parsing / error showing info http://spacex2.slapbet.org/

Just to be clear, for people who want to help cleaning up the p-frames, you have to put 0:0:-1 to all previous frames to edit the p-frames. The field "MMB for Entire Segment" will look like this for example for frame 172: FRAME0:0:0:-1=FRAME1:0:0:-1=FRAME2:0:0:-1=FRAME3:6:20:-1,7:20:84967

So that's how you work on a p-frame without the previous frames affecting it?

Offline arnezami

  • Full Member
  • **
  • Posts: 284
  • Liked: 262
  • Likes Given: 346
p-frame editor has various bugs fixed and now has log parsing / error showing info http://spacex2.slapbet.org/

Just to be clear, for people who want to help cleaning up the p-frames, you have to put 0:0:-1 to all previous frames to edit the p-frames. The field "MMB for Entire Segment" will look like this for example for frame 172: FRAME0:0:0:-1=FRAME1:0:0:-1=FRAME2:0:0:-1=FRAME3:6:20:-1,7:20:84967

So that's how you work on a p-frame without the previous frames affecting it?
Yes. It would be awesome if we had the option to temporarily "turn off" all previous frames this way in the online editor. When sending the command to the server/ffmpeg your would simply discard/ignore all mmb's from frames previous to the current one and put this series of FRAME0:0:-1=FRAME1:0:0:-1 etc before it.

You would still keep all the mmb's in the online editor, you just won't send it to the server during this "single P-frame mode". ;) And when somebody switches back you send all mmmb's again.
« Last Edit: 05/21/2014 10:19 PM by arnezami »

Offline morningdew76

  • Member
  • Posts: 11
  • McLean, VA
  • Liked: 12
  • Likes Given: 7
Latest mmb for frame 72. Just some more sea macroblocks.

Found another expansion- X:10711:80 -> X:10695:80,X:10709:C0 which appears to remove the weird coloring in 43:2 and 43:3

It also looks like 37:3:13414 can be safely backed up to 35:03:13276, for two more blocks of sea.

p-frame editor has various bugs fixed and now has log parsing / error showing info http://spacex2.slapbet.org/

Just to be clear, for people who want to help cleaning up the p-frames, you have to put 0:0:-1 to all previous frames to edit the p-frames. The field "MMB for Entire Segment" will look like this for example for frame 172: FRAME0:0:0:-1=FRAME1:0:0:-1=FRAME2:0:0:-1=FRAME3:6:20:-1,7:20:84967

So that's how you work on a p-frame without the previous frames affecting it?
Yes. It would be awesome if we had the option to temporarily "turn off" all previous frames this way in the online editor. When sending the command to the server/ffmpeg your would simply discard/ignore all mmb's from frames previous to the current one and put this series of FRAME0:0:-1=FRAME1:0:0:-1 etc before it.

You would still keep all the mmb's in the online editor, you just won't send it to the server during this "single P-frame mode". ;) And when somebody switches back you send all mmmb's again.

This is now implemented as "isolate frame" near "show errors"

FYI sometimes generating the log for the p-frame editor errors and doesn't return anything, not sure what it is yet but be aware.

Offline arnezami

  • Full Member
  • **
  • Posts: 284
  • Liked: 262
  • Likes Given: 346
This is now implemented as "isolate frame" near "show errors"

You keep amazing me. This is exactly what we wanted.

Now we can see the P-frames as is. The way they are shown on the wiki pages like here .

Amazing job!

Offline Shanuson

  • Full Member
  • **
  • Posts: 298
  • Liked: 208
  • Likes Given: 642
My idea is that people can pick on of the 15 and start fixing the TS-headers and more importantly the  P-Frame headers. I wrote how those should look like in the post quoted. First try to find the position of the P-Frame headers, they should be more or less equally distributed through the file. See if the counter bytes are messed up, calculate what they are, compare what they should be and change the header bytes if necessary.

The transport stream headers are mostly correct after the work to suss out the correct program IDs and sequence numbers, right? Are there any details we need to know about that? Also, if you have any other links regarding bitmaps for the packetized elementary stream headers and what have you, that'd be helpful. Thanks for all your work!


All 15 I-Frame headers are ok. with all the right values for the variable parts.
For the 4 Byte TS-Header, there are good parts and bad parts in the ts-file. But at least they are all aligned now i think. so every 188th byte starts a new TS-packet like it should be. That does not mean there is a "47"-byte at every 188th position. Sometimes there are huge gaps spanning 10-20 ts-packets with no correct ts-header.

See the picture below. It shows the first few ts-packets of rawsplit_part_8.ts. You can see where Ts-Packets should be, marked by the black line. Blue circles show the correct CC bytes, in Ts-Headers in green are not correct. there are some bytes wrong, due to biterrors. Red shows the I-Frame Header. Cyan marks the two variable parts of the header, which are increasing.

Here they are in dez:
0x329c2B -> 3251243
0x8d228d -> 9249421

The following P-header is: 47 43 E8 3E 07 10 00 31 A7 E6 7E 00 00 00 01 E0 00 00 81 80 07 21 01 8D 51 79 FF FF 00 00 01 B6 51
coloured are both variable parts: there value in dez is:
31A7E6 -> 3254246 -> 3003 more than before, this is correct, because 20*3003=60060, the difference between I-Frames

The second is more complicated I just found out: There is a no equal distance between all PTS values of the I-Frames, but there is a pattern: So i would suggest to decode all PTS values and try to find a pattern. Apply it to all those where the pattern brakes.

       PTS Hex   PTS Dez   Difference to the last value
IF1    59797D     5863805   
IF2    6123ED     6366189   502384
IF3    67CE5D     6803037   436848
IF4    6F78CD     7305421   502384
IF5    77233D     7807805   502384
IF6    7DCDAD     8244653   436848
IF7    85781D     8747037   502384
IF8    8D228D     9249421   502384
IF9    93CCFD     9686269   436848
IF10   9B776D    10188653   502384
IF11   A321DD    10691037   502384
IF12   A9CC4D    11127885   436848
IF13   B176BD    11630269   502384
IF14   B9212D    12132653   502384
IF15   BFCB9D    12569501   436848


I hope this helps some.

What I have written above about the PTS is wrong, I forgot that the PTS is 33bits spread over 5 bytes with some marker 1s in between.
Please ignore that, the PTS itself is increasing by 120120 between I-frames and 6006 between frames.

Attached you find a python-script I use the find P-Frame headers, there are some routines that de(en)code the PTS. Maybe that is usefull for someone else too.

Cheers,
Shanuson

Offline Shanuson

  • Full Member
  • **
  • Posts: 298
  • Liked: 208
  • Likes Given: 642
Finished part_1.ts so far that it is now producing 19 P-Frames.

Next i will start work in part_2

Cheers
Shanuson

Edit: Updated 1 File in that zip which I forgot to save before creating the zip.
« Last Edit: 05/22/2014 12:10 AM by Shanuson »

Offline Req

  • Full Member
  • ****
  • Posts: 402
  • Liked: 413
  • Likes Given: 2580
Elon asks who's leading the effort?

Eight Tips for Collaborative Leadership

Quote
Today’s corporation exists in an increasingly complex and ever-shifting ocean of change. As a result, leaders need to rely more than ever on the intelligence and resourcefulness of their staff. Collaboration is not a “nice to have” organizational philosophy. It is an essential ingredient for organizational survival and success.

That's what's been interesting about the effort - each individual has brought their individual talents, experience, knowledge, and skills to bear on different pieces of the problem to the extent that they are able to contribute, and everyone has been learning from everyone else.

Outsized kudos are due, of course, to the people who have set up the framework that allows this collaboration to occur in the first place - Michael Niedermyer for his ffmpeg patches to simplify the work on the frames, Iain Cole for his magnificent online editor, Moralec for his work in maintaining the Wiki page, etc. But are they "leaders" in the traditional sense of the word? They're fellow collaborators.

I think the effort and the fruit that it has borne so far is just a really excellent example of the power of "crowdsourcing" (I think someone will eventually have to add this video repair effort to the Wikipedia page for that term. :D), and a testament to how many people around the world are cheering for Elon and his entire team at SpaceX, and the vision of a multiplanetary human society.

I remember fondly my first Mars Society convention years ago in Boulder, and my work with the California Mars Society around 2000, and it's just magnificent to see how much closer we are to that goal today thanks to the efforts of not only Elon but also countless other people who were inspired - by Ray Bradbury, or Robert Zubrin, or Kim Stanley Robinson, or in any number of other ways - to each work in their own way and in their own capacity towards achieving it.

In a way, this video repair mission is just another example of that same phenomenon. It's great to see it unfold right before our eyes.

He only gets the space of a tweet, why not just have him link to this thread or the wiki page?

Also, having kept up with the thread since the beginning, I think that Arnezami would be on the short list as well.  In fact, if anybody is playing a leadership role, it's probably him.  At least in my opinion.

I read this over again, and I just want to point out that Shanuson, saliva_sweet, Untribium, ummmm, those are the ones that come to mind.  I made this post mainly because I don't want to leave anybody out.  There's probably more.  So, on to my main point:

I don't even like this approach.  I just wanted to point out that if I were to pick a leader, it would be Arnezami.  He's the one that got the people over here from reddit, he provided momentum in the most critical early stages, he's been refining, defining, and requesting paradigms for collaborative work, etc.  My original statement was that Elon should just link to the thread or the wiki page.  Let the masses who care enough read everybodies contribution and come to their own conclusions.

However, if Elon needs the name of a leader, it should obviously be Arnezami, in my opinion.
« Last Edit: 05/22/2014 12:46 AM by Req »

Offline Untribium

  • Member
  • Posts: 32
  • Switzerland
  • Liked: 32
  • Likes Given: 78
Okay, so I spent a not so little while on the p-frames following 52 and I got 1 (one!) frame fixed...and I don't even know if I did this right :))
It's really tedious, because there's almost no difference between frames 52 and 53, so there's almost no information for the lower part of the p-frame (which makes sense, but it makes it a lot more difficult to find markers...). In the lower part of the frame, you can just barely make out the shape of the rocket, but it took a while until I realized that :) I really wish the legs were extended at this point ;)
22:15:36982,30:15:-1,0:16:38358,25:17:-1,41:18:46765

Off-topic: Good luck to people who used to make a living out of repairing video files...I guess customers might no longer accept "a 50% chance of recovering one or two frames" as an answer after seeing the latest gifs from this video :))

Tags: