Recovering lost footage from the ACT20 action camera

July 26, 2014

The ACT20 action camera is a nice and cheap chinese camera. I own one and use it to record my bicycle journeys. However the software included into the camera is a bit unstable, and it happens often that the recording stops, leaving only a TMP file behind, containing the raw data only, unplayable by any media player. I needed the footage badly, so I found a way to recover the video. I took an intact AVI file from the camera and loaded it into wxHexEditor. After some reading about the structure of an AVI file, I found that I could transfer the header onto the TMP file resulting in a broken but maybe still usable AVI file. So I searched for the “movi” tag in the AVI file and remembered the position of the next plus four bytes after the tag:

0x00002280: 00 00 00 00 00 00 00 00 00 00 4c 49 53 54 7c 39 ..........LIST|9 
0x00002290: 28 33 6d 6f 76 69 30 31 77 62 a1 01 00 00 ff fb (3movi01wb......

In my case the result was 8858. So I extracted the header from the AVI file, and concatenated it with the TMP file:

head -c8858 140624-090652.AVI > headerdata
cat headerdata 1F203D26.TMP > 1F203D26.TMP.avi

The resulting AVI file was playable by MPV only, Mplayer or VLC was unable to cope with it. Obviously all length and index data was wrong in the header. I searched for “recover broken avi” and found this:

I applied the mencoder command to my broken AVI file:

mencoder -forceidx -oac copy -ovc copy 1F203D26.TMP.avi -o repaired.avi

That’s it!




Backup with obnam and encfs

April 24, 2014

I fell in love with obnam, a very useful and fast backup program. However, when I encrypt my backup, obnam gets painfully slow, because it simply calls gpg for each chunk in the repository. Since I upload my backup repository to an untrusted server, encryption is necessary for me. Here is where encfs comes to the rescue. Encfs has a very useful –reverse option which enables me to have a container where I can see my files as if they were encrypted. I don’t need to encrypt my files locally since the whole hard disk is encrypted.

Here is my backup scenario:

First I back up my picture files to /Backup which is mounted on another harddisk:

obnam backup --repository="/Backup/obnam_Pictures/" --exclude-from="/Backup/obnam_Pictures_exclude" "/Common/Pictures/"

I can mount the repository to a directory to see all generations and files:

obnam mount --to="/Backup/obnam_mount/" --repository="/Backup/obnam_Pictures/"

Encrypting the repository:

echo mylongpassword | encfs --stdinpass --public --reverse "/Backup/obnam_Pictures" "/Backup/.obnam_Pictures_encfs"

Now the directory .obnam_Pictures_encfs shows the files in an encrypted form. Lastly I mirror this directory up to my server using lftp:

mirror --continue --reverse --delete --verbose=3 /Backup/.obnam_Pictures_encfs /MyBackup/Pictures_encfs

Make sure the directory is mounted, otherwise the –delete option happily deletes everything on the server.

Note that encfs encrypts the files one by one, so only new chunks will be uploaded next time. This way I can make sure that not only obnam backs up incrementally but also the upload is incremental.

Whitelist filter with transparent Tinyproxy and Firehol

December 10, 2010

I’m a dad with 5 small children. Letting them onto the internet without any restrictions would be as irresponsible as not teaching them how to use the internet. So I decided to filter the traffic. First I tried OpenDNS’ FamilyShield. That works great with english sites, but it does let through hungarian porn sites for example. Hence I decided, a whitelist would be a more suitable solution. I looked for some applications, the most obvious choice was Dansguardian. Dansguardian needs a proxy, Squid is the recommended one, but many bloggers recommend tinyproxy as a more light-weight alternative. Looking at Tinyproxy, I soon realized, that it supports filtering even as whitelist. No need for a big gun, Tinyproxy should suffice, I thought. And I was right. But first… and now comes the story, and through this you will see why I love open source.

In Archlinux, installing Tinyproxy takes this:

pacman -S tinyproxy

Here is what I have changed in the /etc/tinyproxy/tinyproxy.conf file:

FilterURLs On
FilterDefaultDeny Yes
Filter "/etc/tinyproxy/whitelist"

For the meaning of those settings please refer to the very file, comments are extensive.

So, I logged into my son’s account and set Firefox’ proxy to localhost:8888. Now nothing but the entries in the whitelist were shown, everyting else was filtered. Nice. There is another browser, Konqueror, which needed to be proxied as well, and we were set. Wait… really? I found some problems with this setup. First, my son can change the proxy settings anytime he wants, and second, I have to do the same procedure for all my kids accounts. Well, the first case can be solved by locking the proxy settings in Firefox and in KDE4 with the awesome Kiosk Admin Tool. The second case just requires some handywork. And, what if I dont know the passwords?

Spoiled by Archlinux as I am (ie. striving for perfection), I decided to look further. I already heard about transparent proxies, so I looked into the README of Tinyproxy, and quickly learned that it does support transparent proxy mode, all it takes is to compile it with the “–enable-transparent-proxy” switch. I checked the Archlinux package, well, it was compiled without that switch. No problem, this is why we have the wonderful ABS system. Using Bauerbill, recompiling a package is easy:

bauerbill -S --abs tinyproxy

I added the switch in the PKGBUILD and couple of seconds later I had my brand new tinyproxy package installed. Next thing was to setup the firewall. I’m using Firehol, and there is a wonderful command to reroute traffic to the proxy:

transparent_proxy "80 443" 8888 "nobody root bin my wife's acc and mine"

That routes all requests to http and https ports to the proxy, except for those users listed. To my surprise, Tinyproxy answered with a page that said “Unknown method or unsupported protocol”, as if Tinyproxy were not compiled with transparent support. I went to the IRC channel, where I lingered for quite some days until I got an answer. There was a guy who just got into Tinyproxy two days before. Long story short, he pointed out that the compile switch should be


I searched google, looked into the README, and every reference I could get my hands on stated the opposite. But that guy looked into the source code and that is what counts, no questions asked (later I asked one of the devs about this, and he admitted that this switch was changed in 2004, but no docs were updated, lol). So, I quickly recompiled Tinyproxy, and lo and behold … well … it behaved differently. Now it stated that the page was filtered, even for urls that definitely were included into the whitelist. Simply every page got filtered. I rechecked the situation by simply setting the proxy again in Firefox, and this time it worked correctly. So Tinyproxy behaved differently when working as transparent proxy. What can a poor man do if he cannot code? File a bug report of course🙂 But as an experienced bug reporter, I first checked if someone else had already reported it. Sure enough there was already a bug report several months old. The last comment was from the developer, saying “I’ll look into it”. But there was another very important piece: a patch! The user who reported the bug was apparently a programmer and provided a patch with the bugreport as well. Whohoo, I immediately applied the patch to the source, and yes, you guessed it, Tinyproxy finally worked flawlessly in transparent mode.

Of course I reported my success to the devs, and the next day the patch was included into the official git repository, ready to be compiled. The README is corrected as well🙂

I have set up an AUR package named tinyproxy-git for anyone who wants to try it. Just use Bauerbill:

bauerbill --aur -S tinyproxy-git

The morals of the story? The developers often just need some people who are interested in using their product and are willing to test, try and report issues, and most importantly, thank them for their work and encourage them to continue. I saw this happening many times, recently with the fish shell for example. I you like a particular product, get involved in any way you can, be an optimist, and never take it for granted.

Rescuing a CD in Archlinux

February 20, 2010

I have a CD with our favorite movie on it: Bud Spencer – Soldier of Fortune. I promised to my son to watch it together with him. Unfortunately the CD was faulty, KDE didn’t recognise it. My son was so sad, I just had to do something about it.

tail -f /var/log/kernel.log

revealed “Buffer I/O error on device sr0, logical block 326084”. After some failed attempts and reading some ‘man mount‘ I found out how to mount the CD:

mount -r -t iso9660 -o nofail,ro /dev/sr0 /media/cd/

After this, ls /media/cd/ did show the file on the CD. Time to use the almighty dd utility. I found this blog entry to be very helpful:

(On a side note, it seems the author of this blog uses GoboLinux, well done!)

So the command to rescue my CD was:

dd if=/media/cd/Zsoldoskatona.avi of=/tmp/Zsoldoskatona.avi bs=512 conv=noerror,sync

And we were able to watch the movie!