When MFA is not enough

I hope you’ll excuse the format of this post. Coffee does strange things to my brain.

In the beginning, there was the username. And it was deemed good enough, because everybody on the machine was a trusted colleague. Then came machines shared by strangers, and with them the password. And it was deemed good enough, because the resources to crack it were beyond most. Then came the Internet, and the world flourished and grew. But the world contains more villains than the password, in its myriad implementations, could withstand. And thus came multi-factor authentication. And it was deemed good enough, because only someone in possession of the authenticated device could gain access.

But they were all deceived, because the service providers foretold that their users would lose their authenticated device, and lock themselves out of their accounts, and would blame them rather than their own misfortune. And so it was that, fearing their users’ wroth, the service providers gave their support personnel unfettered access to override MFA at the behest of a phone call.

Some service providers were wiser, and gave out backup codes after authenticating, thus allowing users to regain access by themselves, and promised not to give access to anybody with a honeyed voice, and reminded their users ever and anon that they should be mindful of their backup codes, and even gave them the option to be told when their account was logged into. And their users were at peace, trusting that their service providers knew what the fuck they were doing, that their files were safe, and being forever loyal to them.

Save your bookmarks on Pinboard!

Why? Check this out:

  • Simplicity
    • List interface:2016-02-24-145849_1366x768_scrot
    • Bookmarking interface:2016-02-24-140756_1366x768_scrot
    • No ads
  • Price
  • Safety
    • Rock solid since I started using it in 2012
    • Backups in multiple portable formats
  • Privacy & security
  • Access via basically any means: browser extensions, JavaScript widgets, apps and API
  • Extensive FAQ
  • Likely to survive for long because of a simple, sustainable business model

Other than being a happy customer I am in no way affiliated with Pinboard.

Delicious data loss

After many years, it looks like Delicious will never have support for excluding tags from a search, so when I saw that they had added a “Remove Tag” functionality I finally decided to get rid of all the “toread” tags from posts already tagged with “read”. This didn’t quite turn out as expected:

From: Victor Engmark
To: feedback@delicious.com
Content-Type: text/plain; charset=UTF-8

Dear Delicious Support,

For ease of searching I decided today to remove the “toread” tag from all bookmarks I have already read – That is, I went to the toread+read tags page <https://www.delicious.com/engmark/toread+read>, selected all, clicked “Remove Tag”, carefully clicked only on “toread”, and clicked Save. This I repeated for each page until there was only a single page left. During this process I discovered several bugs, here listed according to increasing severity:

When selecting a bookmark the links above (except “Rename Tag” since it’s sneaky and doesn’t actually just rename on the selected items) are “activated”. That is, they turn blue. After clicking “Save” the page would refresh, all the checkboxes would still be ticked, but the links would still be gray. Expected behavior: When bookmarks are selected (before or after an action) the links should be colored. In the case of some bookmarks it was not possible to remove the “toread” tag. I repeatedly tried, and every time the same bookmarks showed up on the page after the refresh. Expected behavior: It should be possible to remove tags from any bookmark. After selecting the bookmarks mentioned in point 2 and clicking “Remove Tag”, the “Remove Tags” dialog only displayed a single tag: “toread”. Expected behavior: This should behave like for other bookmarks, that is, show all the tags for all the bookmarks. When trying to edit individial items from the bookmarks mentioned in point 2, the “Edit Link Info” dialog doesn’t display the same tags as the toread+read tags page. For example the bookmark “Ryan let’s assume that you blend fiction and reality only on special occasions” <http://www.qwantz.com/index.php?comic=1992> has the tags “toread”, “read”, “comic” and “funny” according to the toread+read tags page, but only “RyanNorth” in the “Edit Link Info” dialog. Expected behavior: The “Edit Link Info” dialog should display the same title, URL and tags as the bookmarks list.
Somehow, instead of removing the “toread” tag Delicious removed the “read” tag! Expected behavior: Remove the tag I requested to remove, and only that.

I’ve now lost seven years worth of records of what I’ve read online! Please help me fix the state of my bookmarks. In case you can’t obtain a diff between the bookmarks of yesterday and today I can provide you with one for reference (I do daily XML backups in case of just such a situation). This procedure should be nothing more than a reversal of the tag removals done in the last 48 hours or so.

Posted to my blog
<https://github.com/l0b0/paperless/wiki/Delicious-data-loss> for

Sincerely, a long-time user.

Long story short, I’d misunderstood how the feature works: Instead of removing those tags which you leave in the tag cloud, it removes all tags which you remove from the tag cloud! I had thought it was a “remove these tags” feature, but actually it was a “keep only these tags” feature.

Restoring from the XML files was out of question (they only support HTML import now), and they were unable to provide a reference for what such an HTML file should look like and whether tags would be replaced completely (messing up the changes I did since this started) or just added to the existing ones. I guess a migration to Pinboard is finally in order.

Edit: Here’s how I ended up migrating to Pinboard, fixing the bookmarks in the process:

  1. Export delicious.html and create a backup of the file.
  2. Get the URLs and tags of the bookmarks in the XML file which used to have a “toread” tag but have changed since the fuck-up:
    git diff bookmarks.xml | \
    grep '^-.*[" ]toread[" ]' | \
    grep -Eo 'href="[^"]+".*tag="[^"]+"' | \
    sed -e 's/href="\([^"]\+\)".* tag="\([^"]\+\)"/\1 \2/' > url_tags.txt
  3. Make the URLs and tags sed-compatible:
    sed -i -e 's/[\/&]/\\&/g' url_tags.txt
  4. Replace spaces between tags with commas (otherwise “foo bar” is imported as “foo_bar”):
    while grep -q ' .* ' url_tags.txt
        sed -i -e 's/\( [^ ]*\) /\1,/g' url_tags.txt
  5. Loop over each of the URL tag pairs, and create a sed script from it:
    while IFS=' ' read -r -u 9 url tags
        echo 's/\(HREF="'"$url"'".* TAGS="\)[^"]*"/\1'"$tags"'"/;' >> replace.sed
    done 9< url_tags.txt
  6. Replace the tags:
    sed -i -f replace.sed delicious.html
  7. Verify the difference between the HTML backup and the changed file:
    meld delicious.html{.backup,}&
  8. Import in Pinboard

Creating time-lapse videos

This guide shows you how to create high-definition time-lapse videos from series of still pictures, tailored towards Flickr, and using only free and open source tools available on most operating systems (although I’m not sure about Windows). If you’re just here for the meat, or you’re already photographing in 1280×720, you can go straight to the code or example.


High resolution is not everything, but it’s a useful substitute for a good zoom lens if you’re going to show the final result in a lower resolution than your camera produces. This is very common, and will probably continue to be common for a while. CCDs quickly surpassed desktop screens in resolution (and are likely to continue that way), and showing a full-screen, full-resolution video without stuttering is more than many desktop computers are able to do today.

Flickr staff recommend 640×480 (SD) or 1280×720 (HD) resolution with 25/30 FPS. Anything else I presume will be resized and slowed/speeded up. Since most digital cameras nowadays have much higher resolutions than these, I’ll base the instructions on 1280×720, 30 FPS, but it should be easy to adapt them if you want.

If you want to have as much surface area as possible for your video, you should crop it to fit the aspect ratio (16:9 for 1280×720). This is easy in GIMP:

  1. Open the image
  2. Select the crop tool
  3. Set fixed aspect ratio 16:9
  4. Move and resize the crop field until it fits your purpose. This is where you have the chance to get rid of that random stuff in the corners that you don’t want to leave in the finished movie.
  5. Note the crop position (e.g., x=0, y=495) and size (e.g., 4752×2673) in pixels for later.


Your images are probably still larger than 1280×720 pixels, so we’ll need to resize them as well. We’ll force the aspect ratio (! below) to avoid any rounding errors which might occur since our image size is most likely not a multiple of 1280×720 pixels.

Do it!

Now make sure you have a backup! Then replace the crop values with your own, using the geometry <width>x<height>+<x position>+<y position>:

mogrify -monitor -crop 4752x2673+0+495 -resize 1280x720! /path/to/*.jpg
mencoder -nosound mf:///path/to/*.jpg -mf w=1280:h=720:type=jpg:fps=30 -ovc lavc -lavcopts vcodec=mpeg4:vbitrate=9000000:mbd=2:keyint=132:v4mv -o time-lapse.avi


Delicious bookmarks search in the shell

fil.tero.us is no more, long live filterous! After much deliberation (and switching to a more productive platform & programming language), I decided to re-implement the Delicious personal bookmarks search page in Python for the *nix shell. The result is more security (I don’t store your bookmarks for you), more uptime, and more opportunity for parsing with shell tools such as less, wc, sort and grep.

  1. Install / upgrade: sudo apt-get install libxslt1-dev && sudo easy_install -U filterous
  2. Download your bookmarks
  3. Usage: filterous --help

Scientific American sells customers’ addresses!

I dislike paperwork, despise advertising, hate spam, and loathe companies trading addresses without my consent, which is why I’m seriously pissed off after finding a “special” offer piece of snail mail from New Scientist in my mail box this evening.

The address on the spam was wrong, in two ways: It contained the residence name where the street name should be (a mistake I commonly made after moving here), and it was missing the last letter of the residence name. Both of these errors appeared on the letter from New Scientist, and in an email from Scientific American regarding a missing issue. There’s no doubt: Scientific American is selling their customers’ addresses!

I got a subscription which is not yet expired, but I’m not interested in continuing a business relationship with such an immoral company. I’m going to ask them to terminate the subscription and send me a check for the rest of the issues, and then I never want to hear from them again.

Update: Here’s the body of the reply from Scientific American:

We are in receipt of your complaint of July 23, 2007, regarding the sale of your name to New Scientist for promotional mailings. On all of our subscription order forms, we do make mention of the fact that we may share your name with third parties, and offer you the option of “opting-out” and denying that your name be disclosed. According to our records, you did not make that selection when you originally ordered your subscription.

As per your request, we are canceling your subscription and mailing you a refund check for all issues not mailed. Please accept our apologies for the confusion.

I always opt out on these things, but there’s no way to prove that three years after the fact. They don’t say whether the option is on the form, just that it’s mentioned & offered. I’m disappointed that this isn’t more of an issue.