August bug

Remembering a classic “time travel” bug reminded me of a fun time in Bash. How to reproduce in four handy steps:

  1. Split a date string by hyphens into $year, $month and $day.
  2. Use $month in an arithmetic context. I don’t remember the exact code, but it was probably something trivial like [[ "$month" -eq 1 ]].
  3. Wait until August, when the script blows up with
    [[: 08: value too great for base (error token is "08")
  4. Learn about octal in Bash, smack forehead, and curse language designers who thought it was a good idea to make numbers ambiguous in order to save one character instead of using something reasonable like a 0o prefix to fit with the already ubiquitous 0x and 0b.

At no point did anyone profit during this exchange.

As an aside, this is the sort of thing that only exhaustive testing will catch, because “01” through “07” are the same value whether decoded as octal or decimal, and “10” through “12” are decoded as decimal. Only “08” and “09” are actually problematic values.

Shell scripting dos and don’ts

Shell scripting is like a room full of power tools: handy but dangerous.


  1. Build complex systems. There are just too many ways that external state can affect any piece of shell code. Do you know what your script will do if you change IFS before running it?  What if you give it a file name starting with a dash or containing a newline? How do you recover the state of the system if the script crashed somewhere in the middle? Complex shell script environments invariably end up looking like Rube Goldberg machines of chainsaws and power drills. Use languages and frameworks appropriate for the task.
  2. Expose them to the Internet. Safe input handling is just too damn hard. Unless you’re GreyCat or Stéphane Chazelas.
  3. Use eval. Don’t be evil. There are safer ways to do whatever you’re trying to do.
  4. Write portable code. (By this I mean code which works in multiple shells without change, as opposed to code which can easily be ported to other shells.) Writing portable code means limiting what language features you use and adding complexity to make sure it works the same way in all the supported shells. Because of this, the end result will be more complex and less flexible than the simplest script that supports the shell you have.
  5. Minimise the number of characters. The next maintainer will hate you.
  6. Create interactive menus. Very few shell tools like less and top only make sense interactively. Use command-line arguments instead, so that your tool will be useable both standalone and with other tools.


  1. Test everything automatically. This gives you and others the confidence that your script actually works. Bonus: Allows you to modify your code without having to test everything manually. Extra bonus: Experiencing how difficult it is to test shell scripts exhaustively will convince you to never use them for anything complex.
  2. Provide --long-names for every -s -h -o -r -t option. And if you can bear the screams of dogmatic developers, don’t support short options at all. As long as the names make sense this allows people to write readable scripts. Bonus: No wondering whether -n5f0 is two, three or four options.
  3. Use guard statements like the POSIX set -o errexit -o noclobber -o nounset. While there are some caveats to how these work, they can save a whole lot of headache. Bonus: Use -o xtrace to see what the script does in detail.
  4. Add an auto-complete script. The users will be grateful. Bonus: Gives an incentive to keep the structure of your options sane.

Tag cloud shell script

As an interesting challenge, I wanted to output a tag cloud (aka. word cloud) for a text file using standard shell tools. The result is surprisingly fast (2 minutes to create the tag cloud for War and Peace), and surprisingly short: As you can see, less than 10 lines doing anything more complex than echo. The latest version is much more flexible, but the main work is still just some 20 lines (tr -s … and below), and it’s still fast.

If you do anything more fancy with this, I’d be interested to know about it. I’ve got a couple ideas, but I’m not sure if I’ll ever get around to them:

  • Exclude words from another file
  • Multiple word tags from another file

Example usage: < foo.txt > foo.xhtml

Update: The code is now on GitHub. Fork away!

BASH / Bourne Shell which outputs its own documentation

I love Python docstrings – It’s a great application of the DRY principle. The excellent Advanced Bash-Scripting Guide has an example which does something similar, but it embeds the documentation in a variable in the middle of the code. This makes it less readable and findable than ordinary comments, and leads to duplication if you want to have the same documentation as a comment for other developers. Here’s a very simple solution, which I’m sure can be improved to fit your documentation style. It simply prints all non-empty lines from the start of the file:

    # Print documentation until the first empty line
    while read line
        if [ ! "$line" ]
            exit 64
        echo "$line"
    done < $0

A bit of detail: The exit code is from /usr/include/sysexits.h on Linux, which seems to be the standard, and < $0 passes the contents of the current file to the while loop.

BASH prompt galore

Here’s the latest result of trying to make a BASH prompt which might be useful if you use chroot, ssh, su or git:

# set variable identifying the chroot you work in (used in the prompt below)
if [ -z "$debian_chroot" ] && [ -r /etc/debian_chroot ]
    debian_chroot=$(cat /etc/debian_chroot)


# Color support detection from Ubuntu
if [ -x /usr/bin/tput ] && tput setaf 1 >&/dev/null

# Red user if root, orange if su
if [ "$USER" = "root" ]
elif [ -n "$SUDO_USER" ]

# Red host if SSH
if [ -n "$SSH_CONNECTION" ]

# Git branch
if [ "$(type -t __git_ps1)" = "function" ]
    PS1="${PS1}\$(__git_ps1 ' (%s)')"

# PS1 end
if [ "$USER" = "root" ]
PS1="${PS1}${separator} "

Example session (if the above is in ~/.bashrc or /etc/bash.bashrc on both hosts):

my_user@local_hostname:~$ cd git-repos/project/
my_user@local_hostname:~git-repos/project (master)$ sudo -u other_user bash
other_user@local_hostname:~git-repos/project (master)$ ssh third_user@other_hostname
third_user@other_hostname:~$ exit
other_user@local_hostname:~/Desktop/images$ exit