Start test names with “should”

The purpose of a test is not just to enforce some behaviour on the code under test. When the test fails it also should provide enough information to understand which behaviour failed, where it failed, and (at least superficially) why it failed. If the only output of a failing test is just a binary value like “FAIL”, that test is only giving one bit of information to the developer. A good test framework will also print the test name and a call stack. We can’t change the call stack much, at least not without changing the semantics of the test, and how to actually write the test itself is a whole other story, but what does a useful test name look like?

A trick I learned from Dave Hounslow, a former colleague, is to start test names with “should”¹. This has a few advantages over test [function name]:

  • It removes redundancy, because the function name should already be in the call stack.
  • It is falsifiable, that is, a person reviewing the test can decide to which degree the name agrees with the actual test. For example, they could point out that should replace children when updating instance verifies that new children are added, but not that old children are removed.
  • It encourages testing one property of the function per test, like should apply discount when total cost exceeds 100 dollars, should create record for valid input, and should return error code 1 for unknown error. test [function name] encourages testing everything the function does (branches, side effects, error conditions, etc.) in one test.
  • It invites the developer to write something human readable. I usually find “test …” names to be clunky to read. This may just be bias after years of using this technique.
  • It is better than a comment explaining what the test does, because the comment will not be shown when the test fails.

¹ Some frameworks require that test names are formatted in some special way, like starting with “test”, using snake case, camel case or other. I’m ignoring that part of the naming for brevity and to avoid focusing on a specific language.

Unit testing Python + MySQLdb warnings

There seems to be several methods out there, based on elevating warnings to errors using warnings.simplefilter. Here’s another method, based on recording warnings in a variable, and checking that the last one is a MySQLdb.Warning. Hopefully to be integrated in INSPIRE.

import MySQLdb
import unittest
import warnings
[...]
class TestTagInsert(unittest.TestCase):
    def test_too_long_tags(self):
        with warnings.catch_warnings(record=True) as warn:
            [Run SQL statement]
            self.assert_(len(warn) == 1) # Ensures that the next statement won't break the testing
            self.assertEqual(
                MySQLdb.Warning,
                warn[-1].category
                )
            #If you also want to check the text of the warning:
            self.assert_(
                'truncated' in str(warn[-1].message))