Beyond Basic Testing in Go with testify

One of the first ever open source conferences I attended was YAPC::Eu in Amsterdam, in 2001. One talk that stuck in my mind was about testing, and from around that time I made a point of making sure that my work was adequately covered by unit tests. Perl was actually a very early adopter of unit tests, and not many people know that even Perl 1.0 in 1987 had a unit test suite. The Perl test modules in 2001 were already quite good at writing concise and expressive (though not necessarily well-organized) tests.

As such, it’s one of the first things I want to make sure I can do in a new language. It seems that the basic support for testing in go has a few set of useful features, but also in other ways it’s quite bare bones, leading to the existence of extensions. I’ve checked a few out, and the one which most resembles the testing patterns I’ve come to love is Testify. First I’ll show a basic unit test using the standard library module, before giving a basic rundown on testify.

Basic testing with go test

The good news is that all you have to do to write unit tests is to drop functions in the appropriate path and location, and then go test will run those unit. The existence of this convention alone is a fantastic start and lends itself towards comprehensive unit tests.

ie, unit test for bar.go in package foo should be written as functions in bar_tests.go, and named starting with Test, taking a t *testing.T as their only argument, and no return value.

Test functions should return normally to pass. To fail, you can either panic (but remember, don't panic!), or call a method on the testing.T object to give a friendlier failure message. You can also mark tests as skipped, and write messages to the error log. There are versions of the functions which return immediately vs keep going, and versions which log. I’ve attemped to summarize these below:

The “keep going” variants are not always found in other test systems, but they basically just mark the test as failed and allow it to continue. The idea behind this is that knowing that your change broke 100 tests is more useful than knowing it broke at least 1, and possibly more. The downside is that the first broken test may have caused the subsequent failures, making them “carried errors”.

A basic test script ends up looking like this:

package example

import "testing"

func TestSumFunc(t *testing.T) {
    if SumFunc([]int{4, 5}) != 9 {
        t.Fail()
    }
}

When this fails, it looks like this:

$ go test
--- FAIL: TestSumFunc (0.00s)
FAIL
exit status 1
FAIL    _/Users/samv/work/golang-scratch/example    0.006s

Success looks like this:

$ go test
PASS
ok      _/Users/samv/work/golang-scratch/example    0.006s

Showing context with testify assertions

The built-in testing module is simple enough to detect failures, but it’s helpful to know when things fail, exactly how it failed. This avoids you from having to debug your test script just to figure out what went wrong. A good test wraps all its failures with a message saying what was expected, what happened, and what the context of the failure was.

You could do this using intermediate variables, and error strings:

func TestSumFunc(t *testing.T) {
    sum := SumFunc([]int{4, 5})
    if sum != 9 {
        t.Errorf("Expected 9, got %v", sum)
    }
}

This error message at least gives you some information:

$ go test
--- FAIL: TestSumFunc (0.00s)
    adder_test.go:8: Expected 9, got 1
FAIL
exit status 1
FAIL    _/Users/samv/work/golang-scratch/example    0.006s

However, this is fiddly and requires a lot of extra lines. Enter github.com/stretchr/testify. This brings in the sort of test assertion functions that programmers will be used to from Python’s unittest, Perl’s Test::More, Ruby’s rspec, etc.

package example

import (
    "testing"

    "github.com/stretchr/testify/assert"
)

func TestSumFunc(t *testing.T) {
    assert.Equal(t, 9, SumFunc([]int{4, 5}), "Sum of 4 and 5")
}

Down to one line! And we get a better error, to boot:

$ go test
--- FAIL: TestSumFunc (0.00s)
        Error Trace:    adder_test.go:10
    Error:      Not equal: 9 (expected)
                    != 1 (actual)
    Messages:   Sum of 4 and 5

FAIL
exit status 1
FAIL    _/Users/samv/work/golang-scratch/example    0.008s

The last argument, the message, is optional but highly recommended. If you are repeating a test many times inside a loop for example, you can include the iteration number or details of the item being iterated on to give the person who has to deal with the failure a head start in seeing what went wrong. In other instances, the line number of the failure

The library supports a lot of test functions; some of the most useful are described below. I didn’t include the optional messages arguments, but they all take it so make sure to in your tests!

This is just a small sampling! There are negated versions of many of the above which start with Not, as well as various assertions for testing things like length, approximate equality for floating point use, truth (assert.True and assert.False, of course); see the complete documentation for more.

Notably absent in current versions are range functions: things like assert.Greater() is nowhere to be found. For those, you would have to fall back to the basic style of assertion.

Another thing to watch out for is that it’s quite easy to make your test script itself panic when there are failures. These can be annoying to unpack, so always test and guard subtests which dereference returned values that might be nil; assertions handily return a boolean value you can use for this purpose:

x, err := SomeFunc(y, z)
info := []interface{}{"SomeFunc(y, z) with y, z = ", y, z}
assert.NoError(t, err, "y, z = ", info...)
if assert.NotNil(t, x, "y, z = ", info...) {
    assert.Equal(t, "baz", x.Bar, info...)
}

Mocking objects using testify mock objects

Go already has a built-in system for building mock objects: interfaces. Most testing libraries that implement mocking work by temporarily “monkey patching” functions and passing in dynamic objects which do nothing.

In go, you would typically define an interface that describes the object that your library deals with. Your test script would deliver a simple version that doesn’t do anything except perhaps collect information on its interaction with the code being tested.

This is already sufficient for most traditional uses of mocking. It’s also usually possible to convert your functions from ones that take concrete objects to ones that take interfaces. As a bonus, you’ll probably be making the boundary between your code and the other module more explicit and interchangable with alternate implementations.

You may be able to simplify the building of your test classes with the testify/mock library. It allows you to very quickly build out test objects which collect which of these stubbed function calls are used, as well as provide dynamic, per-test stuffing of results.

It works reasonably for this purpose.

Testing Coverage, Benchmarks, Race detection, Profiling, and more…

Go is the promised land of microoptimization, and so there is wide support in the testing toolset for a variety of powerful techniques you’ve been used to. More on this to come!