Go’s excellent tooling extends to support for testing. Nevertheless, there are quite a few packages that exist to make it easier to write tests. In particular, Go’s verbose (although accepted) handling of errors is a weakness here, as writing tests can involve a lot of boilerplate. Compare writing a test case using, for example, Catch2, with what would be required if using solely the standard library in Go. Unfortunately, I have not found a utility package for testing that meet my design criteria.
The main design criteria are:
testingpackage in the standard library.
Working with only the standard library is obviously possible, but note that it only meets the first design criteria (by definition). Another common package is
testify, but that packages does not meet requirement #2.
The goal here is not necessarily to promote the package that I wrote. That packages is still small, and needs more use to evaluate the API. However, others might find the technical tricks that were required useful. Two tricks made meeting all of the design criteria possible.
This technical detail is not that complicated, but was was not available when I first attempted to build a utility library for use with package
testing. The method
Helper was added to the types
T in version 1.9. The effect is to suppress the utility function when reporting an error.
Consider the following test:
The output of which will be:
In the output, the filename and line number refer to the line with the call to
catch.Check in the user’s code. This is also where an investigation into the failed test should begin. However, without the call to the method
Helper inside of
catch.Check, the filename and line number will instead refer to the location of the logging method inside of the utility package, and key information in the messages will be incorrect. This is a serious usability problem.
With the introduction of the method
Helper in version 1.9, it was possible to write utility packages for
testing that met criteria 1 and 3. However, creating the message for failures still required more input from the caller.
If the messages are to be generated completely automatically, the utility packages needs to be able to get a textual representation of the test expressions. In other languages, the preprocessor (in C) or macros can be used to stringify predicate expressions, and then pass those strings as arguments. Whatever its other faults, the venerable
assert provides complete messages with zero effort from the caller. However, Go provides neither a preprocessor nor macros, so we need another approach.
A solution is to load and parse the source files1. This involves using the function
runtime.Caller, which can provide the path and line number of a function’s callers. With this information, the source code can be read and parsed to recover the arguments to the utility function. Since the utility package can assume that the source code is syntactically correct Go, some shortcuts with the parsing are possible, so that the parsing amounts to little more than counting parenthesis.
There is a weakness with this approach, which is that it can be brittle. For example, if you run the earlier example on the Go playground, you will see an additional error message. The file with the source code could not be opened. This also means that the error message could not be completed, and there will be placeholders where normally the expressions in the calling code would be included. In general, the test binary needs to be run in a context where the source files are accessible, and unchanged. Using
go test -c to build the test binary, and then executing that binary at a later date or on a different machine will degrade the error messages.
Please look at the package catch if you are interested in more details. This package is quite small, but does meet the design goals originally stated. Unfortunately, it also introduces some brittleness where the error messages for failed tests can be degraded.