Unit Testing Notes

Here are random notes on unit testing. Random because I'm not giving a final form, but only still collecting them.

Unit testing is damn hard to do well. Anyone who says otherwise is either a liar or doesn't test his code.

Test first, code second.

There's a lot to say about testing, but there really is nothing more momentous to say about it than this. The implications are staggering.

Types of testing...

Unit tests test individual units including modules, methods or functions and classes in isolation from other units (indeed, from the rest of the application).

Integration tests test integration between two or more units. Functional tests are a subset of integration tests because they test all the units of an application.

When unit tests are applied only to the public interface of a unit, these are called black-box testing and tend not to be as brittle as other tests since the details of a public API tend to change less over time than the implementation of the guts. White-box testing is when the test code is aware of the subtleties of implementation details. This is brittle and leads to work and redoing work.

Books not to fail to have on your shelf...

...or on your Kindle.

Anatomy of a good test case...

The test case consists of three sections:

  1. the set-up,
  2. the action, and
  3. the assertion.

What is set up in the first section should be only as much as will be asserted in the third. If there's additional set-up needed in order to accomplish that crucial to the action and assertion, to put it in the set-up section muddies the test case. Options to solve that include doing such set-up in...

The action should be, as much as possible, a single line.

Assertions should be limited rather than numerous. Each test case has the purpose of testing a single feature, state, etc. Multiple assertions muddy the purpose of the test case.

Test cases should emit a clear message as to what failed and maybe why. They should be silent in the event of success.

Test-doubles

Martin Fowler says, "Think stunt double." This is a generic term for replacing production objects with specially rigged ones for testing. Fowler lists the following examples, which are very useful definitions indeed.

Test behavior only

Test behavior not methods. Test behavior not implementation.

Eclipse JUnit

Good that Eclipse provides for this framework, bad that it's trying too hard to help.

Eclipse's wizard for setting up JUnit test cases automatically generates one @Test method per method and object inside the class under test. This is stupid and to be avoided. It gets you into writing tests that are completely wrong-headed. They will be brittle and expensive to maintain. They will not be testing the way you need to test. They will steer you away from writing tests the way they should be written.

Of course, if you had written tests before coding the class in the first place, ...

Pitfalls in writing tests

If run separately, your test methods work, but run together, some fail, it's because the first one (or ones) is (are) contaminating the later one (or ones).

This easily happens when you're using some kind of dependency injection such as (and especially) ServiceLocator.