So I gave this presentation last week at work on the subject of automated tests. Some people in the audience, as well as some who weren’t able to attend, wanted to get hold of my slides. Unfortunately the otherwise awesome reveal.js failed to convert my presentation into a PDF. But I thought I would blog about it instead. So here goes.
I was really pleased with the number of participants: 30 people from a department of 60, three days before a release, isn’t bad. I had insisted on having the talk during lunch, instead of the usual after work schedule. I think that helped a lot, since a lot of my colleagues are parents of small children. The audience consisted mostly of developers, but I was happy to see some testers as well.
I chose to call my presentation “Writing nice tests – Why, how and when”, with the three questions forming my agenda.
Why
I started with the why, trying to explain why test code is as important as production code.
Why do we write automated tests?
I had compiled a list of three reasons that I find extremely important. There are of course more, but these were the ones I wanted people to remember.
First on my list was to reduce fear. I don’t know if it was Robert Martin or Kent Beck that used the word fear, but it applies well to the feeling I have when coding. Good tests reduce this fear allowing me to feel comfortable making changes. I also mentioned that writing good tests is to show that you understand that code must be able to change and that you want to reduce the fear for the person doing it.
The second reason on the list was for documentation. I stated that a well written unit test is the best documentation of a class or a module. I explained that a unit test is living documentation, which lives until the production code no longer works the way the test describes. But unlike other types of documentation a unit test dies screaming, giving you a chance to react. I pointed out that no other kind of documentation works this way: if you write it on a wiki, in javadoc or even worse as inline comments, it will die without anyone noticing it. And dead documentation will mislead readers, rather than helping them.
Third and last on my list was to detect code smells. I’m a big fan of the way a test can tell you that your code sucks. For example it is painfully obvious that your class has lots of collaborators when you’re forced to create a long list of mocks, which might be a sign that the class has too many roles. It is equally obvious that the method under test is doing too much if you have lots and lots of setup. And more generally if your code is hard to test, you have a serious design flaw, since testability is a key aspect when designing code. What I was trying to tell my audience was that you should rather change the production code than write complex unreadable tests.
Properties of a good test
I went on to describe what properties a good test should have. Again, I had chosen a subset I felt was important.
I started with trying to convey how important it is with fast tests. Fast feedback is a prerequisite for being able to work test driven, but I also expressed how frustrating it can be when you have to wait minutes for a test to execute. It makes you lose focus which is always a bad thing.
Next on my list was that a test must be readable. You should be able to open a test case you’ve never read before and within moments understand what is being tested, how the test works and how the code under test works.
Last I said that also the output of the test should be readable, i.e. the test should be informative when it fails. I have wasted huge amounts of time trying to fix broken tests where the only hint I’ve gotten is “assertion failed”.
How?
Part two was about how to achieve the things mentioned in part one.
How do we write fast tests?
So writing a fast test, I claimed, is really about not writing a slow test and the best way to slow down a test is to have it depend on a database. Not only does it increase the execution time of the test but it also adds a lot of complexity making the whole process slow. For example a test fixture suddenly becomes something really hard to write. You will also have to deal with tests depending on each other if you don’t clean up in a proper way. And of course, anything which forces you to write SQL is always boring.
Another slow factor I mentioned was remoting, e.g. driving the code under test using for example an http client like a rest client or Selenium. Not only do you get network latency, but you will be forced to package and deploy your application before running the test.
So was I saying that you shouldn’t use a database at all? No, I was trying to say that you should be pragmatic. If the logic is in stored procedures then of course you will have to test it. But you should identify places where the logic is not in the database and consider testing it with unit tests, perhaps with a complementary integration test.
Remoting on the other hand should be avoided or kept to a minimum. It’s usually unnecessary and just makes the test run slow, adds duplication and makes it harder to measure coverage.
How to write readable tests
For the readability I started with reminding everybody of how important it is to name your tests well. The name should convey the goal and purpose of the test in a clear way. It’s so important that it’s worth taking five minutes to think about it and discuss it with your partner/neighbour. I gave an example of the benefits of well named tests by telling the story of how I’d been working on this large assignment last month, where I was a bit uncertain of my integration test coverage. I gave my list of test names to a tester in my team and by comparing them to his test ideas he found ten to fifteen tests worth implementing. Even better was that one of the tests described a missing feature and one found a bug. That was a really nice experience and I recommend everyone to try this approach.
I continued with explaining the need for a Domain Specific Language, DSL, in your tests. You want your tests to be as expressive as possible and this is accomplished by reducing the boiler plate, choosing a nice abstraction and emphasizing the essentials in your test. An easy way to accomplish this is to use factory methods or the builder pattern, but it’s also possible to do some freaky stuff in test code which would never pass a review of production code.
I often struggle with huge tests so I wanted to highlight that it’s permitted to split test cases. The benefit of splitting tests cases is that you give each test case its own namespace, allowing for easier naming of test methods. It also grants you the possibility to use different DSL:s making it possible to tailor fit the DSL for the aspect tested in the test case.
I ended with imploring the audience to minimize the number of assertions. I explained that too many asserts will shadow the purpose of the test, especially when asserting on stuff that already has been checked by other tests. But whereas zealots say that you should have one assert per test I adhere more to the school of asserting on one concept per test.
Smarter assertions
So for the next part of the presentation I wanted to show how much you gain in readability for both the test itself and its output by just switching assertion library. I started by showing the difference in expressive power between standard jUnit asserts, Hamcrest and Fest assertions. For the code samples I borrowed heavily from the Fest assertions starting guide.
jUnit
I started by showing how my test would look like if I used the standard jUnit Assert methods.
assertTrue(gollum instanceof Hobbit);
assertEquals("Frodo", frodo.getName());
assertTrue(fellowshipOfTheRing.contains(frodo));
assertFalse(fellowshipOfTheRing.contains(gollum));
assertEquals(9, fellowshipOfTheRing.size());
I pointed out the lack of readability, e.g. that the expected and the actual value is in the wrong order and that it’s hard to see the difference between assertTrue and assertFalse.
Hamcrest
Hamcrest is a matcher library which comes integrated with jUnit, i.e. if you download the standard jUnit distribution you will get Hamcrest as well (although not the latest version). It is built around a method called assertThat which takes two arguments, the object to match against and a matcher. I showed the following code as an example of how the previous test would look like rewritten with Hamcrest (and with some added features).
assertThat(gollum, instanceOf(Hobbit.class));
assertThat(frodo.getName(),
both(startsWith("Fro")).and(endsWith("do")));
assertThat(frodo, isIn(fellowshipOfTheRing));
assertThat(gollum, not(isIn(fellowshipOfTheRing)));
assertThat(fellowshipOfTheRing,
allOf(hasSize(9), hasItems(frodo, sam), not(hasItem(gollum))));
I discussed the pros and cons of Hamcrest, where the pros include a more natural and concise language. The cons are that all the methods in the code above are static imports from a range of Hamcrest classes, which you must know of to use. Also, the language is a bit strained at times as in the “not(isIn())” construction above.
Fest Assertions
I then showed the expressiveness of the Fest Assertions library. This is a library I’ve just recently stumbled upon, but instantly have started to like. My test would look like this:
assertThat(gollum).isInstanceOf(Hobbit.class);
assertThat(frodo.getName()).startsWith("Fro").endsWith("do");
assertThat(frodo).isIn(fellowshipOfTheRing);
assertThat(gollum).isNotIn(fellowshipOfTheRing);
assertThat(fellowshipOfTheRing)
.hasSize(9).contains(frodo, sam).doesNotContain(gollum);
It seemed like everyone could clearly see the benefits of using a library like this. The language is as close to English as you can get and the chaining of assertions makes for really expressive one-liners. Further more, it only requires one static import: the assertThat method. The rest of the methods are discoverable, i.e. your IDE will suggest the available methods depending on the type of object you’re asserting on.
More informative test output
I continued by comparing the output of a failed assertion, starting with jUnit:
assertEquals(10, fellowshipOfTheRing.size()); assertTrue(fellowshipOfTheRing.contains(gollum));
The assertions above would produce the following output
expected:<10> but was:<9>
for the first assert and an empty message for the second.
Next up was Fest Assertions:
assertThat(fellowshipOfTheRing).hasSize(10); assertThat(fellowshipOfTheRing).contains(gollum);
giving the output
expected size:<10> but was:<9> for <[Frodo, Sam, Merry, Pippin, Gandalf, Aragorn, Boromir, Gimli, Legolas]>
and
<[Frodo, Sam, Merry, Pippin, Gandalf, Aragorn, Boromir, Gimli, Legolas]> does not contain element(s):<[Gollum]>
It was easy to show that the output of the standard jUnit asserts would probably need some debugging whereas the output from Fest Assertions was much more informative.
When?
The final part of the presentation was about when tests should be written, i.e. just before committing or before there’s any production code written at all.
I gave my perspective on how hard I think it is to write tests when I’m “done” with the implementation. When everything is functional I just want to push my awesome code to review to show it to my colleagues. I mean, the goal is to have it running and you don’t want to be stuck with boring stuff when you’ve reached the goal, do you? I also find myself having a hard time poking holes in my beautiful creations. I find it much easier to write the tests first.
This lead me to Test Driven Development, TDD. I began with presenting the three laws of TDD as written by Robert Martin in Clean Code:
- You may not write production code until you have written a failing unit test.
- You may not write more of a unit test than is sufficient to fail, and not compiling is failing.
- You may not write more production code than is sufficient to pass the current failing test.
These laws leads to a process known as Red/Green/Refactor:
- Red: Write a failing test.
- Green: Fix the failing test in the easiest way you can think of.
- Refactor: Clean up the code created in the previous step.
Since I have been striving to work test driven for over a year, I tried to give my reflections on what TDD provides me. It goes without saying that it gives me testable code with high coverage. It would be impossible to work test driven without achieving this. I also believe that it gives me guidance. I find it much easier to figure out what do to next and I never implement any unnecessary features. Since I’m locked in the red/green/refactor cycle I feel focused on what I’m doing. In fact, I discovered just recently that it’s excellent to take breaks at “red”. This means that if I want to have a cup of coffee, I always write a failing test first. That way I know when I come back what I was doing.
I wanted people to know that TDD isn’t something you can be good at after reading a book. It really requires a lot of practice and determination. I still have to remind myself every day to write the test first. Sometimes I discover myself writing code without a failing test. I try to be hard when this happens by just doing git reset and writing the test instead.
I went on telling them that I do believe that TDD requires skills. Not only in writing tests but in writing well designed applications. It doesn’t hurt to know your design patterns and your enterprise patterns.
I closed the subject with a book tip: Test-Driven Development by example by Kent Beck. A fun book which gave me a good insight into how a test driven developer thinks.
Summary
I love giving presentations where I get to talk about something that is as close to my heart as tests. We had a pretty good discussion/Q&A afterwards about, among other things, unit tests vs integration tests. I think it went well even though I didn’t get much feedback, this being Sweden and all.
I was however pleasantly surprised some days afterwards when I received an email from one of my colleagues. The subject was “Fest!”, which is Swedish for “Party!”, and the mail body contained a link to a commit and the word “Woohoo”. Reviewing the commit I realized that it was a conversion of an entire module from standard jUnit assertions to Fest assertions. Party indeed!
Driven by Code Quality 