Do you write good unit tests? If you believe so, then how do you know they're good? It's easy for us to fall victim to confirmation and self-serving biases because there are so many bad examples of software developers in our industry. So, how can you objectively determine if you're writing good unit tests?
It turns out there are 3 criteria for identifying good unit tests:
- They're Automated
- They Verify Behavior
- They Support Refactoring
And that's about it! These seem simple enough, but how can you tell if your unit tests meet these three criteria?
They're Automated
Tests should be automated, but automated to what degree exactly? The first question to answer is this: "Is a human executing the system or component in order to verify behavior?" If so, then you're not automated at all. It was Robert C. Martin that said "manual testing is immoral."
The second question to answer is "are your tests executed on every check-in, or at least daily?" This is getting into the DevOps practices of Continuous Integration and Continuous Delivery, but your tests should not require a human to kick them off; they should run as a part of your delivery pipeline. These tests can run as a part of the Build, as a Post-Build validation step, and/or Post-Deployment (by running in a target environment after the system is deployed). You'll likely need a combination of these approaches to achieve the level of confidence your team requires, although unit tests are typically executed in the first two parts of your deployment pipeline: Build or Post-Build.
The third question you'll want to answer is "can your tests be executed in any environment?" Good unit tests become a fantastic way to exercise the system to ensure everything works as expected (can someone say 'regression testing?'). Being able to execute tests in any environment can be a difficult goal to achieve, but it is an ideal worth exploring. If you can truly design your tests to run in any environment, you minimize the likelihood of the dreaded "Works on My Machine" mantra of development.
They Verify Behavior
There are two human philosophies which help us prove things to be true or false. Mathematics (specifically mathematical proofs) can help us prove something to be true and correct, while Science (and the scientific method) can help us prove something to be false. Tests are much more like science than mathematics: testing cannot prove the absence of bugs (which is proving a negative), it can only prove that the software behaves, or doesn't behave, as expected for anticipated inputs or behaviors (assuming your system is deterministic to a large degree).
The fourth question to answer is "do your tests verify that the system is designed correctly, or do they verify that the system works as the user expects?" Tests that verify that the system is designed correctly are largely waste (more on this in They Support Refactoring). Ultimately what is "correct" is highly subjective, and the definition of "correct" in any situation will likely change for a myriad of reasons, including the growth of developers, changes in policy, and the evolution of our technology stack, just to mention a few.
Instead, we should focus on verifying that the system functions the way our users expect. If you're operating in an Agile environment, there's good news: you likely already have a source of these expectations! The source is the acceptance criteria from your user stories. Certainly, good acceptance criteria don't spell out the exact test cases you need to write (since good acceptance criteria are negotiable by definition), but good acceptance criteria give us the foundation for writing good test cases.
From a Lean perspective, we define waste as anything that does not provide value to the customer or user. The customer or user wants to know that the system operates as expected for them, and frankly, they couldn't care less about what design or architectural patterns we chose to implement. Don't waste your time (and therefore their investment in us) by testing the design of your system (such as your public API or the implementation of specific classes or methods). Instead, provide value by ensuring your unit tests verify the functionality as identified by the acceptance criteria.
They Support Refactoring
The ability to support refactoring is the hardest criteria to meet when writing good unit tests. Too many unit tests are written which actually solidify the initial design of the system and therefore make it more difficult to change the code. Remember, as Martin Fowler told us, "refactoring is a change made to the internal structure of the code that doesn't modify its observable behavior." In other words, change the design, but don't change the behavior.
So, the fifth, and arguably most important, question to answer is this: "do your tests allow you to make wholescale changes and refactorings without causing you to spend equal or more time changing the tests?" If you create a class with a switch statement, and now you want to introduce an inheritance hierarchy, can you make that change by making at most one change in the test code? If you change the type of a reference from an array to a Sequence or a List, can you do that without changing your tests? If you move a method from one class to another or change a field to a method, do you tests allow you to do that without changing the reference in more than one or two places in the test code?
This is the hard part! We need to find a balance in our unit tests where they verify specific policies and behaviors but they aren't tightly-coupled to the design of the code base under test. There are a number of patterns and approaches for doing this, but it may require you to rethink how you approach creating unit tests.
In Summary
Your goal is to be able to answer these 5 questions:
- Is a human executing the system or component in order to verify behavior? The ideal answer is "no".
- Are your tests executed on every check-in, or at least daily? The ideal answer is "yes".
- Can your tests be executed in any environment? The ideal answer is "no".
- Do your tests verify that the system is designed correctly, or do they verify that the system works as the user expected? The ideal answer is "they verify that the system works as the user expected".
- Do your tests allow you to make wholescale changes and refactorings without causing you to spend equal or more time changing the tests? The ideal answer is "yes".
Do you have questions about how to do this? Or, do you disagree with me completely? I'd love to hear from you!
