Part of a short sequence of posts about various testing practices. They could be used as pointers in pull request reviews.
Here's an assortment of test description examples I come across too often:
it("imports records successfully", () => {
@Test public void testParsesInputCorrectly() {
context "when parsing malformed input" do it "does the right thing" do
def test_works():
This last one is my favourite. For example, a test for an ETL routine that creates records and invokes procedures. A screenfull of code doing something, no asserts. It worked indeed, very hard.
Anyway, what is the problem with these expressions?
- It works.
- It does the right thing.
- … correctly.
- … as expected.
- … as required.
- … as requested.
- … as documented.
- … successfully.
- … gracefully.
- … properly.
Vagueness
Being vague in human conversation is fine; meaning can be reasonably inferred from the past conversations or the situation. If anything isn't obvious a human can ask to clarify.
Not so in tests. Tests should strive to assume as little context as reasonable. There is nobody around to clarify vagueness on a change
antirez • 11 years ago • Cluster: fix redis-trib --from all.
Fortunately, antirez's code is always crystal clear, and he keeps working on Redis to this day. But clarity does not come naturally to everyone. People forget. People come and go. What was meant by "as expected" three years ago?
Correctness
With my "ackchyually" hat on, I could argue these statements are wrong. All code has bugs; it follows that nothing ever works "properly" or "as required". Perhaps to the best of your knowledge today CSV parsing works "as expected" (by whom?). But the expectation or inputs might change tomorrow, and the test description becomes a lie.
Laziness
The developer did not care about the name of the test.
What do we do instead?
Document what the test is doing:
- "It works" → "It converts dollars to euros according to the current exchange rate"
- "It does the right thing" → "It throws an error if the balance would go negative"
- "It parses input correctly" → "It decodes special characters"
- "It degrades gracefully" → "It returns the email only when the full name is missing"
- "It creates a new user successfully" → "It creates a new user" (can it create a new user unsuccessfully?)
These suggestions are not 100% specific, nor eternally correct. If someone needs to know what "decodes special characters" means, they can read the test code. The aim for test descriptions is to be good documentation. Reading their listing
$ bundle exec rspec --format documentation
RspecPreloader
rspec-3.8
clears the screen
runs specs
preloads the spec_helper.rb file
preloads the rails_helper.rb file
runs before-suite hooks during preload
does not run before-suite hooks after preload
reloads FactoryBot factories if they change before running specs
should give one a good idea what the thing does. Please tailor the specificity accordingly.