Agile Zone is brought to you in partnership with:

Jared Richardson works at Logos Technologies As a recognized expert in the software industry, Jared has worked with both start-ups and software giants. He's been involved with various open source projects, with roles from contributor to founder. Jared co-authored the best selling book Ship It! and Career 2.0, and founded the Agile RTP user group as a local outlet for the agile community in North Carolina. His personal blog is Agile Artisans

Jared has posted 52 posts at DZone. You can read more from them at their website. View Full User Profile

Test Behavior, Not State

08.25.2010
| 14358 views |
  • submit to reddit
Many developers and testers write automated tests. Sometimes they're unit tests, other times package level, and occasionally integration. There are many different types of tests, but there are a few characteristics of great tests. Today let's look at one specific characteristic: behavior versus state.

A common trap that many test crafters fall into is letting their test see too far into the class. Delving into the code inside the method, checking data structures, seeing what routines were called, and looking at variables are all signs that your test has gone too deep. By coupling the test so completely with the exact implementation of the method, you've made it impossible to refactor the code inside the method without changing the test as well. This creates more work for you (generally a bad thing), but also forces you to update the test to match the changes you've made. Adjusting the test gives you a change to change what's being tested, which allows bugs to creep in.

For example, say I've got a routine that grabs a list of students in a class, calculates each student's grade, then returns an average for the entire list. The method signature might look like this:

public Integer calculate_classes_average_grade( String class_name)

You pass in a class name and get a number in return. There are several good ways to test a routine like this. One way would be to pass in a class name, have it pull in known data from a database, let it calculate the average GPA and return it to your test harness. You'd then verify the return value. (You'd probably want to use Dave Hussman's "Nuke and pave" idea on the database to ensure a clean, known data set.)

Another way to test it would be to mock the database connection, have the mock return a known data set, then again, test the return value.

A third way would be to drive the code that uses this routine, testing it from a higher level. Perhaps a package level test or even a GUI test would work for this scenario.

In each case, we're using the code and checking the results. The test doesn't know anything about how the grades are retrieved (apart from the mocking). It only knows what the code should return. This frees up the development team to change the code inside the routine. Perhaps you want to make the code more efficient, change the data structures being used, or completely rewrite the internals of the class. If you've fallen into the trap of overly coupling your tests to your code, these changes will fail your tests, and you'll have two classes to rewrite.

Sometimes a developer will write a test that runs this routine, then checks the internal list of students to see if the correct number of students is in the list, then checks if the right students are in the list, and so on. For the moment let's ignore that that bit of code should be broken out into a subroutine so that you can do that check properly. Instead, let's look at what happens if you write a code that peeks into the class, grabs that data structure, and verifies the structure's internal state. When you give the test that level of insight into the class being tested, what happens when the data structure changes from an array to a hash table? What happens when the order of the students in the list changes? What happens when you want to run the test against a different data set?

In each of these scenarios, the test fails, so you have to update the code. This creates unnecessary work, wastes your time, and introduces the potential of "fixing" the test in a way that lets errors creep in.

So take a step back. Never let your automated tests, no matter their type or flavor, see inside the code they're testing. Use the class, the package, or the routine. Pass in the data values that make it run, and then check the results. This is one of the best ways to write solid, reusable tests and ensure you can refactor your code without having to update your tests at the same time.
Published at DZone with permission of its author, Jared Richardson.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

Comments

François Sarradin replied on Thu, 2010/08/26 - 6:48am

Interesting post! Some of your ideas correspond with the Dan Notrh's BDD approach. Especially the fact to insist on the behaviour. Indeed this is a way to write better and more understandable test. But I was not thinking to decouple the test from the implementation... You are rigth! A test is not an excuse to break the encapsulation principle; this leads you to some test smells.

 

Peter Veentjer replied on Thu, 2010/08/26 - 9:38am

I think that with the introduction of easy mocking tools, a lot of inexperienced developers think it is the only way to test. I have seen way to many tests where the only reason of existence (I could tell) was causing the coverage go up. You still had no idea if the code actually worked, but Sonar didn't complain.

Combine this with (too) many dependencies in classes and you have the following  problems:

- tests not useful while refactoring because it doesn't find bugs(you alread explained this)

- tests themselves need to be refactored, so refactoring takes even more time!

- the tests don't work as documentation because test code is way to long. In a lot of cases I even don't know what the test is supposed to do (or was creating another clone of the method using mocks the goal).

- only the happy flow is tested because the tests are too long and too painful.

- limited interaction with the api, so the test is not going to help you with writing a good api.

I think mocking is a nice tool, but in a lot of cases I'm asking myself if the world would have been better off without it.

 

And unit tests in a lot of cases is the only thing some developers know. So you still don't have any clue if the system works like expected (or that it starts up at all).

 

Jared Richardson replied on Thu, 2010/08/26 - 12:35pm in response to: François Sarradin

Thanks François. I'm glad it made sense to you. It's something that I've seen happen over and over, but I never got around to writing it down. :)

Jared Richardson replied on Thu, 2010/08/26 - 12:41pm in response to: Peter Veentjer

Hi Peter,

I agree with you about mocking. It's a very powerful technique, but it's frequently abused. When I teach it, I try to be very sure the team understands proper mocking versus over mocking.

Unit tests are the same way. Far too many developers think "automated tests" must be unit tests. I prefer to teach multiple approaches, then let the team get the right tool out of their toolbox. The right tool to solve the problem at hand.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.