Unit Testing 101 - Part 2 - Why must we be careful?

So if Unit testing is a safety net why must we be careful?  The answer is two part  in my experience. 

The first reason is quite simple.  Imagine that you are swinging along on your trapeze without a care in the world. You have a safety net underneath you after all so why worry right?  You have the freedom to do whatever you want.  Until you fall.  And then all of the sudden you find out that  our safety net we had invested so much trust into doesn’t actually work as well as we had thought.  There are holes everywhere.  We didn’t notice them before but they are definitely there. 

Maybe your smoke testing found them.  Or if you are lucky enough to have QA (seriously.. I’ve worked without QA more then I have with) then they are flagging defects left and right in whatever system you have setup.  But much worse would be for your customers to be reporting defects left right and center.  This is the worst possible outcome since it will damage the trust our customers have in us.

If you find yourself in this situation my recommendation is to seriously consider stopping all new work until you can get everything under proper test coverage.  As long as you have large holes in your code coverage you are always at risk for introducing bugs in related & unrelated parts of the system.


The second reason for being careful is even more insidious. In this instance as we make changes we find that instead of a nice safety net we’re actually created a big sticky spider web.  Every change we make causes tests to break in ways that they shouldn’t.  We change a single line and suddenly we have 5 broken tests.  We find ourselves in a world of fragile tests.  This is bad because we are sapping away time and energy from our project.  And more importantly we are setting ourselves to be in a situation where we avoid making changes because we don’t want to have to fix a lot of tests for every minor change. 

For me this problem has actually had a more negative affect on my efforts then the former.  When there are not enough tests in place as long as you have a good understanding of how they should be written (leaving aside for a moment why they weren’t created in the first place) it’s not that bad of a problem to find the missing pieces and fill them in.  But fragile tests are much worse because now you can’t trust any tests you have.  You have to go through each one and figure out what is wrong with it and repair it.  More importantly you have to gain the knowledge as to what makes these tests bad versus what makes a test good.  Education is always great so this is definitely the one bonus of dealing with this problem but until you have it dealt with your project is carrying around a big weight on its shoulders.


In both of the reasons I have just given there is one thing in common.  In the eyes of our peers (other developers, managers) the time we have taken to write these tests seems like a waste.  And in the second it’s even worse because we’ve made it so that when you make a change you now have to fear all the unit tests you will inevitable have to fix.  For many people (maybe yourself even) unit testing is a new concept.  It’s a much over due concept but like everything new it is regarded with either hope or mistrust depending on the viewer.  So those of us who are practitioners need to be very careful  in how we do things.  We need to be careful that we are showing the good side of testing and not the bad side.  Start small and when you find you’ve made a major mistake go back and fix it everywhere.  You don’t want to end up like I did with a project full of tests where every mock was an explicit mock (I’ll get into that more in the future). 

Next time we’ll look at what State Based testing is and how it provides the bedrock for our testing efforts.

posted @ Saturday, December 20, 2008 8:52 AM

Comments have been closed on this topic.