A Trial of Test Monday
I was once a part of a software project started from scratch and lucky enough to see it grow from our company as our first client and then outside clients afterwards. We took the lean software approach where we deployed often, GUI’s frequently contained large amounts of changes, and we communicated regularly with our clients to make sure our goals were still meeting their needs. We weren’t deploying as much as Flickr, but we had a twice a week deployment schedule.
We were a small group (3 developers and a manager) involved in the project at first and no QA resources. Our manager tested, deployed, and worked on the database. Although he wasn’t classically trained, he picked up quickly. I remember one Wednesday morning I logged in and saw a new build just before our deployment the night before and I saw that a DAO had been changed by our manager. My stomach dropped when I realized he had changed the code, but when I looked at the changes they were valid changes, fixing an issue before it went into production. This is how good he was, but he was also spread thin with demos, testing, deployments, managerial things, etc. It was also around the time I was sent to my first No Fluff, Just Stuff conference in 2010. This conference showed me just how far behind we were with automated testing.
[Unit, integration, functional, state, behavior, fuzzy, etc.] testing was missing from our project. Luckily for us we were still a couple months before we had enough features to be in production. I met with my manager and we discussed how we might approach getting the rest of the team on board with testing. We were pushing as fast as we could to get our company using our software (this alone would save the company hundreds of thousands of dollars by using our own home grown software instead of renewing our license for our future competitor). Writing tests would at the beginning slow down progress and we had to learn the frameworks chosen. My manager was excited about the benefit of testing especially with the tests integrating into each build. We came up with the idea for Test Monday.
Every Monday for a month the entire team would dedicate the day to learning the testing frameworks and updating our “legacy” code to have tests. The first two weeks were exciting to watch the test code coverage slowly make it’s way from the single to double digits. By the third week I realized that although we made progress every Monday to catch the code up the rest of the week the code coverage would remain the same or go slightly down.
Our weeks started to look like an oscillating wave where Monday would peak our test code coverage and then slowly throughout the week as the codebase changed and increased the test code coverage would go down. By the time we hit Friday it would sometimes be lower than the numbers we saw on Tuesday morning. This was due to issues where when tests failed at least one and possibly the other developer were removing the tests instead of updating them. After the second month of Test Monday we decided it was a good start and it was beneficial to focus and learn the new frameworks as a team, but setting aside an entire day for testing caused the unexpected side effect of creating a mindset where tests only had to be written on the designated day.