STOPLIGHT REPORTS

A Picture is Worth a Thousand Words…



On occasion, I read an article or blog that really hits home or addresses a problem we have and I get excited about it. I’d like to bring your attention to an article called “Simple Strategies to Keep Quality Visible" by Jeff Patton on StickyMinds.com on their Process Improvement tab. I apologize for not having the link available, but inserting simple links in this format continues to be a challenge.

Now what Jeff’s team was addressing and what we need to address have similarities and dissimilarities, but what I particularly like about this concept is the idea of “quality at a glance” provided in a visual format that can be easily absorbed and understood not by the project team, as in Jeff’s example, but by executive management.

Executive management is not involved in the day-to-day activity of the team, and yet when schedules get crunched and production dates are looming, they need to get up to speed quickly and absorb issues ASAP in order to make some tough decisions or take action.

In my company, questions asked are as follows:

1. How far along is the testing effort? Note that unlike Jeff’s example, executive management
does not especially care about “depth”; they assume that number is rolled into the overall
“how far along” number. But the underlying question we’re really being asked is “What
kind of shape are we in right now?”. We express that verbally in meetings. Some sort of
visual “Quality Meter” that has mutually acceptable “this is how we determine where we
are” logic behind it is an interesting idea that might work for us. Executive managers find
information such as “we’re half-way done, and what we’ve seen thus far is pretty buggy”
tells them what they need to know if there are only 3 days left in the test cycle and the
testing effort was scheduled for 20.

2. Based on what we’ve seen thus far, how many errors are likely to occur? (This number is
used to determine roughly how much more development work is going to be required, and
when/how many additional patches are going to be required. Nobody expects it to be
hewn in stone, but if the error rate for new stuff has been running at 40% throughout the
project, assuming it will continue to average 40% is at least somewhat better than pulling a
number out of your…ear).

3. What are the urgent defects and do they block any testing? If they block testing, how
much testing is blocked and is it user-critical functionality?

4. What defects must, from the business/user perspective, be fixed prior to production?

5. Considering 1-4, how long will it take to complete the testing effort? Note that this really
means “When will a) testing of all currently untested stuff be done b)fixes to errors found in
(a) made and turned over c)retests performed and d) everything be ready for
production?”. Executive management realizes that if we are testing ten minutes before
production, we can (and have) located more defects.

We produce a Stoplight report right now, which lists errors and highlights them in green, yellow, and red according to urgency. This does allow both the team and executive management to immediately focus on “red” items, but it never really answers the questions “where are we, overall, right now” and “what needs to happen to get where we need to be (what’s left to do and how long will it take)”. The Stoplight report is also an excel spreadsheet, attached to the normal status report, which means half the recipients don’t even open it up. For those more “old school”, it doesn’t print nicely – the colors don’t show, and it’s 37 pages long. As usual with highly detailed individuals, it’s also become mutated so that there are multiple shades of yellow and green. This means the colors have to be explained.

So what does this mean and why am I blogging about it? Because I think this is a common problem and because I feel inspired to tackle it by something I’ve read; I thought some of you might feel the same way. We’re going to take these ideas, our status report, and our stoplight report, armed with what we know we get asked for every migration, and brainstorm those ideas into one simple, easy-to-read non-excel format . It should be interesting, since the questions that get asked involve what has happened, what is happening, and what is likely to happen. Predictive numbers tend to make my staff break out in hives, so some bravery and a bottle of Calamine Lotion will be required. But what the heck – you have to be unspeakably brave to work in the testing field anyway….

When we’ve developed something we think is going to work for us, I’ll share it and let you know if it addressed any of our issues. If it doesn’t, I’ll tell you about that too….

In the meantime, if any of you have come up with something unspeakably clever that works for your shop, I hope you’ll share it as well. If nothing else, this blog will probably illustrate how we glom onto an idea, play around with it, and make it into something that works for our particular environment. I wouldn’t call my normal business environment agile and I wouldn’t call it structured either. It’s a combination that is primarily iterative and that works for this specific business. One of the things I like best about this company is that they could care less about the technique du jour. They never waste time or money jumping onto some bandwagon just because someone else said the shiny red one is the best. They’re willing to try anything, modify (whatever) and use pieces and parts of any idea, as long as it results in either time/cost savings, or makes something (quality, the user experience, etc.) better. It makes working here worth the effort.

Now this idea is just a small thing – a bagatelle – but it addresses an annoying issue that comes up for us on a regular basis, the solution is simple (Simplicity is the Ultimate Sophistication), and it gives us something new to play around with to make things just a touch better for our next go-round. We have a tendency (go figure) to be too detail-oriented and somewhat anal; I think it will be good for us to pull back and look at how to simplify what we’re communicating.

Kudos to Mr. Patton.