TESTING MYTH-TAKES - PART THREE

“IT ISN’T A BUG UNLESS IT BUGS SOMEONE”


My apologies to the originator is this quote; I’m willing to give credit where it is due if someone lets me know where this originated. It now (sadly) gets quoted all the time.

Let’s talk about this for while. It may end up being a question like “If a tree falls in the woods and no one hears it, does it make a sound?”.

I’m one of those people that says “Yes, sound is made whether something is around to hear it or not”.

Generally speaking, I’m not especially philosophical. It’s just a truth – neither good nor bad. I get impatient contemplating or debating stuff that either doesn’t have an answer or the answer doesn’t matter much in the Grand Scheme of Things.

But certain statements immediately set off some weird switch in my head; for lack of a better description, I’m going to have to refer to it as my internal “bullshit meter”. This statement hit it right away; my entire cranium cavity was reverberating with the equivalent of air raid sirens.

OK, let’s say I find an error. The nature of the error is that the color and size of a given icon is incorrect. Functionality is not affected. The user is willing to take the system as it is.

Is it a bug?

Oh yeah. It’s a bug all right. Unless the user comes back and says they PREFER it that way, all they’re doing is saying “I can live with this bug right now”.

OK. Say I find another bug. The problem is that an internal app that the user never sees is using up twice the amount of cache as normal. There’s plenty of space right now, the user doesn’t care about it, and fixing it is not a priority.

Is it a bug?

Oh yeah. If your company continues to make bad decisions of this kind and lets this type of bug go, eventually your app is going to tank. To quote a famous movie, “Maybe not today, and maybe not tomorrow, but soon….”.

Let’s take a look at a successful company with a relatively successful testing organization and a good history of giving their end users what they want.

This company reviewed every bug that was found during the testing period and included the end user in decisions as to whether to fix or not fix each bug. Since their testing periods were relatively brief, they normally fixed every urgent or serious bug, deferring less important bugs to future migrations. What actually happened, however, was those deferred bugs might or might not have ever been fixed, depending on their priority and severity and staff available. It was normal for development staff to ignore anything they didn’t personally feel was patchworthy.

The company in question might have fixed 250 bugs and deferred 45 “cosmetic” fixes. For every migration.

Over time, what happened was that certain areas of certain applications became “fragile”. That means there were so many little bugs everywhere that touching anything broke something else. There were areas of the code that developers hated to even look at. The users had given up on even reporting smaller errors, regardless of how much they “bugged” them, because they knew they would never get fixed. A few areas of the company had even committed their tribal knowledge of “how to get around” errors to procedures manuals. Imagine writing procedures manuals on work-arounds for bugs!

What eventually happens in such cases is that applications become so unstable they require a complete rewrite. But if the original habits never change - if nothing but urgent and critical bugs are “cared about”, the same situation is going to occur with the new code.

My point is here is that bugs don’t have to “bug” everyone. It’s a bug regardless of whether someone “cares” about it at that time. Have any of you ever worked for a company that regularly tries to “talk the user out of” a bug? I’ve worked for several. This means strong-willed individuals can actually change the behavior of less assertive individuals to the point they won’t even bother to tell them what “bugs” them. The normal scenario here is development dictating what the user wants or needs. Are you going to put your company’s future into the hands of a 28-year old developer who has as much in common with your users as a rock does to a porpoise? I’m not particularly moved if Binky BadCode thinks something isn’t important. Binky hasn’t been in the field long enough to know what is important or to have seen the fallout from bad decisions. I’ve had prod support managers in defect review meetings talk about deferring bugs that “aren’t important” and in almost the same breath complain about the number of bugs found in production. There’s a link between the two.

How about those of you who work in shops where if the user doesn’t report it, it isn’t a bug worth fixing? It doesn’t matter that by the time the user finds it, it will cost 200 times more to fix or may have corrupted something else. What a waste of time and money!

Proper care and treatment of bugs should be addressed in order to protect the viability and future health of your organization.

Can a bad situation like the above, where the apps become fragile over time be halted or reversed? Yes. The team can change their Evil Ways and set some quality standards (I can see a lot of people wince over the word “standards” from here…). Standards are not always bad and they can help turn around a problem. I’m not suggesting the same standards for every company. Maybe you want to state that you cannot break existing code and anything that breaks existing code must be fixed before production. Maybe you want to set an 80/20 rule for new projects – 80% of all defects have been fixed and the remaining 20% are non-critical. Maybe you want to say that all defects found in a migration are fixed within 3 months of production in severity order. A variety of defect management techniques can help; what solves your issues will depend on your environment.

Do I realize some shops don’t have the staff or time to fix everything? Yes, of course I do. I don’t live on Planet Bizarro. I understand the reasons, financial and otherwise. They may be valid for a given migration. Long-term, however, they’re a myth. A Very Expensive Myth. I like the concept of “technical debt”. Google it and see what you think.

Overall, what I’m saying here is that you’re in a TESTING field, dammit. Have you stopped caring about some bugs? Then you’ve lost your edge. If QA doesn’t push for improved/better quality, then there won’t ever BE improved/better quality. You HAVE to care and that means you have to push to get things fixed. Not in an obnoxious way and not to the level of becoming a bottleneck, but there has to be someone who champions excellence and most executive managers are expecting that someone to work in QA. One of the saddest things I see in our field is people who have become ineffective due to complacency or who have fed a downward spiral for their own company through sheer apathy. I’ve known analysts that don’t even write or execute tests any more for things they “know” development won’t care about.

Argh.

Listen, if there’s a bug in the tree and it falls in the woods…….

Well, you understand what I mean.