DOOM, DESPAIR, AND AGONY ON ME....

Doom, Despair, and Agony on me -
Deep Down Depression, Excessive Misery!
If it weren't for Bad Luck,
I'd have no luck at all....
Doom, Despair, and Agony on me!

- Hee Haw

Long Time No Blog. Miss me? Hope you can sense I'm smiling; I can hear some of your comments from here...

On occasion, Life interferes with Art. I had a very serious medical issue - the past 4 months have pretty much been filled with fear, pain, and a lot of Yucky Stuff I'll spare all of you. Having never had surgery for as much as a wart before, it has been Quite the Experience.

I'm happy to say I've only missed a total of 10 days of work, and even that I spent "plugged in" via BB (Blackberry), but I haven't had the time or energy for much else.

Like many people in my position, having a medical issue means the Vultures Have Been Circling and besides facing the reality of my own mortality, I have a new appreciation for the depth of callousness of that permeates some Corporate Mindsets. I'd heard stories, of course, but had never experienced it first hand. It has been an experience I could have happily done without.

But I've survived. A bit battered and considerably more jaded, but I've survived. And I'm fine.

The experience has changed me somewhat, however. First of all, the friendship, support, and caring of perfect strangers has blown me away. On the other hand, some people I cared about and trusted acted like absolute pigs. It was confusing, to say the least.

I did keep up with reading blogs, etc., but found a lot of what I've read has been same-old, same-old and generally uninspiring. The same self-promoting groups and people keep self-promoting. Blogs are written about horeses that were flogged to death years ago. I spent some serious time contemplating whether it was worth my time to be involved any more, and what I really care about.

Well, overall, I find I haven't changed THAT much. I care, in essence, about testing. Not politics or affiliations. I still find that people who care more about themselves than their contributions to the field, their coworkers, peers, or staff are an offense to the nostrils. I'm still bored (and occasionally depressed) by people who jump on bandwagons with no experience or understanding of what they're touting; they just want to be on the "good list"; it's a whole lot safer.

I read the article on the ten top women testers during this time and all I could think was that we women needed to work a whole lot harder. All had contributions that were worthy of note, but tell me the truth - how many names did you recognize? And how many were TESTERS? If I had written that article about men, the hardest part of the piece would be picking only ten. And you'd recognize their names. I thought it was a sad statement on our field. The best woman tester I've ever known is someone you wouldn't know either. But I've known thousands of testers, and she's the best I've ever worked with; she's scary and her skill sets border on magic. And she's a tester; not an author, speaker, or manager. Her name is Carrie Smith. So to all the Carrie Smiths out there, a tip of the hat, a curtsey, and an "I'm not worthy" under the breath. Because we AREN'T worthy if we can't be honest, leave our prejudices and affiliations out it, and offer everyone something we know, from the depths of our hearts, to be the truth as we know it.

So, overall, is it worth it, really worth it, to bother to try any more? To fling my professional, personal, and occasionally irreverant thoughts out into the blogosphere?

Well, why not? I've never tracked my readers - there might be 2 and there might be 200,000. I write like I'm talking to a friend; I write plenty of impressive reports like PhD dissertations and it's not much fun; blogging should be fun.

I figure some of you MUST be like me. Years of slogging in the trenches. Some of you MUST be practical people in charge of QA, QC, or testing, surviving the corporate jungle on a daily basis and doing good things for your company and your people.

So I'm back, and I'm re-dedicating my blog to all of the Real Heroes out there, doing your job and kicking ass, day after day.

And to start things off for what is MY new year, here's a problem I'm dealing with right now and what I'm doing to solve it. Maybe you've faced the same problem and have come up with other solutions - I'm hereby inviting you to tell us about it.

When I started here 5 years ago, there were 365 test cases for a ginormous system. The tests sucked and most were useless. The error rate in production was 49%. No one (staff of 4) had any test training and they were working (and being treated) like dogs.

So we trained people. We got more people. We worked hard on analysis and coverage. We have respect.

There is no impact analysis here and even the programmers have no idea how their code affects other code. And the system is highly inter-related. So the most obvious way to cut error rates was through coverage. Our error rate in production is now under 4%.

So far, so good.

But with maturity has come other problems. We now have over 17,000 test cases. While my staff has grown, we aren't a Microsoft shop, and we can't afford to constantly be adding staff to handle the ever-expanding regression test case base. And that base doesn't find the bulk of the errors - tests for the "new stuff" finds the bulk of the errors. But due to the nature of our business, which involves human life, it's critically important to ensure existing functionality still functions. Because everything is so inter-related, a single error in one function can rock 22 other functions and render our systems useless.

At the same time, it's obvious we need to pull back and be smarter about what tests get run during a given test period. My company simply cannot sustain the personnel growth associated with running every test every time.

Add into that mix an offshore team of ten,, who need stepped-out test cases to operate effectively. It takes one of my analysts an hour to step out a test case.

So what does the solution look like? Well, if you have a batch of regression test cases you've either written or run,, don't you find you pretty much know what the test does by the title? We do. So we added a field to the test case, went through the titles, which was fast, and ranked each test according to criticality. Not rocket science, right? A kind of risk-based endeavor. For a ranking of one, if the test case failed it means the business user would be unable to work. For a two, the business user would be strongly inconvenienced and want a patch. For a 3, an error would be minor and in all likelihood would be deferred to a later release or patch.

We've discussed it with upper management, and we're going to drop the regression tests ranked as a "3" for this release. We'll closely track error levels in production and report back results using our standard metrics. Yes, we have standard metrics. They come in handy for improvement exercises like this.

If there's little or no difference, which is what we expect, AND our business users are satisfied, we'll whittle back the 2s. We'll do that cooperatively with the business users and development. We're also gong to check the 2s against our defect log to see how many errors were found in the past few years with those tests. This will help ensure we don't 'retire" any tests touching code that has been historically problematic.

Any of the tests we "retire" for a given migration may be acive for the next - it depends on what is being changed. And the development/business users folks will see and approve the list.

All "new" stuff will be tested as usual.

What I'm working towards here is changing the nature of the work so that all tests are analyzed and intelligent decisions made as to what needs to be covered for each migration. Not every member of my staff is that evolved. We're going to grow into it.

Eventually, I want a "robust" smoke test to replace the ginormous regression test case base. Let me explain what I mean. If my staff have to test an OS upgrade, we can do that in 4-5 days. If we're testing a standard migration, it takes 5-6 weeks. In essence, my staff "know", through experience, what is and is not "important". We're formalizing that knowledge through our rankings and gradually reducing the test bank until we hit the sweet spot. That is, just enough tests (regression) to get the job done without impacting our operational excellence.

We're going to eventually end up with an intelligent tour of a given piece of functionality, complete with notes and comments about where/how functions interact. My apologies to founders and proponents of the "tour" concepts. We're backing into it by degrees.

You know the old saying that it takes longer to turn a cruise ship than a small speedboat? Well, that's true. I'm estimating it will take us a year - I have to prove each step hasn't negatively impacted production, change my staff's modus operandi, and get the rest of the company on board with us, to say nothing of keeping upper management happy.

I believe, however, the results will be the ability to handle more "new stuff" with current staff, and to keep the need for additional staff just to handle the ever-increasing regression test base to a minimum. Just that much alone would save us around 400K per year. Chicken feed to some of you, I know. But it's a major Chunk of Change to us.

In addition, I think the majority of my staff will enjoy the additional analysis work and responsibility, particularly if it means cutting back repetitive tests that bore the snot out of them anyway.

So that's the issue and our solution in a nutshell - if you have other/better ideas, feel free to drop me a line and I'll certainly give you the airspace!

My next blog is going to talk about cost/benefit analysis for automation. This is something I've done many times for many companies (and the final recommendations aren't always what you think), but I'm going to talk about where we are in my present situation and the questions we're being asked by upper management.

And it's nice to be back. Now that I think of it, it's nice to anywhere!