LEARNING TO TEST

Enjoying the dark side of the force for fun and profit….

I’ve been surprised lately by the number of blogs/articles out there talking about learning how to test. There are SO MANY books and classes out there on basic topics like this that I really felt that I wouldn’t ever be posting on basic topics here. It was my intent to focus on QA management issues, since there’s less material available with that type of focus.

Just like the rest of you, however, whenever I read something about a topic I’m passionate about, it inspires me to put in my two cents.

I’m not going to comment much about Michael Bolton’s blog about the similarities of learning to drive and learning to test (find it on stickyminds.com). First of all, it was obvious to me the memories associated with learning to drive had some personal significance to him, which means a bit of sensitivity wouldn’t come amiss. I understood some of the points he was trying to make and agreed with them, if not, perhaps, in that context. I did disagree with his conclusion. The concluding thought was “You wouldn’t want someone new driving with someone else’s script, would you?”. Yes, I definitely would, but that’s because I equate a script more to a GPS system or map, rather than pamphlets on rules of the road or training/mentoring.

So I was thinking about how I learned to test and how I teach testing to others. Here are my thoughts. I’m not going to try to equate testing to learning to cook, astrology, quantum physics, or plumbing, although that might be fun sometime.

I started in the field in systems analysis. I caught the attention of the QA Manager when I worked in a coordinator position between development and QA on a major project. When I turned in my notice, my boss begged me to go anywhere but QA. Back then, QA reviewed code and was one of the most hated organizations in the company. They’d go back to your lead developer and most valuable resource and tell him his code was inefficient and that they could rewrite 12 lines of his code in 3. Large projects would bog down in QA for months. No wonder they were despised.

Shortly after I joined up, they went through a major change in focus. It had become obvious to executive management that the end user really didn’t care what language a system was written in, how many lines of code it took to write it, etc. So the testing became user/specification oriented, rather than code-based.

The first day I went into the office, there was stack of documentation on my new desk about a foot high. Those, I was informed, were my new projects.

I was 23 and, as I look back fondly at the beginnings of my career, can reliably report that I was pretty much unable to find my own butt with both hands.

I was assigned a mentor – a good one. The entire QA team, which was very tight and supportive of each other, pretty much “adopted” me. I haven’t seen that change much over the years, by the way. It’s a highly dysfunctional QA/QC group that doesn’t take care of its own.

The first “task” I was given was (you guessed it) running regression test cases for someone else’s project. I realize some people think that is a heinous way to start. But it wasn’t. First of all, I learned the application under test by following the guidelines set down by someone who was an expert on that system – the author of the test cases. Second, I learned how I was expected to format my own test cases. Third, I learned what kind of things an expert tester looked for by running those test cases. And fourth (and probably most important to my employers), I was immediately useful and contributing to the team.

As this was going on, I was also given instruction on testing technique and reading specifications for the first project on my stack. The first set of tests I wrote were peer-reviewed by my mentor, who, I might add, wrote all over them in bright red ink, pointing out what I had forgotten or tests I had missed. That first effort was a sea of red ink.

I was pretty bummed out that first time through. At the same time, I was told what a great job I’d done for a first effort and what a great addition I was to the team. That was the first and last time for that process, by the way. You were put through the wringer one time, and were thereafter expected to understand what was required and to do it.

That meant all the rest was up to me. I went through all of that red ink and learned to ask questions; of my mentors, BAs, and developers.

But more than that, I fell in love with testing. I was obsessed with testing. I tested EVERYTHING. During lunch time, I’d pull up the word processor and test it. I’d test the time reporting system. I spent literally weeks of my own time figuring out how to break things. I drove them all crazy; pretty soon I was finding errors in everything I touched. I was lucky enough to work for a company that believed strongly in education and fostering new talent; I believe I probably attended at least 24 classes and seminars while I was an employee there. That testing “love affair” never ended; after all this time, I still love to test. I’ve been a manager for some time now, so it’s no longer my “job”. But it’s still my avocation.

When I look back at it, I was pretty lucky. It’s fine to talk about mentors and training, but I have to say I’m in the minority, not the majority. I’ve met very few testers that have ever had any formal training, let alone a mentor. Anything they’ve learned, they’ve had to learn on their own.

So how do I teach testing? Well, my current company has highly complex, interrelated applications on a multi-tiered system and operates 24X7, with branches all over the world and staff with immediate access to some critical applications via Blackberry. We have unique security considerations. We also directly interface with a number of vendors. Our application systems are not “like” other systems, in that I can’t easily go out and find resources that have a background that is similar. So everyone pretty much “starts from scratch” in terms of knowledge base.

Our first step is an overview of our applications from a high level; we spend about 3 days on that. New resources are immediately given access to our regression test base and are expected to execute those that impact their assignments to get familiar with specifics for the application and our test case format. Low-level staff (new to testing) will be executing existing tests for a while – it provides value to the company while they’re learning.

Everyone, experienced or not, goes through a bootcamp, which takes about 3 days. The bootcamp reviews both QA and QC, with specific emphasis on how things are set up in our QA organization, trains everyone in basic test technique (both structured and agile), and reviews how automation is used in our shop. The bootcamp includes a testing “test”, which we review in order to determine who “gets” it and who needs more help. Those that need more help are given a mentor to work more closely with them.

The bootcamp ensures everyone understands our terminology, understands the basics in terms of technique, knows what is expected from them, and the overall flow of work through our group.

From there, staff move on to their individual assignments. Everyone has their work reviewed the first time they produce deliverables for us. New people who are also new to the testing field execute regression test cases for several migrations. They are then eased into small projects that are somewhat related to the test cases they’ve become familiar with, followed by projects of increasing size and complexity. New people who are experienced will start out with small projects and progress more quickly. Note: we do not normally produce test cases prior to a testing effort. We work from a structured outline – a kind of hierarchical, organized “list” of what we want to examine. Test cases are written after the testing effort and become part of our regression test case base.

We are also self-regulating in that we examine every bug found in production for several weeks after a migration and try to figure out how we missed it. We’re not a zero defect shop, so we expect some errors (right now it’s about 4%); no one is ever “punished” for missing something. If someone misses significant error repeatedly over multiple releases, however, we’re going to first try to mentor the shit out of them. If that doesn’t work, we’re going to bump back their responsibilities until we find the “sweet spot”; something that resource can handle effectively.

One thing I’ve found is that all the training in the world doesn’t give someone a passion for testing. That’s one thing they have to have on their own. We can give them the tools and the training; they’re the ones that have to take it from there.

And not everyone is good at testing. I’ve known many talented, earnest, hard-working individuals that NEVER “get” it, no matter how hard you (and they) try. So what do we do in those cases? To be honest, everyone in my group has to be competent; we “manage them out” if they can’t handle the work. We always try mentoring first; it’s expensive to find and train someone new; but everyone on the team has to pull their own weight. That said, there’s a world of difference between “competent” and “awesome”. I have some of both; most managers do. If I could instill my own love of testing in everyone, I would, but it just doesn’t work that way. I put my competent people on projects that require competence (which is the majority of small projects), and save my awesome people for projects that we know are going to require some awesome testing skills. Chances are also good that those “merely competent” people are going to be awesome at something else that needs to be done in the group – I talked about this a bit in my blog about building a team.

By the way, have you ever noticed that your awesome testing personnel are the ones that pretty much started that way out the gate? Some people just show a definite, rare instinct in regards to finding problems that training and mentoring just enhances. They grab whatever you give them and take off with it. These are the folks with the real passion for the field; I suspect that many of the testing bloggers out there fall into this category.

There are several individuals in the QA/QC field that focus specifically on figuring out what makes a great tester great – WHY are they so good? HOW do their minds work? WHAT do they do differently? I have to leave that weighty research for others; perhaps one day that type of scholarship will help us provide better training or identification of talent across the board.