THE ACCIDENTAL TOURIST

I’m going to confess right here that I’ve had nothing but a healthy contempt for the idea of “tours” for software testing for quite some time.

First of all, going into a test session cold is, in my opinion, stupid beyond words to begin with; something I never do unless I have no other choice and have communicated the risks to management (and therefore bought myself a Get out of Jail Free card). I’ve talked about the difference between testing code and exercising code before. If you don’t know what results are expected, you don’t have a clue as to whether what you’re experimenting with is right or wrong. Some results might give you a pretty good idea, but many results will just be a crapshoot and whether you consider them interesting enough to write up or not will be based on your own personal opinion and your knowledge base. In other words, you’ll be guessing. Some of you are probably pretty good guessers. Some of you probably suck. And in my experience, people who really stink don’t know how bad they are. Every tester I’ve ever met has thought they were great at their job, even if when they only thing they were good at testing was your patience.

So the idea of being a software testing “tourist”, wandering about exploring, maybe tripping over a bug through sheer luck, was a Really Bad Analogy for the type of testing I advocate. I believe testing is an intelligent, analytical process that involves choosing what to examine and having some understanding as to what results are expected. I (personally) like to think about these things in advance of the actual testing and then expand those ideas once I actually have my grubby little hands on the app. And yes, I’m intuitive and curious and need opportunities to indulge in that type of creativity extemporaneously as part of the process.

I travel that way too. I read everything I can about where I’m going to go, decide what I really want to see and do, and set those things up in advance. Once I arrive somewhere, I have a loose itinerary – and I do keep it loose, because I don’t like to be boxed in if something else more interesting comes up when I’m actually at my destination. But my itinerary “forces” me to make some choices sometimes – which is more important? Seeing the Alamo or taking a tour of the old missions? Can I do both? If not, which is more important? My itinerary also keep me on track. For example, if I continue to sit here and watch the flair bartenders at Harrah's, I'm going to miss the show I paid for at Caesar's. Those decisions are similar to what we do as we're testing every day.

All of the information available on software testing “tours” was really, really vague. And I have no use for something totally vague as a teaching tool. Nor can I hand a “tour” to someone and say “run this tour”. Each tour was just a vague bunch of stuff to think about. I’ve found that new testers (and even old dogs in search of new tricks) need something more explicit and better examples. Everyone who works as a tester is NOT a deep thinker, NOT analytical, and maybe are simply average. There’s nothing wrong with being average. Most people are average. If a “tour” can’t help people who are average (the majority) at their jobs, or is only useful to people who are testing superstars to begin with, then they aren’t useful at all. Interesting, maybe, but not useful. You really can’t force people to be thinkers. A lot of people aren’t wired that way. Should they be? Well, how snobbish are we going to be? Demand that our field, alone of all fields out there, only hire and recognize people who are outstanding? That’s a nice thought, but it’s not realistic. I know I’m not the only manager that knows that’s not The Way Things Are In The Real World. Most of us have to deal with a variety of different people, at all different levels of skill set. The types of learning tools I need are those that can help all levels of testers do better work.

So overall, I thought “tours” were just another group of “hoo-haw” ideas by impractical (but brilliant) people who didn’t have a clue as to reality in terms of normal testing resources in the field, and who were in imminent danger of falling out of their ivory towers and hurting Important Parts of their Anatomies.

All of this is buildup for my next statement –

I THINK I WAS WRONG.

That’s right, I think I was wrong. First of all, training testers has thus far not been the easiest job in the world, primarily because it’s difficult to teach someone to not just think, but to question and analyze.

That said, it’s really easy to teach someone boundary testing. There’s never been a need for heuristics, fancy mnemonics or anything else for boundary testing. It’s easy to teach, easy to learn, easy to remember, and easy to apply. Screen testing is also easy to teach. That’s pretty much anything that is an “attribute” of an object or involves visual inspection/operation of a screen. Icons, colors, sizes, locations, layout, validating links, etc., are also easy to learn, understand, and execute.

Where things get complex goes beyond the mere field or object level. It’s how those field and objects interact with each other, get stored and retrieved, behave under different environment/browser conditions, and are used by other apps. Complexity is added by stuff that runs in the background, or periodically. Or how your end users actually use the app. And quality considerations that are more difficult to measure, like performance, reliability, maintainability, etc. It’s also a challenge to teach someone the fine art of ferreting out bugs (error guessing). Again, all of these areas customarily require someone to analyze, think, and question.

At this moment, “tours” are too vague to be useful to me right out of the box. But I’ve been looking for better ways to teach “the hard stuff” for quite some time and the ideas behind “tours” and recent work done to refine those ideas is starting to look like something that could, with enough attention, become revolutionary in our field. How long has it been, really, since we’ve had “new” test techniques? Almost everything we’ve had up to this point has just been a rehash (renamed of course) of stuff that’s been around for years. Like 25 years.

But tours are, or at least have the potential to be, different. I didn’t think so until I started reading James Whittaker’s series of blogs (“tour of the month”). He’s changed my mind about tours, and that’s quite an accomplishment. I think the real reason I’m starting to take an interest is that he is explaining in more detail the thoughts behind each tour. What I can see is the beginnings of something really exciting; a way to teach “the hard stuff”.

Take for example, his latest entry (“the landmark tour”). That blog is imminently sane and makes perfect sense. We don’t have a sales team here that demos software. Our B.A. staff demo software. But the basic idea, of working with someone that is going to perform a demo to determine what is considered “most critical” to the end user, establishing “landmarks”, is primo. I can take those basic ideas, turn them into a kind of list of information that would typically be useful, add examples, and make it “easy” for any of my staff to comprehend, learn, and execute.

Some of the info in various tours is too vague to be useful. The “intellectual tour” would not be useful unless I could provide a list of “hard questions to ask the software” to use as an example. Just saying you should ask hard questions is not enough. Many staff don’t know what the hard questions are. They need more help than that in order to learn how to do that and apply the ideas.

But overall, I’ve become enthused about the potential of tours and am voraciously reading everything that comes my way about them. I’m going to start working with my leads to see what we can do with the concept here.

That said, I’m now going to address terminology and selling of the concept overall. I cannot sell the concept of “tours”. It sounds frivolous. It becomes even more so when you have “Super-model tours”, “Obsessive-compulsive tours”, and so forth. I report to a VP/CTO and he reports to a Senior Executive VP (who is also the CIO). Anything that sounds frivolous and unprofessional isn’t even going to get a hearing. And I find this is one of the biggest criticisms I have of new ideas in our field.

There are a myriad of people you need to “sell” on a new idea. I think having Fun names for serious work is geared towards appealing to young practioners in the field. The problem with that is that you may get plenty of practioners that want to use your ideas/techniques in the field, but they can’t get anyone to buy off on them, and it adds to both their frustration and the problem of being taken seriously by their management.

I think it would be a huge step forward for our field if we focused in on the desired end result. We not only need and want new techniques in the field, we need them to be something we can present and sell to executives who are very serious indeed about their work and their money. I can’t sell “tours”. And when you think about it, the tour is the actual activity. It’s the guidebook, etc. that helps you decide what you want to tour in the first place. The idea of a “guide” I can sell. There are other more professional-sounding ideas I could sell. But talking about testing guides won’t be as much fun as “super-model tours”.

So take this as a kind of challenge. Can we, as a community, sacrifice fun for acceptance/funding? Can we achieve both in some way?

All I know is that I want to continue to work with the idea of “tours”, and that unless the field as a whole comes up with some way to present these concepts professionally to those who pay the bills, I’m going to have to rename everything myself, for the business biases in this region and in order to get them sold. So is there a problem with that? No. But it doesn’t advance those ideas overall in the QA/QC field and just adds to the terminology conundrum. Everyone, whether they like or use the technique themselves, recognizes what the term “boundary testing” means. It would be nice if we could come up with new techniques that were ALSO as widely recognized and accepted. That means we would have to have the same words for the same things.

Along with our technical, professional growth as a community, couldn’t we add some business maturity? I believe it would move us forward much more quickly….