TESTING MYTH-TAKES

"EXPLORATORY TESTING IS THE OPPOSITE OF SCRIPTED TESTING"

The statement “exploratory testing is the opposite of scripted testing” is like saying “thinking is the opposite of thinking”.

Would someone please explain to me why there is any difference whatsoever in the thinking/analysis process involved in formulating tests whether you have a screen mockup in a spec or a screen mockup in a prototype? Does intelligence magically get lost if you write your thoughts down in a test case or structured outline before you test?

Am I the only person in the field that thinks “What is WRONG WITH THESE PEOPLE????”?

So you need to have a system to actually explore before you know what you want you explore? Pardon me, but that’s complete excrement from a large male bovine. If someone drew me a screen on a bar napkin, I’d already be formulating tests and figuring out what I wanted to look at and finding potential holes. Do you think I’d lose anything in the process by writing those thoughts down in advance? No. If my tests are smart when I’m thinking them, they’ll be smart on paper too.

What is it, exactly, that exploratory test proponents are afraid of? And make no mistake, I truly do believe it’s fear. It IS possible to script out tests prior to looking at a single screen. Could it be some don’t have the training, background, or imagination to think something through without actually exercising it? Just a dislike of documentation?

A creative and intelligent tester is a creative and intelligent tester regardless of whether or not they write up their tests in advance. There are actually advantages to writing up tests in advance. First, you don’t forget or “lose” that thought. Second, if you get pulled onto something else, your successor has something to “go on” without going through the same level of analysis you did – there may not be time for that. In my opinion you have more time to THINK before the madness of the testing phase is upon you. Logic and experience tells us that most us don’t do our best work when we’re rushed. And testing sessions are always rushed….

There’s also a silly myth that once you have written test cases, you become an ignorant robot that just blindly executes what is on the written page. My message to those who think that way is to get a clue. Good testers are adaptable. They’re curious. Of course they’re going to execute what their devious little minds thought of before the actual testing. And as they actually get into the system, they’re going to modify some tests, toss some tests, and add tests.

Whenever someone in our field tells me they can’t test without looking at the app, I pretty much think they’re either average or below average in terms of skill set. It’s unlikely someone who lacks the creativity and skill set to design tests in advance is going to magically become creative and knowledgeable once an app is in front of them. Mostly I just think they’re lazy and/or lack experience. It’s hard to know whether you’ve arrived at your destination without a roadmap. The journey itself might be fun as long as you don’t care if you ever reach your destination…

I read some analogy regarding testing as applied to golf. The object was to estimate how many strokes it would take to complete a given hole. The “lesson” was that you’d give a better estimate if you could see and walk the course, making observations.

I understand the point, but overall I thought the analogy was poor. Wouldn’t a schematic of the course tell you enough about the layout to draw at least SOME conclusions? Many questions wouldn’t require “walking the course”. Who will be playing? Men, women, children? What club will they be using? What will the skill levels be? What about weather conditions? Etc. All of those questions or “tests” could be scoped out in advance.

I think the real fear with “scripted” testing has nothing whatsoever to do with test creation. The process of test creation is the same regardless of whether or not the tests are written down or detailed. I think the real fear is monotonous execution of the same pack of written tests. I’ll talk about that in a future blog about building test teams. My only comment is that only those who find that task a monotonous anathema fear running an established test case base. But these are people that wouldn’t enjoy testing the same app over and over again REGARDLESS of whether the tests were written or not written. Any repetitive task can be boring to a creative person. That doesn’t mean, however, that regression testing is bogus. It means you have to have an intelligent strategy for performing it. More on that later….