Do any of you read on-line discussions/blogs by James Bach? Oh, I do. He’s so deliciously arrogant, don’t you think? How do the rest of you feel about the “Contextual School”? I can’t make myself align with a “school” – particularly that one. I just can’t. Because just when I’m ready to concede that maybe it just might be possible, one of the self-proclaimed leaders of some “movement” will publish something so incredibly heinous that I can’t bring myself to do it. Can’t sign any manifestos, can’t bend a knee, can’t drink the Koolaid, can’t become one of the Stepford Wives. I just can’t. On occasion, I’ve equated the Contextual “movement” with “bowel movement”. Overall a good thing, but involving a great deal of ……..
So back to the Erudite and Highly Opinionated James Bach. Did any of you catch the on-line argument between James Bach and Jim Pensyl? James called Jim a “scripthead”. Oh man, I laughed until I think I broke something. Not content with that, he made a bunch of other insulting statements about the uselessness of detailed procedural test cases, blah, blah, blah… By the way, in terms of that original argument, I do feel obligated to point out that it is not only financial concerns that need to be SOX compliant. Any firm OWNED by a financial concern also needs to be SOX compliant. There are candy companies out there that need to be SOX compliant…there, that made me feel better.
Well, I might as well ‘fess up right here. My name is Linda Wilkinson and I’m….I’m….I’m a Scripthead.
I wasn’t always a Scripthead.
My own understanding and reasons for embracing written test cases are as follows:
1. I do not want my most talented analysts rerunning the same stuff over and over again. Nor do I want inexperienced or offshore testing personnel attempting to recreate what was already done brilliantly by someone else much closer to the project. That means my best people need to document what they did in sufficient detail that it can be turned over to a less experienced person to run, thus freeing them up for the next complex effort.
2. My experience is that regression testing finds between 3 and 5 percent of the total error in a given project. I’ve led or handled roughly 250 projects in all kinds of fields at this point. Unfortunately, that 3 to 5 percent is usually critical. That means existing functionality MUST be tested to ensure it still does what it is supposed to to. At the same time, I don’t WANT my people spending a lot of time thinking about it. The original analyst on the project thought about it. I want them to buzz through those tests and focus on NEW STUFF, which finds most of the error. So (and here’s the eye-popper!), I DO NOT WANT REGRESSION TESTING TO BE A PARTICULARLY THOUGHT-PROVOKING EXERCISE. I want it to be an easy no-brainer that we can pass around to whomever has the time, ship off-shore, or….
3. Automation really requires written test cases that have sufficient detail for a coder to produce desired results. You can provide them with less and you’ll get…..something, but you might find you’ve actually been testing X when you wanted to test Y.
4. Most companies for which I’ve worked have standard migration patterns (no, not like elk); such as 1 per month or 4 per year. If you never know how many tests you have, how long they take to run, etc., you won’t even be able to figure out regression testing numbers, let alone new stuff. Guessing is peachy, but analogous or parametric estimating (any PMs out there?) are better and more accurate. Management wants to know “how long will take, how many people, how much will it cost”. They want progress reports – usually in percentages complete. Those numbers should be easy to pull for regression testing, which should be a no-brainer. This frees you up to discuss that pesky new stuff that’s held up due to catastrophic error you found on the first day….
5. People leave, people get sick, people move on, up, or over. If you’re dependent on tribal knowledge (it’s in everyone’s head and passed on through word-of-mouth), then the company for which you work is at financial and quality risk. Are test cases expensive to produce? Yes. About an hour per; maybe two. But it provides for the future of that company, protects against loss of experienced personnel, allows for hiring of less expert resources, and feeds automation. Most companies for which I’ve worked have, in fact, ponied up the money when it is sold to them in that way. Any consultant that DOES NOT PRESENT that data to a given company, or encourages them to embrace a methodology that does not leave them with any intellectual property with which to continue the testing process WITHOUT EXPERTS is, in my opinion, criminally negligent. In fact, over time, I’ve come to regard such people as non-professionals and put them in the same category as I would a rodent, with my sincere apologies to lemmings everywhere. Anyone can “test”. Good testing and responsible documentation – professionalism - requires discipline as well as brilliance. And you have to care what happens to the company when you’re gone. Will they be better or worse off than when you started? Can they carry on?
So by now, you’re equating me with Tyranosaurus Testus, a dinosaur that can’t recognize or appreciate new techniques.
Au contraire, mon frere. First of all, exploratory testing is not a new technique. It’s been around for ….oh…..25 years. What is different (and what is good about the infamous “movement”) is recognizing realities and wrapping some decent strategies and legitimacy around the practice.
I do not assign my most experienced resources to regression testing. I train up new people by starting them on regression testing. It’s a good way to begin in the field and an equally good way to learn the applications under test. My staff do not attempt to write detailed test cases before a migration. They have an intelligent outline of tests they’d like to run based on
We’ve also modified JIT (Just-in-Time) testing methodology for our small projects and have experienced all kinds of Good Things as a result. We still, however, write up test cases at the conclusion of those projects, during our “down” time.
I do agree, however, that the type of test case that is more like a script (with every keystroke documented) is a situation where the juice isn’t worth the squeeze. While we do not have to spend a lot of money on maintenance right now, those types of test cases ARE very expensive to maintain. Our test cases require enough detail that they can be turned over to another tester, developer, or end user and be understood.
Finally, there ARE tests that are one-time only efforts and there are businesses out there that do not require significant regression test suites to stay successful. Normally these are companies that produce products with a limited “shelf-life”. The industries with which I’ve worked (finance/banking, insurance, state government, federal government, non-profit, retail, telecom, aviation) do not fall into that category, with the exception of one-time mass conversion projects. In the case of applications with limited shelf life, intelligent outlines would meet most documentation and auditing requirements without investing significant dollars into an effort that would be discarded after a year or two.
So, what about you? Are you a closet Scripthead? Walk towards the light, Carrie Ann….. The Contextual School may revoke your “Contextually Cool” t-shirt, but there are advantages to recognizing the value of “testing artifacts” and being willing to use them to get the job done…
Posts
0 comments:
Post a Comment