TESTERS REPEAT AFTER ME – “I AM NOT AN END USER!!”

OK, raise your right hand. Now boldly proclaim to the world:

“I am not an end user!”.

Most testing personnel identify strongly with the end user. It’s a fine thing to care about your clients, as long as you don’t allow that allegiance to adversely impact your Real Job, and through your attitude and beliefs give your coworkers and management the mistaken impression that you represent the end user.

You don’t.

It is somewhat unusual for a tester to spend much time talking to an end user, let alone do their jobs day in and day out. This means end users are going to use the system in ways you do not anticipate, no matter how long you spend interviewing or shadowing them. And is it your job to conduct those interviews, ensuring their requirements are incorporated into the system design?

I sincerely hope not.

While it is “nice” to test the system “just like a user”, is it really efficient?

No.

YOUR job is to drive out error, whether that manifests or is of immediate concern to the user or not. That means combining of conditions and use of test data that would likely not even occur to an end user. And you have to be extremely aware of time; chances are that you’ll set up and execute your tests for maximum utilization of the test time available. For example, consider load testing. The impacts of overloading the system might not impact your user base for 5 months. It is also unlikely your end users will methodically test every aspect of an application looking for errors. They will use the system as they would in their daily working environment. So while you may examine features A-Z, your primary users might only use “D”, with 3 other windows up and some other mystery app running in the background. Oh, they may use those other features occasionally (or eventually), but you get the idea….

Remember, by the way, that if you’ve conditioned your peers and management to believe you test like an end user, they are going to believe you’ve failed in your mission when errors are found in production. Since it’s “normal”, even with outstanding testing efforts, to have an additional 10% error manifest in production, you may be shooting yourself in the foot. Good testing efforts include unit testing by development, structured testing by you, and user testing performed by actual users. That way, when error is found in production, it was missed by 3 groups with 3 different perspectives.

By the way, “testing like a user” is not something I’d want to brag about. There are multiple studies out there and I believe one actually found user testing discovered 18% of the total error in a given app, but after 25 years and more than 250 projects, my finding are that user testing finds 10% or less of the total error resident in a given application. Do you still want to test like a user?

Often, too strong of a user-tester bond will actually cause a tester to be a POOR customer advocate. “How can this be?”, you ask….

An end user may not recognize the severity and overall impact of a given error, particularly long-term. Or they may be willing to “live with it” for now. This can cause a tester to lose their objectivity and not push for fixing relatively serious errors. In other words, the tester may get caught up in the ongoing negotiation of “what matters”, even going so far as to either assist other team members to “talk a user out of” considering an issue serious, or conversely convincing other team members and/or management that a given serious error is OK.

I’m not saying such negotiation should never take place; I’m saying a tester’s job should be finding and reporting error, with a strong bias towards getting errors fixed. Negotiation of what should or should not be fixed based on user opinion is best determined by the entire project team or a team dedicated to defect triage. Further, a good severity/priority process for defects can take the emotionalism and guesswork out of the equation.

Another consideration is that when an end user says they can live with an error or workaround, they probably mean “for now” and not “for 3 months” or “forever”. Whenever a user states they can live with an error, they should be asked if they can live with it for 3-6 months. If the answer is a horrified no, it means at best you’re talking about a post-production patch and at worst, you’re impacting development’s ability to complete work for the next migration.

So step back from the ledge and retain your objectivity and focus. And raise your right hand and repeat after me….

“I am not an end user!”