Comments on PRACTICAL QA: DOOM, DESPAIR, AND AGONY ON ME....

Linda, Welcome back and glad you are feeling bett...

2010-04-19T18:07:47.265-04:00

Linda,

Welcome back and glad you are feeling better. Being a "Veteran of the Testing Trenches" myself I can relate to what you say.

I think what you stated with this "Just that much alone would save us around 400K per year." is the key to your future success. When talking with upper management if you can relate a dollar value to the value of what you are doing you will definitely get their attention. You are working towards an efficiency and effectiveness gain that has monitary repercussions. Not that you are looking to cut costs or testing effort, but better focus testing and because of that you are "containing" that cost associated to it.

And I totally agree with your ranking idea/system. I've done that on projects to get things distilled down so I can have an effective Smoke Test set. Then using the ranking to run other tests as needed according to impact/risk.

Best to you.

Jim Hazen

Hi Linda, Welcome back and I wish you continued he...

2010-03-31T16:42:48.235-04:00

Hi Linda, Welcome back and I wish you continued health.

I discovered your blog just recently. I have read every post to date and have thoroughly enjoyed your views, you are insightful, refreshing and witty. I found myself chuckling often.

One suggestion is a form of Regression testing which is “Stress testing”. It’s nothing earth shattering, goes by other names I am sure, and it may already be a part of your #1 tests already.

Since your Regression tests are for existing features you have previously characterized and don’t expect to be affected, running the most stressful tests first, the ones that really push the limits of the system, is a good way to perform a fast sanity test. Stress tests should always Pass. Chances are good even small changes in code or performance will ripple through the system and allow you to identify it quickly with a "Stress Test". If it fails you can then explore further with other Regression tests in the same functional category.

Paul

Welcome back, Linda. Are you able to report depen...

2010-03-30T12:29:15.913-04:00

Welcome back, Linda.

Are you able to report dependencies from your code base? You could reduce the number of tests run by executing only against changed code and its dependents.

Hi Linda, glad you are well. A lot of us missed yo...

2010-03-29T07:28:02.289-04:00

Hi Linda, glad you are well. A lot of us missed your wit and wisdom.

Great to have you back in the land of the living L...

2010-03-13T03:23:51.739-05:00

Great to have you back in the land of the living Linda. Not seeing your blog among the various ruminations on testing in recent months was akin to not having the tabasco in a bloody mary. Pleasant, some people may even like it that way, but not enough to make you shout yee-haw and reach for another!

Keep 'em coming.

Shane

Shane

Hi Linda, In a way I'm glad that the silence...

2010-03-06T08:31:27.188-05:00

Hi Linda,

In a way I'm glad that the silence was due to illness rather than deciding to not blog any more, though I'm sad to hear you were unwell.

Our little testing community would be very hollow without you.

Glad your back.

Anne-Marie

Glad to see you back on the scene. Rob..

2010-03-05T09:05:04.841-05:00

Glad to see you back on the scene.

Rob..

Instead of a smoke test why not just run your enti...

2010-03-05T01:45:19.661-05:00

Instead of a smoke test why not just run your entire suite in parrallel? Reduced time to full feedback.

Just a thought ;)

Frankly I was looking up Google Reader every few d...

2010-03-04T00:04:18.895-05:00

Frankly I was looking up Google Reader every few days and finding no unreads made me wonder. I was glad to finally see that blackened "(1)" and gladder when I read through...

There is a chutzpah about your writing and strong truthful emotion that pours through which makes me nod my head in agreement and smile at the wicked wit interspersed...

Welcome! Wish you good health.

P.S. If Yoga & Pranayama has not inspired you yet, may it!

I've enjoyed the honesty and wit of your blog,...

2010-03-03T23:24:55.664-05:00

I've enjoyed the honesty and wit of your blog, very refreshing compared to the rather bland approach many people have on other professional blogs.

To your questions, I used to work on large pieces of shrinkwrap software so allow me to brain dump some other ideas I've had over the years (some of these are theoretical, not proven :-):

For the entire department:
Fold new stuff into bigger releases spaced further apart. Its always the case that final releases (with to CD or production) have an expense – with large/complex this can be very large. Do less of them, the total test cost over several releases goes down. I know an intelligent 7 year old could understand this concept but sometimes its worth restating as ppl forget the simple stuff in drive to get new features out – especially as it concerns testing. Sure, a release with more stuff in it will take longer to test, but not in a linear fashion.

Start suggesting that the app should be analyzed for stuff to pull out. Are all the features REALLY used? Again, as an app grows, entropy and costs go up, no way around it. Pruning is a viable approach often overlooked. With big complex applications... an entropic event horizon exists where its too expensive or time consuming to actually get to release quality, but again, execs often seem far too concerned on next quarter, not this mythical point of implosion (suddenly though... “well, we have to rewrite the entire application.....”. See you in five years....)

If there are particularly unstable or badly engineered areas of the app, get statistics around this and try and persuade the eng team to get a good developer to go in and rewrite it.

Hire a white box test programmer – someone really sharp and who will 'get' the workflows/use cases, who can start building up some sort of unit test framework or harness that tests the business logic and data flow across modules. Run this all time, and have the errors on each build emailed to the dev team so they can see when they are breaking stuff. (this is like TDD on agile webapp teams)

For the test team:
Allow testers to take turns retesting the old stuff. Really good testers are generally going to get very bored testing the old stuff. If possible lets them go a year before having to test the same stuff again so they have the 'hunger to break it' again.

Bring in sharp interns and anyone else talented who doesn't mind being cheap but will do a good job going through old features. In this economy, there should be plenty out there. The games industry is full of very underpaid testers and I've often thought that would be a good place to recruit testers into business software.

Use orthogonal array analysis to help create powerful est cases and maybe err automating them (or work with the whitebox tester).

Have some of your testers exploring tools to provide more error oracles (if thats the right term) – debug builds, MS application verifier, Rational Purify etc – these types of tool can be used with normal manual testing but catch a lot of silent errors and assertions, and 'random' bugs/crashes due to memory conditions.

welcome back!

2010-03-03T17:45:00.818-05:00

welcome back!

"I figure some of you MUST be like me." ...

2010-03-03T10:02:28.857-05:00

"I figure some of you MUST be like me."

You figured correctly!

Glad to see you back, and writing, Linda.

Welcome back, glad to hear you are feeling better....

2010-03-03T09:02:15.623-05:00

Welcome back, glad to hear you are feeling better.

I agree, sometimes its the folks in the trenches who do the most but get the least recognition. Some because they don't want it, need it or are too busy working.

Good to have you back

2010-03-03T08:22:28.867-05:00

Good to have you back