PRACTICAL QA

Linda, I just found/read this post. OMG, spot on...

2010-07-27T17:16:58.100-04:00

Linda,

I just found/read this post. OMG, spot on and your last line is absolutely hilarious. Can't wait for that entry. Guess you couldn't call it "Jobs in BFE".

Keep it going, great stuff.

Jim

Nice post! Titles are just one more of those "it ...

2010-07-20T16:01:42.993-04:00

Nice post!

Titles are just one more of those "it depends" things we always seem to encounter.

With my current company, I'm a Director of Quality Assurance. It's virtually the same job as my last company, where I held the title of Manager of Quality Assurance. (Here, a better parking space comes along with the Director title though! And some of our Sales people have a Director title, but with nobody to direct.)

And as you point out, you need to dig hard to find out what the role actually entails, rather than what the Title might imply. This is true on both sides of the hiring equation - sadly I've seen from personal experience that interviewing someone with the title of "Lead Tester" doesn't always mean that the person can actually test.

Good luck seeking out that Queen title. I haven't seen that Title on a job posting yet, but you never know!

Jim, it doesn't surprise me that you're on the sam...

2010-07-12T14:55:07.124-04:00

Jim, it doesn't surprise me that you're on the same page I am in regards to budget questions; I'd put you in the category of "experienced". That's a more polite way of saying "previously burned", of course!

Thanks for the good wishes; although there's no such thing as a "perfect" job, I have high hopes for something that's at least challenging, interesting, and/or fun. To say nothing of paying that pesky mortgage...

- Linda

-

Einzige, I'm not sure what you mean about "giving ...

2010-07-12T14:49:25.692-04:00

Einzige, I'm not sure what you mean about "giving too much away". I don't think it's a bad thing for any company to know what kinds of questions you'll ask and why. I normally do some research on any company I'm going to interview with and have some questions written down beforehand. But I don't necessarily stick with my list; it depends on how the interview flows. Sometimes my questions will end up going in a totally different direction.

I don't think I'd be too concerned about a propective employer reading my blog. I've yet to meet a single VP or CIO that reads QA/QC blogs. Although I did have one incident that gave me nightmares for quite some time - an interviewer used the word "fungibility" in a sentence. For one terrible moment, I thought I was interviewing with Matt Heusser...

- Linda

Been there and done that. I hear you Linda. One ...

2010-07-09T12:39:06.300-04:00

Been there and done that. I hear you Linda. One of my first questions is to ask about the budget and if I control it. If they respond "No budget yet" and "no you don't control it" I will stop them there and say thank you. If you want me to be a Manager, then give me what I need to be the Manager. Not that I have companies beating on my door, but in an interview you are checking them out as much, or more, than they are you.

Best to you and hope you get the job you really want and will be happy with. Look forward to more on your blog.

Jim

Do you have a list of standard questions you ask a...

2010-07-09T11:33:00.821-04:00

Do you have a list of standard questions you ask at every interview.

I was thinking about posting a list of mine up on my blog, along with explanations about why I'm asking the question and what I'm hoping to learn from the answer, but I worry then that I might be giving too much away. What if a prospective employer reads that blog post prior to the interview?

Is this a legitimate fear?

Well testers are usually underrated, but its getti...

2010-07-09T11:02:04.486-04:00

Well testers are usually underrated, but its getting better I think. And this shows also in earnings and in budget.

Hi Linda, I value work and the people/environment ...

2010-07-09T07:56:42.344-04:00

Hi Linda,
I value work and the people/environment the most in any job. I have quit 2 jobs for lack of good work in the past. I can sense how it feels when you know that you are not working to your full potential simply because that is not your responsibility, but someone else's.

People matter a lot. If we tend to spend most part of our wakeup time at offices, and we don't quite enjoy working with people or the environment we are in, it's hard to stay there for longer and yet to quality work.

Money is very important too, but comes third on my list.

I do think about medical benefits and other incentives I would get. But, I think I would be ok with the titles as I am not very particular about them as long as they pay me reasonably well.

I really admire the honesty with which you say bitter-sweet truths about QA/QC in general. Thank You for your posts.

Regards,
Parimala Shankaraiah

Anne-Marie, I was a consultant for 7 years and I e...

2010-07-08T17:52:42.856-04:00

Anne-Marie, I was a consultant for 7 years and I enjoyed it. Eventually, however, I got tired of someone else piloting my ship. When you're a consultant, you need to take whatever work is available when you happen to be available. So sometimes you'll be a queen bee and sometimes you'll be a drone. Actually, the variety kept things pretty interesting. There's also the issue of having to look for a new assignment every 6 months to a year; most firms don't carry benches any more and (go figure!) many firms cut their consultants loose right around the holidays (when no one is hiring) - at least in this region of the country.

I've found, for myself, that what I miss the most when consulting is having a staff. There's something I really can't explain that comes with commiting to the success of a company for longer than a few months, and to the growth and well-being of a group of people long-term. Mentoring and training the next generation of QA experts is something I view as an investment in the field and "payback" for everything the field has given to me. I wish I'd had a mentor myself when I was starting out; it would have made such a difference! If even one person had an easier time than I did because of my help, then my entire career has been worthwhile. Ditto with companies I've worked for. I've had past executives run into me who've told me they learned what the testing field was really all about because I worked for them. It changed them, their understanding, and their company for the good. Those kinds of things don't really come with a price tag or appear in a job description; it's just a side-effect of doing what I do and being who I am. I've never left a job where the company wasn't better off because I had worked there. I hope I never do.

And that's as serious as I ever get, Anne-Marie - I don't think anyone has ever asked me that question before!!

- Linda

Well, Rob, I'm not sure a job search is what I'd c...

2010-07-08T17:26:53.947-04:00

Well, Rob, I'm not sure a job search is what I'd call "fun". I like the interviews; I love meeting new people and talking about QA/QC. The rest is just anxiety-ridden stress. I just handle it differently than some people, I guess!!

- Linda

Hi Linda, Nice to see you blogging again. I have...

2010-07-08T16:28:48.547-04:00

Hi Linda,

Nice to see you blogging again. I have a question for you.

Why do you prefer full time employment to consulting?

Or "what floats your boat" about permanent work?

Anne-Marie

Hi Linda, Glad to hear your having some fun looki...

2010-07-08T15:29:45.315-04:00

Hi Linda,

Glad to hear your having some fun looking for your next role. Sounds like you are absolutely clear about what you want in your next role.

I've never really sat down and thought about what order I rank what I look for in a role but quickly thinking about it now; it wouldn't be too far from yours.

I'm looking forward to the next few blog posts.

Rob..

Just wonderful Linda. There is so much in there t...

2010-07-08T11:26:59.646-04:00

Just wonderful Linda.

There is so much in there that I have experienced.

For me the order of importance is the same as yours and for the same reasons.

work; people/environment, money, title, and extras.

regards

John

Hey Linda. I'm currently doing a fair bit of writ...

2010-06-30T08:53:01.225-04:00

Hey Linda. I'm currently doing a fair bit of writing about doing more with less - yes, I know, it's a cliche, I have a fair bit to write about this.

What I'm curious on, though, is the millions of dollars a year in cash savings from automation. I'm curious what actual expenses you cut in order to decrease outgoing cash flow.

Did you lay off staff, or are the savings 'cost avoidance', or what?

Inquiring minds want to know! :-)

Also, I noticed this looks like it was posted an hour /after/ the layoff notice. Was the layoff rolled back, or are you just following through as a good person? If I had been asked my management to provide a report on the effectiveness of program X just after layoff notice, my response would be hard to predict. Likely, it would be something like "I can get you that report in THREE weeks... or you can get it verbally right now. :-)"

("I can't believe you yelloed at the CEO, Matt!" / "He literally asked me to 'give it to him' verbally ...")

Thanks for the reply, Shane! If your shop produce...

2010-05-27T08:57:23.456-04:00

Thanks for the reply, Shane!

If your shop produces updates quickly (less than 3 weeks of test time), or has multiple OS variants, I can see how automation (particularly if done by one resource) eventually has a return on the investment.

For example (this is for readers), if you have to run 500 tests through 4 configurations, 2000 tests would require 4 manual testers. You're just paying one automator. Still, it would take a good 3-5 years or so to see any financial benefit. Again, a long-term vision is required; many companies don't have that.

So congratulations on making it work for your company; don't feel bad about the first attempt; I'd say 95% of all first-time automation efforts fail. Your initial "learning experience" and subsequent success makes you invaluable in terms of the market...

- Linda

Great post Linda. While I agree with all your maj...

2010-05-27T07:01:05.207-04:00

Great post Linda.

While I agree with all your major points, I don't agree that automation can't be done successfully by small software houses without millions to spend under certain circumstances. The reason I say this is that I run a small software house, and we have been successfully running automated regression tests for a number of years. Yes, our first attempts at automation were a failure. Yes, the cost of automation is an addition to other software development, including manual testing. And yes, maintaining and developing scripts is an ongoing process, as is analysis of results.

I'd also agree that developing a framework is time consuming, about 4 developer months in our case, and that each test case averages about 3 hours to implement in a manner that is robust enough to survive a significant number of releases. So the 500 test cases and framework will take a skilled automator a little over a year to get up and running. We also need continued input to write new test scripts, maintain existing test scripts, and analyse the results. This amounts to an extra full time employee, who provides no return on investment until year two, and perhaps little visible return going forward.

The problem is that for a product with a long shelf life, running on a range of O/S varients, there simply isn't much of a viable alternative. You're choices come down to automate, increase the amount of manual regression with each release to cover existing and new functionality, or drop old tests that have consistently passed in the past and at the same time prey to your deity of choice that any bugs will be restricted to new functionality. I suspect most small software houses go with the third option, and at some point become badly unstuck as a result. Been there, done that, got the scars to prove it. Staying with manual testing with adequate coverage knocks your testing to development cost ratio out of kilter, to the extent that you will quickly lose the development capacity to meet market demands, so that just leaves automation. In my case, a four person software house, I reckon automation cost about €110k to set-up, and costs about €30k p/a to maintain. If I hadn't made such a hames of the first few attempts to set it up, the initial cost would be much less. Yup, I was that fool that believed the automation sales hype.

There are a load of caveats on the above. We are working with just two applications that are both mature and largely stable. We have a customer base paying maintenance contracts that want upgrades but don't want existing work flows broken. We allow how we automate to constrain how we develop. As per you example with the grid control from hell, we had the same problem and changed grid control. We look at the automation cost of making UI changes, and avoid large cosmetic changes that don't add enough other significant value to warrant the change. We add APIs to the software specifically to better support testing. Bottom line though, I don't believe we'd still be in business if we hadn't invested in, and persisted with, good automated regression testing.

Apologies for waffling on, but i felt it was worth highlighting a slightly different experience.

All the best,

Shane

Great post and forwarded to manager in my company....

2010-05-19T11:59:29.407-04:00

Great post and forwarded to manager in my company.

This post, along with "PRACTICAL QA LINDA WRITES A FAIRY TALE" is up in the top 10 best blogs of all time. Thank you.

Linda, Spot on target with all of it. This is de...

2010-05-18T16:32:46.118-04:00

Linda,

Spot on target with all of it. This is definitely being saved as a Favorite on Test Automation and will be a source for discussion with colleagues and clients.

Nice job!

Jim

Hi Linda, Just wow! The most thrilling review of...

2010-05-18T08:43:57.442-04:00

Hi Linda,

Just wow!
The most thrilling review of a test automation lifecycle.

I have "Test Automation Problems" page on my blog. From now on, link to your article goes Number One amongst recommended readings.

Thank you,
Albert Gareev

Sorry to hear that, Linda. With your talent, I'm ...

2010-05-18T07:33:18.961-04:00

Sorry to hear that, Linda.

With your talent, I'm sure you'll find a new company to admire, with management you respect. And then you'll build a new staff that is second to none.

Please come back and tell us how it goes?

Good luck, and best wishes.

-joe

I'll keep Buckwheat firmly in mind; thanks! - Lin...

2010-05-18T03:06:35.723-04:00

I'll keep Buckwheat firmly in mind; thanks!

- Linda

Linda, Sorry to hear about the sordid mess. But ...

2010-05-17T19:49:23.108-04:00

Linda,

Sorry to hear about the sordid mess. But feel good to know that you will not have to deal with the aftermath of the 're-org'. Blessing in disguise is best way to look at it.

Just remember you can always go "wooking pa nub in all da wong pwaces..." (remember Eddie Murphy doing Buckwheat sings on SNL).

Best to you.

Jim

Linda, Welcome back and glad you are feeling bett...

2010-04-19T18:07:47.265-04:00

Linda,

Welcome back and glad you are feeling better. Being a "Veteran of the Testing Trenches" myself I can relate to what you say.

I think what you stated with this "Just that much alone would save us around 400K per year." is the key to your future success. When talking with upper management if you can relate a dollar value to the value of what you are doing you will definitely get their attention. You are working towards an efficiency and effectiveness gain that has monitary repercussions. Not that you are looking to cut costs or testing effort, but better focus testing and because of that you are "containing" that cost associated to it.

And I totally agree with your ranking idea/system. I've done that on projects to get things distilled down so I can have an effective Smoke Test set. Then using the ranking to run other tests as needed according to impact/risk.

Best to you.

Jim Hazen

Hi Linda, Welcome back and I wish you continued he...

2010-03-31T16:42:48.235-04:00

Hi Linda, Welcome back and I wish you continued health.

I discovered your blog just recently. I have read every post to date and have thoroughly enjoyed your views, you are insightful, refreshing and witty. I found myself chuckling often.

One suggestion is a form of Regression testing which is “Stress testing”. It’s nothing earth shattering, goes by other names I am sure, and it may already be a part of your #1 tests already.

Since your Regression tests are for existing features you have previously characterized and don’t expect to be affected, running the most stressful tests first, the ones that really push the limits of the system, is a good way to perform a fast sanity test. Stress tests should always Pass. Chances are good even small changes in code or performance will ripple through the system and allow you to identify it quickly with a "Stress Test". If it fails you can then explore further with other Regression tests in the same functional category.

Paul

Welcome back, Linda. Are you able to report depen...

2010-03-30T12:29:15.913-04:00

Welcome back, Linda.

Are you able to report dependencies from your code base? You could reduce the number of tests run by executing only against changed code and its dependents.