This is blog post #4 in a series about job hunting. This piece of the Continuing Saga deals with the job climate overall and salaries.
Nothing has changed in terms of companies trying to snatch up bargains. If they have two candidate that are equal in other respects, they'll choose the cheapest.
What HAS changed is that their expectations are lower.
The market is flooded, and I do mean flooded, with poorly-qualified personnel who have 5-10 years of experience. That's work experience, not necessarily "good" experience. These are the people who have hopped from job to job all over the country, never holding one job for longer than a year. All of them consider themselves "leads" or higher. You know who they are - you've interviewd or worked with them. The problem is that in terms of pay, they will undercut you every time. This glut of unqualified warm bodies has degraded the market.
That would not be true if all interviewers and hiring managers were as experienced and discerning as you are, dear reader. Unqualified people would not get work and the market would turn around.
Unfortunately, our field is still not necessarily well understood. Experience is not necessarily valued above certifications, degrees, and above all, cost. And there is little understanding of what is (and is not!) important in terms of experience. Consider a resource who has had a new job every 6-12 months for the past 5 years. They are applying for "lead" jobs. You cannot gain the type of maturity and experience you need to be a lead when you're starting anew at every position and only stay for a matter of months.
At my level, competition comes from Project Managers, and managers/directors, etc. in other IT areas. I don't know about your region, but here in the midwest U.S., PMs are a dime a dozen. There are a bizillion of them. They have no background in QA/QC specifically and they've never actually done the work. But they present themselves as managing development, operations, QA, etc. - which isn't exactly untrue. On a project basis. Many interviewers do not understand that there's a big difference between a matrix manager and a line manager. Matrix managers do not handle the same problems as a line manager, and they rarely have to have detailed knowledge about the problems facing a test team. Nor do they have to worry about career paths, mentoring, training, process/procedure, automation, tools, and solutions to knotty testing issues. Some have never even had to handle personnel evaluations. The same is true with managers/directors of other areas of IT. Managing a team of DBAs is different than managing a team of testers.
So if the interviewer doesn't know the difference between a qualified and an unqualified resource, they will inevitably choose the cheapest resource or the one with the best patter. At my level, that's an issue; one thing PMs are extraordinarily good at is patter. Considering their entire career is based on negotiation, that's not a surprise. So if you're at a manager level or above, you HAVE TO BE ON YOUR GAME when interviewing.
Particularly if you're not this week's special at K-Mart.
Overall, if an interviewer goes with "cheapest" every time, one can reason that at least they do their company a service financially, if not in any other way. Going cheap in leadership positions is a false economy, whether you're talking about a full-time employee or a consultant. It's the best, most experienced personnel that are going to know how to save you money and maintain service levels. This is an advantage you'll have over other applicants. Talk about the Great Things you've done to increase efficiency and cut costs while improving testing services. They may ask those questions of the next applicant, who won't be able to answer them.
The problem of "cheapest" has been fueled by offshoring and poor policies on the part of consulting firms. While I've certainly seen some differences in the FTE market, consulting has changed significantly in the past 5 years. And I have to say that I'm totally sympathetic to their woes. You've got to be both tough and smart to remain profitable in today's market.
Consider that local consulting firms have to compete for dollars against offshoring concerns. That means cutting their rates to make their applicants more attractive to potential clients.
In order to keep their admin and marketing arms alive, most consulting firms prefer a 40% margin or better on their billing rates. That means if they want to lower their rates to their clients, it is infinitely preferable to hire cheaper people than to lower their profit margins.
Again, many consulting firms don't know the difference between a good or bad QA/QC resource. They will hire resources on which they can realize the highest profit margin. And they're smart about it - normally you aren't hired at all until you've interviewed and have been selected by a client. The problem is that the highly-talented sales personnel will be presenting the candidate in glowing terms, and the client may not know what they're buying and just trust those things are true. Or they may not get ANY candidates they really like and just pick the best out of a bad bunch.
Regardless, client expectations are lower now than they used to be, staff are less experienced, and companies have become accustomed to paying less for resources. Their lowered expectations have bled into the FTE side as well.
So where does that leave today's job hunter?
Well, the good news is that you're still going to get plenty of opportunities. The market is good for QA/QC. The bad news is that you'll have a lot of competition from less experienced and cheaper resources.
I can't give advice in regards to compensation, because it depends on you, your situation, and your region. If you have a familly, mortgage, etc., maybe you'll want to take what you can get and just keep looking until the right thing comes along.
If, however, you can afford to be selective for at least a period of time, I'd suggest you do what your momma always told you. Don't give it away. Why not look for a company that offers you something that makes you look forward to going to work every morning?
I'm pretty practical, myself, and it's important to know the market. What's the going rate for your position in your region? Trying to get more isn't going to work in most cases. And if you're considering relocating, you wouldn't want to lateral into a position where the cost of living is double what you're accustomed to. You might have to move into a double-wide and take on roommates. When I'm offered a relocation opportunity, the first thing I do is spend some time on the net, researching the area.
Talented resources are, as they have always been, at a premium. It's easy to get depressed or discouraged when looking for something new, but nothing outside of you need affect the inside of you. Your attitude is something you come up with all by yourself. You need to honestly assess the market, yourself, and set your expectations accordingly. Then you need to hang tough. Every company won't be looking for the best. But some will. And those are the people you want to talk to, because chances are good you'd get a chance to work with others that are also the best at what they do. There's nothing like working for a company that gives "excellence" more than lip service.
You just need to tell yourself that you are not the Blue Light Special in Aisle 9. You are a BMW and not a rusted-out Pinto.
In terms of salary, I have a bottom line and if it isn't met, I won't even consider the position. Beyond that, I know the market, and I consider the position being offered before I decide what is fair compensation for that particular position. Just because I have a bottom line doesn't mean I want to come in at that level. My bottom line is not necessarily what I made last year plus 5K. My bottom line is what I require in order to maintain some level of comfort and pay my bills. I negotiate on terms until the company and I either reach an agreement or decide we won't be able to reach an agreement.
This is where your own faith in yourself, your worth, and your abilities kicks in.
You have to be willing to walk away if a company can't meet your expectations. And don't count on that company calling you back if you do. Yes, some will sweeten their offer. But some won't. You have to be willing to give it up without regret if the offer isn't what you feel is fair.
Your best chance to get what you want, need, and feel is fair is during the hiring process. If they aren't willing (or can't) pay you now, do you think that's likely to change once you're an FTE? Promotions, bonuses, etc., are not guaranteed. Ever. So if it's not "right" now, it may never be right. Do you want to spend the next 4 or 5 years just getting back to where you were before you joined them?
I'm probably like most you; I always feel bad about turning down an opportunity and wonder if I did the right thing. Second-guessing yourself is a favorite pasttime of job hunters. But I have to say, I've never actually regretted anything I've turned down. But I have, in the course of my career, regretted several opportunities I've accepted.
So overall, it's a jungle out there. But I'm wearing my trusty pith helmet (the one that matches my Coach bag), looking out for quicksand and hoping I don't get torn apart by a pack of rabid hyenas....
Thursday, July 29, 2010
Tuesday, July 20, 2010
A ROSE BY ANY OTHER NAME....
I had to take a few days to actually work on my job hunt, but here's #3 in a series of blogs about job hunting. I'd like to talk about titles...
I don't think this will be particularly applicable to consultants. If you're a consultant, your title at a given company is "The Consultant", whether you're acting as a tester, lead, manager, or director. Your authority will be derived from whomever hired you. The client is not going to care what your actual title is and they don't have to worry about fitting your position into their permanent corporate hierarchy. Any title they associate with a consultative opportunity is merely an indication of the level of expertise they require, if they even bother with that. For example, if they say they need a lead, they don't necessarily care if you have 5 or 25 years of experience, as long as you can get the job done within the rate they have in mind.
If you're looking for a full-time opportunity, however, your title is important whether or not you personally care about such things. In the corporate world, your title establishes your placement in the corporate hierarchy. This isn't just for you or for your ego. This tells OTHERS in the company where you fit in, what you control, and what kind of authority/responsibility you bear. Your title alone tells others how they need to interact with you. It can smooth your path, or place difficult obstacles in your way.
Every person at your level is your peer. So if you work in a company with 200 test managers, it's likely your responsibilities will be similar to that of a lead and you will not have authority that extends beyond the testing products of a group responsible for one application or suite of apps. In addition, the way your group operates may be dictated to you.
So even if you've worked as "just a lead" elsewhere, if it was a small shop and you ran the show, you might find the position somewhat restrictive, to say the least. Still, if you're presently a lead, this might be a good career move, if other aspects of the job are appealing. Having the word "manager" in your title can open other doors.
Because titles are unique to each company, they should be largely disregarded when exploring a job opportunity. But the title's placement within the hierarchy should never be disregarded. The opportunity you accept should place you in a slot that gives you the best opportunity to be successful within that environment.
Consider, for example, that the actual job that needs to be done is to establish and manage a testing function, with all that implies. The position is called "QA Manager". Pretty standard, right? But say upon further investigation, you find the highest-ranking development and operation managers with whom you would work are senior managers and directors. They report to a VP who reports to the CIO. The QA Manager position reports to a director who has no specific QA or testing experience. They may report to the same VP that handles operations and development. What does this tell you?
Well, a prudent advisor would tell you that it would be tacky to leave a job interview screaming "Noooo!!!!!" while fleeing the premises. It doesn't really tell you "DON'T TAKE THIS JOB, FOOL!!". It does, however, tell you in clear and honest terms, what kind of issues you'll be working through if you choose to take the position. Your position is at least 2 levels below the highest-ranking managers you'd be working with and need to influence. In short, they're gonna outrank you.
Anyone at a level above you will have more clout and a closer link to those that control your world. They are not, in unspoken corporate reality, obligated to negotiate with you, listen to you, or treat you like a peer. In fact, if they don't like your shoes, they might decide to undermine your efforts. You need the authority to do what you need to do, and a title automatically sets the expectations of OTHERS to a certain level. If you're a peer, even if others are predisposed to not want to work with you (and this can be based on many things, not because you're not a Great or Talented Individual), your status within the hierarchy requires them to treat you as an equal and try to work things out. Or else you have the same options for escalating problems that they do.
So if you don't have the title, it means you need to understand and accept that successfully performing the job is going to require you to be exceptionally confident, persuasive, and determined. It's possible that even those qualities won't help; you won't know unless you take the position. If you have not historically been an outstanding negotiator with people predisposed to dislike you or testing groups in general, you might want to consider this opportunity carefully. One of the realities of life is that people often dislike, fear, or mistrust change. Your presence in the company is going to represent some level of change. You need to be ready to act as an agent of change, and that's not easy if you don't have the authority to do so.
And promises are just that - promises. They are not what represents actual reality right now and they may or may not come to pass. I've found (for myself) that it is infinitely better to look at a position as it exists and assume it will remain that way. Are you accepting a position you want right now, or the position you hope it might become? What if your wishful thinking never comes to pass? Will you still be satisfied? Ruthless management of your own desires and expectations can help you find the right position.
Likewise with being swayed by title and money. What if that Executive Director of Quality or Associate VP position has no direct reports and no budget? That makes you a cat with no claws. You'll still be able to yowl, but you won't win any fights. And that's OK, if you prefer yowling to fighting anyway. As long as your expectations are met and you have the authority to do what you need to do, it's All Good.
Generally, I think you need to "trust your gut". If something smells bad, it probably IS bad. If, however, the job makes you salivate, you continue to think about what you'd do and how you'd tackle the job at all hours (as opposed to just thinking about how you'd spend your paycheck), then you're In Love and it's probably best to ignore titles, if you can, and just go for it. I passed by director and AVP titles for my last position, which was a "mere manager", because the job itself was what I wanted to do. And I have to say it was the right decision. For ME. Sexy titles might be a deciding factor for you - there's nothing wrong with that if it makes you happy. I know MANY people where status is spice that makes everything else worthwhile. I think they're nice-to-haves. I certainly wouldn't pass by a Juicy Title if everything else was in line with my career goals, but it doesn't tempt me otherwise.
I spend a good half of my time sorting out what a given opportunity's title actually means; if you fling yourself into the job market, you'll probably find yourself in the unenviable position of doing the same. It doesn't help that some contracting and placement firms send you every opening they've got available without actually reading your resume. I've found that being clear in your cover letters is helpful, as well as requesting a job description from (whomever) contacts you. From there, questions regarding "who reports to whom" are best covered in the actual interview (s).
So here's hoping I find a "Queen of QA" position that doesn't end up being either "Untouchable Goddess With No Staff", or "Wretched Serf With Less Authority Than a Rutabaga". Now if you'll excuse me, I've just received an inquiry as to my interest and availability for a "tester" position for which I'd have to relocate to
Upper Mongolia. I've always wanted to live in a yurt.....
I don't think this will be particularly applicable to consultants. If you're a consultant, your title at a given company is "The Consultant", whether you're acting as a tester, lead, manager, or director. Your authority will be derived from whomever hired you. The client is not going to care what your actual title is and they don't have to worry about fitting your position into their permanent corporate hierarchy. Any title they associate with a consultative opportunity is merely an indication of the level of expertise they require, if they even bother with that. For example, if they say they need a lead, they don't necessarily care if you have 5 or 25 years of experience, as long as you can get the job done within the rate they have in mind.
If you're looking for a full-time opportunity, however, your title is important whether or not you personally care about such things. In the corporate world, your title establishes your placement in the corporate hierarchy. This isn't just for you or for your ego. This tells OTHERS in the company where you fit in, what you control, and what kind of authority/responsibility you bear. Your title alone tells others how they need to interact with you. It can smooth your path, or place difficult obstacles in your way.
Every person at your level is your peer. So if you work in a company with 200 test managers, it's likely your responsibilities will be similar to that of a lead and you will not have authority that extends beyond the testing products of a group responsible for one application or suite of apps. In addition, the way your group operates may be dictated to you.
So even if you've worked as "just a lead" elsewhere, if it was a small shop and you ran the show, you might find the position somewhat restrictive, to say the least. Still, if you're presently a lead, this might be a good career move, if other aspects of the job are appealing. Having the word "manager" in your title can open other doors.
Because titles are unique to each company, they should be largely disregarded when exploring a job opportunity. But the title's placement within the hierarchy should never be disregarded. The opportunity you accept should place you in a slot that gives you the best opportunity to be successful within that environment.
Consider, for example, that the actual job that needs to be done is to establish and manage a testing function, with all that implies. The position is called "QA Manager". Pretty standard, right? But say upon further investigation, you find the highest-ranking development and operation managers with whom you would work are senior managers and directors. They report to a VP who reports to the CIO. The QA Manager position reports to a director who has no specific QA or testing experience. They may report to the same VP that handles operations and development. What does this tell you?
Well, a prudent advisor would tell you that it would be tacky to leave a job interview screaming "Noooo!!!!!" while fleeing the premises. It doesn't really tell you "DON'T TAKE THIS JOB, FOOL!!". It does, however, tell you in clear and honest terms, what kind of issues you'll be working through if you choose to take the position. Your position is at least 2 levels below the highest-ranking managers you'd be working with and need to influence. In short, they're gonna outrank you.
Anyone at a level above you will have more clout and a closer link to those that control your world. They are not, in unspoken corporate reality, obligated to negotiate with you, listen to you, or treat you like a peer. In fact, if they don't like your shoes, they might decide to undermine your efforts. You need the authority to do what you need to do, and a title automatically sets the expectations of OTHERS to a certain level. If you're a peer, even if others are predisposed to not want to work with you (and this can be based on many things, not because you're not a Great or Talented Individual), your status within the hierarchy requires them to treat you as an equal and try to work things out. Or else you have the same options for escalating problems that they do.
So if you don't have the title, it means you need to understand and accept that successfully performing the job is going to require you to be exceptionally confident, persuasive, and determined. It's possible that even those qualities won't help; you won't know unless you take the position. If you have not historically been an outstanding negotiator with people predisposed to dislike you or testing groups in general, you might want to consider this opportunity carefully. One of the realities of life is that people often dislike, fear, or mistrust change. Your presence in the company is going to represent some level of change. You need to be ready to act as an agent of change, and that's not easy if you don't have the authority to do so.
And promises are just that - promises. They are not what represents actual reality right now and they may or may not come to pass. I've found (for myself) that it is infinitely better to look at a position as it exists and assume it will remain that way. Are you accepting a position you want right now, or the position you hope it might become? What if your wishful thinking never comes to pass? Will you still be satisfied? Ruthless management of your own desires and expectations can help you find the right position.
Likewise with being swayed by title and money. What if that Executive Director of Quality or Associate VP position has no direct reports and no budget? That makes you a cat with no claws. You'll still be able to yowl, but you won't win any fights. And that's OK, if you prefer yowling to fighting anyway. As long as your expectations are met and you have the authority to do what you need to do, it's All Good.
Generally, I think you need to "trust your gut". If something smells bad, it probably IS bad. If, however, the job makes you salivate, you continue to think about what you'd do and how you'd tackle the job at all hours (as opposed to just thinking about how you'd spend your paycheck), then you're In Love and it's probably best to ignore titles, if you can, and just go for it. I passed by director and AVP titles for my last position, which was a "mere manager", because the job itself was what I wanted to do. And I have to say it was the right decision. For ME. Sexy titles might be a deciding factor for you - there's nothing wrong with that if it makes you happy. I know MANY people where status is spice that makes everything else worthwhile. I think they're nice-to-haves. I certainly wouldn't pass by a Juicy Title if everything else was in line with my career goals, but it doesn't tempt me otherwise.
I spend a good half of my time sorting out what a given opportunity's title actually means; if you fling yourself into the job market, you'll probably find yourself in the unenviable position of doing the same. It doesn't help that some contracting and placement firms send you every opening they've got available without actually reading your resume. I've found that being clear in your cover letters is helpful, as well as requesting a job description from (whomever) contacts you. From there, questions regarding "who reports to whom" are best covered in the actual interview (s).
So here's hoping I find a "Queen of QA" position that doesn't end up being either "Untouchable Goddess With No Staff", or "Wretched Serf With Less Authority Than a Rutabaga". Now if you'll excuse me, I've just received an inquiry as to my interest and availability for a "tester" position for which I'd have to relocate to
Upper Mongolia. I've always wanted to live in a yurt.....
Monday, July 12, 2010
NO PROBLEMS HERE, OFFICER....
This is my second in a series of posts about the search for a new job, particularly about the interviewing process.
Have you ever watched a police show where a car driving all over the road or in an accident is pulled over by a policeman? The Nice Man asks the driver to get out of the car and politely inquires as to whether there is a problem. The driver stumbles out of the car, falls down in the middle of the road, and drunkenly slurs "No problems here, officer!".
There's a corollary there to some interviews. If a company is looking for a manager or a director, chances are pretty good they want someone who can deal with whatever problems, issues, and challenges they've got going on. But if they won't tell you (or what's worse, don't know!) what their problems are, it's pretty much impossible for them (or you!) to determine whether you're really a good fit for the job. Some of the worst mistakes I've ever seen in the field are from lack of disclosure on either side in regards to expectations, experience, and potential handling of major issues.
Frankly, if a company has no problems, I don't want to work for them. In fact, I don't even want to talk to them.
Why is that?
Well, first of all, chances are good they're lying. My level of interest in working for liars is pretty much non-existent.
Secondly, it's possible the person doing the interview doesn't know there are problems. I'm not interested in working for the Perpetually Clueless either. Why? Because it means when I bring existing problems and potential solutions to them, they either won't believe me, or they'll think I caused them myself. Then they'll seek consensus for their lack of understanding from others as clueless as themselves.
But still, it's human nature, particularly when presented with a talented candidate, to downplay any issues. This means, once again, you'll need to don your Sherlock Holmes hat and ask some probing questions. There's a very fine line between trying to attract someone and misrepresentation a job.
I don't think I'm particularly unique in the QA/QC field, and I want someone to give it to me straight. Most of us aren't very interested in sugar-coated pills; it's our job to find and report problems. Those of us that are managers or directors handle problems of one sort or another every day, all day long. Or 24X7.
So I want to know that you have so many errors in production, your user base has taken out a contract on your life. Tell me your automation efforts have failed 23 times, and that your best tester just committed ritual seppuku in the server room.
I want to know that I can make a difference to you and your company in a positive way. I can do that by having a realistic picture of what you need to have done and an equally realistic picture of whether those things are within the scope of my experience, interests, and abilities.
It's likely that the uglier your situation is, the more I'm going to be excited about it. I'm looking for an opportunity that needs someone like me. Any schmoe can handle something that runs well without them. All they have to do is show up for work and Look Pretty. I want to be loved for my mind....
We both need to assess each other and determine if we have a good fit. That means full disclosure on both sides.
Are there problems/issues I'm not qualified to handle? That would NOT excite me much? I'd have to yes. Not many, but there are a few. What's more likely, however, is that there are some challenges I might not want to take on. I'll ask more questions before I make any decisions. One of the most common?
"We need to automate more scripts.".
I'm going to ask a LOT of questions if a VP or CIO tells me this. Often, I find their staff is highly qualified and expert, but expectations in regards to what can and should be automated is skewed. Can I "unskew" them? Yes. But changing an executive's mind in regards to what needs to be done is a long-term and sometimes impossible task, particularly if someone they trust has convinced them that this is the problem. And if they hire you expecting you to "solve" this problem and you can't (even if no one can), the only thing that will be skewed is you.
What kind of problems do I like? Too many errors in production. Unskilled (or no) staff. Or skilled staff that are unhappy and unfulfilled. No organization or an organization that isn't working well. Lack of respect for the testing function. High turnover rates. Staffing issues. No career paths. No concept of any type of testing methodologies other than what someone once read in a book. No (or poor) automation. Chaos. No understanding of where they've been, where they are, or where they need or want to go. No tools or tools that aren't doing the job. Lack of management understanding of what's happening when. Any type of organizational dysfunction with how testing interfaces with other groups. Problems with 3rd party vendors. Problems with on-site/off-shore interactions. Processes that don't flow smoothly. Too much work, too little time.
I like those problems because I know I can solve them. Oh, not necessarily instantly or easily, but I know they can be solved and that I can do it. A really great testing organization is the result of cooperative effort and support; if you don't get either, your ability to "fix" things will be limited. But your time in an interview will also be spent trying to gauge how serious the company is about improving/supporting their testing efforts. The fact that they're interviewing you and are prepared to spend money to make it happen is a good step in the right direction. It's really up to you to figure out if they really just want a figurehead or the real deal.
What's a figurehead? It's a position, not a job. The company can point at you and say they have a manager, director, or VP, but you're just expected to do what you're told (often by the development organization) and not rock the boat. I'm afraid I personally view COE (Center of Excellence) positions this way. That's a position that normally comes with a fancy title and an equally fancy salary. The expectations are normally that founts of good advice and recommendations will flow down from Mt. Olympus (that will be you up in the rarified air) for the edification of the unwashed masses. Unfortunately, since you won't have anyone actually reporting to you, the adoption of any of your advice or recommendations is somewhat iffy. But I do know many people who would love such a position; I'm just not one of them. I like my hands dirtier than that. To say nothing of the fact that I don't like heights...
Does any of this ring true with the rest of you? I personally know several people who have been extremely disappointed by jobs they felt were misrepresented. Then they feel "stuck" in bad situations. That's a lesson to all of us; we need to ask as many questions as we can up front!!
Have you ever watched a police show where a car driving all over the road or in an accident is pulled over by a policeman? The Nice Man asks the driver to get out of the car and politely inquires as to whether there is a problem. The driver stumbles out of the car, falls down in the middle of the road, and drunkenly slurs "No problems here, officer!".
There's a corollary there to some interviews. If a company is looking for a manager or a director, chances are pretty good they want someone who can deal with whatever problems, issues, and challenges they've got going on. But if they won't tell you (or what's worse, don't know!) what their problems are, it's pretty much impossible for them (or you!) to determine whether you're really a good fit for the job. Some of the worst mistakes I've ever seen in the field are from lack of disclosure on either side in regards to expectations, experience, and potential handling of major issues.
Frankly, if a company has no problems, I don't want to work for them. In fact, I don't even want to talk to them.
Why is that?
Well, first of all, chances are good they're lying. My level of interest in working for liars is pretty much non-existent.
Secondly, it's possible the person doing the interview doesn't know there are problems. I'm not interested in working for the Perpetually Clueless either. Why? Because it means when I bring existing problems and potential solutions to them, they either won't believe me, or they'll think I caused them myself. Then they'll seek consensus for their lack of understanding from others as clueless as themselves.
But still, it's human nature, particularly when presented with a talented candidate, to downplay any issues. This means, once again, you'll need to don your Sherlock Holmes hat and ask some probing questions. There's a very fine line between trying to attract someone and misrepresentation a job.
I don't think I'm particularly unique in the QA/QC field, and I want someone to give it to me straight. Most of us aren't very interested in sugar-coated pills; it's our job to find and report problems. Those of us that are managers or directors handle problems of one sort or another every day, all day long. Or 24X7.
So I want to know that you have so many errors in production, your user base has taken out a contract on your life. Tell me your automation efforts have failed 23 times, and that your best tester just committed ritual seppuku in the server room.
I want to know that I can make a difference to you and your company in a positive way. I can do that by having a realistic picture of what you need to have done and an equally realistic picture of whether those things are within the scope of my experience, interests, and abilities.
It's likely that the uglier your situation is, the more I'm going to be excited about it. I'm looking for an opportunity that needs someone like me. Any schmoe can handle something that runs well without them. All they have to do is show up for work and Look Pretty. I want to be loved for my mind....
We both need to assess each other and determine if we have a good fit. That means full disclosure on both sides.
Are there problems/issues I'm not qualified to handle? That would NOT excite me much? I'd have to yes. Not many, but there are a few. What's more likely, however, is that there are some challenges I might not want to take on. I'll ask more questions before I make any decisions. One of the most common?
"We need to automate more scripts.".
I'm going to ask a LOT of questions if a VP or CIO tells me this. Often, I find their staff is highly qualified and expert, but expectations in regards to what can and should be automated is skewed. Can I "unskew" them? Yes. But changing an executive's mind in regards to what needs to be done is a long-term and sometimes impossible task, particularly if someone they trust has convinced them that this is the problem. And if they hire you expecting you to "solve" this problem and you can't (even if no one can), the only thing that will be skewed is you.
What kind of problems do I like? Too many errors in production. Unskilled (or no) staff. Or skilled staff that are unhappy and unfulfilled. No organization or an organization that isn't working well. Lack of respect for the testing function. High turnover rates. Staffing issues. No career paths. No concept of any type of testing methodologies other than what someone once read in a book. No (or poor) automation. Chaos. No understanding of where they've been, where they are, or where they need or want to go. No tools or tools that aren't doing the job. Lack of management understanding of what's happening when. Any type of organizational dysfunction with how testing interfaces with other groups. Problems with 3rd party vendors. Problems with on-site/off-shore interactions. Processes that don't flow smoothly. Too much work, too little time.
I like those problems because I know I can solve them. Oh, not necessarily instantly or easily, but I know they can be solved and that I can do it. A really great testing organization is the result of cooperative effort and support; if you don't get either, your ability to "fix" things will be limited. But your time in an interview will also be spent trying to gauge how serious the company is about improving/supporting their testing efforts. The fact that they're interviewing you and are prepared to spend money to make it happen is a good step in the right direction. It's really up to you to figure out if they really just want a figurehead or the real deal.
What's a figurehead? It's a position, not a job. The company can point at you and say they have a manager, director, or VP, but you're just expected to do what you're told (often by the development organization) and not rock the boat. I'm afraid I personally view COE (Center of Excellence) positions this way. That's a position that normally comes with a fancy title and an equally fancy salary. The expectations are normally that founts of good advice and recommendations will flow down from Mt. Olympus (that will be you up in the rarified air) for the edification of the unwashed masses. Unfortunately, since you won't have anyone actually reporting to you, the adoption of any of your advice or recommendations is somewhat iffy. But I do know many people who would love such a position; I'm just not one of them. I like my hands dirtier than that. To say nothing of the fact that I don't like heights...
Does any of this ring true with the rest of you? I personally know several people who have been extremely disappointed by jobs they felt were misrepresented. Then they feel "stuck" in bad situations. That's a lesson to all of us; we need to ask as many questions as we can up front!!
Friday, July 9, 2010
QUALITY IS FREE. OR NOT.
This is the first in a series of blogs about what I've experienced during my job hunt and it deals with a peculiarity that is somewhat unique to the QA/QC field.
Whether you've worked as a PM, a line manager, or an executive in the IT arena, you know that all thing corporate flow from a budget.
It stands to reason that if you're applying for any type of management position, you're going to want to get a feel for the size of the IT area, the number/nature of the software projects they produce, and what kind of budget you'll have to work with.
I am no longeer amazed when the answer is there isn't a budget for QA/QC. Oh sure, I might start whimpering uncontrollably, but I'm no longer surprised. Development managers and PMs have budgets. Accounting departments have budgets. Operations organizations have budgets. But for some unknown reason, testing groups, regardless of size, apparently work for free.
Or not.
If, in answer to your questions, there appears to be no budget, it behooves you to ask more questions. Contrary to those vicious rumors spread in the 70s, quality is not free. Even monkey testeers want to be paid in bananas. And while bananas DO grow on trees, unlike money, it's highly unusual for such a phenomenon to manifest in Philadelphia, PA.
There are, of course, answers to your questions, and each of those answers contains vital information and a variety of pitfalls. It doesn't mean you shouldn't accept a position, if other elements of the job appeal to you, but it does add to your knowledge base and expectations, possibly ratcheting them down to reasonable levels.
First, it's entirely possible the company doesn't know how much they need to budget and have decided to just hire someone who knows what they're doing and worry about it later. They'll tell you they want their new hire to assess the situation and tell THEM what's needed.
(Cue romantic violin music. Pop champagne corks.) Oh baby. I'm in love.
This is really, really seductive. Who wouldn't want to assess what needs to happen and have it handed to them on a silver platter?
But slow down there. Squint through that rose-colored haze and really take a look at your interviewer. Are they wearing a red suit? Drop your pen on the floor and surreptitiously check out their feet. Are they cloven hooves? Is it hot in the office? Have you noticed a jar of Vaseline on the desk?
If so, these are Very Bad Signs and you need to ask more questions before you sign away your soul.
Find out how much they've put aside to kick things off. If the answer is "nothing", then you have no people and you're it. You might be it for a Very Long Time. Like eternity. A good opportunty will have (some) thought behind it, and some sort of funds put aside to begin the work. Thereafter, you'll have to go through all the customary hoo-haw of making recommendations, obtaining funding, and justifying your requests. You may or may not get what you need to do the job. So make sure you're on the same page before you're committed to what might be a lesson in futility. If there are 300 people in the IT area, it's unlikely a staff of 3 will be able to handle the workload. But if 3 is what executive management has in mind, it's going to be a hard, hard road to get something more reasonable approved. Try to ask questions that give you an idea of what they have in mind. I can assure you they've talked about it. You may have to dig a little, but there's some expectations in there somewhere; use all of your hard-won testing analysis skills aned ferret it out.
In some companies, there will be a budget all right. But you won't control it. This makes you a lead, not a manager. In the corporate world, management staff control budgets, and the bigger your budget, the more power you have. You might not like hearing that, but it's true. If you don't manage a budget, you don't manage squat from a corporate perspective. Whomever is holding the pursestrings can (and wil) steal important pieces and parts of your funds for whatever they see fit. Often it will be for a mismanaged project or a new development resource. You'll find your abiity to provide promotions, raises, or bonuses will either be constrained or dictated to you to ensure there's enough dough for some other group that has more clout than you have. Having a budget, by the way, doesn't keep these things from happening, but it makes it harder and they have to ask you first.
Overall, a budget controlled by someone else is an insoluable issue. It often requires an Act of God to change corporate culture. This is a good reason to make sure you like a given corporate culture before you get on board.
The best opportunities will have some sort of budget, be able to tell you what it is, and will expect you to be responsible for managing it. You will be responsible for setting it for future years.
So if you're a hiring firm with no budget for QA/QC, please have the decency to stock Kleenex for your hapless interviewees. I hate weeping openly all over my navy blue power suit....
Whether you've worked as a PM, a line manager, or an executive in the IT arena, you know that all thing corporate flow from a budget.
It stands to reason that if you're applying for any type of management position, you're going to want to get a feel for the size of the IT area, the number/nature of the software projects they produce, and what kind of budget you'll have to work with.
I am no longeer amazed when the answer is there isn't a budget for QA/QC. Oh sure, I might start whimpering uncontrollably, but I'm no longer surprised. Development managers and PMs have budgets. Accounting departments have budgets. Operations organizations have budgets. But for some unknown reason, testing groups, regardless of size, apparently work for free.
Or not.
If, in answer to your questions, there appears to be no budget, it behooves you to ask more questions. Contrary to those vicious rumors spread in the 70s, quality is not free. Even monkey testeers want to be paid in bananas. And while bananas DO grow on trees, unlike money, it's highly unusual for such a phenomenon to manifest in Philadelphia, PA.
There are, of course, answers to your questions, and each of those answers contains vital information and a variety of pitfalls. It doesn't mean you shouldn't accept a position, if other elements of the job appeal to you, but it does add to your knowledge base and expectations, possibly ratcheting them down to reasonable levels.
First, it's entirely possible the company doesn't know how much they need to budget and have decided to just hire someone who knows what they're doing and worry about it later. They'll tell you they want their new hire to assess the situation and tell THEM what's needed.
(Cue romantic violin music. Pop champagne corks.) Oh baby. I'm in love.
This is really, really seductive. Who wouldn't want to assess what needs to happen and have it handed to them on a silver platter?
But slow down there. Squint through that rose-colored haze and really take a look at your interviewer. Are they wearing a red suit? Drop your pen on the floor and surreptitiously check out their feet. Are they cloven hooves? Is it hot in the office? Have you noticed a jar of Vaseline on the desk?
If so, these are Very Bad Signs and you need to ask more questions before you sign away your soul.
Find out how much they've put aside to kick things off. If the answer is "nothing", then you have no people and you're it. You might be it for a Very Long Time. Like eternity. A good opportunty will have (some) thought behind it, and some sort of funds put aside to begin the work. Thereafter, you'll have to go through all the customary hoo-haw of making recommendations, obtaining funding, and justifying your requests. You may or may not get what you need to do the job. So make sure you're on the same page before you're committed to what might be a lesson in futility. If there are 300 people in the IT area, it's unlikely a staff of 3 will be able to handle the workload. But if 3 is what executive management has in mind, it's going to be a hard, hard road to get something more reasonable approved. Try to ask questions that give you an idea of what they have in mind. I can assure you they've talked about it. You may have to dig a little, but there's some expectations in there somewhere; use all of your hard-won testing analysis skills aned ferret it out.
In some companies, there will be a budget all right. But you won't control it. This makes you a lead, not a manager. In the corporate world, management staff control budgets, and the bigger your budget, the more power you have. You might not like hearing that, but it's true. If you don't manage a budget, you don't manage squat from a corporate perspective. Whomever is holding the pursestrings can (and wil) steal important pieces and parts of your funds for whatever they see fit. Often it will be for a mismanaged project or a new development resource. You'll find your abiity to provide promotions, raises, or bonuses will either be constrained or dictated to you to ensure there's enough dough for some other group that has more clout than you have. Having a budget, by the way, doesn't keep these things from happening, but it makes it harder and they have to ask you first.
Overall, a budget controlled by someone else is an insoluable issue. It often requires an Act of God to change corporate culture. This is a good reason to make sure you like a given corporate culture before you get on board.
The best opportunities will have some sort of budget, be able to tell you what it is, and will expect you to be responsible for managing it. You will be responsible for setting it for future years.
So if you're a hiring firm with no budget for QA/QC, please have the decency to stock Kleenex for your hapless interviewees. I hate weeping openly all over my navy blue power suit....
Thursday, July 8, 2010
MY INDIAN NAME IS "RHUMBAS WITH RECRUITERS"....
Well, the job hunt is finally in full swing (I took a vacation in San Antonio first!), and I thought I'd share the joys and pains of my discoveries in my blog. Whether you're part of a hiring team or seaching for your own version of DreamCorp, maybe some of my experiences and thoughts will help you either hire better candidates, or find a better job.
There's so much to talk to about I'm going to break it up into a series of blogs and have some fun with it; if I ever stop laughing at myself and the world around me, I'll know it's time to hang up my gloves...
Now I've been honored by any number of calls from firms that would like to discuss my qualifications and I'm not going to reward their interest by besmirching their Good Names. If you read something and say "My God!!! She's writing about US!!!", kindly remember I've talked to a lot of people. It's unlikely I've picked you, personally, out of the pack. But if you recognize some of these issues, well, maybe it's a sign you might want to work on your courtship methods. And for those looking for better opportunities, maybe it will help you decide what questions you'd like to have answered before you say "I do.".
I think it would be best to set a little context before I start; what I'm looking for and what Floats My Boat might be different from yours. There are some general considerations when looking for work; money, title, people/environment, and extras. Everyone puts those in a different order.
For me, the job comes first. I have to be attracted to and excited by the opportunity itself. Why is that? Because I have a lot of experience and have been in jobs that paid Very Well Indeed that I didn't like much. The money wasn't worth it. I've had a few jobs where no amount of money would have been worth it. Money is not my primary motivator. I like what I do for a living, and I expect to look forward to getting up and going into work every morning. Since I have a lot of experience, what interests me most are positions that let me me use that experience. I like complex environments with a lot of interfaces, a lot of people, and a lot of problems. I'm used to being in charge and I'm easily bored, so when I say I need challenge, it's not a bunch of hokey applicant BS. Problems are ennervating; that's why I'm in this field.
People and environment come next. When you're interviewing, you need to connect with your interviewer. Sometimes, what they forget is that have to impress you as well. If I don't like the hiring manager or my peers, I'm not going to deliberately put myself into a position where I'd have to work with or for them every day. The environment has to resonate with me. Many of you have "been there, done that", so you know what I'm talking about. Nothing is worse than working for/with people who get on your last nerve. Again, there is no level of compensation that will ever make that kind of opportunity anything but Heinous.
Money is third on my list. It's not my primary motivator, but it IS a motivator. Look, I've got more than 25 years of experience. I'm just not going to be cheap. That's reality. A company doesn't really want or need to hire me if someone with 5 years of experience can do the job. Believe me, I'm not going to be resentful if I don't fit into someone's budget. So it's useless to try to talk me into a ginormous pay cut just for the privilege of getting my foot in the door at (some) company. I'm not 25 years old. I don't want to spend the next 15 years moving up a largely mythical corporate ladder. I've already Done My Time. If I really want the job you have to offer, I might negotiate a little, but generally, if my financial situation reaches critical mass, I'll just sign on with some local consulting firm and consult for a while. I've taken jobs in the past that cut my salary - it has never, not once, been worth it. And promises that salaries will miraculously go up at some unspecified time in the future are pretty much worth the paper they're written on. I know there are a lot of you out there that, if they're temporarily out of work, would and have taken the first thing that popped up. But were you happy about it? Or did you resent it? Did you just stay there, or did you continue looking for something better? There's a lesson there, by the way, for hiring firms. Don't lowball talented candidates. Even if they accept, they won't stay.
Next on my list is title. Titles generally only mean something in relation to other positions at a given company. The primary players in the QA arena should be in keeping with similar positions in the DEV arena. I know the difference between a lead, a manager, a director, and a VP. So the likelihood of me being impressed by a sexy title that any competent senior analyst could handle is slim and none. But I think companies that offer such titles are smart, smart, smart. There are many talented leads out there that would really like the word "manager" or "director" in their title. For me, the job title has to match the job responsibilities. That said, my ego is just as healthy as everyone else's - if the job is great and the money is good, having a sexy title like "Supreme Ruler of the Quality Universe" is a nice, albeit unnecessary, perk.
Last on my list is extras. You can never count on an extra; it's usually the first thing taken away if the company has a bad financial year. I expect "normal" benefits and will actually give up some salary for vacation, but generally anything else is just a "nice to have". I've never had a stock option I could actually exercise and show a profit. Bonuses aren't guaranteed. So I'll be interested in any extras, but they're unlikely to sway me much. There are, of course, some extras that fall into a different category altogether. Hiring bonuses or stock grants are more like money in the bank.
So that's my personal list; how does it stack up to yours? Many people I know put money and title first; some choose people/environment.
Regardless, I hope you'll join me on my safari through the jungle that is the Job Market; coming this next week is:
Quality is Free. Or Not.
No Problems Here, Officer...
A Rose By Any Other Name
Blue Light Special in Aisle 9...
I Smell Brimstone
Save the Baby!
Stenography for Fun and Profit
Job Opportunities in Asswipe, North Dakota
Hope you enjoy it; feel free to add your own experiences!!
There's so much to talk to about I'm going to break it up into a series of blogs and have some fun with it; if I ever stop laughing at myself and the world around me, I'll know it's time to hang up my gloves...
Now I've been honored by any number of calls from firms that would like to discuss my qualifications and I'm not going to reward their interest by besmirching their Good Names. If you read something and say "My God!!! She's writing about US!!!", kindly remember I've talked to a lot of people. It's unlikely I've picked you, personally, out of the pack. But if you recognize some of these issues, well, maybe it's a sign you might want to work on your courtship methods. And for those looking for better opportunities, maybe it will help you decide what questions you'd like to have answered before you say "I do.".
I think it would be best to set a little context before I start; what I'm looking for and what Floats My Boat might be different from yours. There are some general considerations when looking for work; money, title, people/environment, and extras. Everyone puts those in a different order.
For me, the job comes first. I have to be attracted to and excited by the opportunity itself. Why is that? Because I have a lot of experience and have been in jobs that paid Very Well Indeed that I didn't like much. The money wasn't worth it. I've had a few jobs where no amount of money would have been worth it. Money is not my primary motivator. I like what I do for a living, and I expect to look forward to getting up and going into work every morning. Since I have a lot of experience, what interests me most are positions that let me me use that experience. I like complex environments with a lot of interfaces, a lot of people, and a lot of problems. I'm used to being in charge and I'm easily bored, so when I say I need challenge, it's not a bunch of hokey applicant BS. Problems are ennervating; that's why I'm in this field.
People and environment come next. When you're interviewing, you need to connect with your interviewer. Sometimes, what they forget is that have to impress you as well. If I don't like the hiring manager or my peers, I'm not going to deliberately put myself into a position where I'd have to work with or for them every day. The environment has to resonate with me. Many of you have "been there, done that", so you know what I'm talking about. Nothing is worse than working for/with people who get on your last nerve. Again, there is no level of compensation that will ever make that kind of opportunity anything but Heinous.
Money is third on my list. It's not my primary motivator, but it IS a motivator. Look, I've got more than 25 years of experience. I'm just not going to be cheap. That's reality. A company doesn't really want or need to hire me if someone with 5 years of experience can do the job. Believe me, I'm not going to be resentful if I don't fit into someone's budget. So it's useless to try to talk me into a ginormous pay cut just for the privilege of getting my foot in the door at (some) company. I'm not 25 years old. I don't want to spend the next 15 years moving up a largely mythical corporate ladder. I've already Done My Time. If I really want the job you have to offer, I might negotiate a little, but generally, if my financial situation reaches critical mass, I'll just sign on with some local consulting firm and consult for a while. I've taken jobs in the past that cut my salary - it has never, not once, been worth it. And promises that salaries will miraculously go up at some unspecified time in the future are pretty much worth the paper they're written on. I know there are a lot of you out there that, if they're temporarily out of work, would and have taken the first thing that popped up. But were you happy about it? Or did you resent it? Did you just stay there, or did you continue looking for something better? There's a lesson there, by the way, for hiring firms. Don't lowball talented candidates. Even if they accept, they won't stay.
Next on my list is title. Titles generally only mean something in relation to other positions at a given company. The primary players in the QA arena should be in keeping with similar positions in the DEV arena. I know the difference between a lead, a manager, a director, and a VP. So the likelihood of me being impressed by a sexy title that any competent senior analyst could handle is slim and none. But I think companies that offer such titles are smart, smart, smart. There are many talented leads out there that would really like the word "manager" or "director" in their title. For me, the job title has to match the job responsibilities. That said, my ego is just as healthy as everyone else's - if the job is great and the money is good, having a sexy title like "Supreme Ruler of the Quality Universe" is a nice, albeit unnecessary, perk.
Last on my list is extras. You can never count on an extra; it's usually the first thing taken away if the company has a bad financial year. I expect "normal" benefits and will actually give up some salary for vacation, but generally anything else is just a "nice to have". I've never had a stock option I could actually exercise and show a profit. Bonuses aren't guaranteed. So I'll be interested in any extras, but they're unlikely to sway me much. There are, of course, some extras that fall into a different category altogether. Hiring bonuses or stock grants are more like money in the bank.
So that's my personal list; how does it stack up to yours? Many people I know put money and title first; some choose people/environment.
Regardless, I hope you'll join me on my safari through the jungle that is the Job Market; coming this next week is:
Quality is Free. Or Not.
No Problems Here, Officer...
A Rose By Any Other Name
Blue Light Special in Aisle 9...
I Smell Brimstone
Save the Baby!
Stenography for Fun and Profit
Job Opportunities in Asswipe, North Dakota
Hope you enjoy it; feel free to add your own experiences!!
Monday, May 17, 2010
I CAN NAME THAT TUNE IN....
Do any of you remember the Game Show “Name That Tune”? Two people went back and forth trying to trump each other as to how they could successfully name a tune in the least number of seconds. The person who bid the lowest number of seconds had to name the tune and if they did it successfully, they won Fabulous Prizes.
The arguments between manual and automated testing staff are a lot like Name That Tune. Automated testing inevitably runs faster than a manual tester can work. But sometimes faster isn’t better. Sometimes, you can’t name a tune in 2 seconds.
I was recently asked for a cost/benefit analysis for automation. This isn’t my first cost/benefit analysis for this company. It’s possibly my tenth.
I’ve been asked for many similar analyses at other companies. So I thought I would share some of what I’ve learned from these exercises.
First, there are various types of “automation”. For the purposes of this blog, I’m going to focus on automation of manual regression test cases. This is outside of JUnit, CUnit, internal-to-the-code, white-box testing. This is not performance, stress, or load testing. Those, by their very nature, need to be automated.
I’m talking about the typical company, who wants to automate the test bank run by their manual testers in order to save time and money. They’ve talked to some tool vendors and while they like the look and smell of the KoolAid, they aren’t drinking it quite yet. Hence, the cost/benefit analysis.
It’s infinitely easier to do such an analysis if you are NOT an employee of the company and do not already have an automation team, which I do. I already know the constraints that exist in this environment, and believe that given the direction of the company, they must invest in automation if they want to retain current coverage, keep headcount down, and maintain current error rates (which are low) in production. If they’re willing to give any one of those things up, we can move to a manual model and save some money.
That’s right. Save some money. Right now, today.
If you’re experienced, and you’re NOT an employee of a given company, the first thing you know is that automation is expensive in terms of time, resources, and money. Many companies come into it believing what those tool vendors tell them – that in 6 months, their 100,000+ regression tests will be fully automated and will run in one day. Unattended.
Not.
I spent ten years working on nothing but automation projects, and we have an automation effort underway right now. In my experience, it will take at least 2 years to get your automation program running smoothly and to the point where it starts paying for itself. That’s right. 2 years.
And for those two years, the company will do nothing but fling money down what may appear to be a giant black hole.
Automation is simply not for the faint of heart. It’s not for the tight of budget, either. And above all, it’s not for the inexperienced.
Automation of this type is not, ever, a short-term proposition. Automation, particularly in terms of benefit, requires mature management willing to INVEST in a project that, if done intelligently, can pay off Big Time in the long term.
Overall, many companies will not benefit, at all, ever, by automating existing manual regression tests. That said, some companies will benefit to the tune of millions (that’s right – millions) of dollars per year.
So what kind of environments benefit from automation, and what kind of environments don’t?
When you’re considering cost/benefit, the first thing you need to look at is how the company operates, what kind of projects/products you produce, and what you’ve got in the way of test assets right now.
For example, if your company produces products with a “shelf life” of less than 2 years, and your extremely short-staffed team works exclusively on agile projects, funded reluctantly by a company that continually wants all of you to do more with less and perform heroics to get a 10-man job done by 3 people and the janitor, and your test assets consist of notes you took on a toilet-paper roll when you finally got to take a body break after working 12 hours straight, you have a problem.
If your company produces products with a longer “shelf life” and has more than 10,000 tests that HAVE to be run for every migration, or you have one set of tests that have to run through 10,000 different configurations, it’s a different story. If your company spends at least a million dollars per year on testing, you’ve got some traction. And if your company is interested in long-range planning and savings, I’d say you should explore the possibility of automation further.
Here are a few realities as I’ve experienced them:
1. Your automation team, whether consultant or FTE (full-time employee) will cost you plenty, particularly if you want your team familiar with one particular tool set. You might want to focus on programming skills, rather than skill with one particular tool in order to keep things realistic. Someone who kicks automated butt is likely going to be equally skilled, motivated, and driven to kick butt on any tool set.
2. Since good automators are tough to find, whether you’re hiring FTEs or contractors, expect the search to take you some time, and expect to pay whatever top buck prevails in your part of the country. Most companies that have their hands on someone talented is going to keep them, so finding great people is a combination of determination, skill, and luck. In addition, make sure you have people making hiring decisions that can ask the kind of technical questions that separate the wheat from the chaff. You need programmers, not people who can set checkpoints. I’ve found, particularly in the past few years, that there’s a strong division of talent. More than half do work that involves MODIFYING what has already been set up. What you want is the kind of talent that can DO the set-up. It’s not uncommon for anyone who can spell “test tool” to put test tool experience on their resume.
3. It’s common for managers or executives to believe that the bigger the team, the more work will be produced. And often that’s true. However, the first steps in a good automation effort are to hire the right people to do the job, and to get your framework and processes “stood up” before you throw a bunch of people at it. It doesn’t take ten people to design and develop an automation framework. Plan on two for 3-6 months. THEN staff the rest of the team. And consider investing in a lead who really knows what they’re doing.
4. The tools themselves will cost you some money, and the add-ons, maintenance contracts, etc. will continue to cost you money. HP/Mercury tools are really “famous” for this. It would behoove you to check your region and find out what tools are most prevalent, because it will make it easier for you to find people with the right skill sets.And you’ll need more than just tools for your automation staff. If you want them kicking off multiple runs, you’ll need additional PCs. You’ll need space to store tests and results. Since they’ll be working as auto tests are running, they’ll likely need dual PC’s and monitors. Some things can be set up virtually, some can’t. All of it will cost money, time, and resources.
5. It takes TIME to automate a test. Around 3 hours. That’s a blended estimate, of course. Some will take 3 days. Some will take 5 minutes. But your team will probably never automate as many tests per year as you think they will when you start the automation effort.
6. Before anyone automates a single test, you’re going to have to decide what kind of automation framework you want to build, and then you’re going to have to build it. Forget what you’ve heard about capture/playback. You can go that route, but maintenance is an expensive nightmare. It’s more likely you’ll settle for a keyword/data driven model, just for ease of update. How long will that take? 3-6 months.
7. Before anyone automates a single test, you’re going to have to establish object repositories and function libraries, if you opt for a keyword/data driven model. How long will that take? It depends on the size of your apps. We have about 15,000 tests and it took us a year. We were somewhat held back due to a ginormous technology change project; they converted screens in stages and we had to follow them.
8. Before anyone automates a single test, you’re going to have to analyze your tests and determine what can, cannot, should, and should not be automated and come up with a plan that prioritizes the work. Contrary to popular belief, the work should be structured so that the easiest, low-hanging fruit is tackled first. Why is that? Because you’re going to run into technical snafus and you want those solved before you tackle the more complex and critical functionality.
9. You’re going to need to manage expectations and do a lot of education. Many companies believe you’ll automate everything and will be able to fire your entire manual test team. Generally speaking, a good automation effort will allow you to maintain your CURRENT headcount. We were adding 4-5 resources per year just to handle the expanded test case base. We’ve been able to maintain the current headcount for 2 years because of our automation effort. Without it, we’d be paying another 7 or 8 personnel right now.
10. Another common misconception is that you can hire people to do the automation and then they can all go away after the project. Nope. By then you’ll have new tests to automate. The existing auto tests will have to be maintained. And it’s unlikely your manual testing staff will have the knowledge base to make program changes for complex technical issues.
11. In regards to expectations, you’ll also have some managers that think all you do is a press a button and everything will run itself in an hour. Ahem. This is a common misconception encouraged by tools vendors. Yes, kicking off the job is merely the press of a few buttons. But results HAVE TO BE ANALYZED. Errors will either need bug reports or the auto script itself might need to be modified (due to data, programming errors, etc.).
12. Some managers and executives will think everything can be automated. It can’t. For example, we have a set of Grid Controls from Hell that do not lend themselves to automation. You will run into similar issues. Some tests SHOULDN’T be automated. Most efforts, if you’ve got a great automation team, can be 60% automated or more. That said, we have some apps that are at less than 20%. We have some over 70%. We don’t have a single app at 100%.
13. Some managers and executives will think that not only can everything be automated, it can be automated before the application is available in (some) sort of integrated environment. Not. It is a Big, Fat, Honking Waste of Time to automate what is going to become a regression test before it’s been run manually at least once and the results are as expected. There may be similar issues with people who think a test bank designed to run in one environment is going to magically work in some different environment/configuration. Unlikely, to say the least. Most automated tests are technology-driven and are somewhat dependent on internal code or controls that are transparent to a manual tester.
14. Your cost avoidance (not having to hire additional people) is a cumulative savings. So if you don’t have to hire another 4-5 people this year, and don’t hire another 4-5 people next year, your savings in cost avoidance will be 8-10 resources next year.
15. Your automation resources and assets are pure expense. You could, given sufficient bodies, do the work manually. Automation is normally considered an internal efficiency effort.
16. You’ll probably have to change the way you do business in order to accommodate the needs of an automation team and automated test bank. It’s more difficult (not impossible, just more difficult) if you have test data/prerequisites that constantly change. It’s better to create test data than to work from a production overlay. Changes to functionality and screens, etc. have to be communicated to the automation team ASAP. That can be difficult if communication of changes isn’t handled well in your company.
Whew! All of that said, why does anyone automate?
Money, my dears.
All of that expense and time is worth it if you’re saving hundreds of thousands of dollars per year.
So how does one start determining whether you can and will save hundreds of thousands of dollars per year?
Sometimes it’s quite obvious. You have more than 5K regression tests which must be run for every migration, your annual budget for QA is at least a million dollars, the growth of either your work or your team has been over the top for at least the past two years, and you’ve determined that model is not supportable long-term. This all points towards automation as a partial solution. Why partial? Remember that automation will not address every test in your regression test bank. If you can automate 60%, that leaves a 40% growth rate for your manual testers that you’ll have to address in some other way.
Or maybe you have 500 tests, 2 people on your staff, and an annual budget of less than 200K. There is no point in investing in automation for this scenario. Your costs to get an automation program in place and running will far exceed what you can hope to recoup. And this, to my mind, is what is most disgusting about tool vendors and what is told to people that lack experience with automation of this type. It has to be worth the money. If you’re small, it quite simply IS NOT GOING TO BE WORTH THE MONEY. You’ll spend a million or two on any effort that’s actually good – just to get everything going and reach the point the team starts to pay for itself. If your company doesn’t have a million or two to spend, you’re usually going to either end up with something incredibly bad, or you’ll end up abandoning the effort partway through. Running those 500 tests manually will be cheaper. And unless the two manual testers can build an automation framework and do the automation themselves in their spare time, the effort will never be cost-effective.
Where have I seen automation used effectively? Large banks. Insurance companies. Large software houses. Large companies that offer world-wide services of some kind. Telecom companies. State, federal, and heavily-regulated industries. Any type of company involved in travel or logistics. Several types of scientific research organizations. Companies with large client bases. Manufacturing concerns that are heavily regulated. Web-based companies with a large market share. See the theme here? These are the types of organizations that typically have large regression test banks.
Where have I seen it fail? Most automation efforts fail the first time. It’s incredibly difficult to hire someone who really knows what they’re doing or who can handle the inevitable snafus that will come up along the way. But the companies I’ve seen who have abandoned the effort altogether have all been small companies with limited budgets, or excessively large companies that don’t care if they have to hire ten quadzillion manual testers, especially if they’re consultants. In many large companies, consultants are not considered (financially) in the same way as FTEs (Full Time Employees); the bucket for “bubble” resources is easier to tap. That means they’re more likely to hire contractors and less likely to invest 2 million dollars in automation and a permanent automation team.
What kind of questions have my management asked me? Most recently, my management has asked to understand the costs and benefits of automation, by quarter, for the next 3 years. Fortunately, our investment (i.e. “throwing money down a pit”) years are behind us. We will be “in the black” this year, and savings will be racking up to the tune of over 500K in the next two, and close to a million thereafter. And THAT is what makes automation of a manual regression test bank worthwhile. I’m not even mentioning some intangibles, like running repetitive tests that would take 8 people 5 weeks is now done in 2 days. Or that the automated scripts find as many or more errors than human beings did. Of course, the type of testing I’m talking about would probably be referred to as “checks” by some people and I wouldn’t necessarily disagree. I, personally, am quite appreciative of any automated procedure that takes away what is a relatively thankless and mind-numbing process for an expensive and easily-bored human being and actually does it better.
Overall, however, the warning here is that you need to remember that Naming that Tune might be achievable in 2 seconds, as long as you’re willing to pay $500,000 per second to make it happen. Is it worth it? Well, if you have thousands of tunes to name, yes. When you make good choices as to when and how to automate, and you see a plan actually come together (and many thanks to my own A Team!), it truly is a Beautiful Thing. Cost-effective, too.
The arguments between manual and automated testing staff are a lot like Name That Tune. Automated testing inevitably runs faster than a manual tester can work. But sometimes faster isn’t better. Sometimes, you can’t name a tune in 2 seconds.
I was recently asked for a cost/benefit analysis for automation. This isn’t my first cost/benefit analysis for this company. It’s possibly my tenth.
I’ve been asked for many similar analyses at other companies. So I thought I would share some of what I’ve learned from these exercises.
First, there are various types of “automation”. For the purposes of this blog, I’m going to focus on automation of manual regression test cases. This is outside of JUnit, CUnit, internal-to-the-code, white-box testing. This is not performance, stress, or load testing. Those, by their very nature, need to be automated.
I’m talking about the typical company, who wants to automate the test bank run by their manual testers in order to save time and money. They’ve talked to some tool vendors and while they like the look and smell of the KoolAid, they aren’t drinking it quite yet. Hence, the cost/benefit analysis.
It’s infinitely easier to do such an analysis if you are NOT an employee of the company and do not already have an automation team, which I do. I already know the constraints that exist in this environment, and believe that given the direction of the company, they must invest in automation if they want to retain current coverage, keep headcount down, and maintain current error rates (which are low) in production. If they’re willing to give any one of those things up, we can move to a manual model and save some money.
That’s right. Save some money. Right now, today.
If you’re experienced, and you’re NOT an employee of a given company, the first thing you know is that automation is expensive in terms of time, resources, and money. Many companies come into it believing what those tool vendors tell them – that in 6 months, their 100,000+ regression tests will be fully automated and will run in one day. Unattended.
Not.
I spent ten years working on nothing but automation projects, and we have an automation effort underway right now. In my experience, it will take at least 2 years to get your automation program running smoothly and to the point where it starts paying for itself. That’s right. 2 years.
And for those two years, the company will do nothing but fling money down what may appear to be a giant black hole.
Automation is simply not for the faint of heart. It’s not for the tight of budget, either. And above all, it’s not for the inexperienced.
Automation of this type is not, ever, a short-term proposition. Automation, particularly in terms of benefit, requires mature management willing to INVEST in a project that, if done intelligently, can pay off Big Time in the long term.
Overall, many companies will not benefit, at all, ever, by automating existing manual regression tests. That said, some companies will benefit to the tune of millions (that’s right – millions) of dollars per year.
So what kind of environments benefit from automation, and what kind of environments don’t?
When you’re considering cost/benefit, the first thing you need to look at is how the company operates, what kind of projects/products you produce, and what you’ve got in the way of test assets right now.
For example, if your company produces products with a “shelf life” of less than 2 years, and your extremely short-staffed team works exclusively on agile projects, funded reluctantly by a company that continually wants all of you to do more with less and perform heroics to get a 10-man job done by 3 people and the janitor, and your test assets consist of notes you took on a toilet-paper roll when you finally got to take a body break after working 12 hours straight, you have a problem.
If your company produces products with a longer “shelf life” and has more than 10,000 tests that HAVE to be run for every migration, or you have one set of tests that have to run through 10,000 different configurations, it’s a different story. If your company spends at least a million dollars per year on testing, you’ve got some traction. And if your company is interested in long-range planning and savings, I’d say you should explore the possibility of automation further.
Here are a few realities as I’ve experienced them:
1. Your automation team, whether consultant or FTE (full-time employee) will cost you plenty, particularly if you want your team familiar with one particular tool set. You might want to focus on programming skills, rather than skill with one particular tool in order to keep things realistic. Someone who kicks automated butt is likely going to be equally skilled, motivated, and driven to kick butt on any tool set.
2. Since good automators are tough to find, whether you’re hiring FTEs or contractors, expect the search to take you some time, and expect to pay whatever top buck prevails in your part of the country. Most companies that have their hands on someone talented is going to keep them, so finding great people is a combination of determination, skill, and luck. In addition, make sure you have people making hiring decisions that can ask the kind of technical questions that separate the wheat from the chaff. You need programmers, not people who can set checkpoints. I’ve found, particularly in the past few years, that there’s a strong division of talent. More than half do work that involves MODIFYING what has already been set up. What you want is the kind of talent that can DO the set-up. It’s not uncommon for anyone who can spell “test tool” to put test tool experience on their resume.
3. It’s common for managers or executives to believe that the bigger the team, the more work will be produced. And often that’s true. However, the first steps in a good automation effort are to hire the right people to do the job, and to get your framework and processes “stood up” before you throw a bunch of people at it. It doesn’t take ten people to design and develop an automation framework. Plan on two for 3-6 months. THEN staff the rest of the team. And consider investing in a lead who really knows what they’re doing.
4. The tools themselves will cost you some money, and the add-ons, maintenance contracts, etc. will continue to cost you money. HP/Mercury tools are really “famous” for this. It would behoove you to check your region and find out what tools are most prevalent, because it will make it easier for you to find people with the right skill sets.And you’ll need more than just tools for your automation staff. If you want them kicking off multiple runs, you’ll need additional PCs. You’ll need space to store tests and results. Since they’ll be working as auto tests are running, they’ll likely need dual PC’s and monitors. Some things can be set up virtually, some can’t. All of it will cost money, time, and resources.
5. It takes TIME to automate a test. Around 3 hours. That’s a blended estimate, of course. Some will take 3 days. Some will take 5 minutes. But your team will probably never automate as many tests per year as you think they will when you start the automation effort.
6. Before anyone automates a single test, you’re going to have to decide what kind of automation framework you want to build, and then you’re going to have to build it. Forget what you’ve heard about capture/playback. You can go that route, but maintenance is an expensive nightmare. It’s more likely you’ll settle for a keyword/data driven model, just for ease of update. How long will that take? 3-6 months.
7. Before anyone automates a single test, you’re going to have to establish object repositories and function libraries, if you opt for a keyword/data driven model. How long will that take? It depends on the size of your apps. We have about 15,000 tests and it took us a year. We were somewhat held back due to a ginormous technology change project; they converted screens in stages and we had to follow them.
8. Before anyone automates a single test, you’re going to have to analyze your tests and determine what can, cannot, should, and should not be automated and come up with a plan that prioritizes the work. Contrary to popular belief, the work should be structured so that the easiest, low-hanging fruit is tackled first. Why is that? Because you’re going to run into technical snafus and you want those solved before you tackle the more complex and critical functionality.
9. You’re going to need to manage expectations and do a lot of education. Many companies believe you’ll automate everything and will be able to fire your entire manual test team. Generally speaking, a good automation effort will allow you to maintain your CURRENT headcount. We were adding 4-5 resources per year just to handle the expanded test case base. We’ve been able to maintain the current headcount for 2 years because of our automation effort. Without it, we’d be paying another 7 or 8 personnel right now.
10. Another common misconception is that you can hire people to do the automation and then they can all go away after the project. Nope. By then you’ll have new tests to automate. The existing auto tests will have to be maintained. And it’s unlikely your manual testing staff will have the knowledge base to make program changes for complex technical issues.
11. In regards to expectations, you’ll also have some managers that think all you do is a press a button and everything will run itself in an hour. Ahem. This is a common misconception encouraged by tools vendors. Yes, kicking off the job is merely the press of a few buttons. But results HAVE TO BE ANALYZED. Errors will either need bug reports or the auto script itself might need to be modified (due to data, programming errors, etc.).
12. Some managers and executives will think everything can be automated. It can’t. For example, we have a set of Grid Controls from Hell that do not lend themselves to automation. You will run into similar issues. Some tests SHOULDN’T be automated. Most efforts, if you’ve got a great automation team, can be 60% automated or more. That said, we have some apps that are at less than 20%. We have some over 70%. We don’t have a single app at 100%.
13. Some managers and executives will think that not only can everything be automated, it can be automated before the application is available in (some) sort of integrated environment. Not. It is a Big, Fat, Honking Waste of Time to automate what is going to become a regression test before it’s been run manually at least once and the results are as expected. There may be similar issues with people who think a test bank designed to run in one environment is going to magically work in some different environment/configuration. Unlikely, to say the least. Most automated tests are technology-driven and are somewhat dependent on internal code or controls that are transparent to a manual tester.
14. Your cost avoidance (not having to hire additional people) is a cumulative savings. So if you don’t have to hire another 4-5 people this year, and don’t hire another 4-5 people next year, your savings in cost avoidance will be 8-10 resources next year.
15. Your automation resources and assets are pure expense. You could, given sufficient bodies, do the work manually. Automation is normally considered an internal efficiency effort.
16. You’ll probably have to change the way you do business in order to accommodate the needs of an automation team and automated test bank. It’s more difficult (not impossible, just more difficult) if you have test data/prerequisites that constantly change. It’s better to create test data than to work from a production overlay. Changes to functionality and screens, etc. have to be communicated to the automation team ASAP. That can be difficult if communication of changes isn’t handled well in your company.
Whew! All of that said, why does anyone automate?
Money, my dears.
All of that expense and time is worth it if you’re saving hundreds of thousands of dollars per year.
So how does one start determining whether you can and will save hundreds of thousands of dollars per year?
Sometimes it’s quite obvious. You have more than 5K regression tests which must be run for every migration, your annual budget for QA is at least a million dollars, the growth of either your work or your team has been over the top for at least the past two years, and you’ve determined that model is not supportable long-term. This all points towards automation as a partial solution. Why partial? Remember that automation will not address every test in your regression test bank. If you can automate 60%, that leaves a 40% growth rate for your manual testers that you’ll have to address in some other way.
Or maybe you have 500 tests, 2 people on your staff, and an annual budget of less than 200K. There is no point in investing in automation for this scenario. Your costs to get an automation program in place and running will far exceed what you can hope to recoup. And this, to my mind, is what is most disgusting about tool vendors and what is told to people that lack experience with automation of this type. It has to be worth the money. If you’re small, it quite simply IS NOT GOING TO BE WORTH THE MONEY. You’ll spend a million or two on any effort that’s actually good – just to get everything going and reach the point the team starts to pay for itself. If your company doesn’t have a million or two to spend, you’re usually going to either end up with something incredibly bad, or you’ll end up abandoning the effort partway through. Running those 500 tests manually will be cheaper. And unless the two manual testers can build an automation framework and do the automation themselves in their spare time, the effort will never be cost-effective.
Where have I seen automation used effectively? Large banks. Insurance companies. Large software houses. Large companies that offer world-wide services of some kind. Telecom companies. State, federal, and heavily-regulated industries. Any type of company involved in travel or logistics. Several types of scientific research organizations. Companies with large client bases. Manufacturing concerns that are heavily regulated. Web-based companies with a large market share. See the theme here? These are the types of organizations that typically have large regression test banks.
Where have I seen it fail? Most automation efforts fail the first time. It’s incredibly difficult to hire someone who really knows what they’re doing or who can handle the inevitable snafus that will come up along the way. But the companies I’ve seen who have abandoned the effort altogether have all been small companies with limited budgets, or excessively large companies that don’t care if they have to hire ten quadzillion manual testers, especially if they’re consultants. In many large companies, consultants are not considered (financially) in the same way as FTEs (Full Time Employees); the bucket for “bubble” resources is easier to tap. That means they’re more likely to hire contractors and less likely to invest 2 million dollars in automation and a permanent automation team.
What kind of questions have my management asked me? Most recently, my management has asked to understand the costs and benefits of automation, by quarter, for the next 3 years. Fortunately, our investment (i.e. “throwing money down a pit”) years are behind us. We will be “in the black” this year, and savings will be racking up to the tune of over 500K in the next two, and close to a million thereafter. And THAT is what makes automation of a manual regression test bank worthwhile. I’m not even mentioning some intangibles, like running repetitive tests that would take 8 people 5 weeks is now done in 2 days. Or that the automated scripts find as many or more errors than human beings did. Of course, the type of testing I’m talking about would probably be referred to as “checks” by some people and I wouldn’t necessarily disagree. I, personally, am quite appreciative of any automated procedure that takes away what is a relatively thankless and mind-numbing process for an expensive and easily-bored human being and actually does it better.
Overall, however, the warning here is that you need to remember that Naming that Tune might be achievable in 2 seconds, as long as you’re willing to pay $500,000 per second to make it happen. Is it worth it? Well, if you have thousands of tunes to name, yes. When you make good choices as to when and how to automate, and you see a plan actually come together (and many thanks to my own A Team!), it truly is a Beautiful Thing. Cost-effective, too.
LOOKING FOR LOVE IN ALL THE WRONG PLACES....
Well, it was bound to happen eventually; I am, as they say "open to new opportunities"; I've just been re-orged into oblivion.
Well, five years is a good chunk of time and I do take comfort in knowing there is no one they could have hired, anywhere, that could have done more with less. The staff is top-notch, and I can move on knowing the company is infinitely better off because I was there.
Yes, there was drama, treachery, filth, and lies - hey it was a re-org! At least much has now been explained. I knew something was up, just not quite what that something was. I don't think I'll ever become that political; I wouldn't be able to sleep at night and I think I'd hate myself.
So I'll see what turns up; I actually enjoy going to new places and meeting new people, so other than worrying about starving to death, I might actually have a Good Time....
In the meantime, thanks to all of you who were part and parcel of "the best job I ever had"; many people go through their entire careers without having such an experience!!
I do have a post I've been saving regarding automation and I'll get it out shortly. In the meantime, I think some Jack Daniels and a straw would be in order...
Well, five years is a good chunk of time and I do take comfort in knowing there is no one they could have hired, anywhere, that could have done more with less. The staff is top-notch, and I can move on knowing the company is infinitely better off because I was there.
Yes, there was drama, treachery, filth, and lies - hey it was a re-org! At least much has now been explained. I knew something was up, just not quite what that something was. I don't think I'll ever become that political; I wouldn't be able to sleep at night and I think I'd hate myself.
So I'll see what turns up; I actually enjoy going to new places and meeting new people, so other than worrying about starving to death, I might actually have a Good Time....
In the meantime, thanks to all of you who were part and parcel of "the best job I ever had"; many people go through their entire careers without having such an experience!!
I do have a post I've been saving regarding automation and I'll get it out shortly. In the meantime, I think some Jack Daniels and a straw would be in order...
Subscribe to:
Posts (Atom)
Posts
