Friday, October 31, 2008

OFFAL-SHORING

Tough Times for Testers



Well, the economy is definitely having an impact in my region; we have two major employers laying people off. I wish I was hiring FTEs (Full Time Employees) right now – there’s some good talent available. Several small firms are following suit, but they’re “permanently” downsizing. The larger firms are trying the offshore route.

I think it’s time for me to talk about offshoring. I’ve been avoiding it for a while; it’s not a politically-correct topic, and I have a lot of people who work with me reading this blog.

I’ve been a manager for quite some time now and those of you who are also managers know that you can’t avoid the issue of offshoring or the necessity of learning to successfully manage offshore teams. So I’d like to talk candidly about the pros and cons and how to set yourself and your team/company up for success.

In many ways, it’s much easier to find qualified development staff offshore than it is to find good QA/QC personnel. Programming is the more highly regarded and paid profession, therefore it is likely your offshore QA/QC staff will be failed programmers. I don’t know about you, but I don’t customarily hire failed programmers. The thought processes used to logically deconstruct software are quite similar to those used to construct software. Regardless, there is a large percentage of offshore workers who have no particular talent in the field. I realize that’s a harsh statement, but I said I was going to speak candidly and that statement is true from my experience. If you are extremely lucky, you can assume your offshore staff will have the same qualifications as someone in the U.S. straight out of college with maybe a year of experience. You need to tailor the work you want to send offshore accordingly. No amount of wishful thinking will allow someone with a year of experience to perform at the same level as someone with 10 years of experience.

In addition, many workers would like the opportunity to work in the U.S. If your work does not give them that opportunity, they will leave your assignment and hop onto another more lucrative opportunity. Turnover is significant with offshore teams.

Offshore workers are also accustomed to doing things (like producing deliverables for you or testing) in one way and one way only. It is the way they were trained. If what you need deviates from the very traditional “receive specification – write test cases – test – report” scenario, you are going to have major problems.

Communication issues are always significant. Offshore personnel are going to take everything you write and say quite literally and you will have to meet with them often to ensure you are all on the same page and moving forward as expected. Holidays are different overseas and may vary by region. Certain “life events”, like funerals and marriages, require a great deal more time and ceremony than similar events here in the United States and you need to be prepared for that. There will also be cases of illnesses that are extremely rare in the U.S. Malaria, for example.

There is a marked reluctance to bring up problems or issues – you’ll have to work hard to establish the level of trust that will allow an offshore team to speak to you honestly. As a QA Manager, you’ll probably find that positively bizarre, since you probably have the opposite issue with your on-site personnel.

So how do you deal with all of this?

First, when your executive management says your company is going to offshore, DO NOT FIGHT THAT BATTLE. The worst situations I’ve ever seen in a company revolve around fighting for lost causes, appearing insubordinate to your management, and (worst case) attempting to woo other managers to your “side” – which is in direct opposition to executive management. You’ll damage yourself and you’ll damage your team. You will lose that battle. This is a case where the battle cannot be won.

If you work for unenlightened (OK, maybe “flamingly ignorant” is a better term) executives, they might want to offshore EVERYTHING. I can’t help you with that. You’re screwed and although they don’t know it yet, so are they. Smile politely and go find another job as quickly as you can. But if your management is just looking for ways to utilize offshore personnel and cut whatever costs they can in the most efficient way possible, you can do that. I’m lucky in my current position; I work for very smart, very experienced executives who have been there, done that. I have a small offshore team and I still constantly look for ways to save my company money by using them. They know this and are satisfied that I do everything I can without sacrificing quality.

First, force yourself to embrace the concept of offshoring. Yes, I know. It’s like yakking up a hairball. But if you don’t voluntarily, eagerly, cheerfully jump on this new “opportunity” and set the rules of engagement, you might find rules you don’t like much IMPOSED on you. What would you rather do? Offer a plan to management that utilizes offshore personnel in a way that can be successful and that protects both your company and your best staff, or have management get pissed off by your lack of cooperation and just tell you you’re offshoring 60% or more of all available work – a move that will irrevocably damage both your team and your company?

So take charge, and take a hard look at what your group does and produces to determine what can be offshored with a reasonable degree of success. That is, work that can be adequately performed by someone with a year or two of limited experience. A “natural” for this type of work is repetitive regression testing that is not especially dynamic (doesn’t get changed much), manual testing that cannot be automated, kicking off automated test runs, and running a standard set of tests through a large number of different equipment/browser/OS configurations. Anything sent offshore must be extremely well-documented and something your staff has done many times successfully.

I’ve been successful with offshoring efforts that keep analysis tasks onsite and move any repetitive, manual testing tasks offshore. First, in my experience, regression testing finds 3-5% of the total error resident in a given app or migration anyway, which makes it lower risk. Still, the tests have to be run unless you have a really dynamite system analysis group doing impact analysis on every change. Second, your best-trained, experienced onsite personnel will probably welcome the opportunity to move the repetitive work elsewhere and will jump at the chance to work on the newest, latest-and-greatest project work.

To help ensure success, it’s a good idea to pull your senior offshore personnel onsite for training – preferably during a migration. This will give them some idea as to what to expect, what your processes are “really” like, what deliverables are required, and how to interact with your team. I’d also advise you to select a lead for the offshore team and that you interview for this position ONLY. Why is that? The future and success of the lead is dependent on the talent and work of their team. It stands to reason the lead will hire people who will do the work well. In addition, the lead will “know” their own environment and people better than you will. Attempting to impose your normal requirements and expectations onto staff immersed in a completely different culture are doomed to failure.

So have I had problems with moving even regression testing tasks offshore? Yes. You will too. But you can anticipate some glitches and take steps to minimize them. Here at my current position, our first migration with an offshore team involved was pretty heinous – it took the offshore team more than twice as long to do the work. This totally negated our savings. We perservered and told the offshore management team that the work had to either be stepped up, or we would have to reconsider our options. The work for the second migration went very well. And what is most important to me and to my company is that quality levels (errors found in Production) did not increase in either number or severity.

We consider this exercise a success and I can say without fear of contradiction that it is unusual in this region to be able to make that claim. The reason? Most companies do not put much thought into what they offshore. They blindly go forward with either percentages (we will offshore X% of the work) or “everything”, laying off their hottest talent or losing them. Losing them? Oh yes. Once a company embarks on a large offshoring effort, those with the best skillsets will get out of Dodge in a hurry; there are plenty of employers out there looking for the “best of the best”.

Many failures will occur because you cannot, in my opinion, offshore analysis tasks. Your specifications – and you will have to have specifications – would have to be airtight. People who do not speak your language take the written word literally. In addition, there’s a distinct reluctance on the part of offshore personnel to ask questions and raise issues. This means the specifications have to be so clear there aren’t any.

Good luck with that.

The #1 “excuse” offered by offshore personnel for poor testing (or development) efforts is that the specifications were bad. Which is often true and therefore irrefutable. But the rest of us get inadequate specs all the time – that excuse doesn’t hold water locally.

Another common reason for failure is that it is highly likely the company will offshore everything to the same firm. Development, QA/QC, the works. This is a mistake companies make when they outsource locally as well, but the issue is magnified offshore. QA/QC must be independent to be truly effective. Remember, QA/QC people are less well-regarded offshore. It is likely they will ask developers if a given error is really an error, and if the answer is “no”, it won’t get reported or fixed. So your software product is going to be more buggy. QC is Quality Control. If you “give away” your control, you’re going to get whatever you get. Some of the more successful companies I’ve worked with give everything BUT quality control away. They keep QC in-house to ensure they get what they paid for. It’s that final checkpoint (I can see people cringing about “final checkpoint” – yes, I know it shouldn’t be that late in the process, but it usually is regardless of your convictions) before production.

It’s difficult to find depth of experience offshore. That doesn’t mean there’s no talent whatsoever. It means, however, that any company that expects someone who has the experience equivalent of two years to perform at the same level as someone with ten or fifteen is in for somewhat of a shock. With all of the job-hopping and influx of new talent overseas, it’s a huge mistake to offshore work that requires experience and knowledge of your applications to perform well. You’ll finally get someone trained, things will be going well, and –ZAP- they move on. So the work sent offshore needs to be something that could always potentially be handled by someone completely new and not too experienced.

The company doing the offshoring has to have a working, documented system development lifecycle, complete with templates and examples of deliverables. Coding standards are a good idea. The more structured your environment, the better off you’ll be. In addition, the company needs to have people in place to perform training and to review the work. Onsite personnel. That’s not going to be easy, because large offshoring efforts usually result in an exodus of all of your most talented staff. If you want to retain them, it might be a good idea to think about encouraging them to stay, like with a large pile of money.

Several large firms in this area have attempted massive offshoring efforts more than once, and in each case have had to pull the work back in-house, after millions of dollars spent and poor success rates. By then, of course, all of their best people have flown the coop, requiring massive hiring efforts at significantly higher rates.

So what is the answer? I’m not sure there IS an answer that either side the equation will love. Personally, I’m like every other manager I know. I hate offshoring. It’s not that I hate the offshored staff; I don’t hate very many people on a personal basis. It’s more that I can’t really “manage” people I don’t work with face-to-face every day. I can provide some guidance and coordinate activities at best. I’m not immersed in their culture. They aren’t immersed in mine. It’s bloody inconvenient and incredibly difficult to do successfully. That said, however, I think it can work if you choose what to offshore wisely and your expectations are reasonable.

And for those of you who ARE on offshore teams, I’m sorry if this blog has offended you. I realize some of you are both experienced and extraordinarily talented. But that is not true of everyone and it is almost impossible to find entire teams with similar attributes.

Back to the “tough on testers” thought…. My company is certainly not recession-proof, but it is less vulnerable than others, and while changes will undoubtedly take place, I expect my team will weather the storm. For those of you in other industries being hit hard right now – take heart. Experienced people in our field are always at a premium. If you get stuck trying to find a full-time position, consider consulting until the economy improves. I’ve talked to a lot of our consulting vendor reps – they are not feeling any pinch thus far and they’re cautiously optimistic in thinking it’s possible their business might even grow next year. While companies may not be investing in full time staff for a while, they probably will not be cutting back on projects and the need for “bubble resources” (consulting staff) might actually increase…