How NOT to Ruin Your DIY Employee Survey


by HR Toolkit staff

buried in survey dataSo morale sucks, your management team has decided to conduct a survey of your 400 employees, and you just got the job of designing it, collecting the data and reporting the results to senior management. All on a budget a fraction of the one set for the last company picnic. Easy right? If the slick marketing efforts of the DIY survey people are to believed, then it should take no more effort than a copy & paste, a few clicks to upload emails and a download of fancy reports at the end. Cheap, easy and results guaranteed! Even a monkey could do it, right?

Well maybe not. In reality, a lot can go wrong and believe us, you don't want an employee survey disaster on your hands that destroys your employees' trust and collapses morale completely.

Cheap isn't always better

When it comes to DIY surveys, you certainly can't argue with the cheapness. Some tools have monthly subscription rates as low as $30 a month for unlimited responses, others have flat rates and some even have free options (but trust us, nothing is ever free). And the tools themselves are robust, technically solid, easy to use without a PhD in computer science and will allow you to slice and dice the data you collect in a variety of ways. So far so good. The problem, however, is not with the tools themselves, but with the already overworked, underappreciated HR managers who get assigned the task of using them to collect data on complex topics and then analyze that data correctly to produce actionable results. Easy in theory, not so simple in practice.

Survey research is not easy. Sure, everyone thinks they can write good surveys. But the reality is, most surveys written by novices are bad. Really, really bad. And bad surveys get bad results which lead to lousy management decisions. There is both an art and a science to good survey writing that requires experience and practice. At the very least, anyone considering a DIY survey tool for measuring organizational climate should have a few years of survey design experience under his or her belt. But it doesn't stop there. Even if you've written a few half decent surveys (and no, the last survey you created for planning the menu at that company picnic doesn't count) do you know how to design an effective survey that examines all the drivers and motivations that together paint a picture of your organization's cultural landscape? That needs more than just survey design experience, that needs a deep understanding of workplace culture as well.

"No problem", you say. "We'll form a committee and have everyone pitch in ideas for what they want to see in the survey. Then at the end of the survey we'll just ask if there's anything else the employee wants to add."

Really? If there's anything worse than a survey designed by an amateur, it's a survey designed by a committee of amateurs. If you really want a mish-mash of vague ideas, contradictory concepts and irrelevant babble dressed up like a survey, then a committee is your best bet. In other words, please don't do it. Better to invest a little more money and hire an experienced consultant who can at least give you a good, valid survey framework and help you avoid the most common mistakes that amateurs make. And by the way, that "Please tell us anything else you'd like to add" question is probably the single most useless survey question you could ever ask, ridiculed by professional researchers for good reason (read How to Write An Employee Survey for rules on writing a good survey).

Good or Bad compared to what?

Another drawback of DIY employee surveys is the lack of benchmark data. Yes, you can write and deploy your employee survey quite cheaply, but without comparative data how do you know if your scores are high or low based on national or industry averages? Benchmarks, or normative data, allow you to compare your own scores to those of similar types of organizations in your industry or region. Analyis of survey data without using norms is looking at only half the picture and can often be very misleading. The good news is that many research vendors have normative data available to purchase. You should, however, do your homework on norms before you design and conduct your survey. Normative data is only useful if you are using the exact same wording for questions and the same rating scales. It would be a waste of time and effort to find out after your survey has run that none of the norms available match the questions your "committee" insisted be in the survey. So the rule here is -- find valid normative data first, then design your survey using the norms available to you.

So once you've disbanded your commmittee (hopefully), lined up your survey designer, sourced your benchmark data, written, tested and launched your survey, you're home free, right? Not quite. You now have the problem of how to encourage good participation and honest feedback. Employee surveys generally take 2 weeks to run and a good participation rate is 80% or higher. In order to acheive this you'll need to be monitoring the survey daily and sending out email reminders to the people who've not yet done their survey (or quit halfway through). You'll probably need to email reminders 3 or 4 times before you hit your target. Online survey tools will give you the ability to do this, but here's another problem. They also give you the ability to look at individual surveys -- and your employees know that.

They probably already believe that since you're doing the survey in-house that their comments won't be 100% anonymous and right off the bat that will discourage 100% honesty. But if you now start badgering them to complete their survey (since you know who has and who hasn't) you'll give them the impression that they are being tracked individually. (Big Brother is watching!) And that will erode trust in the survey even further. This is a problem with in-house DIY employee surveys that's not easily solved, and that DIY survey vendors have not adequately addressed. If possible, you should send email reminders to everyone on the list and not just the people who haven't completed the survey. This way there's less of a feeling that people are being "tracked". Better yet, use an external third party to monitor the process and clearly communicate that fact beforehand.

We need HOW MANY reports?

The last step in the process is tabulating the data and producing the reports. Here again, online tools do a stellar job of crunching numbers and producing charts quickly and easily. What they can't do, however, is give you any kind of meaningful interpretation of the numbers or give recommendations for action. That's still your job. Hopefully you'll be able to take the normative data you've aquired and combine this with your survey results to highlight gaps or deficiencies in performance and look for patterns. If your survey was designed well and covered the major drivers of employee satisfation (you know what those are, right?), you should be able to zero in on the areas of concern and opportunities that will give you most bang for your buck in subsequent action plans. The bad news is that you'll also have to present these results to senior management, and no-one likes to be the messenger that gets shot! But if your methodology was sound, your participation was high and there were no major complaints from your survey takers, then your results are far more likely to be taken seriously by management.

If you plan to do a follow-up survey in a year to measure progress your action plans (and why wouldn't you?), you're going to want to create some sort of spreadsheet now with a summary of all your scores so that you can easily track movement from year to year. We're not aware of any DIY survey tool that will let you produce trending reports across multiple surveys, so you're probably on your own here. And management will definitely want to see some measure of progess in your yearly reports. Typically it's common to show results from at least 2 prior surveys in a trending report in order to get a sense of movement. Keep in mind, though, that if the content of the survey changes from year to year (and it almost always does) that makes trending the data a bit trickier.

OK Senior Leaders, over to you!

The final challenge, then, will be to convince management to take action on the survey results. Because in the end, not taking action on an employee survey is far worse than not doing one in the first place. And that will cost your company much, much more than the money you saved by doing it yourself.

Good luck!




Copyright © HR ToolKit Solutions