The Danger of the Open-Ended Response

As I’ve mentioned in previous posts, I will happily take any survey, especially ones that directly affect me or my family…although I don’t always enjoy the experience.

The one pictured above is one from our local school district’s ‘strategic plan’…yep, full of those dreaded open-ended responses. Lots of fill-in-the-blank vague questions about improvement opportunities, academic excellence and other edu-speak buzzwords.

I hate surveys like these…open-ended questions such as the ones shown here are as enjoyable as trying on bathing suits in the dead of winter when I’m still digesting all of those holiday meals.

Why do I mock these questions like others ridicule pie charts? Because they are useless.

A bold statement, I know, but just think about this:

  1. Someone has to read each and every response
  2. That person (or someone else) has to organize the responses into manageable categories
  3. Then, only then, can the responses be tabulated into something that remotely resembles a list
  4. And finally, one can draw conclusions and plan a path forward

Oh, did I mention there are upwards of 40,000 students in our district? You do the math of how much time this is going to take to analyze and report…boggling, eh?

Best practice tip: have a few focus groups to ask these kinds of questions and develop a short response set based on the focus group feedback. This will get rid of almost all of steps 1 through 4.

Don’t forget to allow for an “Other” response because you can’t always capture every possible response, but I bet they could have come up with at least 80%–more than enough to offset the costs of the focus groups.

And, you’ll find most folks will be happy to fill in an “Other” open-end, including me.

Share

3 thoughts on “The Danger of the Open-Ended Response

  1. The other best practice, for surveys like this with large sample sizes, is to show respondents only one or two of these open-ended questions. You don’t need large samples for such questions and, as you point out, large samples are tedious to analyze and may not offer more insight for the effort.

    • Good point re: one or two questions…in this case that was just page 1. There were still more on subsequent pages. It kind of amazes me that they don’t have folks on staff who raise huge red flags when these surveys are constructed. Unless our district is just ‘going through the motions’ of engaging with their population.

Leave a Comment