When soliciting feedback (whether it be a survey, poll or discussion), if you want useful, objective feedback it’s a good idea to leave emotions out of the research design.
I do a lot of work, the majority of it pro bono, for our local school district. It’s not that I’m totally altruistic, although I like to think I am at times. Really, it’s because I want folks to have good, useful, objective feedback so they can make smart decisions about children’s education. And, in order to make smart decisions–whether they are about advanced learning programs or how the PTA should spend generous parental donations, unbiased feedback is required.
I’m learning the hard way that the topic of children’s education is up there with money, religion and politics as a minefield of emotions.
So, how can one best approach this emotional battlefield?
First off, please, please don’t use “Pros and Cons” to elicit feedback. One person’s pros are inevitably someone else’s cons…A better choice would be to use an unbiased statement and ask people whether they agree or don’t agree (in market research terminology, a likert scale). You can determine whether a topic is a ‘pro’ or a ‘con’ by how much someone agrees, or conversely does not agree with your statement.
Avoid ‘loaded questions/statements’. You know the ones where the author’s inherent bias is already worked into the question…like “have you always listened to that awful, noisy rock and roll?” Neutrality is your friend…think like Switzerland.
Finally, be sure to listen to all stakeholders in the process. Even though I balked at collaborative survey-writing, getting buy-in from everyone ensured my own biases were kept in check.
So…even if it’s ‘for the kids’, be sure your research design is impartial.
What is it that compels researchers to anthropomorphize objects? I recently took the survey below asking me to give personality traits to cereal…yep, cereal.
I understand from an advertising point of view that the client would like to learn what respondents associate with a particular cereal brand, but sorry, a cereal brand is not “Intelligent, Independent, Approachable, or Trendy”. It’s cereal. Oats and flakes. Nothing more.
OK, maybe ‘Healthy” would be a more appropriate choice…
So, what about assigning descriptions that actually fit the product? Then I could rate a cereal brand on being healthy, convenient, vitamin-rich and other characterizations that fit the product.
Not “Fun”…breakfast really isn’t all that much fun.
Consensus is one of those words that can go either way–you either love it or hate it. If you live in the Pacific NW (especially Seattle) it’s a way of life.
After my last, somewhat harsh post of collaborative survey design, I’m happy to report that not only did we manage to create a pretty useful survey (OK, there’s a teeny tiny issue of bad parallel constructs, but heck…it satisfied some loud constituents), but in 24 hours we have achieved almost a 20% response rate. And, no loud complaining about the survey on our local school blogs.
So, while gaining consensus can be long and painful (think root canal pain), it really can offer some positive results.
“Does this survey make my rear end look big?”…and the perils of collaborative survey design.
I’m currently working on another pro-bono project for a task force of…30 people. Yes, you read correctly–thirty people. When I heard there were that many folks my thoughts immediately turned to the perils of ‘group-think’ and collaborative decision-making.
Turning a major initiative over to thirty people is like asking all of your friends (30 of them!) whether your new dress is flattering. Plan for 30 different opinions. In my case, I’m steeling myself for line edits galore and massive wordsmithing of characteristic statements in the survey. Let’s just hope everyone is OK with a 7-point Likert scale.
In our initial meeting, task force members were somewhat clear on the overall objective, but were definitely not in agreement as to what they ‘needed to know‘ in order to make recommendations — my first step in research design. We spent so much time trying to figure that out, that we had very little time to explore the characteristics of a successful program. In fact, while we did manage to come up with a long list of ideal program characteristics, we didn’t have a chance to rank them, so the task force is expected to accomplish this ranking exercise via email this week.
I’m envisioning thirty nit-picky changes to each statements. I hope I’m wrong.
Lesson learned: if you can’t make the group smaller, make the task timelines very short and concrete. Consensus among 30 people is not going to happen, so keep them focused on the outcome, not the steps to get there.
For the last 4+ months I’ve been seriously training for my Kenpo karate black belt…well, really I’ve been training for the past 3-1/2 years, but for the last few months I’ve trained at the expense of everything else (except maybe eating and drinking…Gatorade, lots of it). Aside from learning 4 forms (katas), 4 sets , as well as 64 self-defense techniques with great names like “Cross of Destruction”, “Blinding Sacrifice” and, one of my favorites, “Squeezing the Peach”–we also were required to read two books, perform community service, and, finally, write a thesis.
For my thesis, investigating the use of martial arts metaphors in business intrigued me. Often terms like “Judo Economics”, “Coding Dojo”, “Black Belt Power Networker”, and the latest “Gmail Ninja” illustrate this proliferation of martial arts metaphors both in the business world as well as life in general. Through my research I learned that much of the use of these metaphors came out of the rise in Japanese manufacturing prowess in the 1980′s–when the US economy was stagnating. In addition, Bruce Lee’s “Enter the Dragon” and David Carradine in “Kung Fu“, along with “The Karate Kid” all combined to make martial arts both mainstream and just a bit glamorous.
In addition, the rise of Six Sigma manufacturing strategy and it’s cadre of black belts (a naming convention that was a feat of marketing genius ) contributed to ‘legitimizing’ the use of martial arts metaphors in the business world.
I would have been remiss if I hadn’t taken the metaphor and applied it to my own consulting business. If Google can make me a Gmail Ninja, I figure Pallas Research can use some Kenpo to make market research better.
- Smart decision-making: Gather as much information as you can before making a decision. Part of our student creed is “I will use common sense before self-defense”. In Kenpo, assessing whether you can remove yourself from the situation before resorting to using your well-honed self-defense skills is using that common sense. In market research, gathering data from every available resource before embarking on a custom research project can sometimes negate the need for the potentially expensive undertaking or at least will make the project more focused.
- Be as efficient as possible: One of the three ‘pillars’ in Kenpo is the idea of using gravity to help make your blocks and strikes stronger. This ‘marriage of gravity’ helps us more petite individuals defend ourselves against much larger attackers. For research surveys, try to make them as brief as possible in order to get your information. Sure, you can use lots of questions to really hone in on a granular level, but you risk bad responses/respondents or high abandonment rates.
- Be flexible: I can’t expect that real-world attacks will come exactly the way we have practiced over and over in my karate classes—being able to shift quickly to compensate will be imperative in a self-defense scenario. Market research projects rarely go perfectly as planned. Similarly, relaxing respondent quotas, going to other sources for sample, running different analyses are all potential mitigating activities that can help keep a project on-track.
- Use all your tools: Many of our self-defense techniques give us ‘tools for our tool belt’ that can be used against an attacker. Whether it’s a block, strike, kick or take down—they can all be used individually or together to extricate us from any situation. Quantitative surveys, focus groups, text analytics, ethnography, social media monitoring and neuromarketing are just some of the tools market researchers can use to help companies make smart decisions. And, don’t rely on just one tool to make your decision—triangulating on an answer tends to yield better results.
- Continuous learning: Getting a black belt is just a milestone in my ever-evolving martial arts career. I watch my sensei continue to work and hone his skills (he’s a 4th degree black belt!) and feel that, while I have my lovely, fabulous black belt, I still have much to learn. My market research career, too, is continuing to evolve. While I write surveys and reports, analyze data and partner with super-smart folks, I still need to learn and improve my skills. Whether it’s a seminar, articles or simply reading other market researchers’ blog posts, I’m continuously working to improve my skills as a research consultant.