Consensus is one of those words that can go either way–you either love it or hate it. If you live in the Pacific NW (especially Seattle) it’s a way of life.
After my last, somewhat harsh post of collaborative survey design, I’m happy to report that not only did we manage to create a pretty useful survey (OK, there’s a teeny tiny issue of bad parallel constructs, but heck…it satisfied some loud constituents), but in 24 hours we have achieved almost a 20% response rate. And, no loud complaining about the survey on our local school blogs.
So, while gaining consensus can be long and painful (think root canal pain), it really can offer some positive results.
“Does this survey make my rear end look big?”…and the perils of collaborative survey design.
I’m currently working on another pro-bono project for a task force of…30 people. Yes, you read correctly–thirty people. When I heard there were that many folks my thoughts immediately turned to the perils of ‘group-think’ and collaborative decision-making.
Turning a major initiative over to thirty people is like asking all of your friends (30 of them!) whether your new dress is flattering. Plan for 30 different opinions. In my case, I’m steeling myself for line edits galore and massive wordsmithing of characteristic statements in the survey. Let’s just hope everyone is OK with a 7-point Likert scale.
In our initial meeting, task force members were somewhat clear on the overall objective, but were definitely not in agreement as to what they ‘needed to know‘ in order to make recommendations — my first step in research design. We spent so much time trying to figure that out, that we had very little time to explore the characteristics of a successful program. In fact, while we did manage to come up with a long list of ideal program characteristics, we didn’t have a chance to rank them, so the task force is expected to accomplish this ranking exercise via email this week.
I’m envisioning thirty nit-picky changes to each statements. I hope I’m wrong.
Lesson learned: if you can’t make the group smaller, make the task timelines very short and concrete. Consensus among 30 people is not going to happen, so keep them focused on the outcome, not the steps to get there.
For the last 4+ months I’ve been seriously training for my Kenpo karate black belt…well, really I’ve been training for the past 3-1/2 years, but for the last few months I’ve trained at the expense of everything else (except maybe eating and drinking…Gatorade, lots of it). Aside from learning 4 forms (katas), 4 sets , as well as 64 self-defense techniques with great names like “Cross of Destruction”, “Blinding Sacrifice” and, one of my favorites, “Squeezing the Peach”–we also were required to read two books, perform community service, and, finally, write a thesis.
For my thesis, investigating the use of martial arts metaphors in business intrigued me. Often terms like “Judo Economics”, “Coding Dojo”, “Black Belt Power Networker”, and the latest “Gmail Ninja” illustrate this proliferation of martial arts metaphors both in the business world as well as life in general. Through my research I learned that much of the use of these metaphors came out of the rise in Japanese manufacturing prowess in the 1980′s–when the US economy was stagnating. In addition, Bruce Lee’s “Enter the Dragon” and David Carradine in “Kung Fu“, along with “The Karate Kid” all combined to make martial arts both mainstream and just a bit glamorous.
In addition, the rise of Six Sigma manufacturing strategy and it’s cadre of black belts (a naming convention that was a feat of marketing genius ) contributed to ‘legitimizing’ the use of martial arts metaphors in the business world.
I would have been remiss if I hadn’t taken the metaphor and applied it to my own consulting business. If Google can make me a Gmail Ninja, I figure Pallas Research can use some Kenpo to make market research better.
- Smart decision-making: Gather as much information as you can before making a decision. Part of our student creed is “I will use common sense before self-defense”. In Kenpo, assessing whether you can remove yourself from the situation before resorting to using your well-honed self-defense skills is using that common sense. In market research, gathering data from every available resource before embarking on a custom research project can sometimes negate the need for the potentially expensive undertaking or at least will make the project more focused.
- Be as efficient as possible: One of the three ‘pillars’ in Kenpo is the idea of using gravity to help make your blocks and strikes stronger. This ‘marriage of gravity’ helps us more petite individuals defend ourselves against much larger attackers. For research surveys, try to make them as brief as possible in order to get your information. Sure, you can use lots of questions to really hone in on a granular level, but you risk bad responses/respondents or high abandonment rates.
- Be flexible: I can’t expect that real-world attacks will come exactly the way we have practiced over and over in my karate classes—being able to shift quickly to compensate will be imperative in a self-defense scenario. Market research projects rarely go perfectly as planned. Similarly, relaxing respondent quotas, going to other sources for sample, running different analyses are all potential mitigating activities that can help keep a project on-track.
- Use all your tools: Many of our self-defense techniques give us ‘tools for our tool belt’ that can be used against an attacker. Whether it’s a block, strike, kick or take down—they can all be used individually or together to extricate us from any situation. Quantitative surveys, focus groups, text analytics, ethnography, social media monitoring and neuromarketing are just some of the tools market researchers can use to help companies make smart decisions. And, don’t rely on just one tool to make your decision—triangulating on an answer tends to yield better results.
- Continuous learning: Getting a black belt is just a milestone in my ever-evolving martial arts career. I watch my sensei continue to work and hone his skills (he’s a 4th degree black belt!) and feel that, while I have my lovely, fabulous black belt, I still have much to learn. My market research career, too, is continuing to evolve. While I write surveys and reports, analyze data and partner with super-smart folks, I still need to learn and improve my skills. Whether it’s a seminar, articles or simply reading other market researchers’ blog posts, I’m continuously working to improve my skills as a research consultant.
Very often in market research, we know the answer to our client’s burning question…or at least we think we do.
In a very real-world example (and one that has been all-consuming for me…hence my ‘radio silence’ on Twitter), our local school district is trying to find a solution to a very overcrowded elementary school that houses three distinct programs. Given the option of moving all, or part of one program (Program “A”) brought a flurry of emails, petitions, and you-name-it from angry parents advocating to keep Program “A” intact.
The school district, to their credit, offered to send a survey to parents to gather more feedback. But, with such an overwhelming response to keep Program A together, the district wondered whether they even needed to send out a survey.
This is where I remind folks that even if you think you know the answer…you need to ask the question.
Not just for the obvious reason to ensure all points of view are accounted for, but also, asking “the question” gives you an excellent chance to engage with your constituents. They will welcome the opportunity to tell you what they are thinking–because they feel you are listening…and don’t we all want to listen to our audience/customers/clients?
In the case of the school district, they did send a short survey–three questions, short and sweet.
So, even if you really, truly do know the answer–please take a bit of time and ask the question.
As I’ve mentioned in previous posts, I will happily take any survey, especially ones that directly affect me or my family…although I don’t always enjoy the experience.
The one pictured above is one from our local school district’s ‘strategic plan’…yep, full of those dreaded open-ended responses. Lots of fill-in-the-blank vague questions about improvement opportunities, academic excellence and other edu-speak buzzwords.
I hate surveys like these…open-ended questions such as the ones shown here are as enjoyable as trying on bathing suits in the dead of winter when I’m still digesting all of those holiday meals.
Why do I mock these questions like others ridicule pie charts? Because they are useless.
A bold statement, I know, but just think about this:
- Someone has to read each and every response
- That person (or someone else) has to organize the responses into manageable categories
- Then, only then, can the responses be tabulated into something that remotely resembles a list
- And finally, one can draw conclusions and plan a path forward
Oh, did I mention there are upwards of 40,000 students in our district? You do the math of how much time this is going to take to analyze and report…boggling, eh?
Best practice tip: have a few focus groups to ask these kinds of questions and develop a short response set based on the focus group feedback. This will get rid of almost all of steps 1 through 4.
Don’t forget to allow for an “Other” response because you can’t always capture every possible response, but I bet they could have come up with at least 80%–more than enough to offset the costs of the focus groups.
And, you’ll find most folks will be happy to fill in an “Other” open-end, including me.