Oftentimes when doing a usability test or user interviews, I’m asked by a client to ask participants additional, related questions. I’m often able to say “no” with the reasoning that it falls out of scope of the original research plan. But, there are times where I say “no” simply because the research methodology being used isn’t the appropriate data-capturing vehicle for these questions.
For example, it’s natural — when we hear our customer talk in-depth about how interesting a new product is — to want to know if they would want to buy it. Or, if a participant seems off-put by the color scheme, we naturally want to ask, “What would you prefer?”
Our clients don’t usually get to sit down “face-to-face” with their customers for an hour, so there’s a natural desire to get to know them better. As researchers, we need to finely understand the strengths and limitations of every method piled into our toolkit so that we can appropriately respond to their requests.
User research is primarily qualitative in nature, meaning that we only speak to a handful of people who represent our target audiences. The reason for this is because user research studies are intended to observe behaviors, first, then use interview techniques to elicit deeper detail around one’s mental model, desires, motives and meanings. What’s critical to understand is that we never outright ask questions about these things (like, “What would you like to see?”), but triangulate a series of questions that guide us towards inferences.
Market research, alternatively, is primarily quantitative in nature (via surveys), requiring a statistically-determined sampling of responses from our audiences in order to produce valid data. We ask customers direct questions and provide them a pre-determined set of answers to choose from. The analysis of large data sets reveal customer preferences, attitudes, demographcs, purchase trends, etc. When large data sets are tracked over time, then we can start analyzing for trends.
Referring back to the earlier example, if a client requests I ask a research participant, “Would you buy this?”, I respond that qualitative data is inappropriate and potentially risky (imagine that one overzealous paid participant could remain vivid in a client’s mind). Market research and competitor analyses may be two different ways to learn whether customers will purchase a product; quantitative data is critical.
Or, if a client would like to know if the participant likes the color scheme, I suggest that designing comps and asking “Which do you like better?” in a survey is much more insightful; again, quantitative data is critical.
User and market research aren’t interchangeable. It’s critical that we can clearly relay in plain language which methods are appropriate for what circumstances.
