Why people lie in marketing research interviews and surveys, and how to overcome it

“Everybody lies. People lie about how many drinks they had on the way home. They lie about how often they go to the gym, how much those new shoes cost, whether they read that book. They call in sick when they’re not. They say they’ll be in touch when they won’t. They say it’s not about you when it is. They say they love you when they don’t. They say they’re happy while in the dumps. They say they like women when they really like men. People lie to friends. They lie to bosses. They lie to kids. They lie to parents. They lie to doctors. They lie to husbands. They lie to wives. They lie to themselves. And they damn sure lie to surveys.”

That quote is from the book Everybody Lies: Big Data, New Data, And What The Internet Reveals About Who We Really Are, by Seth Stephens-Davidowitz, a former data scientist at Google.

The book makes the case that behavioral datasets are a far better way to understand how people act or think than asking individuals for their opinions directly.

One of the many examples cited by Stephens-Davidowitz is from an academic study in Denver in 1950. Researchers compared official city data on voter activity to a survey in which residents were asked if they had voted in the recent presidential election.

Even though the study was anonymous, it found that residents exaggerated their voting behavior. In other words, more people said they had voted than had voted in reality.

Another way of phrasing the book’s core argument – primary market research is less accurate than behavioral datasets if you are trying to explore people’s behavior and attitudes.

It might be tempting for market researchers to dismiss ‘Everybody Lies’ because they don’t like the message.

But the job of a marketing researcher is to help clients to understand their customers’ behavior and attitudes better. So we should be learning from the book, not avoiding it.

For some agencies, this might mean delivering data science projects rather than research projects.

But marketing research is sometimes the only way to explore people’s behavior and attitudes, especially in B2B markets, where public datasets are hard to come by.

And this book is probably most useful in helping us to identify ways to improve how we do marketing research projects:

• One key implication is that secondary research should be conducted in all market research projects, whether in addition to, or instead of, primary research. There are a lot of relevant datasets available online. Useful B2B datasets are rare, but any good agency should at least try to look for one
• We have to acknowledge that people lie in surveys. The book helps us to understand why they lie, and therefore to identify ways to minimize lies and maximize accuracy

 

Why people lie in marketing research surveys and interviews

A variety of academics, researchers and data scientists have tried to tackle the question of ‘why do people lie in marketing research?’ Each has different answers, but there seem to be nine overlapping reasons:

• Increasing self-worth by exaggerating or inflating responses. For example, someone might over-inflate their salary to appear slightly better paid. Stephens-Davidowitz gives an excellent example of this: ‘40% of one company’s engineers said they are in the top 5 per-cent’. Sometimes this exaggeration is intentional, but sometimes the individual is lying to themselves. Stephens-Davidowitz identifies a possible cause – people tell ‘white lies’ in their everyday lives, and this spills over when answering surveys
• Providing socially desirable answers. For example, saying that you voted even if you did not. Society tends to disapprove of people who do not vote, so research participants lie about whether they vote or not
• Being defensive about sensitive topics. Respondents are more likely to lie about sensitive issues such as drugs, religion, or personal finance. They may be wary of divulging something about their nature or beliefs they don’t want others to know. They may be embarrassed by the truth because it portrays them as out-of-step with the rest of society. As a result, it may be easier to lie
• Providing an answer that will ‘please’ the interviewer. For example, telling a company that their new product is attractive, even if it isn’t. Research participants are often just trying to help, or to make a good impression, and sometimes that leads them to say what they think an interviewer wants to hear
• Influencing outcomes in their favor. For example, saying that you wouldn’t spend more than $5 for a product when you would pay $10. By doing so, respondents hope that they’ll be able to save money when the product is launched
• Being hurried, annoyed, or malicious. For example, someone may be trying to complete a pop-up survey so that they can read an article. Or a questionnaire may be so poorly designed that the respondent gives up and decides to ruin the results
• Being forgetful. Human memory has its limits. Ask someone to explain a past action, and they might forget it even happened. Or they might recall it, but forget the details
• Not being able to predict the future. In the same way that people struggle to recall the past, they are also incapable of predicting the future. First, situations can change in unforeseen ways. For example, you may tell an interviewer that you plan to buy an expensive laptop the next time you need one. But if your financial situation changes, you may need to select a cheaper model. Second, people are often over-optimistic about how they will act in the future. For example, you may say that aesthetic considerations will drive your laptop purchase. But when you make the purchase, price may play a more prominent role
• Having no incentive to tell the truth. Incentives can be used to nudge people to behave in a specific, desired manner. While you can incentivize someone to participate in research, you can’t always incentivize them to tell the truth. If a consumer is asked to review a website that they will never use, what’s their incentive to provide their genuine opinions?

 

How to minimize lies and maximize accuracy in research surveys and interviews

There are several things that you can do to improve the accuracy of qualitative and quantitative research, both in terms of the research process itself and in terms of the questions you ask.

The research process

• As mentioned above, you can complement the primary research with some secondary research. If you identify some behavioral datasets, you can use them to validate or challenge the insights generated by an interview. For example, if you’re interviewing asset managers about their thoughts on whether an asset class is attractive, consider looking at publicly available information to see if asset managers are investing more or less in that asset. If the data reveals asset managers are spending more, but they say it isn’t attractive, there’s probably a reason why
• If you can’t find a behavioral dataset, can you create one? It is sometimes possible to create scenarios in which you can explore people’s behavior. For example, academics have written extensively about experimental markets and auctions
• People are forgetful. If you want to gather an accurate understanding of their behavior, you need to interview them ‘in the moment’ or not long after it has happened. For example, if you are trying to understand why you won or lost a sale, you need to interview the decision-maker within a few weeks of the decision, not six months later
• Don’t rush people. Give them enough time to reflect on their responses. 30-minute qualitative depths may be cheaper than 60-minute depths, but they are a false economy if all you’re doing is rushing the interviewee. The result will be lower-quality answers
• At the same time, don’t use more of the respondent’s time than you need. Their time is precious, and if your survey/interview is longer than it needs to be, they’ll be more likely to lie
• Emphasize the incentive to tell the truth by articulating why you are doing the research, and how they might benefit. For example, if you are conducting product development research among customers, make clear that their feedback will be used to build a better product. For this approach to work, research participants need to trust that you will act on feedback. You can build this trust by sending post-research communications that outline the research results and commit to actions that will fix issues. And then you have to deliver on these promises
• Ultimately, make sure you follow best practices. If you write a low-quality survey, you will annoy the respondent, and they will be more likely to lie

The research questions

• Avoid leading questions. Respondents who are trying to help will give you the answer they think you want to hear
• Avoid questions that can trigger people to lie, whether to provide a self-aggrandizing and socially desirable response, especially questions regarding personal beliefs
• Avoid language that would shame research participants for telling the truth
• Try to correct for over- or under-exaggeration by ‘calibrating’ the results. For example, if you can work out how much people exaggerate their salary, you can calibrate the responses people have provided. It can be challenging to identify the extent of over-exaggeration, but it is possible in specific scenarios
• Address bias directly. If you are worried about people being over-optimistic about their future intent, you can try to counter it by asking people to do their best to avoid over-optimism. This approach won’t always work, and can introduce a new bias
• Ask research participants how sure they are about their answers (often called ‘certainty scales’). If someone says they are uncertain about their response, you can either discount or change it. However, we often advise against this approach because it is so subjective. For example, what level of uncertainty is enough to make a change?
• Research participants may try to influence research outcomes to suit their needs and desires (see above). We recommend using research questions that use ‘trade-off’ techniques, as they force respondents to choose rather than giving them everything
• Add some ‘red herring’ questions. For example, if you are asking about an individual’s awareness of different logistics brands, consider adding a fake brand to the list. If they say they are aware of the brand, you should discount their response