The problem with B2B research panels: understanding the limitations and challenges

One of the most frequently discussed topics in our industry is B2B research panels. Should you use them?

In short, the vast majority will provide unreliable data. In our experience, most online research panels don’t have senior decision-makers, if they have business-to-business respondents at all.

It’s claimed that on average, roughly 20% of online survey respondents are not who they claim to be. 

Yet sometimes, it’s much worse than 20%. Take Everybody Lies: Big Data, New Data, And What The Internet Reveals About Who We Really Are, by Seth Stephens-Davidowitz, a former data scientist at Google. 

In one survey, he found that “40% of one company’s engineers said they are in the top 5 percent” when asked how much they earn.

Some people like to increase their self-worth by over-inflating their salary. But results like this are often a sign that respondents are pretending to be someone they’re not.

The counterargument is that every research project will have some bad respondents and it’s just the job of the panel providers, or the market research company, to screen them out.

It’s possible to do this manually and as some commentators argue, more sophisticated techniques are available, involving data science or even advanced fraud detection systems.

This approach reminds me of the famous line “half the money I spend on advertising is wasted; the trouble is I don’t know which half” from John Wanamaker.

It seems as if some agencies take the view that: “Lots of respondents aren’t genuine; the trouble is we don’t know which ones.” And even if they do know which ones – why use panel sample at all, if you know so much of it is misleading?

Using advanced technology to screen respondents is trying to solve the wrong problem. It’s like trying to find good apples when almost the whole apple cart is rotten to the core.

Too much time and money is spent trying to fight fraud, but the easier and better solution is to use a technique that’s free from fraud in the first place.

While you can conduct genuine B2B audience research at scale, using a panel company is rarely the way to do it. The risks of using a bad B2B panel include:

• Basing business decisions on low-quality or misleading data
• Damaging your credibility if it’s published
• Missing out on true insights and opportunities
• Wasting your research budget
• Losing valuable time

Why market research panels are useful for B2C research

If that’s the case, then why is panel research a viable option for consumer research?

The answer is – whereas most research panels don’t have real B2B respondents, they do have plenty of real consumers.

The panel business model works by providers encouraging people to sign up for market research studies. 

They pre-recruit respondents based on demographic, household, and behavioral data. The panel company sends them relevant market research surveys, offering a small incentive for completing them.

Some panels have an incredible reach, with access to millions of consumers. If you need B2C research, no matter which sector you’re in, it’s likely they can target your audience.

Panel research in B2C also tends to be very cost-effective. Consumer panelists are very cheap to incentivize – most are willing to receive less than $1 to complete a survey.

Sometimes, much less! Not only that, but B2C panel research is usually very fast. 

Even allowing time for quality control, you can often start and finish a B2C survey in one day.

So why do B2B and B2C market research differ in terms of panel quality? Because the financial compensation required for genuine B2B respondents is too high for most panels.

Why they don’t work in B2B research

While most don’t offer supposed senior B2B decision-makers $1 a survey, it’s not much more than that.

Realistically, if you’re interviewing executives whose time costs $100 per hour, or much more, the incentive for a one hour interview should be over $200 to make it worth their time.

Once, we found out that another agency was quoting under $10 each for online surveys with C-suite decision-makers at enterprises. Now, why would real C-suite decision-makers give up their very limited time for just $10 and no other benefit?

It’s too expensive for panels to significantly increase the incentive, or offer something genuine senior decision-makers would want. It’s more profitable to incentivize consumers. 

Also: 

• Senior decision-makers can move from business to business fairly frequently, changing roles or industries; for a company trying to maintain a panel, it’s costly to keep finding and updating that kind of information for its database
• Even if a third-party panel were to offer attractive incentives for a B2B audience, market research is a low priority; they’re more likely to commit to one-off, ad hoc research than a program of frequent surveys, so they’re still unlikely to join a panel.
• Many panels claiming to have B2B respondents are still selling consumer sample – these consumers may have jobs, so they can try to answer questions about business in a survey. But they’re not in senior roles, at a B2B company, in the right industry!

Lastly, many third-party panels don’t provide sample for qualitative research methods – including video interviews, focus groups, and online communities. 

Increasingly, clients want follow-up qualitative discussions with respondents. Often, that’s just not possible with panel sample.

Therefore, our view is that a B2B panel is only appropriate in a few specific scenarios – for example if you wanted to interview employees who are in junior or very prevalent roles.

Naturally, B2B clients want a quick and inexpensive way to get high-quality data about their target audience. 

But market research agencies need to have honest and in some cases, hard talks with clients about the limitations of a B2B research panel – the typically quick, inexpensive option.

Simply put, it’s ‘magical thinking’ to pretend genuine B2B decision-makers are that easy, quick, and cheap to reach. You can survey true B2B respondents at scale, just not via panel.

The problem is that a lot of agencies aren’t prepared to have these tough conversations. Therefore, they resort to magical thinking, trying to make B2B panels work.

The other limitations of B2B research panels

It gets worse. There are additional factors affecting the quality and reliability of research panel participants.

Many B2B panels…

• Have ‘professional respondents’: They complete several surveys a day and some are fraudsters, pretending to be other people to qualify for more studies and money; B2B studies are particularly lucrative for them, because the incentives are higher
• Are targeted by ‘click farms’: These are fake website traffic locations; users open lots of accounts to click through surveys multiple times
• Have bots: These are automated fake accounts that take surveys and submit random answers
• Don’t have a proprietary database: They’re using the same sources for sample as each other – perhaps sampling marketplaces like Cint or Lucid; if you use several panels, you could get the same respondent twice, or three times, or more

Lastly, while there are several good panel providers in market research, some other panels give the industry a bad name. They let respondents take an unhealthy number of surveys, pay them hardly any money, and run ads that could influence the results, for example.

In B2C research, there are enough trustworthy panel providers and viable respondents that with enough experience, market research companies know which ones to use or avoid.

In B2B research, it’s much harder to find a panel company with even a feasible chance of targeting your specific target audience. 

As mentioned, there’s already a high chance that consumers will take your survey instead – you don’t want to also run the risk of getting fraudsters or repeat respondents!

The challenges of recruiting B2B participants

Why are genuine B2B participants difficult – but not impossible – to find, for panel companies and market research companies alike?

These are the main reasons:

1. The target market for B2B research is small
2. Real business decision-makers have little time for research
3. Incentivization in B2B research is complicated
4. Gatekeepers will protect their schedule

Taking each of these in turn:

#1 The target market for B2B research is small

Everyone is a consumer. Therefore, out of a global population of eight billion people, it’s not surprising that there are hundreds of millions of consumers on market research panels.

It’s a different story in B2B, where your target audience is – relatively speaking – much more niche. 

There are fewer buyers and only a small subset of them will take part in market research studies. 

This makes it more challenging to ensure B2B market research projects are robust.

#2 Real business decision-makers have little time for research

Revisiting an earlier theme, senior business professionals don’t join third-party market research panels.

Why would they? Most work long hours and are very well compensated for giving up so much of their time.

There is a way to give them a valuable reward for taking part in B2B market research, but it’s not straightforward…

#3 Incentivization in B2B research is complicated

Even if there is enough budget to provide a high financial incentive for each respondent – e.g. $100, $200, or more – sometimes B2B respondents aren’t allowed to accept it. It may depend on their industry or the company they work for.

A viable alternative is a charity donation. Again though, you’ll need to set aside enough budget for a meaningful donation – otherwise, respondents won’t think it’s worth their time.

The other option is a ‘soft’ incentive. For example, if your research is a customer satisfaction survey, they may see the benefit of providing feedback – potential service improvements.

Or, they may be interested in or curious about the topic you’re researching. Can you share the final results with them? They may be willing to take part just to read the report.

#4 Gatekeepers will protect their schedule

Even if your target respondent is interested in your research, they may never find out about it in the first place.

Emailing or calling them with an invite to take part may result in their ‘gatekeeper’ screening your inquiry.

In other words – a fellow team member, secretary, or personal assistant may decide on their behalf that the research isn’t a priority.

Best practices for finding true B2B research participants

#1 Build a proprietary database for research projects

If you conduct lots of research, setting up your own database of interested participants is a much better option than using a third-party panel.

Your own branded panel will work better than a third-party one, because you have complete control over respondent quality. 

You only invite people identified as genuine – e.g. by checking their LinkedIn profile – and passing your screening criteria. That way, you can also build rich profiles about each panel member.

It doesn’t have to require extra upfront investment either. As the last question in research interviews we tend to ask respondents if they’d consider taking part again in the future. 

While some say no, many agree. You’re not requiring them to take part in lots of surveys after all, just an occasional one when the topic is relevant to them and if they have time.

This approach makes it easier to find respondents for every future research project, when you’re trying to reach a similar target audience.

There are lots of advantages since you can:

• Save money in the long run
• Get insights quicker on future projects
• Run more frequent research studies
• Control respondent quality
• Capture detailed profile information
• Use any type of research methodology

Over time, some panelists will leave – or perhaps you’ll remove them if they change sectors for example. If that happens, there are several ways to top up your panel again.

It’s essential to adhere to all data privacy restrictions when managing a panel, especially if you conduct international B2B market research. The main ones are:

• GDPR in the European Union
• CAN-SPAM in the United States
• CCPA in California
• CASL in Canada

So, make sure you obtain clear consent to store and use participants’ data, as required by the regulations.

#2 Use alternatives to panel sample

Typically, third-party panels provide respondents for quantitative research, where the main goal is to have a high number of complete surveys; that makes the data statistically robust.

Thinking about quantitative vs qualitative research, if your objectives are more exploratory than validatory, the latter is often a better fit for your needs.

Moving away from third-party panels means you can invite consenting participants to studies that use qualitative research methods. As mentioned, many third-party panels don’t offer this as a service.

Aside from using a panel, what are the other options? You can find high-quality respondents for B2B research via:

• LinkedIn or other social media networks
• Calling and emailing them
• Industry events and associations
• Trade publications and websites
• Independent online forums
• Competitor websites and marketing
• Your CRM, website, or other internal sources

In terms of the latter, it’s always a good idea to leverage internal knowledge before starting a research project – it can save you a significant amount of time.

For some projects, the audience in your CRM will be too narrow – for example, if you need to speak to prospects or competitors’ customers primarily.

However, your colleagues in client-facing roles may know the quickest way for you to find people to interview in the wider market.

#3 Take extra steps to improve response rates during recruitment 

Given that incentivization is challenging in B2B, to encourage respondents to take part in your project, take extra steps to engage with them.

We recommend:

• Personalizing your outreach: Don’t send them a generic message
• Using multiple channels: Some will prefer an approach by email, others via a phone call, LinkedIn, social media, a letter in the post, etc.
• Making a compelling case: Explain the benefits of participating, appeal to their sense of curiosity, and offer the most appropriate incentives for your audience
• Avoiding peak business times: E.g. first thing Monday morning, or at the end of the year
• Being patient: They may be busy when you first contact them, but more willing to take part at a later date
• Revealing the research sponsor, if appropriate: While it’s often preferable to withhold this information when B2B recruitment is tough this may not be possible

On the last point, potential respondents are more likely to participate if they can see a letter from the sponsoring company authorizing the research process.