How to do sequential monadic testing in B2B market research

How to do sequential monadic testing in B2B market research

Testing during a development process and before launching is a no-brainer, whether it’s for a big-budget new product or an inexpensive online ad.

A well-designed, in-demand product can drive growth. Over a quarter (27%) of total revenue on average is attributable to new product launches after three years, according to McKinsey.

Similarly, for marketing, an efficient campaign could generate $5 for every $1 spent, Oracle reports. An excellent campaign might achieve a 10:1 ratio.

However, there are right and wrong ways to test. Badly executed tests could bring you to the wrong conclusion – whether for a new product development process or marcomms research.

One of the most widely used market research tests to test product, service, or marketing concepts throughout the development process involves a sequential monadic methodology. 

When you have several concepts to test, sequential monadic research techniques can give you more accurate and fairer results than other methods. In this guide, we’ll cover several alternatives including comparison testing and simple monadic testing.

But it’s also very important to use sequential monadic testing correctly. There’s a wrong way to do it that can provide inaccurate and unfair results too.

Ultimately, when used correctly, you can use sequential monadic research to test any type of customer-facing concept for your business, including:

Contents

What is comparison testing?

What is monadic testing?

What is sequential monadic testing?

How to do fair sequential monadic testing

Best practices for sequential monadic testing in B2B

 

 

What is comparison testing?

First, let’s look at the common alternatives to running a sequential monadic test.

The simplest other option is to show respondents all of your different concepts at the same time, side by side. This is called comparison testing.

It’s only a viable option if you have a few concepts to test. Also, the concepts should be quick and straightforward to grasp.

Usually, this is the quickest form of concept testing. Respondents see all the options at once and may make swift judgments – for example, by quickly ruling out a concept if they strongly prefer other ones.

This version of testing gives you a ‘winning’ concept:

  • There can be no dispute that the research conditions are the same for each material
  • The format encourages respondents to pick their favorite concept while having all options in front of them

There are caveats though. Overall, while this is a good way to get a topline verdict, comparison testing is less likely to provide detailed evaluations of individual concepts.

Straight away, respondents will start drawing comparisons, rather than considering each concept on its own merit. They’re more likely to overlook any finer points that your research team is interested in getting feedback on.

However, after exploring sequential monadic testing, you’ll see that comparison testing can work well as an additional exercise at the end of a research interview.

What is monadic testing?

In a monadic testing process, you only show one concept to respondents. It means they can give their undivided attention to the concept, without needing to think about any others.

This is a great way to get detailed, comprehensive feedback on one sole concept. Even in a quick 10-minute interview, there should be ample time to explore all the important aspects of a concept and ask every research question.

However, it is also possible to use monadic testing with more than one concept. This involves dividing the respondents into different groups, with each one seeing a different concept:

  • Arguably, this only makes sense as an approach if you have so many concepts to test, there are too many for a respondent to see during one interview.
  • In this scenario, we would usually recommend prioritizing the concepts in advance of the research, and then only doing tests on the most viable ones.
  • In that case, a comparison or sequential monadic test is more likely to provide accurate results, with every respondent seeing all the concepts before giving their verdict.

Monadic testing with multiple concepts is more common in B2C research. With a very large sample size, it matters less that respondents do not see each concept, because the data is robust and has a low margin of error.

In that sense, it is similar to consumer A/B testing, except that there can be two or more concepts in monadic testing.

However, in B2B research, it’s less common to have a high enough sample size to do monadic testing with several concepts. Senior decision-makers tend to be harder to reach and more expensive to incentivize, plus the target market is smaller.

Therefore, with monadic testing, you’re much less likely to have a robust enough sample size in B2B to include your customer segmentation in the fieldwork. The results will usually need to be at an overall level only.

What is sequential monadic testing?

For testing multiple concepts, a sequential monadic test usually provides the best of both worlds – respondents assess each concept on its own merit, and then they can make a comparison.

In sequential monadic testing, respondents see each concept one after the other. Every respondent sees each concept.

This format provides an opportunity to review each concept in detail. As long as there is enough time factored in at the end of the interview, you can then run a comparison test too.

But in contrast to solely running a comparison test, with sequential monadic testing, respondents will base their final evaluation on a detailed consideration of each concept.

This process tends to provide more comprehensive results, with less dispute about the overall winner.

That said, there are still some drawbacks to a sequential monadic design approach – but you can adapt the research to address these:

How to do fair sequential monadic testing

Advantages of using a sequential monadic test over other formats include:

  • More cost-effective research than monadic testing because it requires fewer respondents
  • Shorter fieldwork because you don’t need as many respondents
  • More detailed concept evaluation – providing more information for product or marketing teams if they need to make revisions

Disadvantages include:

  • Longer research interviews – whether quantitative or qualitative – are required to cover each concept in detail
  • Risk of order bias – e.g. respondents forgetting about some of Concept 1’s benefits, or taking them for granted, by the time they study Concept 3
  • Risk of recency bias – e.g. respondents favoring Concept 3 because it’s freshest in their minds

But in most cases, it’s straightforward to solve these problems:

#1 Longer research interviews

You need to allow for more time for respondents to analyze each concept in detail – that’s the case for both sequential monadic survey design and qualitative depth interviews. How long you need depends on the number of concepts you have and their complexity.

If the concepts are very quick to analyze, you could still include quite a few in an interview without causing respondent fatigue or running out of time. If they’re complex concepts, there may only be time to discuss two or three.

As touched on earlier, in most cases it should be possible to narrow down the number of concepts that will need a full evaluation before the research begins.

But if you can’t, there are other ways of doing this. For example, you could present all the concepts to respondents at a very high level first and ask them to do an initial prioritization exercise. Then you could take forward two or three to explore and evaluate in more detail.

#2 Risk of order bias

The order you reveal concepts can affect their popularity in several ways. For instance:

  • Showing a very strong concept first could impress respondents to a point where their interest is not only piqued – it peaks too. After that, other concepts could come across as ‘more of the same’ and fail to match the first impressions of Concept 1.
  • Alternatively, by the time respondents have evaluated Concepts 2 and 3 in-depth, they may struggle to remember some of the finer details of Concept 1.

Order bias is simple to solve – just show all the concepts to respondents in a purely random order.

This is easy to do in quantitative research because an algorithm can guarantee a random order. In qualitative research, it just takes a little extra planning before each interview.

The key thing is to make sure that you don’t keep showing the same concept first, or last, and so on.

#3 Risk of recency bias

Biases and emotions affect B2B buying behavior. Also known as the peak-end rule, recency bias is one of the most common.

It describes the disproportionate impact of the most recent experience on customer behavior or perceptions.

In a sequential monadic test, the risk is that the last concept respondents see could prove to be disproportionately popular because it’s freshest in their minds.

If you show three concepts in the same order and Concept 3 receives the most positive feedback, recency bias may have tarnished the results.

As with order bias, showing all the concepts to respondents in a purely random order solves the problem.

Best practices for sequential monadic testing in B2B

#1 Prioritize respondent quality over quantity

Respondents in B2B should be people with an important part to play in product or service purchase decisions. The challenge is that these individuals are often senior, short on time, and hard to incentivize for market research projects.

In market research for a B2C target audience, research panels are a quick and efficient way to reach respondents. However, most of these don’t have real B2B respondents, even if some claim otherwise – very few people in senior roles who are time-poor and earning well are also market research panelists.

Therefore, reliable B2B sequential monadic concept testing requires a little extra work to recruit relevant respondents who will answer your questions accurately.

You could build a proprietary database of interested customers, perhaps using a CRM if you have clear consent. Other options include finding suitable respondents using:

  • LinkedIn or other social media platforms
  • Trade publications and websites
  • Independent online forums
  • Industry events and associations
  • Calling and emailing

Personalize your outreach attempts – don’t use standardized messages. Also, always offer appropriate incentives for a B2B market research project.

#2 Keep the interviews targeted and focused

Before asking respondents to evaluate the concepts, it’s usually highly beneficial to establish some context. Learning more about B2B buyers’ world and the situation they’re in when making purchase decisions helps explain the reasons why they like or dislike the concepts you’re testing.

But after clarifying this, focus on the concept evaluation exercise – there may not be enough time to cover anything else. This is particularly important for qualitative research.

Often, research teams want to get 30 minutes or an hour out of respondents, and for highly exploratory projects, sometimes that’s necessary. But for concept testing, you may be able to get all the information you need in just a 10-15 minute targeted discussion.

It’s worth respecting the time of senior decision-makers for exercises like this – especially since you may need to schedule a follow-up interview with them later on. Moreover, frustrated respondents are more likely to lie in a market research interview (knowingly or not) if it is longer than it needs to be.

#3 Refine the concepts then retest them

For concepts based on design sprints, or needing a more iterative test-and-learn approach, you may well need a second round of interviews. After the first round of feedback, decide if you’re ready to take forward one sole concept or if several are still in the running.

Refine these based on the feedback, then test them again in a second round of interviews. This provides more reassurance that the revisions you’ve made to a concept were the right ones.

It’s more efficient to recruit as many of the original respondents as possible. That’s a key reason why it helps if you can keep their first interview short – they’re more likely to take part again if you respect their time earlier in the project.

Summary

When used correctly, sequential monadic research can test any type of concept including service or product concepts; product packaging; pricing models; messaging frameworks; marketing materials; brand positioning; logos and straplines; and brand or product naming.

What is comparison testing?

The simplest other option is to show respondents all of your different concepts at the same time, side by side – this is called comparison testing. It’s only a viable option if you have a few concepts to test, but often, it doesn’t work well for getting in-depth feedback.

What is monadic testing?

In monadic testing, you only show one concept to respondents. It means they can give their undivided attention to the concept, without needing to think about any others.
However, it is also possible to use monadic testing with more than one concept. Groups of respondents will see different concepts, without being able to make direct comparisons.

What is sequential monadic testing?

In sequential monadic testing, respondents see each concept one after the other. Every respondent sees each concept.
This format lets them review each concept in detail. As long as there is enough time factored in at the end of the interview, you can then run a comparison test too.

Pros and cons of sequential monadic testing

Advantages of using a sequential monadic test over other formats include: more cost-effective research; shorter fieldwork; and more detailed concept evaluation
Disadvantages: longer research interviews; risk of order bias; and risk of recency bias. But in most cases, it’s straightforward to solve these problems.

Best practices for sequential monadic testing in B2B

We recommend that you: prioritize respondent quality over quantity; keep the interviews targeted and focused; refine the concepts then retest them.

Chris Wells
Share:

Got a B2B market research project
you’d like to discuss?

Contact us

More from the blog

How to optimize your B2B ecommerce strategy using market research

How to

April 23, 2024

How to optimize your B2B ecommerce strategy using market research

We explore the range of B2B ecommerce models and how market research can inform your launch or optimize a current strategy.

How to optimize your B2B sales cycle using market research

How to

April 1, 2024

How to optimize your B2B sales cycle using market research

The Adience guide to getting market research insights that help improve your performance throughout the B2B sales cycle.

How to do in-depth B2B market intelligence research: our guide

How to

March 7, 2024

How to do in-depth B2B market intelligence research: our guide

We explore the different definitions of market intelligence, how it supports your business goals, and how to use it in B2B.