Concept Testing: Methods, Survey Design, and Examples

In this post we look at concept testing, a highly valuable way to screen ideas before they go live.

Table of Contents: 

• What is concept testing? 
• Benefits of concept testing
• Concept testing methods
• Concept testing use cases
• Concept testing survey design
• Concept testing examples
• Get started on your concept testing
  
  
  

What is concept testing? 

Concept testing is a research method that helps a business understand whether its ideas are set up to succeed in the market. It looks at concepts as a whole, as well as each separate element, to work out what appeals to the target audience and what needs work before the concept becomes a fully-fledged product or service.

Back to Table of Contents

Benefits of concept testing 

In today’s crowded marketplaces, it’s more important than ever for products and services to stand out. When a business has new product ideas, it can be hard to know whether they will be successful or not. There might be a gut feeling that something will work, but this isn’t enough to be sure that investment in the product will reap rewards. Concept testing can help a brand decide whether its idea should be something to pursue in full product development or if it should move on to another idea. Concept testing can also be used to identify improvements for existing products - hen product concepts have been lived and breathed for a while, it can also be easy to lose sight of what the consumer really wants.

Below are just a few examples of how concept testing can benefit your business: 

• Improve products and services 

Service and product launches need to be preceded by thorough research to make them the best they can be. There are several areas of the concept that a business might want to research, including:

- Features: what the concept is offering that is useful, original, or otherwise interesting to the consumer

- Fit with the brand: whether the idea feels like it belongs to the brand, and whether it fits with the brand‘s image

- Usage occasions: when consumers can see themselves using the product, and whether these are numerous and varied enough for the product to achieve high enough sales

- Price: what consumers would expect it to be, and why - as well as reactions to any price points included in the concept

- Fit in competitive set: how the concept differs from other products or services in the category, positively or negatively

• Gain actionable insights

Insights from concept testing help guide the process from an early-stage product idea to a proposition that can be translated into the development of a market-ready offer. For example, a product might yield a strongly positive response, but consumers may suggest improving the colors and font on the concept, showing ways this can be regarded even more positively.

Insights feed into the development and refinement of the concepts being researched, as well as help to guide marketing strategy, launch communication, positioning of the offer, and future design projects.

• Be cost-effective and flexible with your research

Concept testing is a flexible methodology in that virtually any concept can be put into research. Set against the potential for financial losses of an unsuccessful product launch (while also considering the positive effects of product optimization research), the investment is well worth it.

• Get buy-in from stakeholders

It’s not unusual for team members to differ in their opinions on which concepts will be most successful with consumers. The only people who can really tell you whether they’d buy an idea are the target market consumers themselves. Even if they don’t like it, once stakeholders have evidence that a concept is appealing to consumers, they are more likely to back it.

Back to Table of Contents

Concept testing methods

Qualitative

In contrast to the first four quantitative methods described above, qualitative research gives greater leeway for spontaneous reactions to a concept. You can ask open questions such as ‘What do you think about this concept?’, ‘What is your favorite idea within this concept?’ or ‘Tell me about when you might use this product.’ This depth of questioning often lends itself to Qualitative research being most effective in the earlier phases of the product development cycle. Qualitative research is also useful when concrete rather than digital stimulus needs to be provided (e.g. packaging research). This tends to be done amongst a smaller sample than is used for quantitative online surveys, but gives rich information that might not be captured using a questionnaire with pre-defined question parameters.

Monadic (A/B test)

With a Monadic (A/B) test, respondents are shown a single concept and typically asked to rate it in a number of different areas (purchase intent, brand fit, uniqueness, etc.). The advantage here is that respondents devote time to giving thorough feedback on the concept they see, with no bias affecting their responses as they don’t have other concepts with which to compare it. The response rate tends to be good, as evaluating just one concept means a shorter questionnaire, allowing them to stay fully engaged for the entire experience. On the downside, this concept testing method means a greater number of respondents must be recruited to get enough feedback for all of the concepts. Costs are therefore higher and fieldwork could take longer than other methodologies.

Sequential monadic

With this method, respondents see a single concept at a time and answer questions about it before moving on to the next one. This still allows them to consider each concept separately and in detail (like a Monadic A/B test), but also introduces the element of inherent comparison - which some brands may be aiming for and others not. Some respondents may also show a bias for the first concept they were exposed to. Because of the introduction of multiple concepts, some sequential monadic tests can become lengthy which has the potential to affect completion rates. 

Comparative

This method involves showing several concepts to each respondent before asking them to answer questions. This approach asks respondents to compare different concepts, drawing out the advantages and disadvantages of each. It can be very effective in understanding key successes and failures in concepts and provides fast fieldwork with high response rates. However, with comparative testing, depth on the finer details of each concept is less achievable.

Proto-monadic

This approach begins with a sequential monadic method - several concepts are seen in turn and each one is evaluated before moving on to the next - followed by an overall evaluation of which concept is the best one. While it has similar disadvantages associated with sequential monadic testing, it does lead to an understanding of preferred concepts.
Back to Table of Contents

Concept testing use cases

Concept testing is so versatile and can be applied to any research objective where ideas need to be screened for further development or for a product launch. Ideally, two or more ideas should be tested per study so that the chances of a successful concept are optimized; one of the ideas might be far more popular than the others (providing a benchmark of sorts), but if that’s not the case, the appealing elements from each idea will provide guidance on what a winning concept would include.

Some areas where concept testing is often used are:

• Product development

Designing new or developing existing products can be a challenge. Which features should it include? Which characteristics are most important to consumers? Does the color matter? Which price point will make the product most competitive? Putting product concepts in front of respondents and gaining their reactions will quickly tell you which areas you need to focus on to create a winning proposition.

• Logo design

A logo is such a crucial piece of branding to get right. It should be recognizable, instantly associated with your brand, and say something about your brand values. Logo testing with your target audience helps ascertain which designs stand out and communicate the right message about your brand.

• Website design

Concepts for website design can be screened by respondents to ensure layout, navigation, content sections, and color themes are right for your brand and for the purpose of your site. Concepts can be limited to screenshots or brands can opt to trial more sophisticated digital mock-ups.

• Package testing

Different options for packaging - whether they are images or mock-ups - can be presented to respondents for their opinions on standout, brand fit, and general appeal. Testing mock-ups has the added advantage of exploring usage, material, and size perceptions.

• Name testing

What’s in a name? A lot, if you’re a company or product that wants to communicate your strengths and make sure people remember you. To be the next Adidas, Hershey’s, or Twitter, it’s imperative that you get feedback from customers to make sure their associations with your name options are positive and what you intended.

Back to Table of Contents

Concept testing survey design

So you have a product, logo, website, or another idea to test. How do you go about it?

1. Choose your concept survey method

We've talked above about the different methods that can be employed to get consumer feedback on an idea or concept. The method you choose depends on how many concepts you would like to test as well as the type and depth of feedback you would like to achieve on each.

If you have just one concept to research, the monadic test would be the best quantitative option. With two or more concepts to test, sequential monadic, proto-monadic, and comparative tests are possibilities. For a greater level of detail on the various merits of each concept - providing there isn't a long list of concepts to test - a sequential monadic approach is generally preferred so a respondent can focus on giving authentic, genuine responses for each concept before moving on to another. 

If concepts are yet to be fine-tuned, qualitative research is useful for early feedback. Spontaneous comments on the likes and dislikes of each concept help guide concept development to create a tighter proposition. qualitative research can also be conducted alongside quantitative solutions to add emotional depth to consumer responses. 

2. Get your concepts into shape

The quality of the research will depend on the quality of the concepts you put into it. As much as you're looking to get consumer feedback to perfect them, there are many steps you can take to improve them on your own. Make sure you have considered the following:

- If the concepts are written, are they written clearly, with no ambiguity?

- Are the concepts concise? (i.e. don‘t pack too many ideas into each concept)

- Are the concepts consistent in what they communicate to respondents? (Make sure the information and benefits of each concept are comparable - so if you’re testing washing machines, make sure the number of wash cycles, type of cycles, size, color etc., are included in each concept)

- Are all concept visuals high quality?

Written concepts should present the overall idea of the product/service, its benefits (in relation to the consumer need), and the unique selling point (USP). 

3. Design your questionnaire

Once you have selected the concepts that you would like to put into research, you need to be clear about your research objectives. What are you hoping your product/logo/website etc. will achieve? This will help you decide on your metrics and survey questions.

One large benefit of this type of testing is that questionnaires can be templatized, making efforts much more efficient each time you are looking to run a similar type of testing.

Some of the areas to include in your questionnaire might be:

- Overall appeal of the concept

- Likes/dislikes

- Potential usage occasions

- Relevance to consumers' needs (does it fill a need gap?)

- Price and value perceptions

- Intention to purchase/use

- Alignment and/or impact on brand imagery or brand consideration

The objectives will obviously differ depending on what you’re researching. For example, logo research will ask about the appeal of the overall design and the values it communicates, while website research might ask about usability and clarity.

While there is room for open-ended questions in a quantitative survey, results are easier to analyze if the majority of questioning uses scales or ratings-based formulations. This might include likert-scales (those with a pattern such as ‘strongly agree,’ ‘somewhat agree,’ ‘neither agree nor disagree,’ ‘somewhat disagree,’ ‘strongly disagree,’ etc.) or other scoring systems.

Within your survey, you will also need to collect information on your respondents’ demographics, attitudes, and category purchase behavior. This will help you understand which consumers are most likely to respond positively to your idea once it has been launched, as well as to target communication effectively by consumer type.

4. Launch your survey

Good online survey tools will guide you through the questionnaire-building process, helping you create a survey that flows well and uses the right measurements to obtain the results you need. It should provide you with a sample of respondents to complete your questionnaire, or provide the option to use your own customer database (though remember this might mean missing out on competitor customers or potential customers not currently buying into your category).

Back to Table of Contents

Concept testing examples

Companies who launch successful products and services, packaging, names, logos, and websites, know how important effective concept testing is. Often they have to discard numerous unsuccessful concepts before they narrow down ideas to the ones that will perform well. Consumers will never understand how many versions of a product will have been scrapped before they are able to see the one in front of them on the shelf. Magazine publishers test their logos and page designs because they know these are instrumental to the magazine’s personality; haircare brands test concepts for new products to make sure they address the needs of their consumers.

However, sometimes companies go on gut instinct rather than research findings - and wish that they‘d done research first. An example of this is Burger King’s Satisfries, which launched onto their menu in 2013. Positioned as a lower calorie alternative to regular fries, Burger King hadn’t foreseen that lower calorie wasn’t necessarily perceived as more healthy by health-conscious customers. They wanted to forget their diets on the rare occasions that they visited Burger King for a treat, and the higher price point compared to regular fries didn’t help either. If these insights had been revealed in pre-product launch concept testing, Burger King would have saved the huge investment it made in the product. Reception of the product was too low, and Burger King had to pull Satisfries from thousands of outlets.

Booking.com, on the other hand, got it right when their team responsible for helping customers book transport decided to test two different versions of a web feature. Their objective was to help customers book return taxi trips through the booking.com app. To do so, they wanted to test two hypotheses: first, whether customers wanted to book their outward and return taxi journeys together; and second, whether they first wanted to check a single journey and then add a return journey once they’d confirmed the first one. The team at booking.com ran usability tests using two booking prototypes, testing each one amongst different groups of respondents. Once this initial round of testing had been done, the two solutions were put onto the app and researched using monadic A/B tests. The concept that worked best for customers was launched as the permanent booking feature.

Back to Table of Contents

Get started on your concept test

Concept testing is an essential component in creating successful brands, products, and communications. It offers valuable insights into how consumers perceive new entrants to categories and how this will affect their uptake.

quantilope is adept at providing high-quality online concept testing using fully automated A/B tests and other questionnaire solutions depending on whether the research is monadic, sequential, comparative, or proto-monadic. Additionally, quantilope's inColor, a video-based qualitative tool, allows respondents to see, feel, and use concepts before giving their video feedback.

The platform's suite of templates help guide questionnaire building while making the survey customizable for your unique concept testing needs. Whether it’s product concept testing, proposing new features of an existing product, new logo testing, or any other idea your business might have, quantilope can guide you through the process so that you have all the information you need to decide which concepts will result in a successful launch. 

To learn more about concept testing at quantilope, get in touch below: