Concept testing is used to develop ideas across a variety of mediums: products, services, copy, branding elements - even dating profile effectiveness.
There are few things more rewarding than being on the precipice of selling your first product.
After spending weeks, months work-shopping and developing your product ideas - you’re finally at a point where you feel ready. Ready to face the world, give them your pitch, and watch as you draw crowds of adoring customers - all clamoring for a taste of your creation.
While it can be extremely tempting to want to rush our projects off of the drawing board and into the hands of our customers, any seasoned entrepreneur will tell you this - it pays to be patient.
The vast majority of new products fail at launch. The reason for it is simple; most new businesses fail to concept test their ideas.
When we test our concepts, what we’re really doing is finding out if an audience exists for our product; meaning, is our product a good idea?
Concept testing is used to develop ideas across a wide variety of different mediums: products, services, copy, branding elements - even dating profile effectiveness.
Concept tests place a mock version of a product in front of a sample audience of consumers - preferably ones from your target market. From there, customers give you feedback on your ideas: evaluating the elements of your design, and sharing their affection (or disaffection) for them.
The primary benefit of concept testing is that they are cheap. Yes, concept testing your ideas is far cheaper than launching a faulty product. Astronomically so.
Take for example, any list of discontinued soda products no longer made by industry titans Pepsi:
These failed product launches have likely cost Pepsi somewhere in the billions of dollars in losses. All because they didn’t connect an audience to their product before launch. Had they tested their concepts more thoroughly beforehand, perhaps they would have realized - there is not a large enough market of people willing to drink a clear, caffeine free cola; let alone one made with yogurt. Yuck.
In addition to saving their users money, concept tests resolve fairly quickly when compared to other forms of survey. Getting pertinent feedback on your designs allows you to make developments to your products with unparalleled speed; tweaking your designs and crafting something your customers are really going to love.
Online surveys allow you to tailor your audiences to your tastes: from a general sampling of the greater population, all the way down to a carefully crafted group of individuals. Most contemporary surveying services have access to thousands of panelists sampled from hundreds of unique demographics; what this allows for is a very precise and efficient form of testing.
There are several forms of concept testing you can utilize to develop your products into a better, more consumer-ready state.
Comparison testing - also known as A/B testing - involves pitting two or more concepts against one another. Survey respondents are asked to compare and contrast two contrasting ideas by way of rating questions, ranked choice voting, or even through direct written feedback.
A/B testing are some of the most common forms of concept testing. They generate immediate and pertinent results. They can often be run at a very low cost. And if panelists are given the option to share their thoughts on their vote, these types of surveys can extract valuable insights from your customers.
One of the many benefits of this type of concept testing is that they require your audiences to make decisive choices about the products you’re testing: which ones they favor, what their sentiment is towards the various product elements, and so forth.
Monadic testing differs from traditional comparison testing in a number of ways. Unlike comparison testing - where one group of audience members are tested - audiences in a monadic test are broken up into several unique groups. From there, each group is asked to give feedback on only one concept.
The benefit of splitting up your audiences and narrowing the scope of each test is that it allows for an intimate, in-depth analysis of each product element. Surveys can be kept short and poignant, asking only questions that are directed at gathering feedback on one avenue of our product’s design. Customers can be asked questions about a product’s look, cost, functionality, and further on.
While monadic tests can oftentimes be one of the best ways to research and develop specific product features, splitting audiences requires that follow-up tests be conducted in order to compensate. Running multiple sets of monadic tests is often more expensive and time-consuming than running one or two simple concept tests.
Just like in typical monadic tests, audiences in sequential monadic tests are splintered off into multiple groups. The primary difference is that each group is shown all of the product’s concepts - not just the one. Each participating group is asked the same set of questions; questions that are randomized in order to reduce the amount of bias implicit in their responses.
Sequential monadic tests are often more affordable to run than several sets of traditional monadic tests. Because each group has a chance to respond to and evaluate the same set of concepts, the overall sample size for your test audience is smaller. For businesses that cater to niche audiences, this type of testing can be a lifesaver.
Compared to other types of concept testing, this method is more time-consuming for the participants; this increases the likelihood that the quality of your results will be degraded by things like non-response bias and incomplete surveys.
Last but not least, comparative monadic testing combines both monadic testing and comparative testing into one unique survey set.
First, participants (within randomly segmented groups) are asked to evaluate the test concepts given to them. After sharing their feedback on the various product elements, they are asked to consider which concept they most prefer.
This kind of testing allows users to reap the benefits of both forms of concept testing, but without the added costs necessitated by running multiple sets of surveys.
The first step in creating a concept test is to identify which of the four types of concept tests you’re looking to adopt for your survey.
In order to decide which type of testing is best for you, here are some questions to consider:
In other words: for what purpose are you testing your concepts?
If you’re trying to get high-quality, qualitative feedback on your design elements, then one of the monadic testing options will be your best bet. If you can afford testing a larger sample audience, then traditional monadic tests will give you the kind of audience insights you need to develop your product: consumer sentiment analysis, usability, etc.
If you find yourself stuck at a design crossroads and unable to make a choice between two or more decisions, then a comparative concept test is what you’re after. Simple A/B tests can be performed quickly and cheaply, and allow you to make executive decisions on the fly by rapidly incorporating the feedback given to you by your survey panelists.
It is good practice to always craft your concept testing questions around your audience. Asking the average consumer to conduct a high-level analysis of the more technical elements of your product will lead you to receive faulty data; panelists will feel unwilling to answer questions they felt were not made for someone with their level of functional expertise. Additionally, long questions or a high number of questions will put off many panelists from wanting to engage with your concept testing survey. This leads to non-response bias, and can impact the validity of your test results.
If your audience has very little familiarity with your product or the market you are operating in, then comparison testing is often best. Comparison tests ask very little of the panelist, and often provide all the context a participant needs to fully answer the question.
If you are looking to survey employees, current or former customers, or even the competition’s customers - splurge for the monadic testing. Gathering valuable feedback from these audiences is critical to the success of your product; using their insights to design and improve will ultimately benefit your project.
Our budget often limits our ability to make the kinds of business decisions we would like. Of course, we would love a bigger office, fancier advertisements, and a better production line; but in business, it’s all about making do with what you have.
Comparative concept tests are quick, cheap, and easy to run. They can be run multiple times for the cost of one traditional monadic test; however, you get what you pay for. Monadic tests are more efficient, and generate a higher level feedback for your analysis. Comparative monadic tests - while more expensive than simple comparative tests - offer the best of both worlds.
Generally, there are four types of questions you can ask your audiences in a survey.
For a more detailed guide on creating survey questions, click here.
Most comparison tests are made up of close-ended static questions, where participants are asked to choose between one or more options in an answer set.
Proof of concept testing in its simplest form presents panelists with images or a description of your product elements and asking them to vote for which one they prefer.
Open-ended questions can be implemented in the form of free-response questions, where audiences can rate your ideas or share their unique perspective about your concept.
Helpfull’s modular surveying platform allows users to develop their own unique surveys from the ground-up; any number of different surveying questions can be seamlessly integrated into your concept test.
For users who are new to surveying audiences, a large selection of pre-built survey templates are available to cover any of your research needs.
For this concept test, we are looking to test the names and logo designs for a hypothetical stationary company. Our core values are sustainability and Eco-friendliness, and so we wanted to incorporate those themes into our name, logo design, and color palette.
We conducted a simple A/B concept comparison test, and allowed users to vote on which logo design & name they preferred the most.
In this last survey, we utilized a comparative monadic test to evaluate the merits of two competing banner advert designs. This time, in addition to selecting their premiere choice, audiences will be given the opportunity to share feedback with their answer.
A close race leaves us with a decisive choice between the two candidates; however, we want to take a closer look at why one design succeeded over the other. So we turn to the audience comments, to see what inspired them to choose our “perfect fit” option as the preferable advert design.
An intuitive user-interface, coupled with the ability to gather hundreds of consumer responses in just minutes, are just a few of the features that make Helpfull the ultimate tool for any artist, designer, marketer, or inquisitive spirit.
Bibliography
Furdyk, Brent. “Pepsi Products That Were Massive Fails.” Mashed, 2020, https://www.mashed.com/251698/pepsi-products-that-were-massive-fails/. Accessed 12 5 2021.
Sauter, Michael, et al. “When product launches go awry: 50 worst product flops of all time.” USA Today, 2018, https://www.usatoday.com/story/money/2018/07/11/50-worst-product-flops-of-all-time/36734837/. Accessed 13 5 2021.
Set up and begin receiving results within minutes.
Sign up for free, no contract required.
Helpfull is the easiest way to get feedback from thousands of people in minutes. Our online focus group platform provides a pool of qualified panelists to give you their real detailed opinions helping you make better, more informed decisions.
