Conjoint Analysis

 

https://www.surveyking.com/help/conjoint-analysis-explained

 

 

A Guide to Conjoint Analysis

Conjoint Analysis Explained: Examples + Survey Template

Definition: Conjoint analysis is a research technique used to quantify how people value the individual features of a product or service. A conjoint survey question shows respondents a set of concepts, asking them to choose or rank the most appealing ones. When the results are displayed, each feature is scored, giving you actionable data. This data can help determine optimal product features, price sensitivity, and even market share.

Why Is It Important? Conjoint analysis goes beyond a standard rating question. It forces respondents to pick what product concepts they like best, helping identify what your audience truly values.

Interactive Conjoint Analysis Example Question

Conjoint analysis is used by any company wanting to do product research; in this example, a restaurant chain. If the chain wanted to release a new ice cream slot on their dessert menu, conjoint analysis would help determine optimal flavor, size, and price, like in the example conjoint survey question below.

If we offered a new menu item for ice cream, which of the following options would be most appealing to you? Please make one choice per set. If no options look appealing, choose "None."

Set 1 / 2

	Option #1 Select	Option #2 Select	Option #3 Select	Select
Flavor	Vanilla	Strawberry	Vanilla	I would not choose any
Size	Large	Small	Large
Price	$2 USD	$5 USD	$5 USD
	Option #1 Select	Option #2 Select	Option #3 Select	Select
Flavor	Vanilla	Strawberry	Cookie Dough	I would not choose any
Size	Small	Large	Small
Price	$5 USD	$2 USD	$5 USD

This is an interactive example of choice based conjoint

When to Use Conjoint Analysis

Without conjoint analysis it would be impossible to ask about product prices along with flavor and size; a separate rating question for each flavor and size combination is needed. Conjoint analysis solves these problems with a straightforward survey question. When respondents evaluate this question, concept features are compared against one another, and a researcher can identify preferences.

Conjoint analysis is useful in two specific scenarios, marketing research and pricing analysis.

Marketing Research

Conjoint analysis is used in marketing research to identify what features of a product or service are most appealing to a customer base. This research can be conducted on existing products to improve advertising engagement or identify areas of improvement to increase sales. Conjoint analysis could also be used to conduct preliminary research for product feasibility.

A conjoint study will usually include demographic questions such as gender. A marketing executive can then segment the survey data by gender, revealing hidden insights used to bolster marketing strategy.

Pricing Research

Conjoint analysis is useful in pricing research because it forces customers to decide using trade-offs, helping to identify optimal prices for various levels. The ice cream example we use in this document has a $5 USD price with the highest utility, which is paired with a "medium" size. Without a conjoint study, it would have been logical to assume the "large" size should be sold for $5. Because of the trade-offs, the optimal size and price combination was found.

How to Conduct a Conjoint Analysis Study

Often, preliminary data needs to be collected before running your conjoint study. An initial survey would include a MaxDiff or a Van Westendorp question to determine important product features or an acceptable price range. The preliminary survey acts as a baseline to reduce the number of conjoint concepts. A smaller number of concepts reduces survey fatigue and increases the quality of responses.

You also want to organize any custom data that you can be used in the survey. Suppose you want to segment your research by country (USA vs European customers). In that case, you need to make sure that internal data is valid, complete, and accessible by your team before running the conjoint study. If custom data is unavailable, you can add additional questions to the survey before the conjoint question.

With the preliminary survey data in hand and custom data organized, you can now create your conjoint analysis study. You can upload the product attributes and levels, include custom data, and you can add follow-up questions to ensure a successful project.

Conjoint Analysis Terminology

Conjoint analysis is an advanced research technique that uses a variety of unique terminology. To help you get a complete understanding, here is a list of commonly used conjoint terminology:

Attribute

The high-level product features that respondents will evaluate are called attributes. Attributes are the first column in the above example question. That example has the following features: flavor, size, and price. If you studied a new car offering, you might have features such as color, make, model, MPG, and tire type. There is a limit of 20 attributes on the SurveyKing platform.

Levels

The items listed within an attribute are called levels. In the example, the "Flavor" attribute has levels of "Chocolate," "Vanilla," "Cookie Dough," and "Strawberry." When you create the conjoint survey, you define an attribute and the levels that go with each attribute. There is a limit of 15 levels on the SurveyKing platform.

Concept

Combining all your attributes and levels, which creates a hypothetical product, is called a concept. In the above example, concepts are the columns that respondents choose. Concepts are sometimes referred to as "cards" in statistical software. There is a limit of 7 concepts on the SurveyKing platform.

Set

Also referred to as a task, a set contains multiple concepts or product offerings. Respondents will choose one concept per set and then be shown a new set of concepts. There is a limit of 20 sets on the SurveyKing platform.

Part-Worths/Utilities

This term is the most crucial in conjoint analysis. It defines how a respondent values each attribute level. When all the utilities for all respondents are analyzed, a researcher can determine an overall product value. Utilities are the output of a regression equation.

Utilities have no scale compared to other conjoint projects you run. They only matter in the context of the current question you are looking at.

Sometimes utilities are called "part-worths" or "part-worth utilities." We use the term "utility."

Types of Conjoint Analysis

Choice-Based Conjoint

This is the most common form of conjoint. The example question above is a choice-based conjoint question. Respondents pick the most appealing concept for each set. Each set contains a random set of concepts that are evenly distributed. This type of conjoint best simulates buyer behavior since each set contains hypothetical products (concepts). When respondents choose a complete profile, a researcher can calculate preferences from the tradeoffs made. (e.g even though "Strawberry" isn't a preferred flavor, if the price were low enough, it would still provide consumer utility")

As with most conjoint studies, preliminary research is essential to reduce the number of attributes and levels to choose from. With fewer attributes and levels, the number of concepts is reduced, which lowers survey fatigue. A MaxDiff or ranking survey can be used to find the top four ice cream flavors.

Currently this is the only type of conjoint offered by SurveyKing.

Best/Worst Conjoint

Sometimes referred to as MaxDiff conjoint. Similar to choice-based conjoint, this method shows respondents a set of concepts. In each set, respondents are asked to pick the most/least (or best/worst) concepts. This approach is used when a product or service has features that cause both positive and negative reactions. An example could be studying how parents select daycare. The number of full-time faculty would draw a positive reaction. The percentage of fellow students that are economically disadvantaged could produce a negative reaction.

This is a future addition to the SurveyKing platform.

Adaptive Conjoint

This method is also similar to choice-based conjoint. Respondents pick the most appealing concept for each set, except with this method, the next set of concepts are not random but are tailored based on the previous answers. This method is more engaging to respondents and can help fine-tune the data.

This is a future addition to the SurveyKing platform.

Full-profile conjoint analysis

This method displays many concepts and asks respondents to rate each one based on the likelihood of purchase. This method is outdated and was primarily used prior to the introduction of survey tools that offer choice-based conjoint. Asking to rate lots of concepts at once is error-prone, quickly causes fatigue, and yields low-quality data.

Rating or Ranking Conjoint

Ranking and rating conjoint was the method used for full-profile conjoint analysis. As software has progressed, it is now possible to conduct rating or ranking conjoint similar to a choice-based conjoint. Respondents are shown a set of concepts and asked to rank or rate each concept. They could rank by entering a value for each concept, which sums to 100 for each set, or they could enter a number based on a scale. This method is also sometimes referred to as "Continuous Sum Conjoint".

Ranking conjoint is a future addition to the SurveyKing platform.

Menu-Based Conjoint

Menu-based conjoint is a new conjoint method. This method gives respondents the ability to pick multiple levels from a menu. For example, a car manufacturer could ask respondents to choose a base model and price, just like choice-based conjoint. But then they could also ask to check a box for each additional feature desired such as "Alloy Wheels for $1,500", "Sunroof for $1,000", or "Parking Assist for $1,500".

This method is much more advanced in terms of front-end programming and back-end statistics than choice-based conjoint. Often custom solutions need to be built for a company wishing to create this type of project.

Creating a Conjoint Survey

Any survey that contains a conjoint question is referred to as a conjoint survey. SurveyKing currently only offers choice-based conjoint. Here are the steps needed to create your own conjoint survey:

1. Navigate to the "Builder" page of your survey
2. Click on the "conjoint" element box, drag it into your questionnaire, or click the "Insert question" dropdown to add a conjoint question at the end of a specific page.
3. To add a new attribute, click "Add attribute" within the conjoint builder. The builder will show levels for the attribute to the right of each attribute.
4. Choose how many sets and concepts you want to display.
5. Select any options to customize the question further.

Conjoint Survey Options

• "None" choice - This option will add one additional card, or column, per set that says "None" This option is marked by default. This setting reflects the real world, where consumers can choose not to buy a product. You should exclude this setting from projects where customers are forced to pick an option, such as a government service.
• Reset choices - With this option, respondents can start back at the beginning. The respondent will clear all answers for the question, and the first set will be displayed when the "reset" button is clicked. We recommend reserving this option for specific circumstances, as it could lead to second-guessing and low-quality data.

How Many Attributes, Levels, Concepts, & Sets are Needed?

An ideal conjoint question will have roughly 5 attributes (rows), 4 concepts per set (columns), and approximately 5 - 10 sets. This will help ensure respondents are not fatigued. A detailed breakdown is below:

• Attributes - Roughly 5 attributes with no more than 10 total levels per attribute. Having fewer levels per attribute ensures the survey will show various concepts more often.
• Concepts - Roughly 4 concepts to show each set. Too many concepts per set, and you risk respondents not making effective choices. The total amount of concepts available is calculated by multiplying the number of levels in each attribute. In the example above, we had four flavors, three sizes, and two prices. Total concepts available would be equal to 4 * 3 * 2 = 24. Ideally, this number should be no larger than 50. The more total concepts, the harder it becomes to draw meaningful conclusions.
• Total Sets - Showing no more than 10 total sets to respondents to avoid survey fatigue. Generally, 3-5 are best.

How Many Responses are Needed?

We recommend collecting at least 100 responses for each segment being researched. For example, if you wanted to research both males and females, you would want to collect 100 responses for both.

Conjoint Analysis Scoring & Results

Conjoint analysis uses regression to calculate how different attributes and levels are valued.

Because conjoint uses categorical data (a name like ice cream flavor) instead of continuous data (a number like a temperature), a particular type of regression is used called logistic regression. Just like any regression equation, the result of this regression calculates coefficients. These coefficients are referred to as "utilities".

Utility is not a standard unit of measure. It can be thought of as "happiness". If a lot of respondents choose concepts containing "Cookie Dough" and only a few choose concepts with "Vanilla.", even without doing the math, you can imagine that the coefficient for "Cookie Dough" would be higher than the coefficient for "Vanilla."

To illustrate this concept, we ran the above ice cream example with 20 respondents. Below is the analysis of those responses. This analysis includes the utilities for each level in addition to the relative importance of each attribute.

Sample Survey Data - Summary Table

Attribute	Importance	Level	Utility
Flavor	61%	Chocolate Vanilla Cookie Dough Strawberry	.44 7.13 14.11 0
Size	17%	Small Medium Large	0 4.03 1.61
Price	22%	$2 USD $5 USD	0 5.06

Walking Through the Analysis

The utilities in the last column are the output of regression analysis. Next to each number is a small bar chart for visual representation.

Remember, utilities are not an actual unit of measurement and could be thought of as happiness. If we look at the above table, the "Cookie Dough" flavor has a utility of 14, and the "Vanilla" flavor has a utility value of 7. We could interpret this as "Cookie Dough has double the happiness of Vanilla."

The importance column is the weighted difference in utilities ranges for the product levels. You can see that flavor has the level with the largest difference of roughly 7. The larger the utility differences for an attribute, the more important they are to consumers. To get a significant difference, as we see with cookie dough, many respondents choose concepts with that flavor. We know the other levels are evenly distributed, meaning that cookie dough was a significant driving factor in decision-making regardless of size or price. Here's how you would calculate the importance:

Take the largest number for each level, and sum: 14.11+4.03+5.06 = 23.02

Divide each of the highest levels by this number. The calculation for flavor importance is 14.11 / 23.02 = 61%

Overall, it looks like "Cookie Dough" is the preferred flavor, "Medium" serving size, with a price of "$5 USD". "Vanilla" would also be an excellent addition to the menu as it brings a high amount of utility. You'll notice that the higher price has a higher utility value than the lower price. Respondents might have a perception that a higher cost has a better taste or better quality. The preference for "Medium" might also be tied to consumers being health conscious. This summarizes why conjoint is so essential. Not only did we find the two optimal flavors, but we also found the right size and correct price point. This data would be almost impossible to capture without conjoint analysis.

Statistical Details

SurveyKing uses ChoiceModelR, a package in the R statistical program to compute conjoint utilities. ChoiceModelR calculates a coefficient using logistic regression with the maximum likelihood for each attribute level by each respondent. When the analysis is complete, utilities for each level are averaged. The output of our example can be found in this Excel file.

We color-coded the Excel file for each attribute level. Row 22 has an average subtotal, which the average utility for a specific level. The regression equations use effects coding to ensure each attribute in total sums to 0. Because of this, you will notice the excel file contains negative utilities. We shift each number by a constant to eliminate negatives and put the baseline to 0. The dark blue flavor columns were adjusted by 5.43 before the results being loaded into our dashboard. Having a 0 baseline makes the data easier to interpret.

Data used to populate ChoiceModelR:

• Data Matrix - See this Excel file, which is the input for the ice cream example. The first row of each card set contains the card number chosen (column G). The first card selected was 4. This is because the "none" option was selected. When the "none" is the chosen option, the highest index + 1 is the card selection. This is the input required for ChoiceModelR. Other programs use an output similar to this file. You'll see it's the same setup, except column G has a "1" if the card is selected or "0" if not selected. An additional row is added for the none column.
• R - The total number of iterations of the Markov chain Monte Carlo (MCMC chain) to be performed. Default value: 4000.
• Use - The number of iterations to be used in parameter estimation. Default value: 2000.
• Keep - The thinning parameter defining the number of random draws to save. Default value: 5.
• wgt - the choice-set weight parameter; possible values are 1 to 10. This parameter only needs to be specified if estimating a model using a share dependent variable. Default value: 1.
• xcoding - A number that specifies the way in which each attribute will be coded. We code each attribute as categorical, which is the value 0. Prices could technically be labeled as continuous, but for ease of calculations and consistency, we code all variables are categorical.

Time Spent Per Set

The time spent on each conjoint set is also included in the results. This data is useful to eliminate low-quality responses. Responses that answered a set too fast (under 2 seconds) should generally be eliminated from the results.

Analyzing Concept Profiles

A powerful benefit of conjoint analysis is quantifying how each concept would fare in the market. We can easily see the product with the most utility would be Cookie Dough, Medium, for $5 USD. But what about the top three products? Or the bottom three products? In the ice cream example, there were 24 hypothetical products. Unique to the SurveyKing platform is the ability to scroll through each concept in ranked order, to see what profiles faired the best or worst (or offer the most utility). The reporting section will automatically include the table shown below:

Rank	Flavor	Size	Price	Total Utility
1	Cookie Dough	Medium	$5 USD	488.02
2	Cookie Dough	Large	$5 USD	436.91
3	Cookie Dough	Small	$5 USD	403.10
4	Cookie Dough	Medium	$2 USD	381.51
5	Vanilla	Medium	$5 USD	341.31
6	Cookie Dough	Large	$2 USD	330.40
7	Cookie Dough	Small	$2 USD	296.58
8	Vanilla	Large	$5 USD	290.20
9	Vanilla	Small	$5 USD	256.39
10	Vanilla	Medium	$2 USD	234.80
11	Chocolate	Medium	$5 USD	200.82
12	Strawberry	Medium	$5 USD	191.44
13	Vanilla	Large	$2 USD	183.68
14	Vanilla	Small	$2 USD	149.87
15	Chocolate	Large	$5 USD	149.71
16	Strawberry	Large	$5 USD	140.33
17	Chocolate	Small	$5 USD	115.90
18	Strawberry	Small	$5 USD	106.51
19	Chocolate	Medium	$2 USD	94.31
20	Strawberry	Medium	$2 USD	84.93
21	Chocolate	Large	$2 USD	43.20
22	Strawberry	Large	$2 USD	33.81
23	Chocolate	Small	$2 USD	9.38
24	Strawberry	Small	$2 USD	-

To get these figures from the Excel output file, you could create a table with all possible combinations, and use sumproduct to calculate to total utility. Here is an example.

Conjoint Analysis by Question Segments

Sometimes it's important to analyze different segments, such as gender. To do this, add a multiple-choice question to your survey for each segment you wish to study. In the reporting section, you can choose "Conjoint Segment Report." From here, select the appropriate question, and the report will output a data table for each answer. Using the ice cream example, you may notice "Males" prefer "Cookie Dough," while "Females" prefer "Vanilla." These are additional data points to fine-tune your marketing efforts.

Here is an interactive example of a conjoint comparison report unique to the SurveyKing platform. The first question asks for gender and the second question asks for a preferred ice cream concept. You can see males prefer "Cookie dough" with a utility of 23.06, while females prefer "Vanilla" with a utility of 25.63. Each gender segment lists flavor as the most important attribute. The report also includes a segmented ranking of concepts.

Conjoint Analysis Tips

• Keep descriptions simple - For both attributes and levels, keep the descriptions as short as possible. This will make picking choices easier and reduce survey fatigue.
• Images - Because of limited space, we recommend using images inside of each level sparingly. When images are used, we recommend that each image be custom-made for this project with a size no larger than 150px X 150px.
• Additional descriptions - Let's say you are researching a new phone. If you have a weight level of 7oz and 11oz, people won't be able to gauge that difference. You would want to say (ideally in the question text), "Use the iPhone 7 as a baseline weight, that weight would be considered average" Then the size product labels would be "Light," "Average," "Heavier."
• Be aware of incorrect conjoint content - There is a popular online video that explains conjoint analysis in Excel. The video uses "Dummy Variables" to compute the regression. This would be incorrect for two reasons. Excel cannot do logistic regression without any addons. Also, removing dummy variables is unnecessary if logistic regression is done correctly. The video codes a three-level attribute with 1's and 0's, which results in collinearity. Logistic regression assigns categorical data to a unique number. Like in our example, a four-level attribute would have the numbers 1, 2, 3, or 4, depending on what concepts were displayed.