Compare versions and segments

Interpret differences using comparable samples.

Purpose

See how a change or audience group affects results.

When to use it

With published versions or groups you need to compare.

Before you start

• Responses in both slices
• Comparable questions

Example

Compare the task on mobile and desktop within the same version.

Step by step

1. Fix the context. In the study, open Results and select the task version to compare. Open More filters and keep shared criteria, such as completions only. Check the current sample before separating mobile and desktop. Same task · Same inclusion criteria.

2. Choose comparison. In More filters, use Segment by and choose Device to compare mobile and desktop. For versions, select one with a previous version and enable Compare v… with v…. Check the displayed group names before interpreting the numbers. Mobile · Desktop.

3. Read side by side. Open Questions and find the same block in the comparisons. Read each column’s sample size, then the difference and any intervals or significance shown. If a group lacks enough responses, record that limitation before concluding there was an improvement. Small samples create more uncertainty.

4. Validate the reason. To investigate why, filter one group at a time and open People, then Open dossier. Read answers and evidence from both groups. Copy the filtered URL for a colleague with access; they should be able to reproduce the same slice. Preserve filters when sharing.

How to read the result

• Columns by segment or version
• Differences interpreted alongside sample size and uncertainty

Cautions and limitations

• A larger number does not prove improvement with small samples or different audiences.
• Changed blocks may no longer measure the same thing across versions.

Continue with the interactive guide

Compare versions and segments

Related guides

• Publish and update versions
• Read results and metrics
• Export data and present findings