When user statements conflict with their actions, or different user groups offer opposing feedback, design decisions become unclear. Simply dismissing data or averaging opinions does not help. A structured approach is needed to understand these discrepancies and inform design choices effectively.
Identify the source of contradiction
Begin by categorizing the contradiction's nature. Is it a difference between what a user states and what they actually do? Or does it stem from feedback given by a new user versus an experienced one? Sometimes, the conflict appears across different tasks or usage scenarios. Identifying this source helps focus your analysis. This avoids treating all conflicting data identically, allowing for a more precise investigation.
Compare contexts and behaviors
Conflicting evidence often depends on context, rather than being inherently incorrect. For example, a hypothetical user might state a preference for a green button, yet consistently click a blue one during a usability task. This observed contradiction could be clarified if the blue button is always in the primary action position, while the green is in a secondary location. Inspect the source, specific context, and initial conclusion for each conflicting data point.
Propose a discriminant test
Once the source is identified and contexts are compared, design a specific test to clarify the ambiguity. This test aims to isolate variables causing the contradiction, not to validate an average opinion. For instance, if the button's position is a leading hypothesis, a team could test variations where both buttons appear in primary or secondary locations. This allows the team to observe actual user behavior under controlled conditions.
Steps to resolve contradictions
- Categorize the contradiction by type: stated preference versus observed action, user group, or task.
- Document the specific context and source for each piece of conflicting evidence.
- Design a focused follow-up test to inspect the contextual hypothesis.
When encountering contradictory evidence, create a small table. For each conflicting observation, list its source, the specific context, and the initial conclusion. Use this table to formulate a clear hypothesis about the contradiction's cause. Then, draft a task for a small follow-up test to inspect that hypothesis.
Your next discovery begins with a question.
Test an idea, observe the experience and bring evidence to your team.
Create my first studyFree · 1 study · 50 responses/month · no card