Distinguishing study abandonment and task failure

Leaving a task and leaving a study are different events. Someone may fail to reach the goal of one activity and continue answering the remaining questions. They may also complete that activity and leave before the research ends. Recording both dimensions prevents every interruption from becoming a single, ambiguous “dropout” number in the report.

Separate task, session, and incident

For each task, record the outcome using criteria defined before data collection: goal reached, not reached, explicit give-up, or unknown result. Separately record whether the study session was completed or interrupted. A technical failure can accompany any combination of these states and needs its own record. Do not assume that closing a page means usability difficulty, or that finishing the study means succeeding at every task presented to the participant.

Work through a hypothetical example

Imagine an activity to locate an invoice. Person A cannot find it, selects “Give up this task,” and answers the rest of the study. Person B finds the invoice but closes the study on the next question. The first person had an unsuccessful task and a completed session. The second had a successful task and an interrupted session. If person C’s screen does not load, record the observed incident without inventing their intention.

Give each rate its own denominator

When reporting task success, explain who received that activity and which results entered the calculation. When reporting study completion, explain which sessions are being counted. Do not automatically use the same denominator for both measures. Show unknown outcomes and technical incidents separately, following documented rules. If you exclude cases from analysis, retain the original record and reason. Also check whether including those cases changes the interpretation you would present to the team.

Classification checklist

• Define task outcomes and session states in separate fields before collecting data.
• Record observed incidents without attributing unverified causes or participant intentions.
• Show the denominator for each percentage and identify cases with unknown outcomes.

Create a table with task, outcome, session state, and technical incident columns. Fill it using the hypothetical examples above, then ask a colleague to classify the same cases. Resolve disagreements and document your definitions before the next round. Keep an explicit option for situations that the available evidence cannot classify.