Research Pilot: Test Your Study Before Recruiting

Ambiguous instructions, prototype flaws, or inadequate session length can hinder useful data collection in a UX study. Launching without prior testing often leads to unclear results and wasted resources. A research pilot helps mitigate these risks, clarifying your process before official participant recruitment begins.

What is a research pilot?

A pilot is a rehearsal before the main study. A colleague can walk through an initial operational check, but that does not replace research with the intended audience. Check whether the script, prototype, and recording process work together. A colleague’s familiarity with the product may conceal difficulties. Keep infrastructure fixes separate from conclusions about the people you plan to recruit, and label the rehearsal clearly in your research records.

Identifying common problems

During the pilot, observe if your colleague understands instructions without intervention. Check that the prototype works as expected, free of bugs that might frustrate users. Time the session to inspect the planned duration, aiming to prevent participant fatigue or incomplete sessions. For example, if a hypothetical colleague takes twice the expected time to complete a sign-up task on a mobile phone app, this suggests the task is complex or instructions are unclear, prompting team revision.

Adjustments and next steps

After this rehearsal with colleagues, revise the instructions and prototype using the operational issues you recorded. Rewrite ambiguous tasks and fix problems that prevent running the study. Do not present observations from this rehearsal as findings about your target audience. A pilot can also involve real participants when that is the chosen design; document who participated, the purpose, and the intended use of the data. Record the changes before starting the main round.

Research pilot checklist

• Verify the clarity of all study instructions and questions.
• Test the prototype's full functionality to identify any bugs.
• Time the session duration to confirm appropriate length.

Use your pilot findings to adjust the study script, tasks, and prototype. Document any changes made to instructions or the prototype for future reference. Only after these revisions, begin recruiting real participants. This step helps the team conduct a more focused study and collect clearer data, allowing for better insights into user experience.