AI UX research for AI products people adopt
6 min read
Research for an AI product should expose the gap between what people think the system can do and what it actually does. A polished answer can conceal uncertainty, so a useful study includes a plausible mistake and a realistic recovery task.
Use a concrete output that someone has to inspect and correct. The KIMEO case gives one example: a generated scientific figure still needs editing and export before it fits the user’s work. These are researchable tasks with observable outcomes.
Start with a decision, not a demo
Before a research session, write down the decision the team needs to make. It might be whether a support agent should receive a draft reply, whether a customer can approve an automated action, or whether a result needs an explanation. A specific decision gives the session a useful boundary.
Invite people who actually do that work. Ask them to walk through a recent example with their own documents and constraints. Notice where they switch tools, double-check information, or ask a colleague for help. Those moments are candidates for a better experience.
Map the expectations around the AI
Show the interface before explaining how it works. Ask participants what they expect it to do, what information they think it can see, and who they believe is responsible for the final result. Compare those answers with the actual product boundaries.
An interface can look confident even when the system has limited evidence. Test labels, source links, and review steps together. The aim is to help a person make an informed decision about this output in this situation.
Test the moment something goes wrong
Prepare an ordinary success case, an incomplete answer, and a plausible mistake. Keep the scenario realistic and tell participants that they are testing a prototype. Observe whether they notice the problem, understand its consequences, and find a way to correct it.
A recovery path might be editing an input, choosing a source, restoring an earlier version, or asking a person to review. Record the steps needed to recover as carefully as the steps needed to complete the happy path.
| Scenario | What to observe | Design response |
|---|---|---|
| Successful result | Can someone understand and use it? | Editable output |
| Missing information | Can they identify what is missing? | A clear way to add context |
| Incorrect result | Can they notice and correct it? | Review, undo, and human handoff |
Turn observations into a small research plan
Separate what someone said from what you observed them doing. Keep the original evidence attached to each finding so another teammate can inspect the interpretation. Then connect the finding to a specific interface decision.
Prioritize questions that could change the scope of the product. A short study that rules out a bad workflow can be more useful than a detailed report about visual preferences.
Use AI to assist the analysis
An assistant can help organize notes or suggest themes, but the team still needs to check those themes against the source material. Keep participant information within the permissions agreed for the study, and remove identifying details when they are unnecessary.
Treat a generated summary as an index into the research. Go back to the recording or transcript before turning a suggested pattern into a product requirement.
Keep learning after the release
Decide what evidence the live product should capture before release. Useful events might include a result being edited, a task handed to a person, or a draft abandoned before approval. Interpret those events alongside direct feedback.
Return to the same important tasks after changes to the model, retrieval, or interface. The point is to understand whether people can still finish their work with the new behavior.
A trustworthy experience helps people know when to continue and when to check.
Make the next iteration testable
End the research with a short decision log: what the team learned, what it will change, and what remains uncertain. Assign an owner to the next test and make the proposed change small enough to evaluate.
A reliable research habit is built from these short loops. Observe the work, change a part of the experience, and check whether that change helps someone complete the task.
Put this into practice
Explore the related work and delivery scope.
- Related project: KIMEO
Inspect the published product workflow and delivery context.
- Explore the relevant service
See the engagement scope and start a specific project conversation.
Scope a study around your product decision
Share the task, likely users, and the uncertainty that could change your first release.
Discuss product research
