UI13: Dana Chisnell’s “Usability Testing in the Wild”

The second day of UI13 was a series of featured talks. I attended sessions by Dana Chisnell and Kim Goodwin, as well as the keynote by Jared Spool.

Here are my Twitter notes on Dana’s session, “The Quick, the Cheap, and the Insightful: Conducting Usability Tests in the Wild”:

• What is usability testing? asked to audience. “Exercising the design with users.” “Seeing if users can complete pre-determined tasks”
• @danachis showing Fidelity Investments’ high-tech usability lab
• @danachis helped write the book on usability testing, “but I’m here to say you don’t have to do it by the book”
• To do usability testing end-to-end in the classic sort of way, it can take a really long time. But quick-and-dirty is good alternative
• “A usability testing is essentially an evaluation…watching someone like your target user use your stuff” — @danachis
• What’s essential? @danachis took a list and crossed things out. What remains is: develop a test plan (critical), find participants, ..
• …conduct sessions, debrief with observers.
• @danachis talking about her work testing voting ballots. No one helping observe, had 3-4 ppl testing at a time. She was a bit panicked
• She had to brainstorm a plan to be able to observe all ppl simultaneously. Was studying whether ppl accidentally vote for wrong person
• Instead of one at a time, she conducted focus group-like tests. Ppl marked up the ballots and she was able to review them later
• Where’s value in testing? 70% watching someone use the design; 20% working w the team to prepare to test. 8% discussing what happened
• Improvise! Say “yes” to whatever the circumstances are. Trust your team and participants. Don’t let team or stakeholders block…

• …Work to the top of your collective intelligence
• What do you lose doing quick-and-dirty? Can’t collect a lot of quantitative data, lose a bit of rigor, biased sample (probably)
• What’s the ROI on quick-and-dirty usability tests? Qualitative data, opportunity, fitting into schedule, timeliness, credibility
• Quick-and-dirty stops being valuable when you need summative data, benchmarks, and answers to hard problems.
• Best to go where the users would use the thing you’re working on. You can pay them less. They won’t be late. You get all the context
• Remote usability testing is a great idea. Cheap and easy to set it up. Use conference calling facilities and online-meeting apps
• Morae; UserVue; Ethnio
• Plan minimally for your quick-and-dirty usability test. Just the what, why, who, when and where
• In quick-and-dirty, 1st participant is your pilot. Interview-based tasks (talk to them 5-10 min to understand how they’d use product)
• Session outline: greet participant; explain study/role, interview, do tasks from interview, debrief w participant, debrief w observers
• You’re moderating, not training. You should be impartial, unbiased. No teaching! Listen and watch. Ask open-ended questions
• @danachis’s session is over. Good stuff to use when trying to convince clients to let me do a quick test for valuable insights
• Download usability testing templates, docs, etc from http://wiley.com/go/usabilitytesting

[Other UI13 sessions I attended: Peter Merholz and Andrew Crow; Kim Goodwin; Scott Berkun]

Related Posts:

• DIY UX at An Event Apart Chicago 2009 October 20, 2009 | 13 comments
• UI13: Kim Goodwin’s “Where Usability Meets Desirability: Goal-Directed Visual Interface Design” November 3, 2008 | 0 comments
• UI13: Peter Merholz and Andrew Crow’s “Design Strategy and Planning” November 3, 2008 | 2 comments
• UI13: Scott Berkun’s “The Myths of Innovation: How to Lead Breakthrough Projects” November 3, 2008 | 2 comments
• The UX Design Process for the Boxee Beta January 27, 2010 | 33 comments