Vale2017 b13-presentation

The launch of usability testing at
the Cheng Library:
What I learned
Ray Schwartz: schwartzr2@wpunj.edu
William Paterson University
VALE Users' ACRL-NJ/NJLA CUS Conference
Friday, January 6, 2017

Outline
1. What is Usability Testing?
2. Why Usability Testing?
3. How is it done?
4. Our ‘Foray’ into Usability Testing.
5. Test Preparation.
6. Results.
7. Response from the Library.
2

What is usability testing?
• “Usability testing is a way to see how easy to use something is by
testing it with real users.”
• "Users are asked to complete tasks, typically while they are being
observed by a researcher, to see where they encounter problems and
experience confusion. If more people encounter similar problems,
recommendations will be made to overcome these usability issues. “
What is usability testing? (2016). Retrieved from http://www.experienceux.co.uk/faqs/what-is-usability-
testing/
3

• “Watching people try to use what you’re creating/designing/building (or
something you’ve already created/designed/built), with the intention of
(a) making it easier for people to use or (b) proving that it is easy to use.”
• “This element of actual use is what makes usability testing very different
from things like surveys, interviews, and focus groups where you’re
asking people for their opinions about things or their past experiences
using things.”
Krug, Steve (2010). Rocket surgery made easy. Berkeley, CA: New Riders, p. 13.
4

Usability testing is not a quantitative study
• Usability testing is conducted solely for the purposes of improving
library products and services. It also does not need to be submitted to
your University’s IRB for review. An IRB only reviews research as
defined by federal regulations as “a systematic investigation, including
research development, testing and evaluation, designed to develop or
contribute to generalizable knowledge.” (45 CFR 46.102(d)).
• With usability testing, the people involved are usually called “test
participants”, not “test subjects” to remind ourselves that we are not
testing them, but the services they are they using (Krug 2010, p. 13).
5

Why usability testing?
• Usability testing brings users into your design process.
• This means that what you make remains relevant to the people who
will have to use it.
• Uncovers problems while they’re still cheap to fix.
6

How is it done?
• In this case we are talking about DIY usability testing. This is not the
“full blown” usability testing that you carefully select your
participants and take 1 to 2 days testing, with a week to prepare a
briefing and then followed by deciding what to fix.
7

Steve Krug’s version of DIY
A morning a month.
• Which includes testing, debriefing and deciding what to fix.
Start earlier than you think would make sense.
• This means to start testing before you even changed anything.
• Start testing other sites. Even the best sites have usability problems.
Recruit loosely and grade on a curve.
• Doing testing frequently is more important than carefully targeting your
audience.
8

Make it a spectator sport.
• As many people should observe from another room (e.g., via video) as
possible. Seeing is believing.
Focus ruthlessly on a small number of the most important problems.
• Intensely focus on fixing the most serious problems first. Pick the three
worst problems.
When fixing problems, always do the least you can do.
• Tweak, don’t redesign.
• Take something away.
9

Basic Tenets
Matthew Reidsma (Web Services Librarian)
from Grand Valley State University.
• Do one test every month.
• Focus on what users do, not what we’d like them to do.
• No more than 5 questions for a 30 minute test.
• Test no more than 3 students or faculty a month.
• Invite everyone from the library to observe. EVERYONE.
Matthew Reidsma : How we do usability testing. (2011). Retrieved September 16, 2016, from
https://matthew.reidsrow.com/articles/13
10

• First, the presentation to Faculty Librarians’ Meeting to inform and
build support.
• Our modified tenets
• One a month – or when a question arises or as part of a project.
• Two to six questions for a 15 minute test.
• Test no more than three to five stakeholders a month.
• Use screen/voice recording software for librarians to observe later.
• Next, test preparation.
Our ‘Foray’ into Usability Testing
11

Test Preparation
• Equipment - Windows laptop and SnagIt software for screen and voice
recording.
• Location – Food Court of the Student Center.
• Opening script and release form.
• Prepared tasks.
12

Examples from the script
“The first thing I want to make clear right away is that we’re testing the site,
not you. You can’t do anything wrong here. In fact, this is probably the one
place today where you don’t have to worry about making mistakes.”
Krug, Steve (2010). Rocket surgery made easy. Berkeley, CA: New Riders, p. 147-148.
13

“As you use the site, I’m going to ask you as much as possible to try to think
out loud: to say what you’re looking at, what you’re trying to do, and what
you’re thinking. This will be a big help to us.”
“If you have any questions as we go along, just ask them. I may not be able
to answer them right away, since we’re interested in how people do when
they don’t have someone sitting next to them to help.”
Krug, Steve (2010). Rocket surgery made easy. Berkeley, CA: New Riders, p. 147-148.
14

Recording consent form
Thank you for participating in our usability research.
We will be recording your session to allow [ORGANIZATION NAME] staff members who are unable to be
here today to observe your session and benefit from your comments.
Please read the statement below and sign where indicated.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
I understand that my usability test session will be recorded.
I grant [ORGANIZATION NAME] permission to use this recording for internal use only, for the purpose of
improving the designs being tested.
Signature: _______________________________
Print your name: __________________________
Date: __________________
15

Creating Tasks
• Figure out what you’re testing and who the stakeholders are
• Set goals and from them, create tasks.
• User goal: Look up grades.
• Good task: Look up the results of your midterm exams.
• Poor task: You want to see the results of your midterm exams. Go to the
website, sign in, and tell me where you would click to get your transcript.
Grguric, Eka (2016) Guerilla Usability Testing and Communicating Value, NCSU Libraries Fellow,
Code4Lib conference, 8 March 2016.
16

Example of a verbal prompt
• "Maybe you should try logging out?”
Better
• "Can you describe to me what you are expecting to see / happen
right now?”
Avoid verbal prompts
17

Our Tasks
• Find peer reviewed articles on the topic of climate change.
• Find books on the topic of climate change
18

Tested three different sites
• http://www.wpunj.edu/library/
• https://catalog.wpunj.edu/Combined
• http://www.montclair.edu/library/
19

Make some action items
Items that were always (3 M’s):
• Missed
• Misunderstood
• Misused
Quickly summarize the key findings
20

Results
• Eight students tested the three sites – 3 were freshmen, 3 were seniors,
1 was a junior and 1 was a sophomore.
• Use of facets varied by class and interface.
• Correct use of tabs varied by interface.
• Drop down boxes were used.
• Search options link was not used.
• Confusion with some labels on the tabs.
21

Response from the Library
• Presented results at Faculty Librarians’ Meeting
• Solicited volunteers for performing usability tests – one volunteer.
• Results and design of future tasks will be main topic of next Library
Assessment Committee meeting.
• Some agreement on what changes need to be made as a result of tests.
• Peer-review checkbox
• Renaming tab labels(e.g., removing the word “Articles”)
• At this time, there is no mechanism for debriefing and how to decide
on fixing usability issues.
22

Videos
• Peachpit TV. (2010). Rocket Surgery Made Easy by Steve Krug: Usability Demo. Retrieved from
https://www.youtube.com/watch?v=QckIzHC99Xc&feature=player_embedded
• Grguric, Eka (2016). Guerilla Usability Testing and Communicating Value, Code4Lib conference, 8 March
2016, https://www.youtube.com/watch?v=Dd04w--7EuY&feature=youtu.be&t=7999
• UX tutorial: What is usability testing? | lynda.com - YouTube. (2013). Retrieved September 16, 2016,
from https://www.youtube.com/watch?v=48dMCEDJ1gM
Books
• Krug, S. (2010). Rocket surgery made easy / the do-it-yourself guide to finding and fixing usability
problems.
• Krug, S. (2006). Don't make me think! : a common sense approach to Web usability. Berkeley, Calif :
New Riders Pub., [2006].
• Nielsen, J., & Loranger, H. (2006). Prioritizing Web usability. Berkeley, Calif. : New Riders, [2006].
• Nielsen, J. (2000). Designing Web usability. Indianapolis, Ind. : New Riders, [2000].
• Rubin, J. (1994). Handbook of usability testing : how to plan, design, and conduct effective tests. New
York : Wiley, [1994].
References and Other Resources
23

Websites
• Advanced Common Sense - Steve Krug’s Web site. (2016). Retrieved September 16, 2016, from
http://www.sensible.com/
• Matthew Reidsma : How we do usability testing. (2011). Retrieved September 16, 2016, from
https://matthew.reidsrow.com/articles/13
Software
• Camtasia Studio. (2016, December 11). In Wikipedia. Retrieved from
https://en.wikipedia.org/w/index.php?title=Camtasia_Studio&oldid=754262442
• Silverback 3. (n.d.). Retrieved March 8, 2016, from https://silverbackapp.com/
• Snagit. (2016, September 5). In Wikipedia. Retrieved from
https://en.wikipedia.org/w/index.php?title=Snagit&oldid=737929379
24

Vale2017 b13-presentation

Recommended

Recommended

More Related Content

What's hot

What's hot (13)

Viewers also liked

Viewers also liked (9)

Similar to Vale2017 b13-presentation

Similar to Vale2017 b13-presentation (20)

More from Ray Schwartz

More from Ray Schwartz (7)

Recently uploaded

Recently uploaded (20)

Vale2017 b13-presentation