On a discussion thread to which I subscribe, a friendly
debate about whether we prefer the term “Information Architect” or “User
Experience Architect” as a job title took an interesting—but by no means unrelated—turn
towards the subject of drinking wine. This prompted a post by IA Donna Maurer, who
offered one of her blog entries describing the detailed taxonomy of her wine
rack. In her explanation, she states that “I thought I should show just one
aspect of my obsession with organizing stuff. People always laugh at me when I
tell them about this. Not sure why – after all, I do organise messy content for
a living.”

Aside from this offhand explanation, the rest of the blog
entry concentrates (justifiably) on the wine rack. But the comparison stuck
with me, because, as anyone who knows me can attest, I definitely do not bring
my work methodology home (or even to my workspace). I exist in spaces of
comfortable, low-rise clutter: skewed drifts of piled papers, off-kilter stacks
of books. This comforts me, that is, until the time I need to buckle down to
some serious work. Then, I can tolerate nothing but sharp, clean and organized
masses of possessions. Then I really have to give in to the inevitable and get
the work done. I’m sure this avoidance is common to many of us; I actually find
Donna’s integration of life and vocation somewhat extraordinary. Leaving aside
personality or aptitude, how much of our own personal behavior influences our
work (or vice versa)? Is this a form of stereotypical profiling or an authentic fact of life? I’m not sure now if I should change jobs or just invest in a really good filing system.

According to a pair of articles on BusinessWeek.com, the
social networking site Facebook.com attracted 11.5 million individual visitors
over the age of 35 in June, more than double the number a year before. The
growth of this segment is remarkable, given that the site was not open to all
until late September 2006, and the increasing “graying” of Facebook is
predicted to change the site’s character and content.

To paraphrase Groucho Marx, I am reluctant to join any club
that would have me as a member. I enjoy the features the site has to offer, and
am a member of several groups, but I am acutely aware that I am not the target
demographic, and I tread lightly. No pokes from me, folks.

On the other end of the social networking spectrum is
eons.com, which is dedicated to “Lovin’ life on the flip side of 50.”  The site is designed to accommodate its
audience: text is large (with an icon in the global navigation that enables
users to toggle between sizes), controls are big and features are presented in
a reassuring and user-friendly fashion, with much hand-holding in the form of
demos and explanations. All site content, whether user-generated or supplied by
the site itself, can be “boomed” by members to indicate popularity and
approval. Advertisements for Medicare supplemental insurance and pharmacies reflect
the needs and interests of the site’s members.

Perhaps the most overt indication of the site’s demographics
is the selection of topic areas. Right up there with top-level categories
including “Love,” “Money” and “Body” are “Lifedreams” and “Lifemap.” The latter
is an area for members to post and share their own life stories. Amidst all
this life is an “Obits” category where members can post tributes to their loved
ones and participate in groups. Continuing this theme, within the “Fun”
category is a game called “Are they alive or dead?” which challenges users to
decide whether famous personalities are above or below ground.

The site is far from senior in spirit, however: member group topics include dirty jokes and Harley hogs. It seems like a nice site to grow into.

“I’m sick of users,” announced Josh Bernoff in
a recent blog entry, leading one to initially believe that he has joined the
ranks of indifferent (or outright hostile) developers, clients, and other uninterested
parties reluctantly associated with producing applications and websites. However,
Bernoff’s distaste is semantic, not social. He argues that the term “user”
emphasizes technology over relationships and encourages a flattened and skewed
view of the people interacting with the products. He challenges the readers to “try,
just for a day, to stop using this word. You’ll be amazed at how differently
you think about the world.”

Unsurprisingly,
Bernoff’s modest proposal unleashed a torrent of unfavorable comments and
accusations that the substitution of more contextually specific appellations such
as “people,” “customers,” or “employees” was an attempt to fix something that
isn’t broken. A few readers were quick to brandish the smackdown stick of a
failed attempt at elaborate political correctness in Bernoff’s suggestion. And
of course, the old joke that only drug dealers and IT professionals refer to
their constituency as “users” was brought up (twice, in fact).

I agree with the
commentators who stated that “user” is a well-understood term, not
intentionally pejorative, and is, in fact, hard to replace without awkwardness
or lack of precision. My one objection regards the use of the term when
referring to usability testing. Often referred to as “user testing,” here the
term—perhaps unintentionally, but surely—puts the focus of evaluation on the
participant and not the system. I even prefer taking the term “testing” out of
the equation, instead calling the process “usability assessment.” I’ve dealt
with too many participants suffering performance anxiety and brightly uttered
the set piece that “we’re evaluating the system, not you” far too many times.
Of course, I’m not advocating that we keep the user out of the assessment, but
instead focus on the point of the activity. This is probably the one scenario
where people are directly and personally exposed to the term “user,” which is
almost exclusively used internally in design and development teams. Although we’re
all users, we are so in the third person only.

Second
Story Interactive Studios, in its own words, “creates informative and
entertaining interactive experiences in the form of media-rich storytelling
presentations, online collections, interpretive installations, and
database-driven applications.” The company comprises a diverse team of creative artists,
writers, producers, animators and programmers who enjoy a work environment that
boasts both a screening room and a technology lab for experimentation,
prototyping and testing of their projects.

They
boast an impressive list of clients, including AIGA, Nike, the J. Paul Getty
Museum, NASA, Virgin Records and virtually dozens of other top-shelf museums,
media corporations, and independent filmmakers.

Perhaps
the most powerful proof of their expertise is their website, whose engaging and
innovative design and exemplary usability bestow instant credibility. Their
multifaceted taxonomy allows users to view project collections by subject or
category. All projects in the selected subject or category appear as thumbnail
images at the bottom of the screen, in a collection termed “Filtered Projects,”
which allows users to immediately and easily access any and all of the related
works.

Individual
project screens feature a Flash movie of the project, a demo when available, a
description of the project, “Accolades,” and a link to the site. Most charming
(and useful) is the featuring of a label indicating the subject, categories,
date and project name that subtly invokes the memory of the long-gone card
catalog. In keeping with the corporate objectives and personality, each project
also displays a list of “Credits,” identifying the team members by name and
role.

It’s
a gorgeous, stimulating  site  custom-designed for effortless user exploration,  and reflects their  experiences in designing  for  museums and other exploratory spaces.  Art  imitates  art.

The Pathfinder User Experience Design team has written and published a whitepaper that was recently distributed at the annual Software 500 Conference in Boston, MA. The paper will show you how to combine Agile and UXD practices to get higher acceptance, fewer support calls, and return customers as well as define four key places in an agile development process where you can improve project success by integrating UXD practices into the Agile software development cycle. These techniques include the use of personas, taskflows, rapid prototyping, and iterative testing, among others, depending on the stage of development.

Registration for the Pathfinder Knowledge Center will give you access to forthcoming whitepapers, all of which are free. We have several scheduled for publication in the near future, on topics such as the value and purpose of taskflows, a comparison and differentiation between market and user research, and a guideline of best practices for integrating user testing into the Agile development process. The techniques are presented through scenarios, in which the appropriate UXD activities are matched to the stage of Agile development, with careful attention to time and resource constraints.

Sign up for the Pathfinder Knowledge Center today and each of these informative resources will be accessible to you as soon as they become available!

 http://www.pathf.com/knowledgeCenter/AgileUXD_Pathfinder.pdf

Even in waterfall-based design and development projects, formal usability testing at the time the system is being finalized is essentially “too much, too late”: a time-consuming, labor-intensive activity yielding results that are frequently unactionable for the current release. Several iterative assessment techniques, however, can be employed throughout any process, providing timely insight and direction. These techniques are readily adaptable to the Agile methodology, and, with some minor adaptations, can in fact be regarded as “agile” usability testing methods. Low-fidelity testing, using paper prototypes (or even hand sketches) can provide a quick benchmarking, and can be incorporated into a scrum meeting.

Why is usability testing important in the Agile environment?

A crucial component of the Agile methodology is the practice of obtaining customer feedback after every iteration and then making functional changes accordingly. Introducing usability assessments at key points in the process can ensure that the users’ requirements will also be recognized and accommodated within the development process, at a point when any potentially off-course decisions can be corrected and improvements implemented.

The nature of Agile development is to focus on individual features and although the project “epic” attempts to preserve the big picture for the product, this view is nevertheless focused on the functional and business requirements, rather than on the holistic user experience. Usability testing can document and validate issues that may only be the opinions of the team.

Usability studies are experiencing a shift in geography. For some time now, there has been some consensus that a traditional, dedicated usability lab is an unnecessary expenditure for an individual organization. Of course, since we specialize in User Experience Design, including usability testing, it’s a moot point for us. But we’re not exempt from the cost factor, either.

The main driver for this movement is the availability of software such as Morae and Spectator. Even Mac users are getting into the game with a product called VisualMark. These products turn a couple of laptops into a portable usability lab at the fraction of the cost of a full, fixed installation.

Mobile labs have been around for a long time, but they were full control decks that were about as mobile as three large suitcases full of building materials—and harder to assemble and disassemble. Because of the portability of the software-based systems, test participants have been freed from the confines of the lab, like a cageful of bunnies confined for experiments.

The arguments for bringing the test to the participant are persuasive. It’s tough to recruit a half dozen people willing to take time out of their day to come to a lab. Users can be tested in the context of their own environment (although usually not on the hardware they are accustomed to). The software is relatively easy to operate and produces data that can be edited quickly.

But most importantly, the ability to port the system need not preclude its use in a (semi-) fixed lab. The setup simply replaces the need, in a fixed system, for a dedicated control room. We are currently in the process of designing a lab (two-way mirror and all) using Morae on three computers. We believe there are benefits to a lab-based study:

• The application or website being evaluated is accessed from a single location, eliminating network or firewall issues

• Participants can access the product under optimal conditions

• Project stakeholders have the opportunity to observe the sessions without intruding on the process

There’s a rationale for having the flexibility of both approaches; let’s just not toss the idea of the lab environment simply because we’ve been given an easy means to an alternative.

Too often, usability testing is judged to be a “nice-to-have,” but dispensable within the time and budget constraints of a full design and development project.

This activity is often incorrectly perceived as being solely a formalized study that is time and labor intensive, with the potential to threaten the timeline and swell the budget. This is especially the case with the increasing dominance of Agile development methodologies, in which usability testing is thought to be a hindrance to the rapid iterative process.

However, there are many methods to elicit user feedback, both during development and pre-release. Here are the pros and cons of a few.

Formal study: usually performed in a usability lab, led by a facilitator paired with an observer/notetaker

Pros: Comprehensive observation of participant actions and behaviors due to audio and video monitoring. Usually, optimal testing hardware (possibly, a variety of computers) Ability for stakeholders to observe the assessment and directly hear the comments

Cons: Costly. Participants must be local and qualified candidates may be difficult to recruit. Participants may be intimidated by the setting, which may not duplicate authentic conditions of use. Does not integrate well into the Agile process

Mobile testing: Takes testing on the road. Usability specialists, equipped with laptop-based evaluation software, travel to the participants to conduct the study

Pros: Allows for assessment within the participants’ usual environment (e.g., office). Especially useful if many participants are located in the same facility or vicinity.

Cons: Opportunities for observation are limited: all findings must be filtered through the UXD team and the video artifacts they construct. Testing must take place on the team’s laptop.

Remote testing: Using a screensharing application such as WebEx, subjects participate in the test from their individual locations.

Pros: Low cost. No need for UXD team or participants to travel to a common location, affording the potential for geographically disperse participants to contribute to the study. Observers can log into the session and observe silently. Usually does not interrupt the Agile process.

Cons: Webcam provides limited visual observation. Body language cues are mostly lost. Can sometimes be difficult for facilitator and subject to establish rapport.

“Hallway” testing: Participants consist of internal stakeholders or a very small number of external participants.

Pros: Quick and casual, works well with low-fidelity paper prototypes, yields immediate feedback. No elaborate test script is required. Can be performed by usability specialist or others on the project team. An excellent way for Agile developers to gain immediate user feedback.

Cons: Participants may not have the characteristics of the true end-users of the product.

The growing partnership between user-centered design practices and the array of agile methodologies faces an impasse when it comes to usability testing. In a traditional waterfall approach, a formal usability assessment generally occurs close to release and is structured–and often perceived by clients–as a culminating Big Event. Consequently, the attempt to insert traditional usability testing events into the iterative agile process is viewed by developers as antithetical to their process.

Several interesting articles have been written that explore this topic. Uzanto’s Jonathan Boutelle takes the novel approach that usability testing should be taken out of the hands of specialists, whom he feels filters the user insights and often fails to communicate them to the design team. His strategy is to solicit "a constant drip drip drip of insights" by recruiting a small number of participants from Craigslist and having them assess the system remotely. This, he feels, allows the developers to experience the insights first hand.

An essay in Dr. Dobb’s Portal asserts the need for user testing in the agile envionment and supports the need for UXD and agile developers to find a middle ground through mutual understanding of roles and responsibilities.

Michael Andrews has a gently humorous blog entry, "agile usability by committee," in which he criticizes the agile practitioners for their reluctance to involve real users in the process, instead inviting a "user surrogate" or "customer representative" into the mix. He further explores the issue in "user testing and agile methods."

Leisa Reichelt has recently published a thought-provoking post on her site, disambiguity, titled "Yes, you should be using personas." She states that she’s "come to the opinion that personas are incredibly valuable, but not for the reasons many people think they are."

Instead of functioning as a tool to justify design, she argues, personas should serve as the vehicle to communicate the user centered process to stakeholders. By involving the members of the project team in the creation of personas, as well as involving them in user research and testing, they will gain first-hand experience with the design process. And that is, I think, more compelling than having the UXD team create beautifully formatted artifacts in isolation and then present them with a flourish (and extended explanation).

So far, Reichelt is espousing complete–even obvious–commonsense. Then, she takes a very interesting approach to personas, stating that once design begins, she never refers to them again. Her stance is that personas are useful for defining user requirements and prioritizing features. Rather than create personas that embody the qualities of the target users, she instead prefers to create personas that exemplify the "edge cases": precisely the users she would not design for. These personas, she reasons, cut down the "elastic user" at the knees.

The concept of the elastic user is Allan Cooper’s, and signifies the tendency of project stakeholders to generalize users into a single, homogenous entity. The edge case "anti-personas" then function as a brake on broad generalizations about what "the users" want or need.

Leisa Reichelt is also the originator of the intriguing and whimisical Waterfall is bad, washing machine is good, a presentation she gave at the IA summit in which she proposes a paradigm for UXD integration in an agile development process.