.comment-link {margin-left:.6em;}

Beyond the Fat Wire

Monday, October 24, 2005

Monday's workshop

Advanced User Testing
Jill Strawbridge, Symantec

Monday, October 24, 2005
User Testing 2005
Boston, MA
  • exp psych./human factors/indust & systems engg
  • currently Symantec designing apps
  • teaches human factors course at a design school (Arts Center) in Pasadena CA
  • and teaches an upper division pysc class somewhere else

(no wifi in this room. argh. I'm blogging in Dreamweaver...)

Nothing in the conference metadata indicated where registration would be, other than a vague mention of "lobby" somewhere.

So, go to the Lobby level ... nothing. Take the escalator up one level. There is a sign pointing up. Walk up the stairs to the 3rd floor. And there is a registration table.

There was breakfast. (hadn't been told that). Where is the meeting room? Mostly down one hallway. But I saw Peter Morville asking where his was (he's teaching an IA workshop here) ... he'd been down the hallway. Turns out his is on a different floor.

When you get your badge at the registration table, they give you a tacky bag. I'm looking at the handouts on the table with the tutorials and their room locations. ("Oh, there's one of those in your bag").

We get a spiral-bound set of the slides for the workshop -- at the next table. I'd vaguely been pointed there, but it didn't register .. so I have to go back to pick up my booklet.

So much for usability at a high-priced 6-day series of User Experience workshops....

At the morning break they told us they'd set up a wifi room. By noon, the wifi had lost it's connection to the internet.

Now we have the around-the-room introductions...

So far, 2 people in a row who should have been at Steve Krug's workshop, or at least should have copies of his book to give to their managers, since one big problem is getting the organization to truly buy in to iterative testing. I suppose mentioning that (in public in the session) would be bad form, given that this is a Jakob event =)

the usual corporate people. a guy w/ a genealogy site. the guy sitting next to me is w/ a company in Irving, and he's a UTA grad. small world. and a guy at Frost Bank in San Antonio.

Usability Magnitude Estimation (UME) and Master Usability Scaling (MUS), methodology from Mick McGee
  • UME - strength of rating, degree of difference
  • MUS - allowing comparisons
Defining Usability
  • a psychological response to using an interface (intuitive, well-organized, consistent)
  • ... that is evaluated to improve or compare designs
    • formative (to identify and solve problems)
    • summative (to evalutate with metrics)
Limitations with task-based usability measures
  • task differences, complexity, etc., make metrics difficult
  • if people fail, they could fail for different reasons
  • if people succeed, we can't know why
Types of data
  • nominal (no value. names of categories of data only)
  • ordinal (rank order, Likert scales)
  • interval (rank order + size differences, but not ratio differences)
  • ratio (rank order + degree of difference)

Usability Magnative Estimation (UME)

  • developed from the limitations of subjective usability measures.
  • developed from psychophysics
  • a psychophysical measurement method for assessing the psychological sensation of a physical stimulus
    • interpreting multidimensional stimuli
    • magnitude of effect is expressed as a ratio
  • generates lots of stuff to put into reports to impress management

We got academic research babble

There is a wifi room. There will be lunch "on our own" for an hour and 15 mins. So, I think, a sandwich from somewhere and either go to my room or go to the wifi room

Since this is another conference, I again have the strong urge to shop for a new laptop -- a lighter and smaller one, but, then I'd probably miss performance and battery-life.

Oh, during the break I had a Jakob sighting.

Yikes. Today's session is turning into the antithesis of "Don't Make Me Think". Which is why I don't go to ASIS. Or even read JASIST the journal anymore. Guacala!

I mean, the first warning was shortly into the pre-morning-break part, where she was going through a methodology for establishing baseline, comparable usability metrics with user testing subjects ... not only do I think that's a waste of time for most situations (see below), in seconds I could identify weaknesses in validity. Well, obviously, it's possible to construct some form of statistical validity for that methodology, at least validity that academics will recognize. But, for common sense... forget it.

Also, later, I'll outline all the usability challenges of this event. (sheesh)

Wait til y'all see the script she gave us to use for our first "practice" test. Mannn. Is she serious??

I quote: "Usability is your perception of how consistent, efficient, productive, organized, easy to use, intuitive, straightforward it is to accomplish tasks within a system".

We're supposed to read that to the test subject.


And I won't mention this "rate the proportional size of 10 individual circles, without seeing any circle more than once" exercise that is supposed to be the practice that you have the subjects do before you have them do the real test.

I mean, the usability testing methods are not usable!

Man... now I'm so glad I never studied HCI ... would have been like linguistics ... good in the thinking about it, but ridiculous in practice

After lunch, she skipped "ethnographic testing" and "international testing" and went straight to "Back to Basics". "Usability Metrics" -- a lengthy set of slides outlining "what you can test" but not "how do you test it". Particularly problematic when the "what" is memory burden. Given that this workshop is titled "Advanced Usability Testing", I would have thought that how to test would have been kind of central to the topic of the workshop.

Jeez, I don't believe it. The exercise is "what 4 measures would you use to measure the effectiveness of Google" and one guy actually said, measure heart rate, respiration rate, perspiration rate of the test subjects while they are doing the test. Oh man. Is that guy serious??


Post a Comment

<< Home