Beyond the Fat Wire

NISO Digital Preservation Forum: Planning Today for Tomorrow's Resources

2008-03-25T07:24:00.004-05:00

[Note: slides have moved here]

A couple of weeks ago I attended the NISO Digital Preservation Forum in Washington, DC -- an excellent event. There were about 100 attendees, with all sessions in one room and seating at round tables -- perfect for meeting and talking to people, for taking notes or getting out a laptop, and generally more comfortable than rows of chairs.

Presentations are now available for download on the forum agenda page. All were interesting and informative, but particularly so were Evan Owens' "Long-Term Preservation and Standards: An Uneasy Alliance", Lucille Nowell's "The Data Preservation Imperative: A Global Challenge", Katherine Skinner's "The MetaArchive Cooperative: A Collaborative Approach to Distributed Digital Preservation", and Deborah Thomas and David Brunton's "Mitigating Preservation Threats: Standards and Practices in the National Digital Newspaper Program".

Deborah was sitting at the same table I was, and in chatting, she talked about their pilot project putting Library of Congress images on flickr. She said they've had a hugely successful response, judged by hits, favoriting, and commenting. She also said they've gotten valuable information from flickr users, correcting or adding information about the images they'd posted.

All in all, this was a terrific event. Can't say I'd recommend the hotel though.

Posts temporarily unpublished

2006-06-21T20:25:00.000-05:00

I'll republish elsewhere later...

Usability of Usability events

2006-06-19T15:19:00.000-05:00

Last time I was at one of Jakob's events, I complained about usability issues related to the event itself.

This time, the event itself is ok, but the venue is another story.

Of all the hotels in San Francisco, they chose one:

with no casual dining options. In fact, since I brought jeans, knit shirts, and black sneakers, I might not even pass the hotel restaurant's dress code.
without a coffee shop/kiosk/whatever
with a single set of restrooms (which can each accommodate about 5 people at a time for the respective genders -- and that includes those waiting in line) to serve two meeting rooms each with a capacity of a couple hundred)
which is located at the top of a hill, the final grade of which has got to be 45 degrees (and, if there were attendees with mobility problems, how were they supposed to go get lunch within one hour?)

SF again

2006-06-18T21:45:00.000-05:00

I had an easy trip and a good flight, even though the flight was full. Actually, there were some empty seats in the back, so a flight attendant offered a change of seat to the guy in the middle in my row, and he took it. So, my luck with flights is holding and I had one of the very few empty seats next to me.

This event is at the Intercontinental Mark Hopkins -- very fancy, but not my style. Free wireless thanks to this event, but a crappy signal and it's slow. I'm not sure there'll be wireless in the meeting room tomorrow, although maybe... there is supposed to be wireless in the lobby, and the room is close to the lobby, so we'll see tomorrow.

There's a six-page attendee list. Most are at companies we know well (as well as those we've never heard of), but there are some public sector people as well (County of Los Angeles ISD) plus some people at universities, and even a couple of others from libraries.

An ironic moment at the event check-in. They check in people to an online system, and the person who was doing the check-in was complaining about that system. It's apparently got serious usability problems. Hahah.

This hotel is at the top of Nob Hill, at the corner of Mason and California Streets. Steep hill down Mason, and on California towards Powell (where the cable car stop is). Being at the top of a hill, it's very windy. I can hear the wind whistling around the hotel.

Thursday's workshop

2005-10-27T21:14:00.000-05:00

Toolbox of Specialized Usability Techniques
Chauncey E. Wilson
WilDesign Consulting

Thursday, October 27, 2005
User Experience 2005
Boston

He'll set up a bog and/or wiki on this topic. Will send us details by email later.
12 people in this workshop. (Monday's had about 25-30)
Interesting that many people are involved with establishing formal usability programs and usability labs

UTest usability mailing list

email him for: UTest subscription info; tips for setting up a usability lab

at least 3 Canadians in this room

apparently, Jakob asked him to do this workshop

This guy (Chauncey Wilson) is a psych/HCI guy, but more on the Steve Krug end of the spectrum regarding how to view usability. The other person sitting at my table was also in the same Monday workshop as me (Advanced User Testing) and she doesn't have a high opinion of that workshop either.

There is going to be a "World Usability Day" on November 3rd. See the UPA web site for details. They've reserved (? I'll check later when I have an internet connection) the Boston Museum of Science.

I've been curious about Verizon's broadband wireless service ... basically, you get a voice plan and then for like $60/month you get unlimited broadband wifi anywhere within Verizon's broadband service areas, which are pretty much the major metropolitan areas in the US. For example, if I had Verizon broadband, I'd be online right now, even though the hotel doesn't have wireless in the meeting rooms. Assuming, of course, that the Verizon broadband is good service.

Fishbone Diagrams

to review factors that might have an effect on or contribute to a problem, process, or goal
the diagram has a main line (spine) that is the effect you want to examine
"main bones" are cause categories that act on the effect
each main bone is a major potential cause
there is also a root cause that would explain a problem, symptom or effect
Major cause categories
- The 4 Ms: methods, machines, materials, manpower
- The 4 Ps: place, procedure, people, policies
- The 4 Ss: surroundings, suppliers, systems, skills
Common categories for usability
- readability (effect)
  - font size (cause)
  - contrast (cause)
  - language/internationalization (cause)
  - line length (cause)
- navigation
- performance
- accessibility
- organization
- perception/credibility/trust

Question from the audience regarding sorting out cause and effect that is valid, versus apparent but not real cause and effect, and/or cause and effect with intervening variables

Even if you have effects where you have no control over the cause(s), it's useful to understand the effect and its causes

Affinity diagram vs. card sort -- an affinity diagram is a group, social activity, where the group comes to consensus about the grouping/categorization/affinity of a set of concepts (e.g., by moving sticky notes around on a wall). a card sort does the same thing, but it is an individual activity. A group card sort is not recommended -- if you want a group activity, organize it as an affinity diagram activity instead

5-Why Analysis for getting from the proximate cause to the root cause (aka "the deep nagging approach")

technique for moving from symptoms to root causes
move from major categories on a fishbone diagram to root causes

Rapid Analyses for Fishbone Diagrams

vote on the most likely cause
rank main causes on importance, fixability, etc
rank the sub-causes within each main cause
do before and after fishbone diagrams

Tools

SmartDraw
RFFlow http://www.rff.com/
RCA-XPress http://www.rcaxpress.com/

Q-Sorting/Repeated Sorts

(see tutorial booklet)

Tool: WebQ

Cardsorting - Tom Tulles at Fidelity says 25 people. Others say around 40. This speaker says 20-100

Forcing choices (that is, through a Q-Sort, or saying "Spend $1000 in this store" gets more differentiation than if you have people rate on a 1-5 scale such as in a survey.

See WebSort for online versions of cardsorting/Qsorting

P Sorting

Closed vs. Open card sorting

Q sorting (along a dimension)

Repeated sorting

Q-Sort, Repeated Sort:

individual card sort
the person sorts based on self-chosen criteria
after, facilitator asks what that criteria was, etc, records responses, etc
facilitator shuffles the cards/papers
repeat

Another tool: EZSort and EZCalc (from IBM)

you want to see what cards appear with which other cards ... that is where you will find your correlations

Freelisting (a variation on brainwriting, which is a variation on brainstorming) -- to see what comes to people's minds first, for a given topic. this is an individual activity

Brainwriting -- write thoughts on sticky notes, or they write stuff and you collect the papers.

Brainstorming is a social event, and all ideas are supposed to be criticism free. with brainwriting, it's similar, but you write instead of speak out loud. People can write individually, and then the paper gets passed along and others add to it.

"Brainstorming is fraught with peril"

Brainwriting generates a larger quantity of ideas (and, the measurement of brainstorming success is quantity)

Braindrawing is similar, but with drawing instead of writing words

"The Icon Book" - about 10 years old, but has good ideas about iconic images

2-5 minute breaks every 15 minutes makes brainstorming more productive

if you give the brainstorming group a goal of X number of ideas (which will be 10-20% more ideas than you think they can generate) the brainstorming session will be more productive

KLM (Keystroke Level Model)

Like, having read the book by Card, Moran and Newell (1980) is the sign that you're a serious HCI person

GOMS - goals, operators, methods (combinations of operators), selection tools

KLM allows quick estimates of task time with a minimum of theoretical or conceptual background

Fitz Law (Tognazzini) -- you have a screen, a pointer, and a target. Fitz Law says, the bigger the target, the faster you get to it. Tognazzini talks about how big the target has to be to most effiiciently get the user to it.

Interfaces and complexity

2005-10-25T21:42:00.000-05:00

Apart from what I or we might think about Microsoft, MS Office, Jakob, or this conference, this evening's plenary was valuable I think.

There are ways in which our web site is similar to MS Office.

Lots of complexity and functionality
It's hard to find functions in that complexity
People have to be trained how to find and use those functions
People use our site because it's what they have to work with and/or someone tells them they have to use it
(now, if only we had Microsoft's billions of $$ heheh)

Because of that, it seems to me that the design goals and redesign principles he listed are applicable to us.

I go to these events -- Gilbane a year ago, ETech last spring -- and each time, I'm thinking: if only we could really get down to the real thing the library does -- provide access to information -- and recognize that anything else we do is mere "commentary" on that ... THEN we could design the site we need to have, create a brand image that will work, communicate to the university clearly about our value.

Tuesday evening - a preview of the new Office interface

2005-10-25T20:02:00.000-05:00

As a preliminary, let me say that I witnessed a Jakob fan-boy moment. Walking toward the room where the event was to be held, Jakob was walking some paces ahead of me. A guy (who was your typical 30-ish traditional corp tech type (apparently)) passed in the other direction. "Jakob Nielsen", he exclaimed, "I'm a big fan of your work!"

Yikes

This was a talk by a guy from Microsoft who talked about the interface design for the upcoming version of some components of Microsoft Office (particularly Word, PowerPoint, and Excel)

Although both Jakob and the Microsoft guy were billed, in reality, Jakob did a 15 minute introduction and then the Microsoft guy spoke for about 40 minutes. 20 minutes of Q&A followed (thus, it ended at 6:30, not at 6:45)

The idea behind the evening was for the evolution of interfaces, away from "command oriented" actions and toward "results oriented" actions.

Jakob's introduction went through a brief history of the user interface:

batch commands
line mode commands
fullscreen text-only terminals
GUIs

All of which, he pointed out, are command based interactions.

Back in 1992, he said he had predicted that by 1996 interfaces would have evolved into a non-command form. Now he's saying maybe that will happen by 2010.

The idea of a non-command interface is what he calls "agent oriented" task performance, where the interface does what the user intends rather than what the user commands. I presume that the system gets trained by the user to know what that intent is. The idea is that complex commands get performed to the user's specification without the user having to detail those specifications each time. (at least, that's my interpretation. Jakob didn't actually say that. However, I can't imagine he means that software intuitively or telepathicly interacts with the computer user)

So, on that note, he turned the room over to the Microsoft guy, Tim Briggs.

More history, as he reviewed the Word interfaces from version 1 to the present (Office 2003, which is 2 years old already. I hadn't realized it'd been that long).

The point of the review was to illustrate the complexity of the interfaces ... Word has grown to have 300 commands and 31 toolbars. People can't find the commands they want. They asked for functionality that already existed (but they couldn't find it, or didn't recognize its function if they saw it). The UI is hard to browse. Core functions took too long for users to accomplish.

Design goals:

Keep frequent and familiar tasks efficient
Help people discover best practices (that is, help them find the fastest way to do things)
Make browsing for familiar goals (tasks, commands) easier
Let people focus on the output (their end document), not on the UI

Redesign principles:

Streamline the core functionality
Consolidate the UI areas
Apply 3-stage formatting, which means:

gather bundles of features together into, sort of, palettes to apply to a given portion of a document
demonstrate what's possible (that is, show previews of what something will look like if they apply a feature/palette)
dialog access for tweaking (that is, give people access to power tools if they want them)

He walked through some of the interface. The Menu bar at the top of the apps is now a "ribbon" with tabs. So, instead of clicking and looking at dropdowns with flyouts from the dropdowns, you get a big "toolbar" (though it's not called that now) with buttons/widgets. These "ribbons" have "function chunks" (meaning only that all the, for example, text formatting widgets are grouped together).

These "commands" are not organized around a scenario or object. There is supposed to be better labeling and feedback to the user. Basically, this means there are now "super tooltips" ... not only does mousing-over display a label, but it's a super-label that include what the widget is for and what it might be used to do.

Tools will be contextually relevant. This means that, for example, if you insert an image into a Word document, if you select that image, you'll THEN see image editing/formatting tools. Then, when the image is no longer selected, those image tools will disappear.

Finally, he went through what he called "Effect of a new experience". That is, that from their pre-beta testing of the new Office interface, that they expect people will experience a drop in productivity at first. Productivity (and user's perception of his/her own productivity) will stay low while the user keeps using the new UI for the old tasks he/she is accustomed to doing. The user's perceived productivity (though, at this point, he started getting fuzzy about whether he was talking about real or perceived productivity) will start to increase as the user starts discovering how to accomplish new tasks with the app.

Actually, this last part sounded suspiciously like a pitch to partners that MS wants to have buy in to the new version of Office.

Not too much of note in the Q&A. One person asked about Mac users of Office, and the answer was, basically, they are still screwed. Someone asked about accessibility for people who don't use a mouse. The answer was "it'll be better" (but, that's easy to say, isn't it?)

Someone asked how they decide what new features to include. This led him to talk about "SQM" (pronounced "squim") -- system quality management, which originally was to provide MS with information about application crashes. Now, they use it to gather feature usage.

You know that feature/option in MS products to participate in a "Customer Improvement Program" or "Feedback Program", sometimes hidden under "Service Options"? Well, by saying "yes", you are agreeing to send Microsoft information about the commands and features you use in the MS product.

One guy asked a question that seems to me to typify the difference between this event (the User Experience 2005 event) and, say, ETech (O'Reilly's Emerging Technology conference from last spring).

The question was, what has the interface done to prevent the questioner's big pet peeve. The pet peeve is people making headings in a Word document by applying text formatting, rather than by applying a structural style. Of course, there's no real answer to this.

And, of course, in the context of html, I also would want my authors to use the markup I specify rather than something else, even if the result "looks" the same on the web page, in the end.

But, the point is, this guy is worried about how to force people to use a piece of software (in this case, MS Word) in a way that HE thinks is correct.

ETech, on the other hand, was all about molding any tool so that THE TOOL does what you want. That the user is the one that rules the software, the hardware, the application, the task -- and not the other way around.

Rain!

2005-10-25T11:55:00.000-05:00

It's raining, 47 degrees, and windy. Hopefully, the weather will be better tomorrow.

This evening there is a plenary with Jakob and somebody from Microsoft, otherwise this is a free day. It's curious to me that they insist on calling this a conference. When it's pay-by-the-day for each day's one-day workshop, and there isn't really anything else here besides the workshops (this evening's "plenary" aside), the fact that the thing lasts for 6 days does not qualify it as a "conference" in my opinion.

Monday's workshop was so disappointing. But, I still have hopes for Thursday's, which is "Toolbox of Specialized Usability Methods". I mean, since he bothered to put his outline up for people to see (or so it seems), maybe he'll actually talk about how to use real methods.

Monday's workshop

2005-10-24T20:35:00.000-05:00

Advanced User Testing
Jill Strawbridge, Symantec

Monday, October 24, 2005
9:00am-5:00pm
User Testing 2005
Boston, MA

exp psych./human factors/indust & systems engg
currently Symantec designing apps
teaches human factors course at a design school (Arts Center) in Pasadena CA
and teaches an upper division pysc class somewhere else

(no wifi in this room. argh. I'm blogging in Dreamweaver...)

Nothing in the conference metadata indicated where registration would be, other than a vague mention of "lobby" somewhere.

So, go to the Lobby level ... nothing. Take the escalator up one level. There is a sign pointing up. Walk up the stairs to the 3rd floor. And there is a registration table.

There was breakfast. (hadn't been told that). Where is the meeting room? Mostly down one hallway. But I saw Peter Morville asking where his was (he's teaching an IA workshop here) ... he'd been down the hallway. Turns out his is on a different floor.

When you get your badge at the registration table, they give you a tacky bag. I'm looking at the handouts on the table with the tutorials and their room locations. ("Oh, there's one of those in your bag").

We get a spiral-bound set of the slides for the workshop -- at the next table. I'd vaguely been pointed there, but it didn't register .. so I have to go back to pick up my booklet.

So much for usability at a high-priced 6-day series of User Experience workshops....

At the morning break they told us they'd set up a wifi room. By noon, the wifi had lost it's connection to the internet.

Now we have the around-the-room introductions...

So far, 2 people in a row who should have been at Steve Krug's workshop, or at least should have copies of his book to give to their managers, since one big problem is getting the organization to truly buy in to iterative testing. I suppose mentioning that (in public in the session) would be bad form, given that this is a Jakob event =)

the usual corporate people. a guy w/ a genealogy site. the guy sitting next to me is w/ a company in Irving, and he's a UTA grad. small world. and a guy at Frost Bank in San Antonio.

Usability Magnitude Estimation (UME) and Master Usability Scaling (MUS), methodology from Mick McGee

UME - strength of rating, degree of difference
MUS - allowing comparisons

Defining Usability

a psychological response to using an interface (intuitive, well-organized, consistent)
... that is evaluated to improve or compare designs
- formative (to identify and solve problems)
- summative (to evalutate with metrics)

Limitations with task-based usability measures

task differences, complexity, etc., make metrics difficult
if people fail, they could fail for different reasons
if people succeed, we can't know why

Types of data

nominal (no value. names of categories of data only)
ordinal (rank order, Likert scales)
interval (rank order + size differences, but not ratio differences)
ratio (rank order + degree of difference)

Usability Magnative Estimation (UME)

developed from the limitations of subjective usability measures.
developed from psychophysics
a psychophysical measurement method for assessing the psychological sensation of a physical stimulus
- interpreting multidimensional stimuli
- magnitude of effect is expressed as a ratio
generates lots of stuff to put into reports to impress management

We got academic research babble

There is a wifi room. There will be lunch "on our own" for an hour and 15 mins. So, I think, a sandwich from somewhere and either go to my room or go to the wifi room

Since this is another conference, I again have the strong urge to shop for a new laptop -- a lighter and smaller one, but, then I'd probably miss performance and battery-life.

Oh, during the break I had a Jakob sighting.

Yikes. Today's session is turning into the antithesis of "Don't Make Me Think". Which is why I don't go to ASIS. Or even read JASIST the journal anymore. Guacala!

I mean, the first warning was shortly into the pre-morning-break part, where she was going through a methodology for establishing baseline, comparable usability metrics with user testing subjects ... not only do I think that's a waste of time for most situations (see below), in seconds I could identify weaknesses in validity. Well, obviously, it's possible to construct some form of statistical validity for that methodology, at least validity that academics will recognize. But, for common sense... forget it.

Also, later, I'll outline all the usability challenges of this event. (sheesh)

Wait til y'all see the script she gave us to use for our first "practice" test. Mannn. Is she serious??

I quote: "Usability is your perception of how consistent, efficient, productive, organized, easy to use, intuitive, straightforward it is to accomplish tasks within a system".

We're supposed to read that to the test subject.

Sheesh

And I won't mention this "rate the proportional size of 10 individual circles, without seeing any circle more than once" exercise that is supposed to be the practice that you have the subjects do before you have them do the real test.

I mean, the usability testing methods are not usable!

Man... now I'm so glad I never studied HCI ... would have been like linguistics ... good in the thinking about it, but ridiculous in practice

After lunch, she skipped "ethnographic testing" and "international testing" and went straight to "Back to Basics". "Usability Metrics" -- a lengthy set of slides outlining "what you can test" but not "how do you test it". Particularly problematic when the "what" is memory burden. Given that this workshop is titled "Advanced Usability Testing", I would have thought that how to test would have been kind of central to the topic of the workshop.

Jeez, I don't believe it. The exercise is "what 4 measures would you use to measure the effectiveness of Google" and one guy actually said, measure heart rate, respiration rate, perspiration rate of the test subjects while they are doing the test. Oh man. Is that guy serious??

Don't Make Me Think: The Workshop

2005-09-30T23:23:00.000-05:00

Just got back from the Chicago edition of Steve Krug's Don't Make Me Think workshop (by the way, it's pronouned kroog (as in goog-le) not krug (as in rug))

It was good, I'm glad I went. The main thing from it, for me, is kind of a good news/bad news thing.

One feature of the workshop is that attendees send in a problem URL and Steve does a 12 minute mini-expert review. I sent in our Research Resources page and he additionally included our Find Articles page.

The good news is, we aren't doing much worse than other library sites he knows. The bad news is, even though everybody agrees it sucks, he has no good solution. He thinks it might be, at heart, an Information Architecture problem (that is, he suggested that he and we consult with Lou Rosenfeld). But, while there may be some IA things to do, essentially the problem is that we have content tied up in 250+ silos that can barely talk to each other. And, OneSearch helps only a little. It lets us peek into some of the silos, but the view into those silos isn't that great, not all silos are viewable, and, overall, it takes such effort to get that peek that it's hard to know which is better -- to have the peek or to stick with choosing one database at a time.

He did like our plan to turn the "Find" pages into some sort of research wizard, where we lead people along a decision path that results in suggestions of resources.

But, people outside libraries don't really get the complexity of what we're doing. One guy wondered why we just don't list the 5 or 6 databases, and was dumbstruck at the concept of 250 3rd party licensed databases and 30,000 ejournals.

Another guy wondered why we can't use cookies or something to allow for re-running a search in another database. When I suggested that would be like doing a Google search and then popping over to Yahoo and expecting to re-run the same search, he got a deer-in-headlights look.

I just hope we can build that research wizard app. I've got a sense that task will be daunting.

Otherwise, the workshop was good. He talked some about the major points in his book.

The best parts were the mini-reviews and mock user tests of attendees sites. Along with this, he gave tips and advice on conducting tests and reporting on them. He's a big advocate of simple testing, simple tools, and avoiding the "big honking report". Rather than a report, he prefers debriefing the design team and/or stakeholders. And even more than debriefing stakeholders, he prefers to get them to observe actual tests as a way to get them to "get it".

And, of course, he's a big advocate of testing early and often, and repeating the early and often throughout the life of the site.

While a fancy lab is not necessary, we really do need some sort of facility. Laptop, Camtasia or Morae, some place private where the tester can feel comfortable, some place where there can be appointments to allow notetaking and decompressing on the part of the test facilitator in between appointments, and a way for the test to be observed by others who are not present (audio or audio-video feed, one-way mirror, or something to allow remote observation of the test)

And scripts. We need scripts.

Oh, regarding that person who hated that fact that we provided a custom "Page not Found" page with explanation and site index for the old web pages we can't redirect individually -- he suggested the book Defensive Design for the Web.

He'll have his slides on his site after the 3rd edition of the workshop (he prefers that attendees not read ahead). He does have other downloads though: a sample test script and a video consent form (both are MS Word documents, so right click and save as)

Additionally, about the mini-reviews, attendees whose sites didn't get a review during the workshop are entitled to a phone consultation with him. In our case, even though we got a review during the workshop, since he didn't have any really good advice, we get to have a phone consulation too. We can do this as a conference call, so we'll see what kind of scheduling we can do for that.

Regarding usability testing tips:

Start doing it before you think you can. This is the "sketch on a napkin" stage.
Continue through the "cubicle" stage (showing ideas to people in your organization) and the "neighbor" stage (getting the ideas from someone totally unrelated to your project)
Test limitedly and often. This means, bring in 3-4 people for about 45 minutes each to do a task. Then fix the problems found by those people. Then bring in 3-4 more people to test again.
Plan on testing on a regular basis, for example, set aside a morning a month (in our case, this would be per project if we are testing/developing multiple projects) to test whatever you have going on.
Don't stress about demographics. All people really need is to have used a web browser before. It's best if they don't have a lot of experience with your site.
Be careful with sequencing tasks during a test. Once a person has completed a task, they have learned about your site, and that can mean that they miss problems with the subsequent tasks you ask them to do
At the beginning of a testing session, get the person to talk: use this time to ask who they are, what they do, how much time they spend online, what they do online, what kind of web sites they like. Part of this is to give you context for when they are performing your tasks, but it also is a "warm up" period. It gets them accustomed to talking to you, it shows them that you are interested and listening to them, it helps them trust you.
Record the sessions with Camtasia or Morae. Have the person sign a permission form for recording the session. This is even if we don't do IRB exemption.
Eye tracking software is cool, BUT ... it's really expensive and provides more data than most people know what to do with. At this point, if you need eye tracking data, it's probably cheaper in the long run to outsource the eye tracking tests.
Ignore the people who say "I don't like color [X]" or other design opinions based on personal preference. On the other hand, if you get several people who all express the same strong opinion on a design element, then you should pay attention.
Each test will be task oriented, but you will get non-task things (such as points of confusion, points where the site loses the trust of the user, etc) from listening to the person's talk during the test and observing points of hesitation, etc.
As a test facilitator, you'll be in "therapist" mode: getting people to talk about what they're seeing and thinking, asking what they expect will happen if they do [X], prompting them to say what they're thinking/reading if they have been silent for 15 seconds or so, and, most importantly, not leading the person to an action or statement.
If the real live task involves a completion point (for example, for an online store, buying something ... in our case, I suppose it could be saving a citation, printing an article, etc.) take the person through to the end of that completion point, even if you set up a dummy situation.
Give the person some choice in the task they'll do. For example, if the task is "find 5 articles for a paper", let the person choose the topic of the paper. The idea is to give the person some personal involvement in the outcome of the task. This also lets them do the task within a conceptual space they are comfortable with.
Always pay the people in some way. In our case, we should always give some kind of gift certificate, maybe a $10 JavaCity certificate, a pair of movie passes, etc. Note that in a typical, non-academic environment (i.e. in the commercial or non-profit worlds), the standard rate for paying people to be testers is $50 a session, unless they are a select group, for which you'd have to pay more. We don't have to pay that much, but we have to give people something for their time.

And, as with all things related to web design and development, in the end "it all depends". Test informally or formally, test random people or user segments, depending on what you're doing, what information you're after, etc.

Conference Wrap up

2005-03-17T21:55:00.000-06:00

There are some session presentations available now -- especially "Rules for Remixing" (get the PDF), the conference opening session with Rael Dornfest and Tim O'Reilly.

The most important session for us was "Creating Passionate Users" -- a 1/2 day tutorial presented by Kathy Sierra, who was a creator of the Head First series of books for O'Reilly. And from this session, the most important message was "allow your users to have an 'I Rule' experience"

That is -- in the end, it does not matter what your users think of you or think of your product. (That is, it doesn't matter what library users think of library resources or services). What REALLY matters is how users think of THEMSELVES as the result of their interaction with your product.

There were other themes of note at this conference:

The O'Reilly Radar and remixing
Innovation and creativity, such as at Applied Minds and the Center for Bits and Atoms Fab Lab
"All Complex Ecosystems Have Parasites" by Cory Doctorow of EFF and Boing Boing -- if a system is complex, there will be messiness. If you try to "fix" the messiness, you will only succeed at breaking the system and removing the complexity. With maturity in a system, you also have to expect and embrace the complexity, and with complexity you have to expect and embrace the diversity, and with diversity, you have to expect and embrace the mess.

Day Four

2005-03-17T11:02:00.000-06:00

Conference Announcements
Rael Dornfest
Time: 8:20am - 8:30am

Location: California Ballroom B & C

Re:MixMe
Lawrence Lessig
Time: 8:30am - 8:45am

Location: California Ballroom B & C

Conversation with Lawrence Lessig
Cory Doctorow
Time: 8:45am - 9:15am

Location: California Ballroom B & C

Military E-Tech
JC Herz
Time: 9:15am - 9:30am

Location: California Ballroom
Track: Social Software

Emerging Massive Media
Paula Le Dieu
Time: 9:30am - 9:45am

Location: California Ballroom
Track: Emerging Topics

The Economics of the Long Tail
Chris Anderson , Joe Kraus
Time: 9:45am - 10:00am

Location: California Ballroom B & C

Conversation - TBD
Time: 10:00 - 10:30

Putting the P Back in VPN: An Overlay Network to Resist Traffic Analysis
Roger Dingledine
Time: 11:00am - 11:45am

Location: California Ballroom A
Track: Peer-to-peer

VC Funding for Geeks; or, How to Get Your Technology to Emerge the VC Way
Marc Hedlund
Time: 11:00am - 11:45am

Location: California Ballroom B
Track: Business

Remixing the Networking Platform: Hacking Hardware and Adding Services
Nikolaj Nyholm
Time: 11:00am - 11:45am

Location: California Ballroom C

Ask Jeeves Alpha
Rahul Lahiri , Apostolos Gerasoulis
Time: 11:00am - 11:45am

Location: Plaza Room B
Track: Products and Services

Trust Me: Adventures in Social Engineering
Jon Oliver
Time: 11:45am - 12:30pm

Location: California Ballroom A
Track: Social Software

Life Hacks Live
Danny O'Brien , Merlin Mann
Time: 11:45am - 12:30pm

Location: California Ballroom B
Track: Emerging Topics

Odeo -- Podcasting for Everyone
Evan Williams
Time: 1:45pm - 2:30pm

Location: California Ballroom A

It's Not Rocket Science: The Brain for Designers
Matt Webb
Time: 1:45pm - 2:30pm

Location: California Ballroom B

Tech That Helps the World
Lee Felsenstein
Time: 1:45pm - 2:30pm

Location: California Ballroom C
Track: Emerging Topics

Making Web Services Personal
Ben Trott
Time: 2:35pm - 2:55pm

Location: California Ballroom B

From the Garage: Lessons Learned Birthing and Building Web Start-Ups
Mark Fletcher
Time: 3:00pm - 3:20pm

Location: California Ballroom B

Public Documents as Weblogs
Mark Simpkins , Gavin Bell
Time: 2:35pm - 3:20pm

Location: California Ballroom C
Track: Web Services

Maker Fair

2005-03-16T23:28:00.000-06:00

About a dozen people had tables showcasing their inventions at this reception, which also was an opportunity for O'Reilly to promote its new magazine Make

I think the coolest thing there was this guy who graduated from MIT in 2002, who is responsible for the original hacked XBox.

Here is Ben Hammersley again. He wore a kilt the first two days of the conference. I saw him in pants earlier, or maybe last night, but today, he was wearing a skirt.

More Makers:

Another maker ... with interesting head decoration:

Some crowd scenes:

The O'Reilly conference staffer handing out free drink tickets at the door:

The door prize (which you can win if your name is drawn AND you are present at the final session of the conference):

And finally... sometimes you just have to get away from the crowds:

Afternoon break

2005-03-16T17:13:00.000-06:00

one hour break (well, more since the previous session ended early) because the vendors exhibit room closes at 4:30 today

Remixing Culture with RDF

2005-03-16T17:10:00.000-06:00

Remixing Culture with RDF: Running a Semantic Web Search in the Wild
Matthew Haughey, Creative Director, Creative Commons
Mike Linksvayer, CTO, Creative Commons

Creative Commons

Licenses and Metadata

tracking the licesne
format of the work
permissions, requirements
extra metadata

Why use the Semantic Web?

small organization
natural language search not good for plain text metadata
decentralization -- other search engines could use it
existing RDF toolkits could be used

Metadata format

publishers and search engine needs
considered html head elements
considered robots.txt hacks
considered data in extra files/link element
chose RDF in HTML

Metadata format II

ease of use primary goal - copy/paste button and rdf in one chunk
any custom elements automated by license app
(one more but i missed it .... it's hard to type balancing an 8 lb laptop on one knee)

Nutch -- open source web crawler/indexer/query interface, aims to be massively scalable, built on Lucene Java text serach library

lucene.apache.org

Oregon State used Nutch to replace their Google search appliances

osuosl.org/news_folder/nutch

the future of Creative Commons metadata: RDF/A, Semantic XHTML, GRDDL (pron: griddle)

Conclusions:

semantic web lets anyone use the entire web as a db
nutch is a mostly prebuilt app for your domain
domain specific search engines without the infrasturcture of a search engine company
solves semantic catch 22: publishing data/consuming data

Ontology is Overrated

2005-03-16T16:30:00.000-06:00

Ontology is Overrated: Links, Tags, and Post-hoc Metadata
Clay Shirky, Decentralization Writer/Consultant, shirky.com

ways we think we understand categorization is wrong. what we're doing to categorize on the web is a leftover from previous models that don't fit now

periodic table of the elements - his perfect classification scheme

library card catalog - the most recognizable classification scheme

of course, he trashes LC, which I sympathize with, but he mistakenly assumes that LC is hierarchical, and he is trashing it because he is looking for hierarchy and isn't finding it.

then, with library problems, he complains about using the classification system as a means for shelfmarking. which is also true

first there was the periodic table of elements, then there was the library card catalog, then there was Yahoo as the first attempt to categorize the web

The idea is -- how do you organize the world -- create a categorization system -- WITH NO SHELF .... that is, with no extenally specified "correct location" for an item

(moving through faceted classification to nothing but lots of links between items -- that is, search rather than browse)

An ontology presupposes a set of users, for whom, the structure and terminology of the ontology are designed

the more you move towards size, scale, nonexpert users, the less useful an ontology is

Voodoo Categorization

most of the world is not amenable to categorization, and when you try to force it you get problems

signal loss (is it Mac, Apple, or OSX) -- you have to create, mandate, and enforce terminology (is it movies, film or cinema? and what do you lose when you collapse terms?)
predicting the future is hard (a book about Dresden goes in the category "East Germany" -- what do you do when there no longer is an "East Germany"?)

Organic Categorization

del.icio.us

folksonomies

======

well, i agree with him but only to a point. users should create their own categories. their own, ad hoc, fully in-the-moment categories are going to be great, at least for them, at least for now.

searching is great, and a massive ontology has problems

but ...

organic categorization is even less useful for finding the UNKNOWN ...

at best, you can only find individual unknown items that are within known categories ... or among the categorizes of people in your network who you trust with their categorization

does the world make sense? if you think so, then you make a categorization system based on that, not regarding those who don't agree with the way you make sense of the world

Day Three PM - Sessions

2005-03-16T15:19:00.000-06:00

Building Communities with Software
Joel Spolsky , Michael Pryor
Time: 1:45pm - 2:30pm

Location: California Ballroom B
Track: Social Software

Ontology is Overrated: Links, Tags, and Post-hoc Metadata
Clay Shirky
Time: 1:45pm - 2:30pm

Location: California Ballroom C
Track: Emerging Topics

Python in your Pocket
Erik Smartt
Time: 1:45pm - 2:30pm

Location: Plaza Room B
Track: Products and Services

Lessons Learned While Building Basecamp
Jason Fried
Time: 2:35pm - 3:20pm

Location: California Ballroom B
Track: Business

Remixing Culture with RDF: Running a Semantic Web Search in the Wild
Matthew Haughey , Mike Linksvayer
Time: 2:35pm - 3:20pm

Location: California Ballroom C
Track: Emerging Topics

Introduction to Yahoo! Search Web Services
Jeremy D. Zawodny
Time: 2:35pm - 3:20pm

Location: Plaza Room B

Hardware Hacks from the Far Side
James Larsson
Time: 4:20pm - 5:05pm

Location: California Ballroom B
Track: Hardware

Forgiveness, Not Permission: Retro-fitting the Semantic Web onto British Democracy
Tom Loosemore , Stefan Magdalinski
Time: 4:20pm - 5:05pm

Location: California Ballroom C
Track: Emerging Topics

The Easier, Faster Route to Building Mobile Applications: Using the Best of Existing Approaches and Overcoming Current Limitations
Rodney Aiglstorfer
Time: 4:20pm - 5:05pm

Location: Plaza Room B
Track: Products and Services

Social Robotics, Scmocial Robotics: Feral Robotics and Some Other Quacking, Shaking, Bubbling (what would the opposite of feral be?) Robots
Natalie Jeremijenko
Time: 5:10pm - 5:55pm

Location: California Ballroom B

BBC Programme Information Pages: An Architecture for an On-Demand World
Tom Coates , Gavin Bell, Matt Biddulph, Margaret Hanley
Time: 5:10pm - 5:55pm

Location: California Ballroom C
Track: Web Services

Independent Individuals and Wise Crowds

2005-03-16T14:15:00.000-06:00

Independent Individuals and Wise Crowds, or Is It Possible to Be Too Connected?
James Surowiecki

author of "The Wisdom of Crowds"

Remixing Wikis with Rendezvous, Web Services and SchoolTool

2005-03-16T13:50:00.000-06:00

Tom Hoffman , Tim Lauer

Tim Lauer is an elementary school principal

Networked Objexcts

2005-03-16T13:35:00.000-06:00

Tom Igoe

alzado.net

http://www.alzado.net/intromx.html

a network is at least 3 objects

KU -- a network app where, one user sits with a cry sculpture, which cries when someone in the social network expresses his/her sadness. then when the user caresses the cry sculpture, a "happiness" light illuminates at the place of the user who cried. as a way to express emotion and response over a network

Phone as Platform

2005-03-16T13:20:00.000-06:00

Clay Shirky

Phone as Platform, or "Please, Mr. Carrier, May I Add Some Value?"

it's a class at NYU

PacManhattan -- a game of PacMan played on the grid of Manhattan streets -- came out of a games class

the urban grid and the game grid could be merged in interesting ways. But they learned that GPS doesn't work that well in an urban environment. the class ended up using the phone not for GPS but for two-way voice (hahah). somebody in the control room on the phone with the person running the physical grid. So, the phone was not the platform for the app, the phone was in the app.

ConQwest -- something about moving large inflatable figures around -- a semicode (2D barcode) is captured by a phone, sent to a server, processed by the server, and then a message is sent back to the phone

Dodgeball -- mobile social software. embedding a social software pattern into a real life social network. after housekeys, the phone is the one item that everyone carries. social scanning -- it knows who you want to meet, it messages your phone when the person you want to meet is in the room (whatever), so you can go talk to the person (hmmmm...........)

Mobjects/HeartBeat -- send a hug. a way to say "I'm here, I'm thinking about you" without composing a message or being too intrusive. strip presence from messaging, leaving the presence intact

the potential of the phone as a device is hardly being tapped

check the Nokia 6630

US carriers are a real barrier

server intrstructure is the key

phone number or Bluetooth ID as a universal/foreign key

Voice is underused

Asterisk expands single PSTN line
VoIP allows geographic distribution
SIP

Mesh (multiple network devices) - hardware support is not here yet, and the carriers are still a problem

Morning Break

2005-03-16T12:58:00.000-06:00

The exhibits room is open today. The O'Reilly table says "take one free book". I wonder how many people refrained from taking more than one, if they were inclined to want more than one?

There were some tough choices.... Head First Design Patterns, Firefox Hacks, XBox Fan Book, other books with "hack" in the title. I chose Head First Design Patterns.

Did I choose an additional book, however illegal and/or unethical that might have been?

You'll have to consider all you know about me and then choose your own answer to that question...

Not so many people here today. I guess yesterday was the big deal day, with Tim O'Reilly and Jeff Bezos on the program

By the way, I need to shop for a small laptop. Maybe a tablet. Battery life notwithstanding.

Folksonomy, or How I Learned to Stop Worrying and Love the Mess

2005-03-16T12:30:00.000-06:00

Clay Shirky, Decentralization Writer/Consultant, shirky.com
Stewart Butterfield, President, Ludicorp
Joshua Schachter, del.icio.us
Jimmy Wales, President, Wikimedia Foundation, Inc.

letting people/users create their own categories and categorization

social computing -- resolving conflicts between the individual and the group

resolving conflicts between different people's views of what the categorization of something should be

now that we have tags, can we connect our tags across different systems? technorati might be doing this now

what are you tagging for? your stuff for yourself? your stuff for other people? other people's stuff? -- remember why you are tagging and who you are tagging for

tagging as a part of RDF applications?

Wikipedia

2005-03-16T12:00:00.000-06:00

Jimmy Wales

social computing

wikipedia.org

since January 2001

people sharing information freely

Wikipedia lets people publish on a widely viewed platform, but provides a measure of quality control and avoids the neglected or lost personal sites made by home page creators

The Swarming Web

2005-03-16T11:50:00.000-06:00

Justin Chapweske, CEO - Onion Networks

Large File Support - higher probablility of failuer
justin.chapweske.com/lfs-shame

swarming -- RAID for the web -- looking for an alternative to TCP/IP with large/infinite bandwidth

fault tolerance? load balancers? commodity servers like at Google? (but, scalability, expense)

an array network? a mirror network? -- even more expensive, security concerns, user experience problems

self-healing data transfer -- integrity checks during the transfer. can store data on untrusted systems and still know if the content remains intact

===

Best Practices for swarming:

static elements, composed dynamically (CSS, RSS, Google maps)
utilize http caching semantics
thing cheap servers, cheap bandwidth, and intelligent software: the promis of grid computing delivered

SwarmStream public edition
onionnetworks.com/products/swarmstream/sspe/