An extended summary of the findings in narrative form.  

For a quicker summary, go directly to Recommended Next Steps.
For excruciating detail, see the raw Transcripts.  

Getting Started


Time. Reviewers spent between 5 and 40 minutes reviewing the site. This disparity reflects the fact that some found it fascinating and others didn't. (As you can see from the questions, shown in Test Overview, we weren't at all directive about how much time to spend.) However, half the sessions (including one of the two double or co-discovery sessions) spent 12-13 minutes: definitely the median time.

Browsers. Offered a choice, 2 chose IE and 6 (including both groups of 2) chose Netscape as their preferred browser for this task. (That's not the breakdown of browsers actually used, due to version problems and inability to access the site from some browsers on some dates.)

URL. Three remarked on the URL itself as interesting, or entertaining, or just a great URL to have gotten for this project. "I thought you were joking."

Site's purpose (on first glance). All read the top summary as the answer to this question. Three read (or pronounced) INTRAnet as INTERNET; two pronounced it INTRAnet, one said "for the web," two didn't specify. Some remained confused about web sites vs. intranet documents for some time. Reviewers seem to be skimming, so we might want to emphasize this distinction even more on the first page.

Reviewers' first click. Six (including one double group) clicked first on About this site. The other double group clicked first on Techniques.

Navigation


Overall, the site's navigation wasn't clear enough for this group. UIE recommends not worrying about navigation apart from the site design as a whole, and I agree, but I couldn't resist summarizing some navigation-related findings here.

Six reviewers had explicit navigation issues and complaints.
 
  • The most common request was to keep the TOC in the left border at all times.
  • The next most common problem was Trellix-generated navigation links ("left over" from auto-generation) that were not working.
  • One suggested a mailto link on every page, for convenience.
  • One couldn't figure out how to get back to the home page to find the samples she'd seen earlier...either because navigation was not consistent, or because home and Techniques looked so similar, or both.
  • Several users objected to the use of "click here" as link text (or not link text).

Quotations and further ideas:
 
  • "There's no persistent navigation, so that's a problem."
  • "This is nice, the little table of contents in the left margin, but you lose that…(demonstrates clicking back and forth to About this site)…so one of my comments…was, I'd like to see that in the left panel on all of these pages, `cause otherwise I have to go back to the home page to switch topics."
  • (to Dan) "Even though you've put this in quotes, it's still [not clear]. I'm not as familiar with the document as you. To you this site is pretty obvious because you're emotionally involved with the document." Dan: This is very helpful.
  • "I don't think I've ever seen a tour of a web page. The concept of a tour would be new to most people, I suppose. One comment I have is, in lieu of a map, something that would be more generic, say, would be to have, instead of next stop on the tour, have the little tour stops along the bottom. The one that you were in could be colored differently. I was wondering where I was along the tour, I couldn't remember the list of stops…don't' know if they'd fit along the bottom or not."
  • "You shouldn't need this, or if you do need it, it should be hot. I think Jakob Nielsen or somebody said, `if you need to say click here, you're doing something (wrong).'"

While on "tour", two users saw an attractive link in the page being discussed, followed it, and went off the tour. I'm not sure this is a problem, just being descriptive. "What I've done here is I've stepped…off the tour for a second." J: How did that feel? "I guess that's what I figured was happening. Stepped off the tourbus for a bit, but you can get back on the tourbus."

Naming


Reviewers felt some page and link names weren't descriptive enough. The biggest problems were with pages that had "site" in their name, including About this site and This web site. Then the related link called Back to `this site' read very confusingly: the reviewer asked if it was the same as the browser back button.

Another reviewer didn't see the difference between Techniques and Samples.

Quotations:
 
  • "About-this-site is a snoozer. Everyone has an about-this-site. I would remove the word site off of there. Use `good documents defined' or something. It could be credits, and thanks to my mom and dad…"
  • (about Write in newspaper style) "You could simply say, use the inverted pyramid, because that's what this whole page is about. It's not about writing like a newspaper. This whole page appears to be, the inverted pyramid." Suggested renaming the page to that.
  • "The tricky thing is trying to come up with catchy names for pages. Like, `this [web] site' isn't" a descriptive name."

Text - Clarity & Layout


This and the following sections contain comments on various aspects of the text in the site, with general comments followed by comments related to specific pages.

In general:
 
  • "When I first came into it this morning, yeah…actually, on UNIX, the fonts were so crufty and small, the default fonts…I got very annoyed. I did actually end up changing them."
  • "This needs a copy editor, just for punctuation and indentation."
  • "The presentation of the content in a way lends credibility or takes away from credibility. So, just the formatting on this page makes me think that [it's an amateurish-looking site]…not that everybody has to have a web designer do their site, but there's some happy medium…he could be [more] consistent….maybe I'm just a little anal." J confirms: you mean people are looking at both form and content? "That's the web… It'd be nice if it was just on the merit of the content. [But instead] the presentation distracts from the content if it's not [just right]."
  • "Kind of an understated web site…on the internet, everything seems to be graphics heavy, pictures and stuff all over the place. It's a little unusual, the site is a little unusual that way."

Page: About this site.
 
  • Read Not for web sites… line twice. "That's an unclear phrase, I don't get that."
  • Just below that: "Unclear what you meant. `Web sites' is too generic a term to be meaningful in this context. And even the way it's phrased…you almost need another `for' in there."
  • "Are these headings?" (J: that's what he's highlighting) "They're a little long too." (other reviewer) "Yeah. The principle (make links descriptive) is good to a point, but at some point, I don't want to read all this in a title, I might just as well read the article, or the section!" (back to first quoted reviewer) "Yeah.""
  • "Right here I'd be confused and I'd go back."
  • "My first thought: `…I'm not gonna read this. This seems awfully dense.' You gotta scroll a whole lot, it's not clear if there's any payback for scrolling, just seems like a lot of text. I actually did not read it this morning, and said, `well, phooey, don't need that.' "

Page: Goals of the design
 
  • Dan: Detail is a technique in the list - some reviewers said final paragraph was poor English, in which `detail' is being used as a noun. This hasn't been gone over by an editor.

Page: Overview
 
  • "Good." J: what? Likes that "you both list how to write the text and how to lay out the pages." But, "I'm not sure what the differences between these two - these three headings are unclear." Thinks the last three headings (how to layout, how to design, how to organize - all but the top heading) are not sufficiently different from each other.

Page: Techniques
 
  • Re Write links that don't have to be followed: "I don't understand that sentence." (other reviewer) "I thought this was like a list of things that you shouldn't do, which it seems to be." Meaning Write links again - since it's first and he didn't understand it, and might disagree, he thought the whole thing was a list of counter- or bad examples.
  • "I didn't think there were links initially, I thought it was just like really bad style. When I see links…like this, I initially thought that it wasn't really a link," but just lots of text formatted like a link, kind of a newbie/amateur error.

Page: This web site: A tips and techniques knowledge base
 
  • Reviewers didn't like the horizontal list being used within a paragraph, they're more used to seeing it, centered, at the bottom of a web page. "Just put a line break!" (other) "Or even use bullets!" (They're both laughing hysterically at this point.) (first reviewer again) "I feel bad…Is this what the software looks like, `cause I'm getting the idea that this is what it's trying to help you do." (Negative reaction) Other reviewer cracks up again.
  • (same feature) "This format for me seems like…and this is more based upon our background…like a textual footer, mimicking a graphical toolbar." Usually centered, "so when I saw…my eyes went…oops."
  • "I don't necessarily like having the bold in the same line. Bold for most purposes is always a header, it's not part of text. I would have Tips and Techniques and then have a break under it, and then have text…"
  • Because of possible confusion finding link and difficulty scanning, doesn't like line with Text here then Start
  • Pulls up HTML source. "Font size six!" (Shocked.) "The header's huge…that's a little big. Can you tell me what you created it with?"

Page: Various tour pages
 
  • (Referring to the page that frames up Techniques) "A little rocky here." Not sure if he liked "seems to becoming" - maybe wants it to be "seems to be becoming."

Page: Tour of a tour page
 
  • "Is this…? This is very confusing." (Goes back and forth.) "This is…a tour of a tour? This gets too meta for me, I guess." (Scrolls left border.) "When you just look at it, it's like, `there's a mistake!' Or if you've messed a lot with frames, constructing frames, making mistakes, you end up with something like this. This struck me as `this isn't what is supposed to happen.'"

Text - Readers' Reactions to Content


In general:
 
  • "I expect it to be mostly a style guide, but maybe some thoughts on what products you might use." (Said with a grin.)
  • "Good basic information. It makes it clear that this is going to be a style guide for writing documents, not for coming up with the next thing on Netscape's `cool sites'."
  • "This is pretty cool. Gives you writing tips. Maybe this can be the next Strunk & White! So often a lot of the stuff you find on the web is just fluff. It's good to find some free stuff that's interesting."
  • (Re deciding what should be on the site.) "I've been thinking a lot about structuring and architecture of web documents. Do we really need a page that says `new.' It depends. You gotta answer that question for each site." Discussion: making each element pull its weight. "You first start with, `what buckets of information do we really have to present to people.' It's really easy to get into, `well, we'll do a new thing, and we'll do links to related sites, and blah blah blah blah blah.' Well, links to related sites, everybody does, but think about it, it takes them away from your content. Do you really want to do that? Maybe you do. In my case, I don't, so I don't want that." J: prompts her to go to goals of the design. Is this what you meant? (Re tour narrations) "You could combine the two in the left column: what were my goals here, and how did I do it." That's what she would like.
  • "I'm not someone who's involved with writing…web documents, like they say here. I haven't really published any documents on intranets or internets; because that's not one of my interests, that site wouldn't be that interesting to me." Dan: Suppose someone told you now you had to publish docs to an intranet. "Then I would probably have an interest."

Page: About this site
 
  • "You make a very important point here, and it's four paragraphs down and not well headed…your target audience may be internal consumers of information, not the general public. We have this sound-bite mentality…you've got eleven seconds of my attention span, grab it. That's not what you're about. There's an important statement here, you say it here, but I missed it."
  • "New mediums need their own forms for writing - I don't know what that means, their own format? Or form like a website form? It started to get vague. Right away I'm like, this doesn't seem like good writing on this site, so I'm already confused, so I don't see how it's going to help me more."

Page: Philosophical discussions
 
  • "I would probably come back to this, given time…I'd even go to the philosophical discussions, I suspect that wouldn't be (a popular destination though)."
  • Didn't go there. J: from the title, how did it strike you? "Just as something that I probably already know."

Page: Techniques
 
  • "Links that don't have to be followed…okay!"
  • "I think the links are too long…that's what confused me on the first page." (other reviewer nods) Disagreed with both the recommendation and the examples of it, based on his group's corporate style.

Page: Write in newspaper style.
 
  • "Inverted pyramid…we all know this… [Reads more] This is a good overview for people who haven't written in awhile."
  • "On the other hand, there's a lot of other information that goes into a newspaper style guide, that you probably don't want to use on the web, that you don't want people to emulate. Such as `telegraphspeak,' they leave out adverbs, and they leave out prepositions, and that can be confusing if you write technical documents for the web." In a newspaper, "you tend to write about halfway like people would write in a telegram. You don't need to do that on the web." Dan should be more explicit about what to copy (from them) and what not to copy.
  • "I read the paper every day but mainly I'm not really reading it, I'm skimming it. I only read a fraction of it. (Reading most of the page) Pretty interesting!"

Page: Various tour pages
 
  • (Referring to the border content) "I guess I would want to see a paragraph about why I thought I needed this stuff, what's my goal here? I guess a little bit more rationale than explication."

Page: News
 
  • "I'm kinda confused, is this for a particular company's intranet, or advice for anyone writing for an intranet?"
  • "What's the rationale for the news? Why do users care about this?"
  • "I've noticed most sites have a news page. I've been going to a lot of corporate web sites…there's always a news page…"

Miscellaneous


Three reviewers with some (previous) Trellix exposure asked why there was no map on the site. (After the first text, Dan added a demonstration map, but just in one sample: they meant, a map for the site as a whole.)

Many reviewers were slow to notice that the site used its own techniques. But most eventually did notice at least some of this.