First Monday
Read related articles on Web sites - Design, Libraries and Electronic Publishing
A Typology for Ready Reference Web Sites in Libraries, and What It Can Tell

Many libraries manage Web sites intended to provide their users with online resources suitable for answering reference questions. Most of these sites can be analyzed in terms of their depth, and their organizing and searching features. Composing a typology based on these factors sheds light on the critical design decisions that influence whether users of these sites succeed or fail to find information easily, rapidly and accurately. The same analysis highlights some larger design issues, both for Web sites and for information management at large.

Contents

Reference Web Sites
Why do Libraries Build Reference Web Sites?
Why Examine the Design of Library Reference Web Sites?
The Designer's Dilemma
Assessing Reference Web Sites: The Right Tool for the Job
Elements in a Typology of Ready Reference Web Site Designs
Types of Reference Web Sites
Conclusion
Notes

Reference Web Sites

The growth of the World Wide Web places more and better content at the disposal of Web users every day. That same growth makes it harder and slower to locate specific content, given the essentially unorganized nature of the Web.

For this reason, schemes to search, label and organize Web resources abound. Web browsing applications from Mosaic to Netscape Navigator have recognized and addressed this tendency: the capacity to bookmark the URLs of specific sites is built into their design. It is a small step from a set of bookmarks to a personal "home page" sharing hot links with other Web users. Another small step leads from personal home pages to "home pages" for institutions, with content that aspires to general rather than merely personal appeal. At the farthest end of this path we find the commercial search engines, of which Yahoo! in particular is noteworthy for its subject tree approach to managing a vast array of links.

For librarians, these functions of selection, organization and presentation are familiar professional tasks. Many libraries create their own organized lists of  Web-based resources to supplement their collections of locally-controlled printed materials and digital resources. Entities like the Internet Public Library are venturing into pure cyberspace, leaving behind traditional collections and the idea of the library as a concept limited by physical location. One of the most frequent, and potentially useful, library experiments with the use of the Web involves the construction of pages that work with reference tools.

A "reference Web site" is an HTML-based page (or system of pages) that provides potentially useful information by assembling hot links to online tools (some sites also include citations to paper tools). Because Web site designers must assume that remote users will be working without direct human assistance or the opportunity to discuss subtle nuances, these sites emphasize factual material: reference information that addresses basic questions of the sort known to librarians as "ready reference" questions ("What is the capital of Vermont?" or "When was Dante born?"). There are scores of "ready reference" Web sites that vary in size and design, most of them created by the reference departments of university libraries. The information content itself -- such as a table of metric equivalents, or an interactive currency converter -- almost always resides on a server at another site. The library reference Web site itself consists of a collection of hot links, plus whatever added value pertains to organizing and describing that content.

Why do Libraries Build Reference Web Sites?

Why are so many librarians committed to creating reference Web sites? After all, Web utilities like AltaVista can find the same content using powerful search engines. And individual Web surfers easily "bookmark" their own favorite, useful URLs: a site for the local weather forecast, an online newspaper, an interactive stock market ticker, and so on. These alternatives are readily available to any user with Web browsing software, at no additional cost.

Relatively little has been published about the purposes and uses of library-created reference Web sites. This does not mean that the use of the Web for reference has been ignored. For example, College and Research Libraries News regularly publishes an "Internet Resources" feature [ 1], and a review of library and information science literature shows how rapidly librarians realized that an important new instrument was theirs to use. 

In 1994, Thomas Pack wrote about an "Online Reference Shelf," citing suitable online tools, but the Web and hypertext did not figure into his conception [2].  In the same bound volume of Database, however, we find an early description of the World Wide Web and an astute prediction about its potential. James Powell fully anticipates the reference Web site while describing experiments with HTTP: "...one of the areas where librarians could make the greatest contribution to the Internet was the document [that is, Web page] listing resources by subjects.  Here I listed a few subjects with links to various resources, such as Gopher servers, other WWW hypertext documents, or Usenet newsgroups related to each subject. ... In this way, seemingly unclassifiable collections of information ... could be presented in an organized manner" [3] . 

By 1996, Greg Notess would be citing specific Web pages and their URLs for his "On the Nets" column, and thinking in terms of "Internet Ready Reference Resources," if not of reference Web sites per se [4]. In the same year, Scott Mellendorf neatly described the evolution of bookmarking into something like a subject-based reference Web site: "As any Web surfer soon discovers, a collection of bookmarks can grow very quickly. Soon the collection becomes so large that it loses any real value. ... After experiencing, 'That's not what I thought it was,' a few times, we split our bookmark collection into subject groupings. This allows easier access and provides a logical organization scheme" [5].  It has taken only a few years for these exploratory ideas to grow into numerous well-conceived reference Web sites maintained by libraries.

Such sites became not only possible, but attractive for libraries. Librarians invest time and resources in creating reference Web sites, because doing so extends four familiar library service functions into cyberspace:

Recent research published in Science further indicates that Internet search engines alone may not discover all relevant materials on the Web.Ê Steve Lawrence and C. Lee Giles found that "individual engines cover from 3 to 34% of the indexable Web" onlyÊ and are challenged to keep up with the growth of Web content.Ê Ready reference pages prepared by libraries offer an additionalÊ hedge against incomplete results [ 6 ].

Why Examine the Design of Library Reference Web Sites?

The principles of librarianship explain the fascination of library reference Web sites for librarians. Why should other Web denizens care about them, either as information seekers, or as students of the Web and its potential?

For information seekers, applied librarianship has benefits: finding what one wants, as rapidly as possible, from the "best" source (accurate, current, accessible), and with a minimum of wasted motion.

For students of the Web, including designers of other Web pages, an examination of library reference Web sites suggests useful criteria for the evaluation of Web sites at large, pointing out the strengths and weaknesses in several organizational plans, and testing ways to make access to content faster and more precise.

The Designer's Dilemma

Anyone using library reference Web pages knows that some sites work better than others, and soon settles on a few favorites. Users may (or may not) recognize what attracts or disappoints them. Some sites fail to offer sufficient content; others combine too much content with insufficient organization, so that the user flounders. These errors show the importance of valid design and selection decisions.

When Richard Einer Petersen identified "subject directories which are manually-maintained collections of Web sites organized by topic" as one of "Five Major Categories" of Internet search engines [7], he also concisely summed up the purpose and design behind reference Web sites. In another general discussion of Web site management, Thomas E. Jevec clearly noted the key decisions - and dilemmas - faced by reference Web site designers: for example, "balancing design, content and functionality;" and choosing between "naming conventions and site structure"  [ 8]. Because they aspire to gather contents from all suitable sources, such sites must accommodate a wide range of Web designs created by diverse hands. Because they are intended to handle high levels of traffic, they must be well organized. Library ready reference Web sites therefore shed revealing light on some general issues of Web site design.

There is a familiar aphorism (sometimes known as Wexelblatt's Scheduling Algorithm) which says that planners can:

A variation for reference Web site designers could be

A good reference Web site designer must choose the first element: failing to do so results in an unusable site. Then a single choice remains: making the full contents immediately accessible on the first page seen by the user, versus including as many potentially helpful resources as possible. The first option leads to a smaller but consequently simpler site. The second option leads to a larger site, burdened by the complexity that comes with size, so that clear labels, logical combination of elements, and effective navigational options play a critical role.

Assessing Reference Web Sites: The Right Tool for the Job

No single Web design or collection of resources can meet every situation: users differ, and questions differ. However, alert librarians and users can gauge the potential utility of given reference Web sites for specific situations, choose wisely, and save time by remembering two simple principles:

When deciding whether to explore a specific reference Web site, users can apply these two principles by asking two simple questions, which can be answered after a brief inspection of the site:

Don't reach for a hand grenade when a fly swatter will do - or vice versa.

Elements in a Typology of Ready Reference Web Site Designs

One can assign most reference Web sites to a typology of designs based on answers to these same questions.

Depth criteria

First, sites can be described in terms of their "depth." In other words, how many times must the user click on a hot link within the original site, before arriving at the server with the actual content?

The greatest difference in design, and in the user's experience, occurs with the transition from Group I to Group II sites, because it is at this point that hierarchy is introduced and with it the need for sharper organization. Group III sites differ from those of Group II only in degree. For the same reason, there is little point in describing sites as "Group IV" (involving four layers, four choices and four clicks), "Group V" and so on. The success or failure of the organizing scheme already will be apparent at the Group III level.

Some sites have mixed or inconsistent depths: one path might require two clicks, another might take three. Because of the increasing burden imposed on users by designs for sites from higher-numbered Groups, it makes sense to classify sites with paths of mixed depth by their higher, rather than lower, designation. Thus a site with a mix of Group II and Group III characteristics is better treated with other Group III sites.

Organizational criteria

Second, sites can be described in terms of their organizing or searching features. Minimal experimentation and observation will disclose which features are available.

Sites may combine options. For example, an Option K search engine might accompany a display of topics by subject. The site might be described as one of Option K+S, but it remains better to analyze and discuss each Option-based Type separately, because the user in practice chooses one approach or the other at a time.

Types of Reference Web Sites

Combining these characteristics - depth and organization - creates a typology with a dozen distinct elements.

Option A Option S Option S# Option K
Group I

Type I-A

Type I-S

Type I-S#

Type I-K

Group II

Type II-A

Type II-S

Type II-S#

Type II-K

Group III

Type III-A

Type III-S

Type III-S#

Type III-K

Each Type has its own characteristics, strengths and weaknesses, and logical application to information seeking on the Web.

Group I sites

Group I sites employ a single Web page, displaying all links to content resources. This quality of "direct access" can be tested by moving the mouse/pointer up and down the list of links to see the destination URLs visible at the bottom of a Netscape browser. All of the content servers will have addresses somewhere else on the Web. (Or virtually all: in rare cases, one or two links on a large site will deviate from the pattern, but too infrequently to have much effect.) Group I sites have no depth, and so no hierarchy.

Effective user access means that Group I sites need to remain small, for two reasons. First, the whole content list must download from the Web before the page can be used. Second, users have to scroll through the whole list to know what resources are available.

Group I sites are useful for answering routine questions, especially in cases where the user already knows of a likely source, and the primary task is to find it on the Web.

Group I sites can be subdivided further according to their internal organization:

Group I sites typically present between thirty and sixty URLs, almost never more than a hundred.  As the number of significant Web resources grow, there is a tendency for Group I sites to evolve into Group II sites as their designers add and then organize additional content.

Type I-A sites

These are the simplest reference Web sites, single pages listing their full contents in alphabetical order.

Some representative Type I-A sites include:

Type I-S sites

Type I-S sites group resources by subject, rather than alphabetizing them by name.

Some representative Type I-S sites include:

Type I-S# sites

While theoretically possible, reference Web page designers in practice have ignored Option S# (Dewey) for Group I sites. Their small size does not demand this amount of detailed control.

Type I-K sites

Type I-K sites offer a keyword search engine, so that the user no longer has to rely on either alphabetical lists or predetermined subject groupings.

A representative Type I-K site is:

In small sites, keyword search capability is less crucial than in large ones. In fact, the "Find in Page..." function on most Web browsers fills the same role, if the site consists of a single page.

Group II sites

Group II sites employ multiple Web pages, arranged in a hierarchy. Links to the content resources appear on secondary pages. This quality of "indirect access" can be tested by moving the mouse/pointer up and down the page: links on the primary page will lead to secondary links on the same local server. URLs for the destination resources will appear only on the secondary pages.

The additional structure in Group II sites allows them to offer larger numbers of resources, without sacrificing accessibility. No single page has to be too large to load quickly or browse reliably, and a variety of organizing devices help users choose intelligently among those pages.  An ability to express the characteristics of each secondary page is crucial. If these pages are identified poorly, or lack consistent focus, users may visit multiple pages before finding the one they need -- or before giving up.   

Group II sites are useful for answers that rely on less common resources, especially if the user has no specific source in mind. The subject-based groupings become a source of added value.

Group II sites can be subdivided further according to their internal organization:

Type II-A sites

Type II-A sites use multiple pages to list large numbers of titles. Typically, pages set up along this model are supplements to other versions of reference Web sites.

A representative Type II-A site is:

Type II-S sites

Type II-S sites are the most prevalent reference Web sites: large enough to accommodate many resources, but not so large as to become confusing.

Resources are grouped by topic or theme on separate pages.  These secondary pages are listed on the main page, often in alphabetical order by their designated subject. The clarity of the labels in that list has a major impact on navigating the site.

Some representative Type II-S sites include:

Type II-S# sites

Type II-S# sites adapt the subject-based page, by using the Dewey Decimal system (or a similar classification scheme) instead of named groupings.

For example, a user who knows that the Dewey number "310" denotes statistical material can move immediately to appropriate materials without having to recognize any particular named resource. On the other hand, users without special knowledge gain nothing from this approach, and have to rely on subject labels that appear in arbitrary order.

A representative Type II-S# site is:

Type II-K sites

Type II-K sites provide prominent keyword searching options for the user. Many sites with this feature combine it with subject-based categories.

A representative Type II-K site is:

Any modern library OPAC (Online Public Access Catalog, or computerized "card" catalog) functions along the same lines as a Type II-K site: see for example the keyword searching screen of Tripod, a library catalog at Bryn Mawr College. With the addition of hot links to Web-based content in new WebPACs, the resemblance will increase. The AltaVista search engine (and others like it) relies on the same principle, in a very sophisticated form and without being restricted to ready reference-oriented URLs.

Group III sites

Like Group II sites, Group III sites employ multiple Web pages, arranged in a hierarchy. These sites have even greater depth: secondary pages refine the categories sketched out on the primary page, and links to the content resources do not appear until the user arrives at a set of tertiary pages.

This additional degree of structure allows Group III sites to offer even greater numbers of resources. In theory, proper organization prevents the loss of ready access. In fact, however, each decision placed before the user increases the opportunity for miscommunication, and the additional number of clicks by the mouse may wear down the patience of some users, especially if pages download slowly. As in Group II sites, clarity in assembling and labelling the groups of resources is critical.

Group III sites sometimes contain thousands of resources, therefore they are appropriate tools when the answer to a question seems to demand an obscure source.

Group III sites can be subdivided further according to their internal organization:

Type III-A sites

In theory, Type III-A sites would display the names or titles of resources, in such large numbers that primary, secondary and tertiary screens would be necessary to keep the individual pages from becoming inconveniently long. No sites of such size seem to exist at this time. The Ohio State site noted under Type II-A suggests how such a site might work, if we imagine an intermediate layer for each letter of the alphabet.

Type III-S sites

Type III-S sites extend the model of Type II-S sites, but add an additional layer of pages to manage larger numbers of resources.  Because the availability of online tools on different subjects determines the need for such a step, and varies from subject to subject, many Type III-S sites are "mixed" in depth. As previously noted, there is little point distinguishing four or more steps from three, however.

Some representative Type III-S sites include:

Type III-S# sites

Type III-S# sites build their extended system of subject categories on the Dewey Decimal system, rather than categories named by the Web site designer. Because thousands of resources are listed, this structure comes across as somewhat more useful at this level of depth, than at the Group II level.

A representative Type III-S# site is:

Type III-K sites

Type III-K sites are typically so large that the set of materials retrieved by a keyword search is itself so large, that it benefits from a hierarchical organization. For example, in Yahoo!, the first choice for the user involves entering the keyword (which permits the user to define a category, to some extent); the second involves selecting among an array groups, and the third choice may involve either actual content or another layer of hierarchy (thus this is another site of "mixed" depth).

A representative Type III-K site is:

Conclusion

A typology of this kind should not be an end in itself: it is pointless unless it leads to analysis. By analyzing specific Web sites, we can assign convenient labels to the ways in which Web site designers try to organize large amounts of varied material. Analysis also makes us aware of which features on a given site critically affect its performance. It is easy to recognize that rotating icons and gratuitous background colors do nothing to improve some sites; it may be less apparent that other devices (including tables or forms) may contribute little to the actual user's experience, including success or failure.

One can draw some conclusions even from a superficial application of typology to Web sites:

The experience of librarians with ready reference Web sites should not only contribute resources from the Web to the practical side of public service in libraries, but also pay off with insights into general strategies for managing information in all its forms.

About the Author

Steven W. Sowards is the Head of the Social Sciences and Humanities Reference unit of the Michigan State University Libraries, East Lansing Michigan, USA. He has been a reference librarian since 1986. His office Web site is http://www.lib.msu.edu/sowards/

e-mail: sowards@pilot.msu.edu

Notes

1. Among the recent examples is Judith L. Hart and Gary E. Hart, 1997. "Biotechnology Resources," College and Research Libraries News, volume 58, number 11 (December), pp. 759-765.

2. Thomas Pack, 1994. "The Online Reference Shelf: Finding Basic Information about People," Database, volume 17, number 2 (April), pp. 61-65.

3. James Powell, 1994. "Adventures With The World Wide Web: Creating a Hypertext Library Information System," Database, volume 17, number 1 (February): pp. 59-66. The quoted passage appears on p. 63.

4. Greg R. Notess, 1996. "Internet Ready Reference Resources," Database, volume 19, number 2 (April/May), pp. 88-91.

5. Scott A. Mellendorf, 1996. "Working the Web with a No-frills 'Work Page,'" Online, volume 20, number 1 (January/February), pp. 21-24. The quoted passage appears on p. 24.

6. Steve Lawrence and C. Lee Giles, "Searching the World Wide Web," Science 280/5360 (3 April 1998): 98-100. Science appears on the Web in fulltext at http://www.sciencemag.org but only for fee-paying subscribers.

7. Richard Einer Peterson, 1997. "Eight Internet Search Engines Compared," First Monday, volume 2, number 2 (February).

8. Thomas E. Jevec, 1997. "Designing and Maintaining Information in the Fast Lane," First Monday, volume 2, number 8 (August).

Contents Index

Copyright © 1998, ƒ ¡ ® s † - m ¤ ñ d @ ¥