First Monday
Lost in gallery space: A conceptual framework for analyzing the usability flaws of museum Web sites by Paul F. Marty and Michael B. Twidale

Abstract
Lost in gallery space: A conceptual framework for analyzing the usability flaws of museum Web sites by Paul F. Marty and Michael B. Twidale
This article reports on a study which used results from 119 scenario–based evaluations of 36 museum Web sites to develop a conceptual framework for analyzing the usability flaws of museum Web sites. It identifies 15 unique dimensions, grouped into five categories, that exemplify usability problems common to many museum Web sites. Each dimension is discussed in detail, and typical examples are provided, based on actual usability flaws observed during the evaluations. The availability of this conceptual framework will help the designers of museum Web sites improve the overall usability of museum Web sites in general.

Contents

Introduction
The users and usability of museum Web sites
Scenario–based evaluations of 36 museum Web sites
A conceptual framework for analyzing the usability flaws of museum Web sites
Conclusion: Improving the usability of museum Web sites

 

 

++++++++++

Introduction

Museum Web sites appeal to a wide spectrum of Internet users, including elementary school students, university researchers, and the general public (Jones–Garmil, 1997). Every day, thousands of museum Web sites offer users the opportunity to immerse themselves in great works of art, conduct scientific experiments, or learn about different cultures (Bowen, et al., 1998). The potential educational value of museum Web sites raises an important question: how usable are the experiences offered by museums online? While the need to evaluate Web sites for usability has been recognized for years (Nielsen, 1999), museum professionals have only recently applied usability analysis techniques to their own Web sites, and no framework exists for guiding the analysis of museum Web sites as a distinct genre in general (Cunliffe, et al., 2001).

This study uses results from 119 scenario–based evaluations of 36 museum Web sites to develop a conceptual framework for analyzing the usability of these sites. This framework consists of 15 unique dimensions, grouped into five categories, that exemplify the major usability flaws found with museum Web sites. In this article, we discuss the importance of usability for museum Web sites, present our conceptual framework, and provide examples of the typical usability flaws exemplified by the framework’s dimensions. Our analysis will help designers build museum Web sites that avoid the usability problems affecting these sites today.

 

++++++++++

The users and usability of museum Web sites

As museum Web sites have become more complex, interactive, and sophisticated, the need to know whether the intended users of the Web site are actually finding these sites both useful and usable has increased in importance (Marty, 2004a). Museum Webmasters find themselves asking questions such as: Are our Web sites meeting the needs of our online museum visitors? Are our users satisfied with the online museum experience we provide?

These are difficult questions to answer. While we have learned a great deal about the demographics of the online visitor, we still have much more to learn about what users want to do when they visit museum Web sites and whether their needs are being met (Booth, 1999; Bowen, 1999; Chadwick and Boverie, 1999; Kravchyna and Hastings, 2002; Ockuly, 2003). This lack of data means that it is not always possible for museum professionals to know whether they are building museum Web sites that meet users’ needs; as a result they risk upsetting and alienating the very users they are trying to reach (Marty, 2004b).

Museum Webmasters now realize that their Web sites need to become more user–centered.

While the long–term solution to this problem will require more detailed, longitudinal studies of the needs of the users of museum Web sites, a simple short–term solution exists: all museum Web sites should be evaluated for usability. Usability analysis is the process of evaluating interfaces to determine the degree to which those interfaces are easy to use (Nielsen, 1993). Poorly designed and hard–to–use Web sites can easily drive away users, and the usability of Web sites in general is of significant concern for all who design and use them (Nielsen, 1999).

Recently, there has been a call for more usability studies to be conducted on museum Web sites (Cunliffe, et al., 2001; Cleary, 2000; Peacock, 2002; Semper, et al., 2000; Vergo, et al., 2001). Museum Webmasters now realize that their Web sites need to become more user–centered (Streten, 2000; Hertzum, 1998). To do so, museum Webmasters are taking steps to evaluate their designs, determine whether they are usable, and use the results of their evaluations to inform the design of their Web sites (Garzotto, et al., 1998; Dyson and Moran, 2000; Harms and Schweibenz, 2001). While existing recommendations for improving the usability of all types of Web sites are certainly helpful for the designers of museum Web sites (Krug, 2000), an analysis of the common types of usability problems faced by users of museum Web sites will help museum and information professionals design more usable museum Web sites as a specific genre.

 

++++++++++

Scenario–based evaluations of 36 museum Web sites

As advocates for increased usability analysis of museum Web sites, we conducted usability evaluations of 36 different museum Web sites at national and international conferences for museum professionals over the past three years. Our principle aim in conducting these evaluations was to illustrate the power of user testing, and to advocate for frequent usability evaluations in the museum Web site development process. As our demonstrations progressed, we realized that the usability problems we identified, as well as the design recommendations we derived from them, had analytic value for museum Web site designers. The results of our demonstrations provided an extensive data set from which we could develop a conceptual framework to account for the common usability problems of most museum Web sites.

Since there are so many different kinds of museum Web sites, accounting for their common usability problems seems at first like an impossible task. Our evaluations covered a wide range of Web sites, from complex, database–driven Web sites to simple, static HTML Web sites, and from this variety we have found certain commonalities which we explore below. The following list provides some examples of the types of museum Web sites we tested during this study:

The specific museum Web sites evaluated during these tests were well–designed Web sites, and included many that had won awards; they were not selected as exemplars of bad practice. They were suggested by conference attendees, who also served as volunteer user testers. While we ensured that these volunteers had no prior experience with the Web sites they tested, they often had substantially more experience with museum Web sites in general than the average online visitor. As a result we would argue that any problems they encountered would likely cause even more trouble for the average museum Web site visitor.

During our evaluations, we used a special method of usability testing developed by the authors to identify a large number of flaws in a small amount of time (Marty and Twidale, manuscript in preparation). While a detailed step–by–step account of the workings of this method is beyond the scope of this article, in essence it is a method for high–speed usability analysis (cf. Thomas, 1996) where usability evaluators spend ten minutes assessing a previously unknown interface and developing representative tasks, ten minutes administering these tasks to representative users, and ten minutes analyzing the results of the tests to identify usability flaws and make recommendations for design.

Our approach is very different from that of traditional user testing, where a single Web site is tested at length, with many subjects, in order to validate the overall usability of the site (in the case of a summative evaluation) or uncover its major and minor usability flaws (in the case of a formative evaluation). The emphasis of our method is on finding as many usability flaws as possible in the shortest amount of time, and not on creating an exhaustive list of all, or even most, of the usability flaws in any given interface. Since each of our evaluations lasted only thirty minutes, and involved only one or two participants, the usability problems we discovered should be viewed as a sampling from the set of all possible problems with each site.

Our method of high–speed user testing relies on the principles of scenario–based design (Rosson and Carroll, 2001) to develop representative tasks that will help us quickly identify usability problems with a Web site. For each of the 36 Web sites tested, we custom–designed representative scenarios of use, developing and administering three or four different scenarios for each museum Web site. The following list provides examples of the types of scenarios used in our usability evaluations (to preserve the anonymity of the sites tested, these examples are only described in abstract terms; in the actual tests, particular details were provided):

Over the course of 36 thirty–minute evaluations, we developed and administered 119 such scenarios representing the typical activities of visitors to many different types of museum Web sites. From these scenario–based evaluations we identified over 500 individual usability flaws during the total study time of 18 hours. Subsequently, to isolate the common usability problems of museum Web sites from this list of specific usability flaws, we conducted a qualitative analysis of our findings, looking for problems common to the majority of the museum Web sites sites tested. The data from our usability evaluations were analyzed using the procedures of grounded theory as outlined by Strauss and Corbin (1998). This kind of analysis is an iterative, on–going process where the results of each evaluation were analyzed as part of a continual process of exploring the data to identify emergent themes and dimensions. This process included the techniques of coding and memoing in order to develop specific dimensions characteristic of the usability flaws of museum Web sites.

When we looked across the different types of museum Web sites we tested, we found there were certain characteristics associated with usability problems shared by these museum Web sites. These characteristics manifested themselves in ways exemplified by the usability problems found during our evaluations. These findings formed the basis for developing a conceptual framework that will help museum and information professionals account for common usability problems with museum Web sites.

 

++++++++++

A conceptual framework for analyzing the usability flaws of museum Web sites

Our analysis resulted in 15 dimensions that exemplify the most common usability problems encountered by the users of museum Web sites. Together, these dimensions provide a conceptual framework that improves our understanding of the common usability flaws of museum Web sites in general. For ease of comprehension and discussion, we have grouped these 15 dimensions into five general categories representing different characteristics that we find account for most of these usability problems. Each dimension can be seen as a manifestation of one of these five general characteristics:

In the analysis that follows, we will discuss each characteristic, analyze each manifestation or dimension of that characteristic, and provide examples of the typical usability flaws exemplified by these dimensions. The availability of this conceptual framework will help the designers of museum Web sites understand the common usability problems faced by the users of their sites. Note that, in order to preserve the anonymity of the 36 Web sites we evaluated, certain details have been withheld from the examples of usability flaws provided. Nevertheless, these are typical examples, based on actual, detailed, and particular usability flaws we observed during our evaluations, and as such are indicative of many other examples that might have been provided for each dimension. For more information about the quantitative results from our research, please see the Analysis Table in Appendix A below.

Too much content ... can be overwhelming to the museum’s online visitors.

Characteristic 1: Museum Web sites have large amounts of rich content

Most museums do not lack for content, and museum Webmasters often try to share as much of this content online as possible. Museum professionals may spend a tremendous amount of time developing this content, often offering multiple interpretations of different concepts. Their efforts frequently result in thousands of database records and numerous Web pages with many interactive features. Too much content, however, can be overwhelming to the museum’s online visitors. Users of museum Web sites may be confused by the sheer number of choices presented to them, make wrong decisions, become frustrated, and find themselves unable to use the museum’s Web site to complete simple tasks. Designers of museum Web sites should consider the following dimensions of usability problems manifested by this characteristic of museum Web sites.

Dimension 1.1: Too much content can frustrate users, making them less willing to spend time with the Web site.

Museum Web sites that offer a great deal of content are generally not confusing to the people who design them. The content is familiar to them, they understand its organizational conventions and they rarely have to cope with all of it at once. Users, however, are often overwhelmed with large amounts of content from the moment they open the museum’s Web site in their browser. This can frustrate users, lead them to believe that they will never be able to make sense out of the site, and make them more likely to abandon tasks they hoped to accomplish with the museum online.

Example: Visiting a museum Web site for the first time, a user looking for some particular piece of information is surprised to see dozens of options linked from the museum’s main page. Noticing a search option in the upper right hand corner of the window, the user clicks on the button and enters his search terms in the text box. To his dismay, the museum Web site returns hundreds of hits, with no obvious method of sorting or narrowing the results — all of which look extremely similar. The user, already convinced that this museum Web site is too complicated, abandons the Web site for a simpler online search engine.

Dimension 1.2: Too many choices may lead users to make selections without considering all options.

Museums Web sites often require users to consider many choices in order to find the information they seek. When faced with what they perceive to be too many options, however, users rarely read their choices carefully before making decisions. Instead, they skim the available choices, and select what appears to them to be the first most plausible choice. Even if a more likely choice appears later on in the list of possible choices, few users will have the patience required to find it.

Example: A parent hoping to have her child’s birthday party at a local children’s museum visits the museum’s Web site to find out how to schedule the party. When the main page loads, she quickly skims her options, immediately clicks on a link labeled "Programs & Events," but can find no information about birthday parties. Unnoticed on the main menu was a link labeled "Just for Kids," which would have led her to a page with information all about birthday parties.

Dimension 1.3: Too much content and too many choices may lead users to focus on only one area at the expense of others.

Visitors to online museums often attempt to cope with large amounts of content and choices by quickly choosing one area of the museum’s Web site to focus on while ignoring the rest of the Web site. Once a choice as been made, many users demonstrate tunnel vision, stubbornly sticking with their selection and refusing to reconsider their original choice. When users have chosen inappropriately, many museum Web sites do not provide clear feedback to users that they will not find what they seek in this section of the Web site. Designers of museum Web sites should realize that users will typically not abandon a line of inquiry, especially if that approach has worked in the past, without being explicitly directed to do so, even in the face of mounting evidence that their original choice was the wrong one.

Example: A school teacher searching for online activities that would appeal to fourth graders visits the Web site of a natural history museum. Selecting the "Education" link from the main menu, she systematically steps through every single page (no small feat) in this area of the Web site, but she finds not a single word about online activities. Eventually she gives up and tries another museum’s Web site, unaware that if she had broadened her search to other areas of the museum’s Web site she would have discovered a section entitled "Online Resources" containing several examples of online activities for grade school students.

Dimension 1.4: Too many perspectives on the same content can confuse users looking for information on a specific topic.

Museum Web sites often offer multiple interpretations of museum content, particularly online exhibits where users are frequently encouraged to explore resources from different perspectives. Users, however, tend to be confused by Web sites offering different paths to find similar answers. Faced with multiple perspectives, orientations, connections, and interpretations, our test users had a difficult time determining what to do; they often wished for the Web site to provide a straightforward method of learning about a particular topic, instead of encouraging them to select one from multiple methods of accessing or interpreting the available data.

Example: A museum’s Web site allows visitors to explore its collections using a detailed historical timeline illustrated by selected artifacts. Users can browse this timeline by specific cultures, by geographic locations, or by historical time periods. One visitor to this timeline, hoping to find one particular known artifact in the museum’s collection, is uncertain how to begin: should he start with culture, geography, or history? Making a random selection, he becomes further confused by this approach to browsing the museum’s collection, and distressed that a simple search engine is not available.

Beauty, however, does not always go hand in hand with usability ... Artistically designed interfaces may bring only confusion to the user who simply wants to know "what do I do now?"

Characteristic 2: Museum Web sites have artistically designed graphical user interfaces

Museums are beautiful places, and most designers of museum Web sites strive to make online museums beautiful as well. Beauty, however, does not always go hand in hand with usability. Museum Web sites that are beautiful and artistic are frequently also confusing, distracting, and hard to navigate. Designers who spend a great deal of time creating a beautiful interface may be surprised to find that users will typically spend less time admiring the Web site as a work of art and more time trying, often unsuccessfully, to use the Web site to accomplish some task. Artistically designed interfaces may bring only confusion to the user who simply wants to know "what do I do now?" Museum Webmasters should consider the following dimensions of usability problems manifested by this characteristic of museum Web sites.

Dimension 2.1 Artistically designed graphical layouts can be disorienting and distracting to users trying to accomplish tasks.

Museum Web sites often feature artistically designed graphical layouts that look good on the surface but can interfere with the user’s ability to complete tasks. "This is very pretty," our test users would often say, "but I don’t know what to do with it." During our tests, it often appeared that designers spent more time making their sites look beautiful and less time testing their sites for usability. These sites often distracted our test users as they tried to look beyond the design to find answers to their questions.

Example: One museum Web site offered users the chance to take an interactive, animated virtual visit to a far–away land. The visit, once successfully launched, was simply exquisite, featuring highly–detailed guided tours, conversations with interesting characters, and the chance to immerse oneself in a different culture. The problem, however, was that users had a hard time determining how to begin their visit: the beautifully designed introductory page for this feature lacked a clear "click here to begin" type link, and few users realized that the way to begin was to click on the formal invitation being offered by one of the onscreen animated characters.

Dimension 2.2 Artistically designed interface elements can be perplexing and meaningless to users trying to navigate the Web site.

There is a difference between a graphical element at which one simply looks and a graphical element with which one interacts. Many of the museum Web sites we tested featured navigation controls that were more beautiful than usable; this was particularly the case when these interface elements violated norms making them inconsistent with the user’s prior experiences online (such as clearly labeled "next page" and "previous page" buttons). Often the only way for users to figure out what an interface element would do was to click on it, a navigation experience that can be especially confusing to first time visitors of a Web site.

Example: One museum Web site featured a virtual tour of the museum’s physical galleries. The navigation controls for this virtual tour were contained within an artistically designed, schematic representation of the museum’s floor plan, found on every page on the museum’s Web site. This schematic, however, was unlabelled, and few visitors found the virtual tour. Unless users were previously familiar with the museum’s floor plan, there would be no way for them to connect that schematic with the virtual museum.

Characteristic 3: Museum Web sites have interfaces that encourage exploration

Museum Web sites, like many physical museums, are often designed to encourage a spirit of exploration. For some users, this approach can be very appealing, but for others, this can turn quickly into an exercise in frustration. Users often arrive at museum Web sites looking to find something in particular: information about a topic of interest to them, perhaps, or an online exhibit recommended by a friend. Museum Web sites that encourage exploration often end up discouraging visitors who come to the Web site with specific interests. These Web sites need to support both guided and unguided exploration; exploratory interfaces designed without this understanding risk manifesting the following dimensions of common usability flaws in museum Web sites.

Dimension 3.1: Exploratory interfaces often emphasize random browsing and discourage users interested in exploring specific topics.

Museum Web sites that emphasize random exploration make it easy for users to explore unfamiliar topics. With many museum Web sites, finding something fun, interesting, or educational is trivially easy; often you can simply close your eyes, randomly click the mouse, and be assured that the Web site will reward you with something worth seeing. It is usually much more difficult, however, to use these Web sites to find something specific. Museum Web sites with exploratory interfaces usually cater to users interested in undirected or unguided visits and implicitly discourage visitors interested in exploring more specific topics.

Example: The Web site for an aviation museum offered a tremendous amount of rich content online about everything from aviation history to the technical specifications of hundreds of aircraft. For users interested in aviation, this museum’s Web site was a treasure trove of information, as long as users did not care what information they gathered. Randomly browsing users were guaranteed to find something of interest simply by following any link or clicking on any graphic. Users seeking information about one particular aircraft or one particular event in aviation history, however, found themselves baffled at every turn, unable to understand their choices and unable to navigate the Web site to achieve their goal.

Dimension 3.2: Exploratory interfaces often require users to make choices without understanding the consequences of those choices.

Museum Web sites with exploratory interfaces often require users to explore the museum’s online offerings step–by–step, without first being presented with an overview of the entire Web site. Unable to see the big picture, these users are often asked to make choices without first understanding the consequences of those choices. Our test users in this situation frequently remarked, "I don’t know what to do here, but they obviously want me to do something," before randomly selecting an option in the hopes that that choice would eventually meet their needs. Excepting cases where such puzzles are intentional, these situations arise when designers forget that those choices which seem so clear to them will be completely mysterious to users unfamiliar with the museum’s Web site.

Example: The Web site for a history museum featured a detailed online exhibit about a particular historical event. The content for this online exhibit was so extensive that the exhibit was divided into four distinct, non–overlapping sections — each labeled with a historically appropriate, but informationally vague, title. The main introductory page for the exhibit required the online visitor to enter the exhibit by choosing one of these four section, yet offered no information other than the title of the section. In this way, users exploring this exhibit were required to make an important choice affecting their online experience before they had any idea what the exhibit was truly about.

Dimension 3.3: Exploratory interfaces often fail to include tools that allow users to quickly locate specific known items.

Users having difficulties browsing museum Web sites in search of known items usually turn to search engines, indices, or site maps as a way of solving their problem; the designers of exploratory interfaces, however, do not always think to provide such mechanisms. If users are lucky enough to find a search engine or a site map, they often discover that less effort went into the design of those features than other portions of the interface. In these cases, site maps usually present few links and little detail, while search engines may be hard to use and offer poor feedback or ambiguous search results. When the designers of museum Web sites are too focused on building exploratory interfaces, they often decide not to provide alternate methods of locating specific known items, such as search engines or site maps, even if those methods would be preferable to random browsing for many users.

Example: A user visits a particular museum Web site in search of an online exhibit that came highly recommended by a friend. This user has never before visited this Web site and knows only the name of the exhibit in which she is interested. Finding a page about the museum’s online exhibits, the user is surprised to find no list of available exhibits by title, but rather a rotating graphic where images of different exhibits fade in and out at a rate of one every few seconds. Unwilling to wait for the desired exhibit to appear (and uncertain of recognizing the exhibit if it did), the user decides to try a different approach. The user is quickly distressed, however, to discover this Web site offers neither a search engine nor a site map, and that the only way to find the desired exhibit is to page through the online exhibits one at a time.

Characteristic 4: Museum Web sites are designed by museum professionals

Museum professionals of all types, including curators and educators, are frequently involved in the design of Web site components or the development of content for online exhibits. While the involvement of museum personnel in the creation of museum Web sites can lend an aura of authenticity to the online museum, it can also impose unanticipated requirements on the users of museum Web sites. Visitors to online museums do not expect to have to learn special skills in order to use a museum Web site; they expect the content to be presented using words familiar to them and to be formatted in ways that they understand. Museum Web sites designed primarily by museum professionals, however, often require users to have "insider knowledge" about how museums work if they are to make full use of the online features provided. Users may need familiarity with organizational schemes used by museum professionals or special knowledge of specialized vocabulary used in museums. When museum professionals are involved in the design and development of museum Web sites, designers risk manifesting the following dimensions of common usability flaws in museum Web sites.

Museum Web sites should organize content in ways that make sense to the Web site’s users, and not require visitors to recognize organizational schemes particular to the museum environment.

Dimension 4.1: Museum professionals often design Web sites that use organizational schemes unfamiliar to non-museum professionals.

The content of museum Web sites is often organized into categories that make sense to museum professionals but not to users unfamiliar with museum practices. A museum Web site’s main menu, for instance, may represent a museum’s internal organizational structure (each menu item associated with a different unit in the museum), an information representation bound to confuse non–museum professionals. Museum Web sites should organize content in ways that make sense to the Web site’s users, and not require visitors to recognize organizational schemes particular to the museum environment.

Example: A visitor to a museum Web site is interested in learning what artifacts are the highlights of the museum’s collections. The Web site’s main menu is divided into several sections, including "collections," "exhibitions," "about the museum," "education," "programs & events," and "planning your visit." The visitor tries the link labeled "collections" but discovers only information designed for researchers and scholars, with no overview of the museum’s highlights. Returning to the main menu, the visitor starts guessing menu items, eventually finding an overview of the museum’s collections under the link labeled "exhibitions." Puzzled, he wonders, "how would I have known that was there?"

Dimension 4.2: Museum professionals often design Web sites that use controlled vocabularies unfamiliar to non–museum professionals.

Many aspects of museum Web sites require visitors to understand the specialized terminologies and controlled vocabularies used by museum professionals. This is often the case when visitors browse collections databases, where arcane classifications and metadata schemes seem designed for specialists and not the average museum user. Visitors who cannot understand the language used on museum Web sites will likely be unable to make effective use of the museum’s online resources.

Example: A museum Web site allows visitors to search multiple collections by entering search terms into one text field and using checkboxes to indicate which collection or collections should be included in their search; by default, all checkboxes are checked on. The collection titles are obscure, however, and no information is provided to explain the purpose of the checkboxes. A visitor to this site, trying to find particular artifacts, enters her search terms, then pauses over the checkboxes. Commenting "I don’t know what any of this means," she proceeds to uncheck each checkbox, then executes her search. When her search returns "no results found," she leaves the museum’s Web site, convinced that the museum has no artifacts that meet her search criteria and unaware that she inadvertently searched no collections.

Dimension 4.3: Museum professionals often design Web sites unable to cope with unanticipated errors that might be made by non–museum professionals.

The designers of museum Web sites are generally unable to predict the kinds of errors likely to be made by non–museum professionals. For this reason, many museum Web sites are unable to provide useful support or feedback to users making unanticipated mistakes. When visitors to museum Web sites make simple mistakes that no museum professional would have made, museum Web sites are unlikely to help them recover from their errors.

Example: A user trying to locate an online exhibit about a particular artist spots a link labeled "search" on the museum’s Web site. Clicking this link, he enters the artist’s name and is surprised when page after page of records are returned. Scrolling through the results, the user looks for a link to the online exhibit but is disappointed to see that every record links to an individual museum artifact. While an experienced museum professional would immediately recognize that this user had inadvertently searched the museum’s online collections database and not the museum’s Web site, this user eventually abandons his search, dismayed by the overwhelming amount of unhelpful information provided by the museum’s site.

Characteristic 5: Museum Web sites are intended to supplement physical museums

With the relatively rare exception of completely virtual museums (online museums with no physical counterpart), most museum Web sites are designed to supplement physical installations. Teachers can use museum Web sites to prepare lesson plans in advance of museum visits, students can download pre– and post–visit activities, and the general public can double–check the museum’s location, hours, and admission fees. Museum Web sites provide access to a wealth of information about artifacts not on display and offer virtual exhibits as online counterparts to installations located in the physical museum. The designers of museum Web sites, however, should be aware that users unfamiliar with the physical museum may experience usability problems with these Web sites. Museum professionals who offer online features to supplement their physical installations risk manifesting the following dimensions of common usability flaws in museum Web sites.

Dimension 5.1: Users often have trouble understanding the museum Web site outside of the context of the physical museum.

Museum Web sites are often designed with the implicit understanding that their users are either familiar with, or sometimes physically located in, the physical museum. This is especially true of museum Web sites that were first created as information kiosks located in the museum’s galleries and only later moved online. Facts that are simply "obvious" to individuals familiar with the physical museum tend not to be made explicit on the museum’s Web site, a mistake that can quickly turn an obvious fact into an obscure one. Even tasks as simple as finding the museum’s location on a map can be made into impossible challenges by Web site designers who forget to provide the museum’s street address online.

Example: An individual planning to return to a museum after several years visits the museum’s Web site to determine whether his favorite temporary exhibit is still showing in the museum’s galleries. He quickly finds the online supplement to this exhibit on the museum’s Web site; although the exhibit is listed as "temporary," the Web site provides no information about whether the exhibit is still physically located at the museum. The user will have to call the museum (assuming he can find the museum’s phone number) to find out whether the exhibit is still there.

Dimension 5.2: Users are often confused by museum Web sites that lack the cues or support mechanisms interwoven by default into physical museums.

When visitors walk into physical museums, they know (at least subconsciously) that the cues and support mechanisms of everyday life will form an integral part of their visit. Gravity tells them which way is up, the entrances to exhibits look (for the most part) like doors, gallery exits are clearly labeled, and — if one is lost — other visitors can be observed and followed; visitors in physical museums rarely try to access the next gallery by crawling through heating vents. The users of museum Web sites, however, often find themselves performing the online equivalent, clicking randomly on the screen in the desperate hope of finding a way out of their current location.

Example: To represent a popular traveling exhibit now showing at their museum, designers decide to use a simple graphical image with no textual label or identifying features. Inside the museum, this logo works well: visitors quickly learn from other visitors that by simply following this logo, they will eventually find the popular exhibit. Visitors to the museum’s Web site, however, lack the physical cues necessary to make this connection; few users associate the obscure logo on the museum’s main page with the new exhibit, and the only users who find the popular exhibit online are those who by chance think to click on the image.

Dimension 5.3: Users are frequently constrained by museum Web sites that seek to emulate the characteristics of physical museums.

The designers of museum Web sites frequently attempt to convert face–to–face exhibits into online exhibits that emulate characteristics of the physical installation. These online exhibits may use metaphors such as animated turning pages to move from Web page to Web page, or require visitors to proceed in a linear fashion from display to display as if they were walking down a hallway in the physical museum. While some may appreciate such touches of realism, online exhibits that attempt to duplicate characteristics of the physical world tend to place experienced users accustomed to the capabilities of the online environment at a disadvantage.

Example: Returning to an online exhibit for a second time, the user of a museum Web site wishes to move quickly to her favorite portion of the exhibit. The online exhibit, however, was designed as a simulation of a physical exhibit; in an attempt to duplicate the experience of walking through a physical museum, the interface requires visitors to page through each section of the exhibit one at a time. With no table of contents, no site map, and no hyperlinks provided, the repeat visitor quickly grows tired of this metaphor and gives up before reaching her goal.

 

++++++++++

Conclusion: Improving the usability of museum Web sites

Our analysis of the usability flaws of museum Web sites covered a wide range of different types of museums: museums of art, science, history, and culture; museums with large, heterogeneous collections and small, focused collections; and museums whose primary audiences are students, scholars, children, and adults. While some of the characteristics and dimensions we identified in this article have strong parallels with issues that concern the usability of all Web sites (cf. Nielsen, 1999), we believe it is important to consider the particular usability and interface design challenges that concern museum Web sites in particular, and hope that our findings will prove of use in informing their future design.

The designers of museum Web sites often invest huge amounts of time and money in developing extensive Web sites with fabulous content. Not analyzing these Web sites for usability can mean users of the site will fail to discover and appreciate this rich content — a frustrating experience for both museum professionals and museum visitors. The fifteen dimensions presented in this article should help Web site designers and usability evaluators focus their efforts on avoiding the most common usability problems encountered by users of museum Web sites.

The designers of museum Web sites often invest huge amounts of time and money in developing extensive Web sites with fabulous content. Not analyzing for usability can mean users will fail to discover and appreciate this rich content ...

We do not mean to imply that the way to avoid these problems is to create museum Web sites without the characteristics listed above. It is not an inherently bad thing for museum Web sites to have rich content, for example, but to maximize the desirable features of this characteristic, designers must acknowledge possible usability trade–offs and ameliorate potential difficulties by providing appropriate navigational cues, a clear information architecture, and so on. Similarly, there is much to be said for providing multiple perspectives, exploratory interfaces, and intriguing, or even puzzling, aspects to museum Web sites. Nevertheless, designers should be aware of the possible problems that users can have with these approaches, and design for more prosaic information needs and usage patterns as well as for more expert or experiential usage.

Predicting and testing the usage patterns of the many possible visitors to a museum Web site can sound daunting, but even small amounts of user testing can be surprisingly informative; it is our hope that considering the features of a Web site in the light of the dimensions presented in this paper will prove useful. Although each dimension was discussed independently in this article, they often co–occurred in relation to the usability flaws that underpin this analysis; even worse, they frequently interacted, adding to users’ confusions when trying to make sense out of museum Web sites. Therefore, designers and evaluators should consider this conceptual framework holistically when designing and evaluating museum Web sites. As more museum and information professionals recognize the need to conduct usability analyses of museum Web sites, we hope that the conceptual framework presented in this article will help guide efforts to improve the usability of museum Web sites. End of article

 

About the authors

Paul F. Marty is Assistant Professor in the School of Information Studies at Florida State University.
E–mail: marty@fsu.edu

Michael B. Twidale is Associate Professor in the Graduate School of Library and Information Science (GSLIS) at the University of Illinois at Urbana-Champaign.
E–mail: twidale@uiuc.edu

 

References

Ben Booth, 1999. "Understanding the information needs of visitors to museums," Museum Management and Curatorship, volume 17, number 2, pp. 139–157.

Jonathan P. Bowen, 1999. "Time for renovations: A survey of museum Web sites," In: D. Bearman and J. Trant (editors). Museums and the Web 1999. Pittsburgh, Pa.: Archives and Museum Informatics, pp. 163–174, and at http://www.archimuse.com/mw99/papers/bowen/bowen.html, accessed 31 July 2004.

Jonathan P. Bowen, Jim Bennet, and James Johnson, 1998. "Virtual visits to virtual museums," In: D. Bearman and J. Trant (editors). Museums and the Web 1998. Pittsburgh, Pa.: Archives and Museum Informatics, n.p., and at http://www.archimuse.com/mw98/papers/bowen/bowen_paper.html, accessed 31 July 2004.

John Chadwick and Patricia Boverie, 1999. "A survey of characteristics and patterns of behavior in visitors to a museum Web site," In: D. Bearman and J. Trant (editors). Museums and the Web 1999. Pittsburgh, Pa.: Archives and Museum Informatics, pp. 154–162, and at http://www.archimuse.com/mw99/papers/chadwick/chadwick.html, accessed 31 July 2004.

Yvonne Cleary, 2000. "An examination of the impact of subjective cultural issues on the usability of a localized Web site: The Louvre Museum Web site," In: D. Bearman and J. Trant (editors). Museums and the Web 2000. Pittsburgh, Pa.: Archives and Museum Informatics, pp. 153–162, and at http://www.archimuse.com/mw2000/papers/cleary/cleary.html, accessed 31 July 2004.

Daniel Cunliffe, Efmorphia Kritou, and Douglas Tudhope, 2001. "Usability evaluation for museum Web sites," Museum Management and Curatorship, volume 19, number 3, pp. 229–252.

Mary C. Dyson and Kevin Moran, 2000. "Informing the design of Web interfaces to museum collections," Museum Management and Curatorship, volume 18, number 4, pp. 391–406.

Franca Garzotto, Maristella Matera, and Paolo Paolini, 1998. "To use or not to use? Evaluating the usability of Mmseum Web sites," In: D. Bearman and J. Trant (editors). Museums and the Web 1998. Pittsburgh, Pa.: Archives and Museum Informatics, n.p., and at http://www.archimuse.com/mw98/papers/garzotto/garzotto_paper.html, accessed 31 July 2004.

Ilse Harms and Werner Schweibenz, 2001. "Evaluating the usability of a museum Web site," In: D. Bearman and J. Trant (editors). Museums and the Web 2001. Pittsburgh, Pa.: Archives and Museum Informatics, pp. 43–54, and at http://www.archimuse.com/mw2001/papers/schweibenz/schweibenz.html, accessed 31 July 2004.

Morten Hertzum, 1998. "A review of museum Web sites: In search of user–centered design,Ó Archives and Museum Informatics, volume 12, number 1, pp. 127–138.

Katherine Jones-Garmil (editor), 1997. The wired museum: Emerging technology and changing paradigms. Washington, D.C.: American Association of Museums.

Victoria Kravchyna and Sam Hastings, 2002. "Informational value of museum Web sites," First Monday, volume 7, number 2 (February), at http://firstmonday.org/issues/issue7_2/kravchyna/, accessed 31 July 2004.

Steve Krug, 2000. Don’t make me think! A common sense approach to Web usability. Indianapolis, Ind.: New Riders.

Paul F. Marty, 2004a. "The changing role of the museum Webmaster: Past, present, and future," In: D. Bearman and J. Trant (editors). Museums and the Web 2004. Pittsburgh, Pa.: Archives and Museum Informatics, n.p., and at http://www.archimuse.com/mw2004/papers/marty/marty.html, accessed 31 July 2004.

Paul F. Marty, 2004b. "The evolving roles of information professionals in museums," Bulletin of the American Society for Information Science and Technology, volume 30, number 5, pp. 20–23, and at http://www.asis.org/Bulletin/Jun-04/marty.html, accessed 31 July 2004.

Paul F. Marty and Michael B. Twidale, manuscript in preparation. "Usability@90mph: Maximizing the benefits of user testing demonstrations."

Jakob Nielsen, 1999. Designing Web usability: The practice of simplicity. Indianapolis, Ind.: New Riders.

Jakob Nielsen, 1993. Usability engineering. Boston: Academic Press.

Mary Beth Rosson and John M. Carroll, 2001. Usability engineering: Scenario–based development of human-computer interaction. San Francisco: Morgan Kaufmann.

Jim Ockuly, 2003. "What clicks? An interim report on audience research," In: D. Bearman and J. Trant (editors). Museums and the Web 2003. Pittsburgh, Pa.: Archives and Museum Informatics, n.p., and at http://www.archimuse.com/mw2003/papers/ockuly/ockuly.html, accessed 31 July 2004.

Darren Peacock, 2002. "Statistics, structures & satisfied customers: Using Web log data to improve site performance," In: D. Bearman and J. Trant (editors). Museums and the Web 2002. Pittsburgh, Pa.: Archives and Museum Informatics, pp. 157–166, and at http://www.archimuse.com/mw2002/papers/peacock/peacock.html, accessed 31 July 2004.

Rob Semper, Noel Wanner, Roland Jackson, and Martin Bazley, 2000. "Who’s out there? A pilot user study of educational Web resources by the Science Learning Network (SLN)," In: D. Bearman and J. Trant (editors). Museums and the Web 2000. Pittsburgh, Pa.: Archives and Museum Informatics, pp. 179–200, and at http://www.archimuse.com/mw2000/papers/semper/semper.html, accessed 31 July 2004.

Katie Streten, 2000. "Honored guests: Towards a visitor centered Web experience," In: D. Bearman and J. Trant (editors). Museums and the Web 2000. Pittsburgh, Pa.: Archives and Museum Informatics, n.p., and at http://www.archimuse.com/mw2000/papers/streten/streten.html, accessed 31 July 2004.

Bruce Thomas, 1996. "Quick and dirty usability tests," In: P. Jordan, B. Thomas, B. Weerdmeester, and I. McClelland (editors). Usability evaluation in industry. London: Taylor & Francis., pp. 107–114.

John Vergo, Marie–Claire Karat, John Karat, Claudio Pinhanez, Renee Arora, Thomas Cofino, Doug Riecken, and Mark Podlaseck, 2001. "Less clicking, more watching: Results from user–centered design of a multi–institutional Web Site for art and culture," In: D. Bearman and J. Trant (editors). Museums and the Web 2001. Pittsburgh, Pa.: Archives and Museum Informatics, pp. 23–32, and at http://www.archimuse.com/mw2001/papers/vergo/vergo.html, accessed 31 July 2004.

 

Appendix A: Analysis Table

#
Dimension
X of 36 Web sites
X of 119 scenarios
X of ∼500 Usability Flaws
Characteristic 1: Museum Web sites have large amounts of rich content
1.1
Too much content can frustrate users, making them less willing to spend time with the Web site.
18
34
∼135
1.2
Too many choices may lead users to make selections without considering all options.
21
35
∼140
1.3
Too much content and too many choices may lead users to focus on only one area at the expense of others.
15
27
∼110
1.4
Too many perspectives on the same content can confuse users looking for information on a specific topic.
14
26
∼105
Characteristic 2: Museum Web sites have artistically designed graphical user interfaces
2.1
Artistically designed graphical layouts can be disorienting and distracting to users trying to accomplish tasks.
18
35
∼140
2.2
Artistically designed interface elements can be perplexing and meaningless to users trying to navigate the Web site.
26
49
∼200
Characteristic 3: Museum Web sites have interfaces that encourage exploration
3.1
Exploratory interfaces often emphasize random browsing and discourage users interested in exploring specific topics.
22
38
∼150
3.2
Exploratory interfaces often require users to make choices without understanding the consequences of those choices.
22
42
∼170
3.3
Exploratory interfaces often fail to include tools that allow users to quickly locate specific known items.
24
45
∼180
Characteristic 4: Museum Web sites are designed by museum professionals
4.1
Museum professionals often organize Web sites using organizational schemes unfamiliar to non–museum professionals.
16
26
∼105
4.2
Museum professionals often use classification schemes or controlled vocabularies unfamiliar to non–museum professionals.
23
37
∼150
4.3
Museum professionals often design Web sites unable to cope with unanticipated errors that might be made by non–museum professionals
17
28
∼110
Characteristic 5: Museum Web sites are intended to supplement physical museums
5.1
Users often have trouble understanding the museum Web site outside of the context of the physical museum
18
36
∼145
5.2
Users are often confused by museum Web sites that lack the cues or support mechanisms interwoven by default into physical museums
15
24
∼100
5.3
Users are frequently constrained by museum Web sites that seek to emulate the characteristics of physical museums
15
28
∼110

 

Editorial history

Paper received 2 August 2004; accepted 24 August 2004.

Contents Index

Copyright ©2004, First Monday

Copyright ©2004, Paul F. Marty and Michael B. Twidale

Lost in gallery space: A conceptual framework for analyzing the usability flaws of museum Web sites by Paul F. Marty and Michael B. Twidale
First Monday, volume 9, number 9 (September 2004),
URL: http://firstmonday.org/issues/issue9_9/marty/index.html