Assessing the accessibility of fifty United States government Web pages: Using Bobby to check on Uncle Sam by Jim EllisonThis study evaluates the current accessibility of U.S. Government Web pages for people with disabilities. Several Federal laws, and specifically Section 508 of the U.S. Rehabilitation Act, require Web pages of government agencies to be accessible to people with disabilities. This investigation built on past studies that used the Web accessibility evaluation tool Bobby to assess various types of Web sites. The home pages of fifty U.S. government agencies were reviewed for accessibility based on Section 508 guidelines. This study establishes that the U.S. government has not met its accessibility goals.
Contents
Introduction
Accessibility for people with disabilities
Legislation and government responsibility for accessibility
Literature review
Methodology
Limitations
Procedures
Data summary
Discussion
Conclusion
Introduction
At the beginning of the new century our society is fully invested in the electronic information age. The World Wide Web has become a ubiquitous source of information for the majority of Americans ... or has it? There has been discussion of a "Digital Divide" shutting out some segments of our population. For one of these groups, people with disabilities, digitized data has opened doors to a larger volume of information through the use of adaptive and assistive technology. Individuals with blindness can use screen readers to access electronic versions of print sources instantly instead of having to wait for a talking book, Braille conversion of text or similar alternate format "translation." A person with a mobility or communication impairment can conduct many day–to–day activities at home with a computer instead of seeking assistance from a human intermediary. The exponential growth in the number of sites on the World Wide Web combined with the growing volume and diversity of Web–formatted information services, would seem to create a new world of opportunity for disabled individuals.
The World Wide Web and other Internet information delivery technologies offer great potential for enhancing access to information. Unfortunately, due to poor page design, much of the Web is not accessible to people with disabilities. The U.S. government, via several initiatives and legal mandates has attempted to mend this pothole on the information highway. Are people with disabilities falling through the Net when they attempt to access Federal agency Web sites? Full accessibility of Federal Web pages has been the mandate since 2001 — has it been achieved?
Accessibility for people with disabilities
What makes a Web site accessible? Chuck Letourneau, at Starling Access Services, provides an excellent definition: "Anyone using any kind of Web browsing technology must be able to visit any site and get a full and complete understanding of the information contained there, as well as have the full and complete ability to interact with the site" (Letourneau, 2003).
How do poorly constructed Web sites affect people with disabilities? The barriers to access are as varied as the variety of Web sites in cyberspace and the types of disabilities a person may have. Several authors (Mates, 1999; Paciello, 2000; Lilly and Van Fleet, 1999) have detailed how a variety of disabilities can cause individuals to stumble on the path in cyberspace as well as how changes to Web design combined with adaptive or assistive devices overcome barriers to information access. Some of the best examples of how people with specific disabilities are limited on the Web and how solutions can be implemented are found in Barbara Mates’ online text Adaptive Technology for the Internet: Making Electronic Resources Accessible to All and the World Wide Web Consortium's document "How People with Disabilities Use the Web".
The Web is increasingly becoming a more visual medium so the general public often only thinks of blind individuals as having difficulty accessing the Web. However, poor Web page authoring practices are not only limiting the blind. "Curbcuts" on the information superhighway are needed for other disabilities as well (Mates, 1999). Some visually impaired individuals have restricted or low vision so they may require content with magnified font size or they may have a restricted field of vision that limits the amount of information to be viewed at one time. A visual impairment frequently forgotten in Web design is color blindness — an "invisible" impairment that can cause a person to miss critical information and prompts within a Web site (Mates, 1999).
Mobility impairments (cerebral palsy and arthritis, for example) affect the ability to use extremities (arms, hands) for functional tasks. Activities that may be limited by mobility impairments include: typing on a standard keyboard, using a mouse, or clicking repeatedly though a Web site.
Auditory impairments affect the ability to hear and process sounds. Web pages relying on sound bites or similar constructions to convey important information or instructions would be problematic for this group of individuals. These examples are just a sample of disabilities and factors to consider when evaluating the accessibility of a Web page for people with disabilities.
The various types of disabilities combine to comprise a large group of affected individuals. Estimates on the number of people with disabilities in the United States vary, but most are derived from U.S. Census.
"In December 1997, based on a census taken during the four–month period of October 1994 to January 1995, the U.S. Department of Commerce’s Bureau of Census reported that 1 in 5 Americans or (54 million people), have some kind of disability. This is about 20 percent of all U.S. citizens, which comprises a larger minority population than African Americans (approximately 30 million).Additionally, the same report highlighted the following breakdown of those same statistics:
- 1 in 10 Americans has a severe disability.
- Among children aged 6–14, 1 in 8 have some type of disability.
- 1 in 2 Americans 65 years and older has a disability.
- 1 in 5 Americans between the ages of 15 and 64 has a disability." [1]
These figures indicate that there are many Americans with disabilities and therefore there is a substantial need for accessible information.
Legislation and government responsibility for accessibility
The government of the United States of America has traditionally provided services, established standards for access to public and private services, and enforced laws affecting people with disabilities. The Rehabilitation Act of 1973 mandated accommodations (wheelchair ramps, etc.) to make government services more accessible to the disabled. Public Law 101&150;336, The Americans with Disabilities Act (ADA) of 1990, "prohibits discrimination and ensures equal opportunity for persons with disabilities in employment, state and local government services, public accommodations, commercial facilities, and transportation" (U.S. Department of Justice, 2003). This Act also mandated the establishment of TDD (Telecommunications Device for the Deaf)/telephone relay services to provide public service for disabled citizens.
Additional laws have dealt directly with the delivery of electronic services to the Public.
"The Telecommunications Act of 1996 established important standards related to the transmission of information involving telecommunications interfaces and their operating environments. It is very likely this will include the Internet and the World Wide Web." [ 2]The Assistive Technology Act of 1998 provides states with funding to promote assistive technology and to develop programs to provide disabled citizens with these technologies. Some, not all, have interpreted this law may be a mandate for state and local governments to ensure Web accessibility (Noble, 2002; Waddell and Urban, 2000).
In 1998, the standard that most clearly applies to Federal Web sites was updated and amended to its present form.
"Section 508 of the Rehabilitation Act requires access to the Federal government’s electronic and information technology. The law applies to all Federal agencies when they develop, procure, maintain, or use electronic and information technology. Federal agencies must ensure that this technology is accessible to employees and the public (Office of Information Technology, 2003)."The Section 508 guidelines were developed by the Access Board, an independent Federal agency that promotes accessibility to all activities, and are available from the Board’s Web site http://www.access-board.gov/sec508/guide/1194.22.htm. Details about the application and practice of this law are available at the U.S. Department of Justice Section 508 home page at http://www.usdoj.gov/crt/508. After a period of interpretation and review, these standards were finally implemented in June 2001. Section 508 requires the home page of a Federal agency Web site, as well as the 20 most visited pages within the site, to meet specific accessibility guidelines. The U.S. Department of Justice is to report on the status of Federal Web site accessibility every two years. In September 2003, the status of the first report, due in August of 2003 could not be determined. It is clear that, by law, Federal agencies must ensure their Web sites are accessible.
In addition to a legal responsibility to ensure accessibility of government resources, there are other reasons for the United States government to make a commitment to accessible electronic information. "The U.S. government is the single largest producer of content for the Web. Several trends are accelerating the pace of government Web publishing including cost cutting efforts and mandates to reduce paperwork" [ 3]. In 2003, it was reported:
"Taken as one brand, U.S. government Web sites, which attracted 44.8 million unique visitors, ranked fourth in overall traffic in February behind traditional leaders AOL Time Warner (89.8 million), MSN (89.1 million) and Yahoo! (80.7 million). Collectively, government site traffic finished ahead of Google (40.3 million) and eBay (36.6 million)" (Mark, 2003).The Government Printing Office ( GPO) is the largest publisher in the world. Various e–government initiatives are pushing more electronic versions of printed government information as well as new resources onto the Web. "Under the Government Paperwork Elimination Act, agencies must enable the public to interact with them electronically, whenever possible, by October 2003" [ 4]. This includes "paperwork" such as forms from the Internal Revenue Service and Social Security Administration that most Americans need access to in order to conduct daily activities, whether disabled or not. Access to government information is a necessity for all Americans.
Literature review
The U.S. Government recognized that the growth of information technology presented many opportunities and well as barriers to people with disabilities. In 2000, the U.S. Department of Justice issued a report entitled Information Technology and People with Disabilities: The Current State of Federal Accessibility. Federal agencies were given a self–assessment report to evaluate their use of electronic information technology (EIT). The checklist for Web pages included in this self–assessment was based on World Access Initiative (WAI) guidelines, a recognized authoritative standard for accessibility created by the W3C. A review of the information submitted to the Department of Justice showed that overall, the majority of government Web pages were fully, or partially accessible, prior to implementation of Section 508. Specific recommendations to make Federal agency EIT more accessible to people with disabilities were presented. The report also pointed out that
"removal of barriers on Federal agencies’ Web sites is simply a matter of good design. It also benefits others, such as those who use low–end technology with lower modem speeds and people who use wireless Internet connections" (U.S. Department of Justice, 2003).Researchers have evaluated the accessibility of many types of Web sites. An excellent source for Web page accessibility studies and a listing of resources is available at the Web Accessibility Survey site maintained by Axel Schmetzke at http://library.uwsp.edu/aschmetz/Accessible/websurveys.htm. Many studies have used a Web accessibility evaluation tool called Bobby ( http://bobby.watchfire.com/bobby/html/en/index.jsp) to test the accessibility of Web pages.
In 1999, Erica B. Lilly and Connie Van Fleet looked at the main library Web pages of the 100 universities listed in Yahoo’s 1998 list of the "100 Most Wired Colleges." Their study, "Wired But Not Connected: Accessibility of Academic Library Home Pages" found only 40 out of the 100 sites were accessible according to Bobby (Lilly and Van Fleet, 1999).
Guthrie, in her study, "Making the World Wide Web Accessible to All Students," looked at the academic home pages of 80 colleges of communication and schools of journalism in the United States and Canada. Using Bobby to evaluate the sites, she found that 63 did not meet the criteria for accessibility (Guthrie, 2000).
Axel Schmetzke has conducted several studies on the accessibility of Web pages. In one of his earlier studies, "Web Accessibility at University Libraries and Library Schools," he evaluated the home pages of the 24 highest ranked schools of library and information science in the United States, based on rankings listed in U.S. News and World Report, as well as the main library page for universities associated with these ranked schools. Using Bobby, Schmetzke determined only 23 percent of the library schools and 59 percent of the main library pages were accessible (Schmetzke, 2001).
Building on these studies, Emily Jackson–Sanborn, Kerri Odess–Harnish and Nikki Warren conducted an assessment of several kinds of Web sites in 2001. In their study, "Web Site Accessibility, A Study of Six Genres," Bobby was used to check six types of sites for accessibility: most popular, clothing, international, jobs, college and government. All sites were selected from the Web site 100hot, with the exception of Government sites, which were not tracked for popularity. The Government sites were selected at random from the "Federal Agencies Directory" created by the Louisiana State University Libraries. This study found that government sites had the highest percentage of accessible sites with 60 percent being accessible, while the most popular sites had the lowest rating with 15 percent accessibility (Jackson–Sanborn, et al., 2002).
In 2002, Genie Stowers, conducted an assessment of the usability and effectiveness of 148 Federal government Web sites. Her assessment of these sites occurred in early 2002 and used Bobby for analysis. Stowers reported 13.5 percent of the sites had zero errors when reviewed by Bobby (Stowers, 2002).
Marty Bray, Claudia Flowers, and Patricia Gibson looked at 120 school district Web sites in 2003 in their study entitled "Accessibility of School Districts’ Web Sites: A Descriptive Study." Their assessment found 74.3 percent of the pages had accessibility problems (Bray, et al., 2003).
All of these cited studies indicated that full accessibility to the Web had not been achieved in any group of sites regardless of type. In particular, academic sites reviewed had low levels of accessibility, in spite of the fact that they were created by organizations most likely to have access to current knowledge on the subject as well as an understanding of the need for accessible Web sites for students, faculty, and staff.
Methodology
Estimates vary on the number of U.S. government Web pages, but the number cited in print and Web sources generally ranges between 20,000 and 30,000. The United States government is the largest single content provider of information on the Web. The volume and diversity of government–produced information is staggering and can be expected to continue to increase.
"The Smithsonian Institution has placed so much information on its Web site that it would require approximately 35 hours to read it all." [ 5]Instead of trying to construct a random sample from an ever–growing number of sites, 50 home pages were purposively selected. These sites were chosen because they represent well–known agencies of the Federal government. These include two Web portals, the White House, as well as 15 executive branch departments and 32 agencies selected from the alphabet soup of acronyms of commonly known Federal agencies. If these high–profile sites have not achieved Bobby approval, then one may consider it unlikely that the government has met its own mandate for accessibility in the thousands of other sites it creates for public consumption.
Bobby, the accessibility evaluation tool used in the studies discussed previously, was selected for use in this study. The studies mentioned earlier all used older versions of Bobby. The data for the last of the cited studies were collected in 2002; since then Bobby has been updated to version 5.0 and is administered by a different organization. Version 5.0 allows the user to evaluate a site specifically using Section 508 guidelines or WCAG (Web Content Accessibility Guidelines) version 1.0. Older versions of Bobby did not offer Section 508 guidelines as an evaluation option.
Bobby uses an automated program to review the HTML code of a Web page to look for common accessibility errors, based on 16 checkpoints derived from Section 508 or WCAG guidelines depending on which option the user selects. If the automated check does not find errors based on these 16 areas, a Bobby Approved icon is displayed. If the automated program does find an error, a not approved icon is displayed and the report lists the lines of HTML code with associated errors and links to descriptions of the errors. Since the program is automated it is limited in its scope and cannot guarantee that Bobby located all errors. Bobby also lists items in the page that would require a manual check to determine if the HTML code in question may cause an accessibility problem. Sometimes these items do not represent an accessibility issue, but only an individual looking specifically at the HTML in question can determine this [ 6].
Limitations
As with most studies, especially ones that evaluate a medium as changing as the Web, there are limitations. A Web page that is accessible today could become inaccessible tomorrow due to a minor change in page construction and vice versa. Accessibility guidelines can be, and are, updated or "tweaked" periodically. Interpretations of potential problems in HTML code by one accessibility evaluation tool could be a non–issue using another tool. This study uses a new version of Bobby, which is slightly different from versions used in previous studies. There are currently several Web site accessibility evaluation tools available for free or for a fee. The functionality of these products is comparable to Bobby in many ways. Each of these tools can differ in how guidelines are interpreted and evaluated. However, Bobby was chosen because it is a widely known and proven product. Additionally, it does allow the user to specifically apply Section 508 guidelines, which is the relevant standard for Federal government Web sites. Bobby has also been the evaluation tool of choice for many past accessibility studies so the use of this product extends consistency among studies.
TIn addition to these general limitations, Bobby has specific limitations, noted both by its creators and by researchers that have used this accessibility evaluation tool in the past. Bobby’s limitations include not being able to look at certain types of formatting in a Web page such as JavaScript and cascading style sheets. Bobby 5.0 is a relatively new product that had not been critiqued as thoroughly as previous versions.
A newer accessibility evaluation tool, Cynthia Says, functions in a manner similar to Bobby. It can be found at the Cynthia Says Web site at http://www.contentquality.com. Both Cynthia and Bobby allow a user to assess accessibility using either WCAG 1.0 or Section 508 standards. Although using similar technology and the same accessibility guidelines, these two products differ in how they report and interpret some 508 accessibility issues. An item–by–item chart comparing the similarities and differences between these two tools is available at the Cynthia Says Web site at http://www.contentquality.com/Standards/CynthiaVersusBobby.htm.
A study conducted in 2002, "Two Falls out of Three in the Automated Accessibility Assessment of the World Wide Web Sites: A–Prompt v. Bobby," by Dan Diaper and Linzy Worman used Bobby 3.2 and A–Prompt, another accessibility evaluation tool, to look at 32 university Web sites in the United Kingdom. This study revealed that A–Prompt did a better job detecting priority one and three errors than Bobby. There was no agreement between the two tools when looking at priority two errors (Diaper and Worman, 2002). This raises questions about the differences in accessibility tools and the reliability of these tools in deciphering accessibility issues.
Paul Jaeger reviews the effectiveness of measurements of accessibility that use analysis tools such as Bobby in "The Importance of Accurately Measuring the Accessibility of The Federal Electronic Government: Developing the Research Agenda." Bobby "is simply not capable of measuring the full range of accessibility and usability concerns that can affect individuals with disabilities" because software analysis tools are only able to check for basic design errors (Jaeger, 2004). The use of Bobby or a similar tool cannot promise 100 percent assurance of accessibility but if an error is found, it is reasonable to conclude that an accessibility problem exists.
Procedures
In December of 2003, the home pages of 50 Federal Web sites were evaluated with the online version of Bobby 5.0, using the Section 508 evaluation option. Interior pages were not accessed because there was no way to determine the 20 most visited pages within the sites. Site evaluation reports that had the "508 Approved" icon were listed as accessible. This icon will display if no errors are detected. Home pages that received the "508 Not Approved" icon after evaluation were considered inaccessible. The Bobby evaluation report for each Web page accessed was reviewed. These results are listed in Table 1. This method for determining accessibility follows procedures similar to those used by many studies cited in this report. Determining accessibility in this study was limited to the errors Bobby could detect but not those would need to be determined or excluded by a manual user check.
Table 1: Federal Web Sites assessed and results.
Web site and agency Bobby Cynthia www.fedworld.gov
National Technical Information Service (NTIS)Bobby approved Cynthia approved www.firstgov.gov
Federal Citizen Information Center
Office of Citizen Services and Communications
U.S. General Services AdministrationExplicitly associate form controls and their labels with the LABEL element.
(3 instances)Cynthia Approved www.gpoaccess.gov/fr/index.html
Federal RegisterExplicitly associate form controls and their labels with the LABEL element.
(1 instance)www.whitehouse.gov
White HouseProvide alternative text for all images.
(4 instances)
Explicitly associate form controls and their labels with the LABEL element.
(1 instance)www.usda.gov
Department of AgricultureExplicitly associate form controls and their labels with the LABEL element.
(2 instances)www.commerce.gov
Department of CommerceProvide alternative text for all images.
(10 instances)
Provide alternative text for all image map hot–spots (AREAs)
(1 instance)www.defenselink.mil
Department of DefenseProvide alternative text for all images.
(1 instance)
Explicitly associate form controls and their labels with the LABEL element.
(1 instance)www.ed.gov
Department of EducationBobby approved Cynthia approved www.energy.gov
Department of EnergyProvide alternative text for all images.
(4 instances)Cynthia approved www.dhhs.gov
Department of Health and Human ServicesExplicitly associate form controls and their labels with the LABEL element.
(1 instance)www.dhs.gov/dhspublic/index.jsp
Department of Homeland SecurityProvide alternative text for all images.
(1 instance)www.hud.gov
Department of Housing and Urban DevelopmentExplicitly associate form controls and their labels with the LABEL element.
(1 instance)Cynthia approved www.doi.gov
Text Only —
www.doi.gov/index-text.html
Department of the InteriorBobby approved —
text onlyCynthia approved www.usdoj.gov
Department of JusticeProvide alternative text for all images.
(8 instances)Cynthia approved www.dol.gov
Department of LaborProvide alternative text for all images.
(1 instance)www.state.gov
Department of StateProvide alternative text for all images.
(1 instance)
Explicitly associate form controls and their labels with the LABEL element.
(1 instance)www.dot.gov
Department of TransportationProvide alternative text for all images.
(16 instances)
Explicitly associate form controls and their labels with the LABEL element.
(1 instance)www.ustreas.gov
Department of the TreasuryExplicitly associate form controls and their labels with the LABEL element.
(1 instance)Cynthia approved www.va.gov
Department of Veterans AffairsBobby approved Cynthia approved www.senate.gov
United States SenateProvide alternative text for all images.
(6 instances)
Explicitly associate form controls and their labels with the LABEL element.
(1 instance)www.supremecourtus.gov
Supreme Court of The United StatesProvide alternative text for all image–type buttons in forms.
(1 instance)
Explicitly associate form controls and their labels with the LABEL element.
(2 instances)www.cdc.gov
Centers for Disease Control and PreventionExplicitly associate form controls and their labels with the LABEL element.
(1 instance)www.cia.gov
Central Intelligence AgencyExplicitly associate form controls and their labels with the LABEL element.
(1 instance)www.eeoc.gov
Equal Employment Opportunity CommissionExplicitly associate form controls and their labels with the LABEL element.
(1 instance)www.epa.gov
Environmental Protection AgencyExplicitly associate form controls and their labels with the LABEL element.
(1 instance)www.faa.gov
Federal Aviation AdministrationExplicitly associate form controls and their labels with the LABEL element.
(1 instance)Cynthia approved www.fbi.gov
Federal Bureau of InvestigationProvide alternative text for all images.
(1 instance)www.fcc.gov
Federal Communication CommissionBobby approved Cynthia approved www.fdic.gov
Federal Deposit Insurance CorporationProvide alternative text for all images.
(2 instances)
Explicitly associate form controls and their labels with the LABEL element.
(2 instances)Cynthia approved www.fema.gov
Federal Emergency Management AgencyBobby approved Cynthia approved www.federalreserve.gov
Board of Governors of the Federal Reserve SystemExplicitly associate form controls and their labels with the LABEL element.
(1 instance)Cynthia approved www.irs.gov
Internal Revenue ServiceProvide alternative text for all images.
(1 instance)
Explicitly associate form controls and their labels with the LABEL element.
(1 instance)www.ftc.gov
Federal Trade CommissionProvide alternative text for all images.
(2 instances)
Explicitly associate form controls and their labels with the LABEL element.
(1 instance)www.gao.gov
General Accounting OfficeBobby approved —
for non–frames version
Frames version not approvedCynthia approved www.access.gpo.gov
Government Printing OfficeBobby approved Cynthia approved www.nasa.gov
National Aeronautics and Space AdministrationBobby approved Cynthia approved www.neh.fed.us
National Endowment for the HumanitiesProvide alternative text for all images.
(2 instances)www.nimh.nih.gov
National Institute of Mental HealthProvide alternative text for all images.
(3 instances)
Explicitly associate form controls and their labels with the LABEL element.
(3 instances)
Provide alternative text for all image map hot–spots (AREAs).
(12 instances)www.nps.gov
National Park ServiceProvide alternative text for all images.
(1 instance)www.noaa.gov
National Oceanic and Atmospheric AdministrationProvide alternative text for all images.
(7 instances)
Explicitly associate form controls and their labels with the LABEL element.
(2 instances)www.nrc.gov
Nuclear Regulatory CommissionProvide alternative text for all images.
(20 instances)
Explicitly associate form controls and their labels with the LABEL element.
(1 instance)www.ntsb.gov
National Transportation Safety BoardProvide alternative text for all images.
(1 instance)www.opm.gov
Office of Personnel ManagementExplicitly associate form controls and their labels with the LABEL element.
(2 instances)Cynthia approved www.peacecorps.gov
Peace CorpsExplicitly associate form controls and their labels with the LABEL element.
(6 instances)www.sec.gov
Securities and Exchange CommissionProvide alternative text for all images.
(6 instances)Cynthia approved www.si.edu
Smithsonian InstitutionProvide alternative text for all images.
(6 instances)
Explicitly associate form controls and their labels with the LABEL element.
(1 instance)www.ssa.gov
Social Security AdministrationBobby approved Cynthia approved www.uspto.gov
Patent and Trademark OfficeExplicitly associate form controls and their labels with the LABEL element.
(1 instance)Cynthia approved www.uscg.mil/USCG.shtm
Coast GuardProvide alternative text for all images.
(1 instance)www.surgeongeneral.gov/ sgoffice.htm
Office of the Surgeon GeneralProvide alternative text for all images.
(11 instances)
Explicitly associate form controls and their labels with the LABEL element.
(2 instances)
Two of the original 50 sites selected for evaluation, the U.S. House of Representatives and the Library of Congress, could not be evaluated using the online version of Bobby after multiple attempts. The specific reason these Web pages could not be assessed is not known. Home pages of two other well–known Federal agencies were selected as replacements. The Department of the Interior home page could not be evaluated on multiple attempts. However, the Interior home page prominently displayed a link to a text–only version of the site. The text–only version is accessible. The General Accounting Office home page offers a non–frames version which tested as accessible, although the frames version did not. Since both of these sites offered alternatives, and these alternatives were accessible, these were counted as accessible sites.
Data summary
Of the 50 home pages evaluated, 11 (22 percent) received the "508 Approved" icon on the Bobby report. Additionally, while the automated Bobby review of these 11 home pages did not find any specific Section 508 errors, each approved site had several suggested user checks. It is thus conceivable that further manual checking would have revealed barriers even among the eleven pages that had passed the automated Bobby check.
Of the 39 home pages (78 percent) that did not pass the automated Bobby evaluation 29 had the error "Explicitly associate form controls and their labels with the LABEL element"; 24 had the error "Provide alternative text for all images"; two had the error "Provide alternative text for all image map hot–spots (AREAs)"; and, one had the error "Provide alternative text for all image–type buttons in forms". These results show a heavy clustering of just two types of errors.
On a positive note, 17 of the home pages that did not receive the Section 508 approved icon on the Bobby report had only one error and one instance of that error. Essentially, a modification to one line of HTML on each of these 17 home pages could potentially make these compliant to Section 508 guidelines. A review of the 17 home pages with one error, shows 12 have the error "Explicitly associate form controls and their labels with the LABEL element." The Bobby report presents this explanation for this error:
"An HTML LABEL specifically associates the label's text with the form control. This allows the browser to tell the user definitively which label applies to the given control. Usually, clicking on the label positions the cursor in the form field, or toggles the value of radio buttons or check boxes. This is intuitive for many users and provides a larger target for the mouse."Making this change in the HTML code would not require an extensive investment of time and could be accomplished fairly easily.
To cross–check these findings, the same fifty home pages were evaluated using Cynthia Says less than 24 hours later. Cynthia Says approved the same 11 sites that Bobby did, as well as an additional 10 sites. Although there is a significant discrepancy between these tools, the Cynthia Says results indicate only 42 percent of the 50 Federal home pages met the Section 508 accessibility standards.
Discussion
Since the enforcement provisions of Section 508 (Access Board Standards) had gone into effect over two years ago, in 2001, and Jackson–Sanborn, et al. had found 60 percent of government Web pages to be accessible that very same year, the low accessibility finding of 22 percent is a surprise. Considering Stowers’ findings of 13.5 percent accessibility early 2002, these results are still surprising since Federal agencies had nearly two years to make changes to their sites. This small–scale study cannot be used to conclude that the accessibility rate for the thousands of Federal government Web sites is 22 or 42 percent. However it can be reasonably concluded the mandate of 100 percent accessibility has not been achieved. In 2000, the Department of Justice report Information Technology and People with Disabilities presented a largely positive review of the accessibility of Federal Web pages. Four years later, accessibility does not seem to have improved, and perhaps it has worsened. This study only looked at the home pages of a small number of agency sites, not the 20 most visited pages within that site, which by law are also supposed to be accessible. It is possible that one or all of the 11 home pages that did receive Bobby 508 approval could have one or more of 20 most visited pages within their sites fail to be Section 508 approved. Each approved site had several suggested manual user checks and it is possible some of the user checks could involve elements that hinder accessibility.
As with previous studies of this nature, there are certainly limitations to this study. However, it is apparent that the U.S. government needs to re–evaluate its effort to achieve its own directive to meet Section 508 accessibility guidelines. By law, all sites should be accessible. This study clearly shows that there are serious accessibility issues with a majority of the selected high–profile Federal home pages.
Does a finding of accessibility using an automated program like Bobby guarantee accessibility for all users? Jim Thatcher provides a good explanation of this at his Web site:
"Any testing for Web accessibility must be viewed as a process which combines automated software tools with human judgement (sic). It is not possible to purchase any Web accessibility evaluation tool, no matter how expensive, run it on your site, and conclude that your site (or even page) is accessible, or whether it complies with the Section 508 provisions. Human judgement is absolutely required. You can conclude that your page is not accessible, or does not comply with Section 508, because in several cases this conclusion can be reached from the fact that the page is lacking some required valid attribute. Software testing tools certainly should pick up these errors."An accessibility evaluation product such as Bobby, Cynthia Says, or A–Prompt can help locate common accessibility issues in a Web site. However even after user checks have been completed and acted on, these tools cannot guarantee accessibility for all users. By making sure that Section 508 or WCAG guidelines are followed in Web site construction, an organization can be confident that most people, whether disabled or not, are able to access the information they need.
Conclusion
While this paper has focused on the accessibility of Web sites, it is important to remember that Web sites are just one part of the accessibility question faced by individuals with disabilities. Web sites often serve as portals to additional information services such as databases and e–learning sites that provide Web–formatted information. Making sure Web sites are accessible is an important step, but it is just a first step in ensuring access to digital information for all people.
As demonstrated by this study, as well as in earlier studies, even organizations that are motivated and mandated to provide 100 percent access have not achieved this goal. Unfortunately, there are many more organizations and individuals that have neither the mandate nor the motivation to make sure that electronic information placed on the Internet and in other electronic formats is accessible. There is great potential for accessibility for all — but a lot of work remains to be done to make this dream a reality. If the U.S. government cannot ensure accessibility of its own Web pages, it cannot credibly advocate that other organizations provide accessible information.
About the author
Jim Ellison is an employee of The Rehabilitation Services Commission, an agency of the State of Ohio that works for people with disabilities. He received a MLIS degree from Kent State University in December 2003.
E–mail: jim.ellison@earthlink.net.
Notes
3. Sherman and Price, 2001, p. 227.
4. Hernon, et al., 2001, p. 7.
6. An excellent resource on the use of Bobby is available from accessibility expert, Jim Thatcher, at http://www.jimthatcher.com/bobbyeval.htm.
References
M. Bray, C. Flowers, and P. Gibson, 2003. "Accessibility of School Districts’ Web Sites: A Descriptive Study," Information Technology in Childhood Education Annual, volume 2003, number 1, pp. 209–221.
D. Diaper and L. Worman, 2002. "Two Falls out of Three in the Automated Accessibility Assessment of World Wide Web Sites: A–Prompt v. Bobby," at http://dec.bmth.ac.uk/staff/ddiaper/hci2003sub.pdf, accessed Retrieved 20 July 2003.
S.A. Guthrie, 2000. "Making The World Wide Web Accessible to All Students," Journalism and Mass Communication Educator, volume 55, number 1, pp. 14–23.
P. Hernon, R.E. Dugan, and J.A. Shuler, 2001. U.S. Government on the Web: Getting the Information You Need. Second edition. Englewood, Colo.: Libraries Unlimited.
E. Jackson–Sanborn, K. Odess–Harnish, and N. Warren, 2002. "Web Site Accessibility: A Study of Six Genres," Library Hi Tech, volume 20, number 3, pp. 308–317.
P. Jaeger, 2004. "The Importance of Accurately Measuring the Accessibility of the Federal Electronic Government: Developing the Research Agenda," at http://www.rit.edu/~easi/itd/itdv09n1/jaeger.htm, accessed 2 February 2004.
C. Letourneau, 2003. "Accessible Web Design — A Definition," at http://www.starlingweb.com/webac.htm, accessed 20 June 2003.
E. Lilly and C. Van Fleet, 1999. "Wired But Not Connected: Accessibility of Academic Library Home Pages," Reference Librarian, numbers 67/68, pp. 5–28.
R. Mark, 2003. "Government Sites Surge in Popularity," at http://boston.internet.com/news/article.php/2120061, accessed 24 June 2003.
B. Mates, 1999. Adaptive Technology for the Internet: Making Electronic Resources Accessible to All. Chicago: American Library Association, and at http://www.ala.org/ala/products/books/editions/adaptivetechnology.htm, accessed 12 July 2003.
S. Noble, 2002. "Web Access and the Law: A Public Policy Framework," Library Hi Tech, volume 20, number 4, pp. 399–405.
Office of Information Technology, 2003. "Federal IT Accessibility Initiative," at http://www.section508.gov, accessed 23 June 2003.
M.G. Paciello, 2000. Web Accessibility for People with Disabilities. Lawrence, Kans.: CMP Books.
A. Schmetzke, 2001. "Web Accessibility at University Libraries and Library Schools," Library Hi Tech, volume 19, number 1, pp. 35–49.
C. Sherman and G. Price, 2001. The Invisible Web: Uncovering Information Sources Search Engines Can’t See. Medford, N.J.: CyberAge Books.
G. Stowers, 2002. "The State of Federal Websites: The Pursuit of Excellence," at http://www.endowment.pwcglobal.com/pdfs/StowersReport0802.pdf, accessed 4 February 2004.
U.S. Department of Justice, 2003. "Americans with Disabilities Act," at http://www.usdoj.gov/crt/ada/adahom1.htm, accessed 15 June 2003.
C. Waddell and M. Urban, 2000. "An Overview of Law and Policy for IT Accessibility," at http://www.icdri.org/CynthiaW/SL508overview.html, accessed 16 September 2003.
Editorial history
Paper received 6 June 2004; accepted 16 July 2004.
Copyright ©2004, First Monday
Copyright ©2004, Jim Ellison
Assessing the accessibility of fifty United States government Web pages: Using Bobby to check on Uncle Sam by Jim Ellison
First Monday, volume 9, number 7 (July 2004),
URL: http://firstmonday.org/issues/issue9_7/ellison/index.html
