First Monday


Students' Frustrations with a Web-Based Distance Education Course

Many advocates of computer-mediated distance education emphasize its positive aspects and understate the kind of work that it requires for students and faculty. This article presents a qualitative case study of a Web-based distance education course at a major U.S. university. The case data reveal a taboo topic: students' persistent frustrations in Web-based distance education. First, this paper will analyze why these negative phenomena are not found in the literature. Second, this article will discuss whether students' frustrations inhibit their educational opportunities. In this study, students' frustrations were found in three interrelated sources: lack of prompt feedback, ambiguous instructions on the Web, and technical problems. It is concluded that these frustrations inhibited educational opportunities. This case study illustrates some student perspectives and calls attention to some fundamental issues that could make distance education a more satisfying learning experience.

Contents

Introduction
Literature Review
Background
Methodology
Students' Frustrations in an Online Course
Understanding Students Perspectives
Discussion
Conclusion

Introduction

Recent cutting-edge technology, such as the World Wide Web and online conferencing systems, enable universities to implement distance education to reach a diverse population and to provide open learning environments 24 hours a day 7 days a week. Some analysts argue that the Internet-supported distance education courses do more than bring new students into online classrooms. In addition they form "a critical pressure point for challenging the dominant assumptions and characteristics of existing traditionally organized universities in the 21st century" (Hanna, 1998).

The number of distance education courses is growing (Hanna, 1998; National Center for Education Statistics, 1998; Rahm and Reed, 1998; Roberts, 1996). There are also substantial discussions about distance education in higher education. The literature about distance education is dominated by enthusiastic studies and accounts. Once consensus is reached, it tends not to be disturbed by a dissonant idea (Heylighen, 1992). Indeed, similar patterns are found in other relatively young fields, for example, in the Business Process Reengineering (Kling and Tillquist, 1998) and hypertext (Unsworth, 1997) literatures. However, the studies in the computer-mediated distance education are more anecdotal than systematically empirical or critical. Distance education advocates argue that the increasing number of online courses will readily expand educational opportunities. This article queries this assumption that these courses may be offered easily by illustrating some of the problems that arose in a specific Web-based course.

This case study examines students' difficulties in learning effectively in a Web-based distance education course. The course, B555 [ 1]s, was offered at a major U.S. university. The entire course was organized through a Web site. This study is based on interviews and observations of the students who were enrolled in B555. B555 was chosen because the instructor permitted observation of the online class and interviews with her and her students. It was not chosen because we expected it to be a superb course for students or a troublesome course.

The following vignette, written by the first author, illustrates the kinds of problems that students experienced in B555.

Every Monday, the instructor of this course usually sent an e-mail message explaining the requirement for a particular module. The messages reflected on the previous week's online discussion, encouraged students, suggested questions for the upcoming online discussion, added resources for assignments, and posted activities. However, when I saw John who participated in my research as a student in B555, the instructor had not sent any e-mail to specify the assignment for the week. John did not seem to know what to do in a computer lab by himself.

John was in a computer lab around 11:30 PM on Saturday. Many papers were scattered around the computer he was facing. Since we were introduced to each other a week before when I conducted an observation and interview, I said hello to him and realized that he was working on the B555 class assignment. He told me briefly, "See we have to make sense out of this," by pointing out this week's assignment page, printed out from the B555 Web site. I remembered that he had been waiting for the assignment specification via e-mail from the instructor since early in the week, and asked, "So, the instructor hasn't sent out the specification message for the assignment?" John said no. I did not want to interrupt his work, so I left the lab. Later, around 1:00 AM, I returned to the computer lab. John was alone in the quiet lab. I was working on a different project, so I picked up my printing job and said to John, "don't work too late." Then, John looked at me with vacant eyes and said that if I had been doing my interview now, I would get a totally different perspective about this course, because he knew that I was studying B555. He seemed so exhausted and frustrated. He looked like he really wanted to talk to somebody to share his frustration.

Some studies report the importance of students' isolation in distance education courses (Besser and Donahue, 1996; Twigg, 1997). Our original research question was: How do the students in B555 overcome their feelings of isolation in a virtual classroom to create the sense of a community of learning? However, during the observations and interviews (including John), we learned that students' isolation was not as big of a problem as was frustration in this course. Possibly because of the small class size, students supported each other and had a sense of a community of learning.

However, during the first several interviews, students frequently reported major frustrations with various aspects of taking an online course. We immediately investigated the research literature about online courses. A few studies mentioned students being frustrated with technical problems, but their authors emphasized the value of the students' learning. We felt that the substantial frustrations reported by our informants in B555 were not incidental and could actually impede their learning. Consequently we shifted the focus of the study to examine how much and in what ways students frustrations in an online course can inhibit their learning.

This article illustrates the frustrations that students can experience while taking a distance education course, and how these frustrations can significantly inhibit their educational opportunities. It is not uncommon for university students to be frustrated in some of their courses. Some students in enormous lecture courses, certain topics (ie., calculus), or with poor instructors express frustration. B555 is the kind of small (6 person) course that is usually well received by students. The instructor was selected, in part, because she was viewed as competent and sufficiently experienced by her department chair - who had taught some distance education courses. There was no a priori reason that we could find in the literatures about distance education to anticipate that students' persistent frustrations would be a significant feature of B555.

Nipper (1989) identifies three generations of distance education. The first generation was provided mostly through paper-based instruction; the second through integrated multimedia such as delivering courses via television; the third was provided through two-way communications media such as video-conferencing. In this study, the third generation of distance education, especially using networked computers, was explored. Thus, throughout this article, we refer to distance education as computer-mediated distance education.

Literature Review

Research on the WWW and Distance Education

Research studies indicate that the achievement and satisfaction of students in distance education courses is not significantly different than the achievement and satisfaction of students in traditional classrooms (Johnstone and Krauth, 1996). Distance education offers opportunities for students who cannot travel to a campus for their classes (e.g. Owston, 1997). Harasim (1993) asserts that computer networks make the world connected, and that this concept is applicable to distance education. However, past studies do not illustrate the details of students' perspectives on distance education. Moreover, research on the effect of distance education has been focused on student outcomes (Ahern and Repman, 1994), but not on the affective aspects of distance education. Johnstone and Krauth (1996) prove the efficacy of technology in distance education, but do not examine surrounding issues, such as students' isolation and effective advising from instructors.

As Windschitl (1998) notes, research on the use of the World Wide Web (WWW) lacks disciplined scholarly articles. "The vast majority of published work is description of technology implementation in classrooms" (p. 28) or reflection of what has been done in distance education. One reason for this is because the WWW is relatively young and is still in a testing stage, not an evaluation stage. Indeed, the lack of disciplined scholarly articles characterizes the field of Computer-Mediated Communication (CMC) as a whole [ 2]. Romizowski and Mason (1996: p. 442) claim that "only some 10% to 15%" of the articles published about CMC by 1991 were research studies. Moreover, they note that little qualitative research based on observation and interviewing in CMC has been conducted. Windschitl also suggests that qualitative studies capture unique phenomena about WWW use. Yet the research literature on the use of WWW is short of analytical studies as well as qualitative studies (Burge, 1994; McIssac and Gunawardena, 1996).

Hidden Phenomenon

Higher education in the U.S. is facing a challenge to meet new demands for the next century. Various criticisms of traditional classrooms appear frequently, such as lack of personal attention, boredom, outdated knowledge, lack of appropriate skills for workplaces and inappropriateness for a diverse population (Diamond, 1997; Gardiner, 1997; Handy, 1998; Roueche, 1998; Wingspread Group on Higher Education, 1993). Many researchers advocate "solutions" such as active learning, learner-centered principles, effective use of technology, and collaborative learning (American Psychological Association, 1997; Bonk and Kim, 1998; Cove and Love, 1996; ERIC, 1998; Schroeder, 1996). The expectations for technology to transform higher education are disproportionately high.

This enthusiastic attitude toward technology is not entirely new. Kling (1994) identifies "technological utopianism," which refers to "analyses in which the use of specific technologies plays a key role in shaping a benign social vision." A similar attitude is found in the history of educational technology:

Most of what we read or hear about computers in education emphasizes only one aspect, usually the good points, but occasionally the bad, to the exclusion of other points of view. This is at least partly due to the screening effect of the popular press, who favor the excitement of extremism over the calm of rationality, preferring in the name of "reader interest" to create what Monosky (1984) calls an artificial dichotomy (Ragsdale, 1988, p. 50).

Both Kling and Ragsdale caution against extreme views of technology, either positive or negative, and suggest that more socially realistic analyses are needed.

When computers were introduced in classrooms in the 1980s, "extolling the computers as a boon to critical thinking, professional educators by and large have been conspicuously uncritical about the computer itself" (Sloan, 1985, p.1). Rather than accepting the enthusiastic attitude toward technologies in education, Cuban (1986) observes an unrelenting cycle of technology promotion and adoption in classrooms by reviewing the literature on the educational use of motion pictures, radio and television since 1920s. The cycle indicates a pattern; technology was introduced in classrooms by enthusiastic advocates, such as administrators and researchers, but teachers failed to effectively use technology because of the lack of equipment, time and training. Cuban cautions us not to expect too much of computers in classrooms because their use may follow the same pattern as other technologies. As some authors (Dreyfus and Dreyfus, 1984; Ragsdale, 1988; Salomon, 1985; Sloan, 1985) criticize Computer-Assisted Instruction in K-12, recently other authors also have criticized educational computing in general, such as information technology in higher education (see Noble, 1998) and computers in schools (see Oppenheimer, 1997).

A systematic search of the ERIC database helped to locate some research about problems of distance education, such as students' isolation, lack of effective advice (e.g., Abrahamson, 1998; Brown, 1996; Rahm and Reed, 1998). However, there is little research about students' frustration in distance education. A few authors identify this issue (e.g. Dede, 1996; Feenberg, 1987; Stahlman, 1996), but these are rather "socially-thin" (Kling and Tillquist, 1998) and do not indicate the problems in social contexts. Even the few researchers who mentioned deeper social factors of the problems in distance education did not focus on students' frustrations (e.g. Burge, 1994; Kang, 1988; Kiesler, Siegel and McGuire, 1984). This topic has not been highlighted in writings about computer-mediated distance education. We question why this phenomenon of students' frustration has not been seriously studied, and identify the following four possible reasons.

First, the researchers who study distance education may be biased toward technology. It is natural because

events are interpreted in the context of experience and expectations. Thus, it might be said that the answer to questions about the effect of computers on education is unobtainable regardless of the researchers' backgrounds, or more precisely, because of their backgrounds. That is, our backgrounds cause us to expect certain results, preventing our clear perception of other results (Ragsdale, 1988, p. 14).

Kling (1984) and Bowers (1988) examine how educational technologies are not simply "value-free tools." Many authors of distance education studies are affiliated with technology-oriented departments, such as educational technology, library and information science or technology support centers. Therefore, they might have a favorable view of technology, such as seeing "distance education via technology as a potential silver bullet" (Twigg, 1997, p. 28). Noble (1998) asserts that "behind this effort [promoting technology in higher education] are the ubiquitous technozealots who simply view computers as the panacea for everything, because they like to play with them." For examples, the special issue of Web-based learning in Educational Technology (Hackbarth, 1997) is devoted entirely to technical issues (e.g. Starr, 1997) and teachers' perspectives (e.g. Berge, 1997). McIssac and Gunawardena (1996) state that "more than 23% of the literature reviewed concerned issues related to technology and the role of the distance educator" (p. 421). Burge (1994) asserts that most of the literature on CMC in higher education is "cautious optimism to hyperbole" (p. 22). Thus, the field has not critically addressed negative implications, especially from students' perspectives in distance education.

The second possible reason that little research reports students' frustration is because few qualitative research studies have been done (Burge, 1994; Windschitl, 1998), so that the fine-grained dynamics of virtual classrooms are unknown. In addition, McIssac and Gunawardena (1996) criticize the research literature in distance education because of lack of research rigor. "Although studies focusing on learners have received attention in the literature (18%), it is largely descriptive" (McIssac and Gunawardena, 1996, p. 423). Wolcott (1990) claims that "informants [students] ... do little talking" (p. 29) regarding qualitative research in education. For example, one qualitative study was conducted by an instructor of the course (Yakimovicz and Murphy, 1995). The study's data are not very reliable, because students are likely to be polite to their instructors. People are even polite to computers and give favorable evaluations of computers when asked about their performance (see Reeves and Nass, 1996).

Third, students may not have had opportunities to express their frustrations with Web-based distance education. At the end of the semester, students might make positive comments about the courses because of a relief of finishing a course and concern about hurting instructors' feelings. For example, for the course we studied, a student posted a "thank you" note during the final week saying how much she had learned, and how much she appreciated these learning opportunities:

"I do believe you all are the best classmates and instructor I have ever met. I can see your hard work, your enthusiasm, and your patience learning along. I'd like to say that the most successful condition I've learned from this class is: warm and supportive class atmosphere." (personal communication, December 1).

Therefore, since little research has studied their learning processes throughout the semester, students' frustrations have received little attention. That is why the results of many studies are positive, including such findings as students enjoying their experiences despite their technical problems (Gregor and Cuskelly, 1994; Yakimovicz and Murphy, 1995).

Finally, it is possible that past studies were conducted only with instructors experienced in distance education (e.g. Gunawardena, 1992). More experienced instructors might better handle students' frustrations, technological problems and ambiguous instructions to reduce the obstacles to distance education.

However, the history of educational technology teaches us that it is necessary to study failures as well as successes.

Provided that we understand the limitations of each technology as well as its capabilities, and more importantly, provided that we understand the people we are trying to educate and the kind of education we are trying to give them, we can use technology in ways that will really help. There is no technological panacea; there are only technological solutions to some educational problems (Simpson, 1985, p. 91).

Bryson and de Castell (1998) urge that we need to pay more attention to failures of educational innovation because it will tell us why success stories are arbitrary. Unsworth (1997) also argues that "many things that we take to be trivial, or embarrassing, or simply wrong, will be of interest to our peers in the future." He claims that people learn from errors and failures, and suggests that recording them is necessary to make progress.

Frustration and Learning

Sustained frustrations impede students' learning. Frustration interferes with pursuing goals (Reber, 1985) and thus it is one of the factors influence learning. Two aspects of learning are affected by frustrations: cognitive and affective (Jonassen and Grabowski, 1993). Research with college students shows that high levels of anxiety decrease the storage and processing capacity of working memory and impede making inferences (Darke, 1988a; Darke, 1988b). In addition, high frustration can demotivate students (Jonassen and Grabowski, 1993). Motivation is a strong factor that influences student learning (Alexander and Murphy, 1994; Covington, 1993; Stage, 1996). Especially, distance education requires that students be self-regulated (e.g. Abrahamson, 1998). In this kind of learning environment where students are away from traditional classrooms, frustration can be a major obstacle for distance learning.

Background

Study Site

B555 was a graduate course whose syllabus characterized it as designed "to give participants a broad general view of computer-enhanced language learning and the place of the World Wide Web in the language learning classroom." It was taught through a Web site developed by graduate students during the previous summer. This site contained reading materials, activities, discussion questions and additional readings organized along eight themes, such as authentic task, control and "time and feedback." The course syllabus and assignment instructions were available on the Web. The grade was based on five assignments: Reading responses (10%); Online discussion participation (15%); Internet address book (10%); Portfolio (50%); and, Final project (15%). After students entered their user names and passwords, they would see the menu screen. This page used the metaphor of a traditional classroom, so that the student could be situated in their familiar environments.

The Students and Instructor

In 1997, B555 enrolled six master's students [ 3]. Four students had only minimal experience with computers, but one of them wasenthusiastic about technology and spent 30 to 40 hours a week for this course at the beginning of the semester. One student was very familiar with computers; she was also familiar with the course's content through friends who had taken B555 during the prior summer. The sixth student trained teachers in integrating computers into a curriculum. She was taking this course far from the university, so that she did not have direct access to university facilities. She was the only student who had taken a distance education course prior to B555.

The instructor was a Ph.D. candidate and an international student. She was an experienced English teacher; but this was her first experience teaching in the U.S. and by distance education. She was selected, in part, because she was viewed as competent and sufficiently experienced by her department chair - who had taught some distance education courses. She had audited B555 over the previous summer and she participated in the design team for the B555 Web site. However, B555 was designed by a faculty member in her program. The faculty member provided content for most of the Web site, such as course activities, and reading assignments. But because of the faculty member's health problems, the current instructor took over B555. Instead of altering the Web site, she tailored the instructions on the Web by sending weekly e-mail messages.

Informed consent was obtained before each of our observations and interviews. Pseudonyms are used in order to protect informants' identities. However, if we cited quotes from e-mail messages with their pseudonyms, it could reveal the subjects' identities to the instructor knew who said what in e-mail messages. In addition, since the class size was so small, as long as we described students' profiles in this article, the instructor would certainly identify the individual student. Because of the availability of the data from computer networks, it is easy to reveal informants' identities [ 4]. Therefore, in order to protect the informants, we decided not to include individual students' profiles, though they would have helped readers understand this case more deeply.

Methodology

Data Collection

We used a case study methodology because we found it necessary to develop a "thick description" (Geertz, 1973) of a virtual classroom. The inquiry was an instrumental case study (Stake, 1995) based on a need for a general understanding of students in distance education. The department where this study was conducted offered several courses online. B555 was chosen because the instructor permitted observation of the online class and interviews of her and her students. She was also interested in learning from this study. Six students, five students at the university campus and one student from out of state, were enrolled, and all agreed to participate in this study.

The study used three different methodologies; observation, interview and document review. First, online classroom discussion was observed to grasp how the instructor facilitated the dialogue among students. Observations in this electronic environment focused on the nature of students' discussion; the instructor's pedagogy, such as online discussion facilitation; the instructor's comments to students; and task assignments.

Secondly, it was observed how each student interacted with the Web site and conducted an interview immediately after the students had finished their tasks. Observation sessions lasted one to two hours. Four of the six students in the course were observed. The interview following the observation lasted about an hour for each student. One student did not allow observing his interaction with a computer because he said that he would not feel comfortable if somebody observed him while he was working on this course, and another student was far from the university location. However, these two students and the instructor agreed to be interviewed for about an hour. Moreover, data were collected from informal conversations with two students as well as the instructor.

Different kinds of observations were conducted. One of the observations was a special event during the semester that provided opportunities for students to interact simultaneously. They had a field trip to SchMOOze University - a virtual university campus accessed by telnet from all over the world. SchMOOze University is a MOO specifically designed for an English as a Second Language learner. MOO is an acronym [ 5] for "MUD Object Oriented" (Bruckman and Resnick, 1998, January) referring to "a multi-user, text-based virtual reality" (Blankenship, 1993). Simply stated, MOO is programmed to provide an electronic space where people can meet online, have synchronous chat and discussion, and play games. The remaining observations were conducted in university computer labs. However, because we were observing our informants working at computers, they were asked to "think aloud" (Ericson and Simon, 1984) while they used their computer. Although this methodology has a disadvantage that researchers cannot take account of every comment from informants during think aloud sessions, the methodology is widely used for usability tests. Dillon (1994) rationalizes the use of verbal protocol during the tasks to gain more accurate information than during retrospective verbal reporting.

Third, we examined various types of documents related to B555, including the course syllabus, reading assignments and the catalog's course description. The syllabus explained materials that students would use, the purpose of this course, format and philosophy, class schedule and requirements/evaluation including readings/responses, electronic discussion forum/participation, Internet address book, portfolios and final project. In addition, with her permission, the instructor's personal log was reviewed.

Data Analysis

Three different kinds of data were analyzed (observation, interview and document review data) simultaneously while data were collected. The analysis was triangulated in terms of methodologies, people, and time (Silverman, 1996; Stake, 1995). Furthermore, each interview transcript and interpretation were validated by informants.

Multiple perspectives allow people to see situations differently. "Our realities as well as our understandings are transient and inconsistent" (Stake and Mabry, 1995, p. 6). Therefore, case studies need to provide multiple perspectives and let readers judge and construct their reality (Prus, 1996; Stake, 1994; Strauss and Corbin, 1994; Wolcott, 1990). In this study, congruence as well as inconsistency were cautiously examined in order to present multiple perspectives.

This present article is organized as follows: the section, "Students' Frustrations in an Online Course," describes situations that students in B555 encountered. The description includes minimal interpretation in order to provide a "vicarious experience" (Stake, 1995) for readers, so that they can relate it to their existing knowledge and participate in a rich experience from this case. The next section, "Understanding Students' Perspectives," offers observation vignettes and interviews followed by a section of commentary. The "Discussion" section presents an assertion generated from this study. Finally, the "Conclusion" section summarizes the study and raises cautions in the study of distance education. The first author conducted the fieldwork; "I" refers to the first author whereas "we" refers to both authors.

Students' Frustrations in an Online Course

In this section we describe some of the situations that students in B555 found to be particularly frustrating. The students did not report that all of their course activities were marked by frustration. However, these illustrate the kinds of events that stood out for the students.

A Virtual Field Trip

SchMOOze University "was established in July, 1994 as a place where people studying English as a second or foreign language could practice English while sharing ideas and experiences with other learners and practicers of English." The B555 students took a field trip to SchMOOze University to experience virtual space. When people join SchMOOze University electronically, they see text-based screens although this virtual university uses metaphors of location. People can explore different virtual buildings (e.g., library, Mall, and meeting rooms) to meet people from all over the world by using simple commands, such as "go to east." Prior to this event, the instructor e-mailed instructions for the SchMOOze University field trip and a map of the SchMOOze University. The following observation was a special event in the middle of the semester that provided opportunities for students to have synchronous interaction.

The class meeting time was set at 8:30 PM. All students and the instructor were supposed to meet at a virtual meeting room, so that the instructor could see who was online. Kathy immediately started the field trip to SchMOOze University when I arrived at her real apartment.

Kathy typed: @knock MMM (instructor's name)
but the computer replied
I don't know

Kathy said, "It doesn't understand. How stupid it is. Let's try with a different name." Then she tried: @knock mmm
two or three more times, but it didn't work.

She murmured " I don't know what I am supposed to do. Maybe I am already in." She realized that several messages showed up on her computer screen. When she saw the following messages:
John smiles
John laughs
on the screen, she typed, "moan." Conversation on the screen went very quickly. It was very hard to catch up with what was happening. A student complained.

Sheryl: Please slow down.

However, the conversation never slowed down. When Kathy saw the following message:
MMM: everybody seems familiar with commands."

Kathy typed: I practiced this afternoon.

When she typed, Kathy seemed very careful about spelling and capitalization; she is an English teacher.
Sheryl: I like the action of calling rows.

Kathy explained to me and said, "I think what she means is 'calling role.' Sometimes it's confusing, the teacher and half of the students are non-native speakers."

Further, she said "I tried this afternoon, but at that time nobody responded to me. So, this is the first time that I see the conversation going like this" by pointing out many messages on the screen.

Then, she saw the message:
Julie: Julie is here

She tried to respond. While she was typing, she said "By the time I type in my response, the conversation is gone."

She tried to say "Welcome" to Julie, but a female Japanese English teacher with 17 years of teaching experience joined the conversation. Several people who were not in the class also joined this discussion.

The first 30 minutes went very quickly, just trying to figure out who was there and what to do.

Kathy explained to me, "This is the first time we talked together ..." and complained "What are we supposed to do," glanced at her watch. It was almost 9 PM and according to the instructor's guideline, students were supposed to leave from the original meeting room, go to different buildings at SchMOOze University, and look for possible student activities.

Therefore, Kathy typed: Are we supposed to move around now?

MMM: chose building

When she saw the instructor's message, she murmured "I'm going to be out, go to lobby, and go to Mall." However, she couldn't find anybody to talk to at the virtual Mall, so she typed commands to go back to the original meeting room. She saw on the screen that there were still students from the B555 class and their discussions were continuing.

She said, "Now, I'm back to the discussion." and typed: Guess I need to stay put.

MMM: go to the building

Kathy said, " I feel like nobody is answering my question," and complained "I've already been around the campus and ..."

While she was deciding what to do next, the online discussion at the meeting room was still going on. When she saw a message referring to Ann, she typed: Who's Ann?

The situation was chaotic in that room because different conversations were happening simultaneously. Before finding out who Ann was, Kathy said, "Maybe I'll explore the campus now." She suggested going to a virtual bar to her classmates by typing:

How about the bar?

Kathy saw everyone's agreement with her opinion on the screen.

When Kathy saw a message saying "Knock, knock," she figured that somebody knocked at her door and she typed: Enter
but she received no answer. Kathy muttered, " What am I supposed to do? I'm confused," and looked at the instructor's guide. Then, she assumed Julie was sending a message that knocks at her door because she saw Julie's message asking her a question, and tried to find where Julie was. Kathy typed a command to move where Julie was and Julie started sending a message to her.

Julie: I don't want to leave you at the bar alone.

Kathy laughed when she read the message. Julie tried to instruct her how to respond to knock in this text-based environment, but Kathy was still struggling. Kathy looked at her watch and said, "This is exactly an hour." She told me that, "If I have one complaint about this class, it is that time goes so quickly. I can be hooked up with a computer for a whole day and then realize that I haven't had a dinner or I haven't prepared my lesson plans."

Although Kathy seemed very well prepared for this special virtual field trip, she was frustrated because she could not figure out what was wrong when she could not effectively perform her intended commands, e.g., simply responding to a knock. She could not ask for help with her problem from a real person, so she had to solve the problem by herself.

Amy commented about this virtual field trip at a computer lab a few days after this event:

At SchMOOze University, I got lost. Before this event, I had to set up software, some special software for MOO, the instructor said, on a computer. So, I downloaded it and set it up. I checked if I could go to the meeting room before the class activity time, then I went there successfully and thought everything was fine. But, when I went there to see classmates at the meeting time, I got lost. I could see their online conversation, but they couldn't see my messages. So, I called Sheryl and she taught me how to use commands and so on. I just forgot to put parentheses when I typed. That's why the classmates could not see my messages. I talked to other people from different places at SchMOOze University, but not my classmates. I was so frustrated because everyone else could do it, but why not me. Not only for the SchMOOze University activity, but I put in lots of time for this course overall, but I couldn't see the results. Like I paid a hundred dollars, but I only got ten dollars back. I probably spend a hundred minutes, but I can get ten dollars worth (personal communication, November 11).

Like Kathy, Amy was frustrated because of the problem that she had with operational commands at SchMOOze University. She expressed her frustration and even anger at herself. It appeared that she felt as if she had been left out of the class because she could not use the commands properly.

Julie also had a negative experience with this virtual trip. Because of the slow connection from her computer, her responses delayed significantly. When I observed Kathy trying to talk to Julie online, Kathy had no response from Julie for more than two minutes. Finally, Kathy gave up the conversation with Julie. Another student also reflected back the feeling in a personal e-mail to the instructor next day after the field trip to SchMOOze University:

I thought your [the instructor's] preparation for our visit to SchMOOze U was excellent ... I did not enjoy our class excursion there however because the technology did not live up to expectations. I also felt more encumbered by knowing people there. I was more cognizant of hurt feelings and other people's frustration, it narrowed my exploration. (personal communication, October, 23).

There were, however, positive comments about the virtual field trip. John was enthusiastic about the SchMOOze University activity:

I'd loved the MOO session. I felt like doing that, we're really sort of like a community. I was totally laughing, at my computer, laughing. It's so weird to laugh at the computer. But I was laughing because I really felt somebody's there talking. And I met a person that was kind of cold to me and asked me weird questions, and they never really answered my questions. That hurt, you know? So it's real feelings that were involved. It's kind of interesting (personal communication, October, 30).

John seemed very excited about the technology. In contrast with Kathy, Julie and Amy, John had a positive experience with the field trip to SchMOOze University. In the interview, he mentioned his favorable view of technology, although he did not have much experience with computers before taking B555. Because Kathy and Amy expressed their uneasiness with the computer, we suspect that computer competency may affect their level of frustration.

Working Alone at Night

It is common for students in many online courses to work alone, often at home in the evenings or weekends. However, it is hard for students who work under these conditions to resolve some of the kinds of potentially frustrating ambiguities that can be resolved more readily in a face to face class meeting.

The dynamics of this issue are illustrated by the experiences of one student, John, whom I met unexpectedly when he was working alone after midnight in a campus computer lab. The topic for this week was "feedback and time." John was working on this assignment - evaluating lesson plans using technologies for language education. He started talking.

J: I am frustrated because I am here too long (laugh).
I [6]: How long have you been here?
J: Ohhhhh, I (.) . . . probably nine o'clock, I guess.
I: Four hours?
J: Yeah. So, my eyes are tired. Of course, a part of the problem is not totally the class's fault. Part of the problem is finding things really interesting. They don't completely relate to the class. I mean, we are looking for things, lesson plans that we have to evaluate, right? And there are all kinds of great lesson plans. I am looking for ideas for my classes and I just get stuck. Then by the time I'm at the place where I really need to be doing my work, I'm totally frustrated because I really wanna go home. I don't wanna be here anymore. That's one thing. There's nothing wrong with that, well maybe just a little. I don't know.
I: Too much information?
J: Perhaps. I mean these links on the B555 Web site have all the lesson plans that we can give to a class. I think this one [pointing to a link], just tons and tons of activities, but most of the stuff on these, I don't like. So, I don't wanna spend my time. Okay, number one, that's not very much stuff. Number two, stuff on there, I don't like. That gives me very little to go with. I mean that stuff I wouldn't use. I mean it could be better if I could make my own lesson plan or something and then, talk about how I would use assessment in it. So, anyway, I've got a couple of things I wanna use for the B555 class assignment, but I feel like it's sub-standard. Or not exactly how I would wanna define it if I would be looking for something to fulfill the requirement. Just all of this together. These all three problems together and that's two, none of them are really terrible, but I am just frustrated in general (personal communication, November 11).

When I met John, he was extremely frustrated and he had more than one source for his frustrations. He still had not received specifications for the assignments from the instructor, and felt confused.He also had doubts about the professional development of the B555 Web site.

J: I just feel like the first part of the class was really well organized, and had a lot of really good information, like the first three chapters on the B555 Web site, but last few chapters, I feel a little bit ...
I: Only this chapter or?
J: No, last few chapters have been ... The readings haven't been as good and they haven't been connected quite as well to the topics that we are talking about. You really have to find something. I just wish that last few, especially this one .... I just don't feel like it's a very interesting chapter ... I wonder, I just have a small nagging doubt in my mind that, maybe when whoever put this together, the class, they really put a lot of work at the beginning, did a good job, but not as good a job at the end of it. (personal communication, November 11).

When I left the computer lab, John returned to work on his assignment and declared:

"I will finish this work anyway. It'll probably take an hour and it may not be a good work. (.) But just do it." It was almost 1:20 am. He said to me in a tired voice, "You have a good night, and I'll have a good night."

Interactive Communication Tool: E-Mail

Students and the instructor in B555 generated quite intensive online discussions through e-mail, and all of the students posted far more than five one-to-two page long messages During the week of October 19th they posted 35 short and long messages; this volume was common throughout the semester. On the surface, this volume of online discussion indicates a lively class. However, we found that there were some underlying problems with the reliance on e-mail.

First some students did not read other people's postings before writing their e-mail messages. One student reported this practice in an interview. Second, some students were unable to make time to read and post e-mail during short intensive discussion periods. For example, one student did not post any comments when the other students intensively discussed a particular topic for two days After another student summarized the overall discussion in his e-mail, she sent an e-mail that had a subject line saying, "Ah . . . I cannot catch up with all of you :( [ 7] " She was one of the students who posted the fewest number of e-mail messages to the online class discussion. Some other students also reported to me that they were overwhelmed by the volume of e-mail, and that they fell behind in reading and responding online.

Some of the students' difficulties were a byproduct of using e-mail differently than many people expect. In the "standard view," students will read their e-mail online and reply immediately from their computer. My observations of Amy, who did not have a computer printer at home, revealed a more complex way of working with e-mail.

Amy logged into a computer system in a campus lab and copied all of her e-mail messages into a word-processing file. She reported that she didn't want to waste paper; if she printed messages directly from the e-mail system, it would look like a lot to read, (although it took time to copy the messages). "After that, I delete the messages because it's too much e-mail." She reported that her routine work for B555 was to: print out all the e-mail messages for B555 in a word-processing document; print out all the readings for this course; read the e-mail messages and reading assignments at home. She would reply to messages on another day when she returned to the campus computer lab.

Another student, Eric, also commented about the overwhelming e-mail messages:

"I don't like, I have to say, I don't really like turning on the computer and finding that I have eleven messages on my e-mail. It's a pain. I mean to answer that things, just talking in conversation would be so much easier, rather than replying and doing all the stuff you have to do. So, that is just time-consuming, but it is a part of at a distance. I think if you are doing that, you have to be aware that you're gonna be spending more time with computer problems, not getting online, software, freaking out, crashing, whatever it's gonna happen, it gonna take you a lot longer, waiting in a line at a lab. There are so many things that make it kind of difficult to do." (personal communication, November 13)

Wegerif (1998) also reports a student's comment of a "daunting prospect" of being behind reading messages. It appeared that students in B555 were competing with each other, or felt a duty to produce a notable number of e-mail messages. Harasim's (1987) reports that students tend to spend longer online than the course requires and students' handling of e-mail may help to explain this phenomenon.

Understanding Students' Perspectives

No Physical Existence, No Feedback

In a Web-based distance education situation, students do not see each other or their instructors. This absence of physical cues led to some frustration among students.

John was working on one of the B555 activities in a computer lab and reading e-mail messages. He pointed out an e-mail message from the instructor and said:

I agree with her, but I am not sure if I should send a message saying, "I agree." That's the problem with this e-mail. If this is the classroom, you can just nod your head to show your agreement. I am not always sure that if I am contributing enough or not. Other people, like Julie and Kathy, are really active. I feel a sense of competitiveness. So, my survival skill is not to respond. In fact, I haven't gotten any feedback about my contribution. I cannot tell from the e-mail. You can tell from the classroom what the professor thinks about you from the body language and the way they talk. So, I am not feeling that I'm getting enough assessment. I haven't gotten any grade for this class, but most of the grade is from the portfolio, so it's OK, I guess. (personal communication, October 30).

Eric also indicated his frustration with not getting enough feedback:

One of the problems is that I'd like to have feedback. A kind of constant feedback. With the class, you don't really (.), especially this distance edu., I guess you don't get that kind of feedback. (personal communication, November 13).

Sheryl expressed the lack of immediate assistance from the instructor as well as the difficulty of finding information on the Internet.

Sheryl was working on her assignment for the B555 course. She started the Netscape browser, went to the Yahoo Education site, and typed, i.e.,"instruction for evaluating electronic learning." The computer responded:
There is no Web site to match your inquiry.
She looked unhappy. There was nothing she could find from this Web site regarding her keywords. (I thought she used too specific a phrase rather than a careful selection of keywords). It would be helpful if one of the class sessions taught tips and techniques about searching the Web.) Then she went to the AltaVista search engine Web site where she could search for Web sites by entering keywords. She explained, "one of my friends told me that AltaVista is much better." AltaVista helped her to locate one Web site, but that page didn't help her. She did another search with the keywords, "educational assessment." Then she found too many matchingpages, so she went to the refine page to narrow down her inquiry, e.g., byadding "assessments" and "education." She seemed to be finding a reasonable list of the sites for her topic.

She clicked the Net Search button on the Netscape menu and went to another search engine, InfoSeek. She explained, "I am not satisfied with these articles that I found so far, so I'm doing more research." She typed, "educational instruction and assessment." The list of Web sites did not satisfy her. Next, she typed "classroom instruction." She glanced over the list, but it did not meet her expectation. Then, she typed, "instruction for online language learning," which brought up a new list. She clicked one link which she did not like, nevertheless, she found a specific reference, the Computer-Assisted Language Learning page. This seemed relevant because this was the title of this class! However, when she tried the links on that page, none of the links were working. Whenever she clicked one, the error message appeared. She could not reach any Web sites and was not able to find the sites she was looking for. She told me that "this part is most frustrating; finding the information from the Web." She compared this experience with the other B555 class taught in the summer. She did not take the class, but had heard about it. "They had more resources. They saw a teacher in person, so they might have had the same problem, but not as much frustration as this.

Commentary

In contrast to his positive comments on the virtual fieldtrip, John seemed to be unsure about the appropriate attitude in an electronic environment. We assume that one of his frustrations was caused by a lack of feedback from the instructor. Because he could not see his instructor physically, and could not determine the instructor's expectations, his anxiety level might have increased. Eric had the same problem. This issue of lack of physical appearances is identified in the literature. Kuehn (1994), Harasim(1987), and McIsaac and Gunawardena (1996) caution that the lack of communication cues is a disadvantage of CMC because people cannot use any nonverbal cues, such as gestures and facial expressions.

We saw the same frustration with Sheryl. Her frustration came from weak skills in effectively searching the Web and a lack of immediate help. We observed her difficulties in skillful Web searching. One gap in this course may be the (tacit) assumption that graduate students in this program all have good online search skills. (The face-to-face version of B555 includes interns who can work at students' side to consult with them if they have technical problems.)

he lack of prompt feedback from the instructor was certainly a major source of frustration for students because they were concerned about their performance. According to McIssac and Gunawardena (1996), "the concept of interaction [including feedback] is fundamental to the effectiveness of distance education program as well as traditional ones" (p. 407). Bonk and Cummings (1998) also suggest the significance of feedback in Web courses. Later in the semester, the instructor realized that she needed to give more feedback to her students. In her weekly announcement on November 10, 1997, she apologized for not providing "enough and prompt feedback."

Technological Problems

During interviews, some informants reported the frustration with technological problems and the absence of personnel to provide technical support. Unlike the other students, the interviewee was taking the course from a distant site. She indicated three areas of frustration, the biggest of which were technology and the inflexibility of the course schedule.

First of all, inappropriate prerequisite statement. For example, there is nothing to say that you should know HTML, but our first assignment was creating a Web site. Fortunately, I knew it. I'd explored learning how to do HTML by myself. If I didn't know, I just cannot imagine how to get through. Secondly, this course is very time specific. The course I took before, I could go in anytime and finish anytime. However, this course is very specific in terms of time. For example, I got into the class a week late and the instructor sent me e-mail saying that they had already started. As an old learner, I felt so intimidated. I felt pressure to catch up. Third, accessibility to technology. This is related to the prerequisite. There is nothing that says we should have access to a Web server. However, when we developed the Web site as an assignment, we had to have the server access. Since I work for a school, one of the technical people helped me to connect to the Web server. If I didn't have these resources here, I would have dropped this course.

I don't have any access to the wonderful computer labs in the university. I don't have a [electronic] student locker and software that are available on campus. (personal communication, October, 31).

In addition to observations and interviews, students' online discussion through e-mail gave me insights into what was happening during the class. Because this course was offered completely through distance education, I had no opportunity to have classroom observation, although I observed students individually while they were working on their computers. Reading students' e-mail served as another substitute for physical classroom observation. Some students expressed honestly their anxieties and frustrations with the course in their messages. For example, on Friday, 17 October at 19:47:56, a student wrote the following message to the instructor regarding a technical problem:

I have spent one hour trying to follow your directions. I am getting an error message. The first time I tried to download it as a zip file, the error says, cannot access this file. I am getting extremely frustrated :(

On Saturday, 18 October at 3:06:15, she wrote another e-mail message regarding the same technology issue:

This computer is very frustrating. I would imagine it is like sitting in a class and only understanding some of what was said, then asked to answer a question. I have felt it ... panic ... isolation .... frustration ... anger. This has been a very good lesson. I will keep trying.

About 30 minutes after this message, on Saturday, 18 October at 3:27:05, this student sent out an e-mail message saying that she had solved this problem.

The instructor's personal logs offered a different perspective, and helped me better understand the class dynamics. The instructor expressed her problems and frustrations in the fact that she could not solve students' technical problems. For example, in one of her personal logs, she wrote:

I may need to understand more about how network and ISPs (Internet Service Provider) work. This to me is a hardware issue that I really did not want to touch and that I don't know how much help I could give to people. But, Julie and the previous two real distance students (who dropped out after several frustrated experiences) keep pushing me to this knowledge domain. (personal communication, October 18).

Because of these e-mail interactions, the instructor knew that students had difficulty dealing with technological problems and felt frustration. During the interview, she commented:

I think computer skill-wise, they [the students in B555] are not able to handle some of the assignments and exercises. And ordering, like we started from building Web pages with very minimum help, even though we provided them with very good, we thought, very good job-aids, but still they had difficulties. Help themselves learn. They are not in that kind of mode yet. They still need help. I guess both them and us, we are not used to this kind of environment at all. If you are in a classroom, a teacher can lead them during the process, so whenever they have problems, we can just fix it, right on a spot. However, if you give them the job-aids, if there is anything wrong there, there is no way we know. There is no way we can fix it right away and make it smooth for them. So that's frustrating for them and also frustrating to me because sometimes you feel that you've done everything you could, but just it doesn't work out that way. (personal communication, November 18).

Commentary

The student without direct access to technological hardware and support had to resolve technological problems that the other students did not experience. She was the only student who did not live near the university, and her frustrations were different from those of other students. She was as concerned about the technology as she was about the course content because her technical support was inadequate. However, some of the students also reported difficulties with technology during observations and interviews. Several studies (e.g. Burge, 1994; Gregor and Cuskelly, 1994; Kang, 1988; Wiesenberg and Hutton, 1995, November; Yakimovicz and Murphy, 1995) report students' frustration with technology during the evaluation of their distance education courses; but they do not thoroughly investigate it. However, the importance of computing support for professional work and even the public's use of the Internet has been well reported in other studies (see Kling and Jewett, 1991; Kling, 1999).

Pedagogical Issues - Ambiguous Instructions

Though I understand each sentence and word in the e-mail that the instructor sent us, I don't know how to use the instructions to compose the programming. Because in her instruction, sometimes I can follow steps 1 and 2, and then I can't follow from steps 2 to 3. So I go back to the beginning and start over. The instruction is all in text, no graphics because she sends it to us through e-mail ... So, when I submit my assignment, I always put a note to her, "please let me know if I need to do more or if I need to delete something" to make sure if I do the things that I am supposed to do. Because I don't know exactly what the instructor wants. (Amy, personal communication, November 11).

In this interview, Amy expressed her frustration in two ways. The first regarded the e-mail instructions that the instructor sent each week. The second frustration was caused by the ambiguous instructions on the WWW site. In our document analysis, we concurred that these instructions were too simple, and could be interpreted in many different ways. For instance, one of the activity instructions on the Web was:

1. Review the sample testware package that you have. What does it test? How do you think it facilitates learning? OR Visit one of these sites: [List of URLs]
2. Look at least two electronic portfolios (student works) in Student Project page [URL]. How would you give feedback to the student?
OR
3. To create a quiz on the Web, here are some tools you can use: [List of URLs]

The instructor intended to give students flexibility. However, some students did not consider this flexibility as an advantage. Sheryl, for example, did not think that she was getting effective instruction because of the ambiguous instructions on the WWW site and in e-mail messages from the instructor.

I: What's the biggest problem in this course?
S: I think the biggest problem is the instruction of our assignments. I usually don't understand what she wants, either e-mail or from the Web site. Actually I shared the print-outs with my friend. He is a doctoral student, and he looked at the instructions. He thought that our instructor was not a very good presenter because he also agreed that those instructions were so ambiguous that it's very confusing. There were no points at all. Sometimes, she takes all kinds of responses and she would say, "it's good you are creative," but sometimes I got her response that this is not what I want. So I felt very frustrated because we were supposed to be creative and that's what I came up with, but she said that's not what she wanted. That's the biggest problem.

I was expecting some theories of those conditions [that were taught in this course], but I didn't learn a lot because there was very little instruction on the Web. We have to learn from the class discussion, but the other classmates in this course are English teachers, foreign language education majors. They have more experience in this area, so they are talking something higher level, but I'm in lower level because I have no background in this field. So, I don't really catch them. I don't really catch the themes on these conditions. I don't even know the clearest definitions of those terms, but I think I learn a little, gained a sense of it. (personal communication, November 5).

Like Amy, Sheryl also gave up trying to clarify the instructor's expectation after asking her a few questions. Further, when I asked her what was the most frustrating thing, Sheryl answered:

Lack of teacher's support and teacher's clarification of her instruction. Usually I e-mail her if I have any questions and her answer is very ambiguous, too. So, I won't ask the second time. (personal communication, November 5).

Kathy's frustration was that she was uncertain what the instructor expected for this course because she could not see the instructor physically. She also gave me an example of how she misinterpreted the instructor's message in an e-mail message.

The instructor has been good about responding immediately when you ask something. However, I have been in school in my life and I didn't realize how much I relied on my knowledge of what teachers are looking for, sort of, you know. You sit in a classroom with somebody and you analyze who they are and what they like and you cannot analyze because you've never seen them. So, you are only guessing it what teacher really wants.

You don't know how to interpret what they say because you don't know the personality. Like one time, the teacher was making the joke and I took her seriously and it really hurt. She was saying that I can't remember what it was now, but something about that nobody is working ... since none of you are working at this, maybe we should do such and such and I wrote her back, "what do you mean we are not working. I am spending six hours a day and she wrote back that said, "it was only a joke." So, things like that, but I think if you are, like, very careful in what you write and communicate often with people, you can put them easily to get to know you. (personal communication, October 22).

While students felt frustrated with this course, the instructor was also frustrated. Overall, the students' evaluations of the instructor were positive. All the students appreciated the support from the instructor. Some students even sympathized with her because she also had to handle so many technological problems in this course. In addition, the distance education format also led to problems interpreting the students' messages. I asked the instructor if she received much e-mail about ambiguous expectations in this course. Her answer was yes. "Sometimes they do give me some of this [complain of ambiguous instructions], right, but you know sometimes you don't really know whether it is just an excuse or it is real. You don't know."

Commentary

The assignment instructions for B555 on the Web site provided opportunities to triangulate interviewees' comments during interviews and observations. The problems students reported about assignment instructions for B555 from the students were confirmed on the Web site. These problems were:

  1. Instructions for the assignments were too vague. Sometimes students did not understand what the instructor's expectations were.

  2. The Web site was developed for the same course offered in summer, and the summer course was offered in a traditional classroom, not by the Web-based distance education form. Therefore, some instructions for activities were for the course in summer. For example, one of the activities instructed students to make teams. However, in the Web-based distance education course, students had to work individually, so the instructions confused them.

  3. Because the Web site was developed earlier, some hyperlinks to the other Web sites were no longer accessible.

In addition, it seemed that students did not understand the instructor's expectations. The instructor knew that the instructions on the Web were too ambiguous, and she tried to make them clear. For the final project, she sent out an e-mail message: "I think we need a set of very clear criteria so that you and I know exactly what you are expected to do and how your project will be judged." (personal communication, November 10). However, this attempt did not succeed. A few students posted questions about her "clear criteria." Not all the students were familiar with technology, and some were already overwhelmed. Therefore, the ambiguous expectations for this course might have increased students' anxiety.

Dealing With Frustrations

The only way for Amy to resolve her frustration was to talk to a classmate of similar ethnicity in her own native language. Amy seemed to give up communicating with the instructor about her problems because she thought there was nothing the instructor could do to help. Thereafter, Amy complained only to her friend.

A: I am calling a friend every week, just to complain. She is a good listener, whenever I complained, she just listened and I felt better.
I: Did you complain to your instructor?
A: Once.
I: Why just once?
A: I complained once about the difficulty of searching on the Web, and she gave me the tips for searching as I told you before. After that, I didn't complain because I felt stupid. I should have spent more time on this, but I couldn't because I'm too busy. If I hadn't taken this many courses and also work, I could (.) if you want to take this course, you have to spend time. I want to complain, but it's not the instructor's problem, or the class's fault. It's my problem. There is nothing she can do about it. (personal communication, November 11).

Despite all the frustrations encountered, John expressed a different view in an informal conversation. He believed that it was a good opportunity for him to have these frustrating experiences because now he understood what his students might experience when he teaches similar courses in the future. Furthermore, the instructor tried to help her students resolve their frustrations. Later in the semester, she started to ask students for their suggestions to improve tutorials and teaching materials. She got the impression that now students felt less frustrated. One of her messages to the students was:

I'm more comfortable to let you face flaws now than before, because I feel that: (1) You have experience solving this level of computer problems. (2) You know that learning can come from failure and frustrations. (3) The communication channel that we happen to so luckily have helped too. (personal communication, November, 1).

Also, she stated the same issue in her personal log:

It was from the MOO week that I started asking them for improvement ideas, and it seems to me that this opened a new door for communication ... . All of a sudden they agreed that it is all right to be frustrated when following instructions that are with flaws, because flaws give opportunities to think and to gain real control ... . (personal communication, November 1).

Commentary

If students could deal with these frustrations, this might not be a negative experience, as the instructor said. In fact, the students supported each other by sharing their frustrations personally or with all their classmates. We also believe that without this mutual support, none of the students would have gotten through this course. Some students felt a community of learning with their classmates. Bates (1994) claims that one of the major contribution of two-way technologies is allowing interactions among students as well as between students and instructors, and it is true that there was active interaction among students in this course. Many researchers assert the importance of virtual community to support students (e.g. Burge, 1994; Jonassen, Davidson, Collins, Campbell and Haag, 1995; McIssac and Gunawardena, 1996; Savard, Mitchell, Abrami and Corso, 1995). In this case study, however, it seemed that some students never overcame their frustrations, despite a great deal of interaction with others. Such frustrations must not be ignored. Hanna (1998), Koble and Bunler (1997), Mason and Kaye (1990), and Rahm and Reed (1998) are among the few researchers who have addressed the lack of student support in distance education.

Discussion

From interviews and observations (thinking aloud), two interpretations were formed in this study. It appeared that there were two foci of frustration among students in this course. The first focus was technological problems; students without access to technical support were especially frustrated. Students whose computer skills were inadequate, like Amy, also faced persistent technological problems. The second focus involved the course content and the instructor's practices in managing her communications with her students. Students were frustrated because of a lack of immediate feedback from the instructor and ambiguous instructions on the Web and via e-mail.

The instructor did not appreciate the duration of the students' frustration. She believed that she had reduced the students' frustration and noted during an interview:

They [the students] thought that the problems they had were basically their own; other people did not have the same problem until we opened up the conversation and they realized that, oh, yeah, we were all in the same boat. Now, they have this peer support coming in. That [problem], I think, we took care of pretty well. (personal communication, November 18).

However, her students still expressed their frustration earnestly during observations and interviews late in the semester. Part of the reason for the instructor's misperception resulted from the fact that the students' e-mail regarding their frustrations were only the tip of the iceberg. Students did not express all of their frustrations.

In summary, in this distance education course, students' frustration originated from three sources:

We found that these frustrations so overwhelmed some students that they gave up on the formal content of the course. The instructor's personal reflection note revealed that two other students who began taking this course from distant sites dropped it because they could not overcome technical problems. In addition, during interviews two students claimed that they will not take distance education courses in the future in order to avoid these frustrations. Therefore, students' frustrations were serious problems in this distance education course.

Conclusion

We conclude by cautioning about advertising only the virtues of computer-mediated distance education. Most of the articles about distance education that are written for people who are not deeply immersed in distance education research emphasize the positive aspects of distance education (e.g., Barnard, 1997; Harasim, 1993; Yakimovicz and Murphy, 1995). Only a few scholars (e.g., Bromley and Apple, 1998; Jaffee, 1998; Wegerif, 1998) examine important limitations and pervasive problems. It is acceptable to fantasize about the future when a field is young, because these discussions can propel the field forward. Distance education has great potential for providing rich environments for students; however, as history has taught us, new technology is not a panacea. It has trade-offs.

It is time to seriously consider the actual experiences among students in distance education courses and to critically discuss the phenomena of distance education. As Bates (1994) states, "it is a relatively untested assumption that advanced technologies, ... , are pedagogically more effective than older" (p. 1577) technologies. We also question if technology can improve pedagogy with little special effort. For more than a decade Clark (1983; 1994) has raised the arguments of whether or not media influence better learning.

It is easy to place the burden of students' frustrations wholly upon the instructor's limitations. With an experienced and skilled instructor, the students would have found the online version of B555 to be a valuable delight! There is good reason to believe that many online courses are taught sufficiently well that the students value them and do not experience the kinds of frustrations that we discovered in this case.

One might argue that this course was a unique case of a poor instructor poorly teaching an online course, and that this "oddball case" tells us nothing about online courses. We differ with this last interpretation. The department chair had taught some online courses and his department had notable experience with online courses taught by several of his faculty. He could have cancelled the online course if he could not find a competent instructor. He could have sought mentoring help for the instructor. Alternatively, she might have sought advice from the faculty about improving her teaching of this online course. None of these alternatives were enacted. It would be remarkable if this were the only time that an academic administrator mis-perceived the pedagogical capabilities of a replacement instructor when faced with the loss of the original instructor. It appears that even an experienced administrator and online teacher also mis-perceived the kinds of pedagogical shifts required from face-to-face teaching, and could underestimate the extent to which mentoring could be critical. Certainly, these issues arise in traditional face-to-face courses.

Unfortunately, a large fraction of the articles about computer-mediated distance education emphasize the potentials of new technology, and understate the extent to which instructors may need to develop new pedagogies as well as different approaches to managing their online courses. High quality online education is neither cheap nor easy. This "technological utopianism" (Kling, 1994) is also found in the history of educational technology in general. Clearly, we need more student-centered studies of distance education. We need research that is designed to teach us how the appropriate use of technology and pedagogy could make distance education beneficial for students. "[I]f failure isn't a possibility, neither is discovery" (Unsworth, 1997).

About the Authors

Noriko Hara is a research associate in Instructional Systems Technology at Indiana University. Her research agenda includes the investigation of organizational learning,communities-of-practice, and online learning within social informatics. Additionally she designs and develops information architecture for educational Web sites. Her recent publications concern the use of information technologies to support learning environments. She is currently writing her dissertation on professional communities-of-practice.
E-mail: nhara@indiana.edu

Rob Kling is Professor of Information Science and Information Systems at Indiana University, Bloomington where he also directs the Center for Social Informatics. He is Editor-in-Chief of The Information Society and has written and edited the second edition of Computerization and Controversy: Value Conflicts and Social Choices. His research focuses on topics within social informatics, such as the roles of digital libraries and electronic publishing in improving professional communication and organizational change in a digital economy. His current research, Scientific Communication and the Shaping of Knowledge Networks is funded by a NSF KDI grant.
E-mail: kling@indiana.edu

Acknowledgements

This study was made possible with the help of the instructor of B555. In addition, we have benefited from discussions Curt Bonk, Andy Feenberg, Carrie Heeter, Linda Mabry, Mike Molenda and Alice Robbin.

This paper will be published in a slightly altered form in the Routledge journal Information, Communication and Societyin 2000; for further details on the journal, visit http://www.infosoc.co.uk/index.htm.

Notes

1. B555 is a pseudonym used in this study.

2. The research on the use of WWW is included in the field of CMC.

3. Originally eight students registered for B555. However, two students who were taking it far from the university dropped after suffering from technological problems.

4. There was a substantial debate about research in cyberspace in 1995, after a research team at Carnegie Mellon University published a study of pornographic picture exchange on the Internet (see Kling, 1996). The study obtained users' information tracking by computer networks without their permissions and violated their privacy (Thomas, 1995, 1996).

5. MUD is an acronym for "Multi-User Dungeon" developed for multi-players to play Dragons game in 1979 (Bruckman and Resnick, 1998).

6. I indicates the interviewer and the initials of the interviewees are used in this paper.

7. This mark is used frequently in text-based communication to show the message sender's sad feeling.

References

C.E. Abrahamson, 1998. "Issues in interactive communication in distance education," College Student Journal, volume 32, number 1, pp. 33-43.

T.C. Ahern and J. Repman, 1994. "The Effects of technology on online education," Journal of Research on Computing in Education, volume 26, number 4, pp. 537-546.

P.A. Alexander and P.K. Murphy, 1994. "The Research based for APA's learner-centered psychological principles," Paper presented at the American Educational Research Association, New Orleans (April).

American Psychology Association, 1997. "Learner-centered psychological principles: A framework for school redesign and reform," at http://www.apa.org/ed/lcp.html

J. Barnard, 1997. "The World Wide Web and higher education: The promise of virtual universities and online libraries," Educational Technology, volume 37, number 3, pp. 30-35.

A.W. Bates, 1994. "Distance education, educational technology," In: T. Husen and T.N. Postlethwaite (editors). The International Encyclopedia of Education. Second edition. Oxford: Elsevier Science, pp. 1573-1580.

Z. Berge, 1997. "Characteristics of online teaching in post-secondary, formal education," Educational Technology,, volume 37, number 3, pp. 35-47.

H. Besser and S. Donahue, 1996. "Introduction and overview: Perspectives on . . . distance independent education," Journal of the American Society for Information Science, volume 47, number 11, pp. 801-804.

L. Blankenship, 1993. "The Cow ate my brain or A novice's guide to MOO programming, Part I," (September), at http://www.cs.rdg.ac.uk/people/mkh/virtual_worlds/MOO/tutorials/mootutor1.html

C.J. Bonk and J.A. Cummings, 1998. "A Dozen recommendations for placing the student at the centre of Web-based learning," Educational Media International, volume 35, number 2, pp. 82-89.

C.J. Bonk and K.A. Kim,1998. "Extending sociocultural theory to adult learning," In: M. C. Smith and T. Pourchot (editors). Adult learning and development: Perspectives from educational psychology. Mahwah, N.J.: Erlbaum, pp. 67-88.

H. Bromley, and M.W. Apple, 1998. Education/Technology/Power: Educational computing as a social practice. Albany, N.Y.: SUNY Press.

C.A. Bowers, 1988. The Cultural dimensions of educational computing: Understanding the non-neutrality of technology. New York: Teachers College Press.

K.M. Brown, 1996. "The Role of internal and external factors in the discontinuation of off-campus students," Distance Education, volume 17, number 1, pp. 44-71.

A. Bruckman and M. Resnick, 1998. "Virtual professional community: Results from the MediaMOO project" (January), at ftp://sunsite.unc.edu/pub/academic/communications/papers/muds/moo/MediaMOO-3cyberconf.txt

M Bryson and S. de Castell, 1998. "Telling tales out of school: Modernist, critical, and postmodern "true stories" about educational computing," In" H. Bromley and M.W. Apple (editors). Education/Technology/Power: Educational computing as a social practice. Albany, N.Y.: State University of New York Press, pp. 65-84.

E.J. Burge, 1994. "Learning in computer conferenced contexts: The learners' perspective," Journal of Distance Education, volume 9, number 1, pp. 19-43.

R.B. Clark, 1994. "Media will never influence learning," Educational Technology Research and Development, volume 42, number 2, pp. 21-29.

R.E. Clark, 1983. "Reconsidering research on learning from media," Review of Educational Research, volume 53, number 4, pp. 445-459.

P.G. Cove and A.G. Love, 1996. "Enhancing student learning: Intellectual ,social, and emotional integration," EDO-HE-95-4. Washington, D.C.: ERIC Clearinghouse on Higher Education.

M.V. Covington, 1993. "A Motivational analysis of academic life in college," Higher Education: Handbook of theory and research, volume 9, pp. 50-93.

L. Cuban, 1986. Teachers and machines: The classroom use of technology since 1920. New York: Teachers College Press.

S. Darke, 1988a. "Anxiety and working memory capacity," Cognition and emotion, volume 2, number 2, pp. 145-154.

S. Darke, 1988b. "Effects of anxiety on inferential reasoning task performance," Journal of Personality and Social Psychology, volume 55, number 3, pp. 499-505.

C. Dede, (1996). Emerging technologies in distance education for business," Journal of Education for Business, volume 71, number 4, pp. 197-205.

R.M. Diamond, 1997. "Curriculum reform needed if students are to master core skills," Chronicle of Higher Education, volume 43 (August 1), p. B7.

A. Dillon, 1994. Designing usable electronic text: Ergonomic aspects of human information usage. London: Taylor and Francis.

H.L. Dreyfus and S.E. Dreyfus, 1984. "Putting computers in their proper place: Analysis versus intuition in the classroom," Teachers College Record, volume 85, number 4, pp. 578-601.

ERIC, 1998. "Clearinghouse on Higher Education: Instruction," at http://www.eriche.org/Library/instruct.html

A.K. Ericson and H.A. Simon, 1984. Protocol analysis: Verbal reports as data. Cambridge, Mass.: MIT Press.

A. Feenberg, 1987. "Computer conferencing and the humanities," Instructional Science, volume 6, number 2, pp. 169-186.

L.F. Gardiner, 1997. "Producing dramatic increases in student learning: Can we do it?" National Teaching and Learning Forum, volume 6, number 2, pp. 8-10.

C. Geertz, 1973. The Interpretation of cultures. New York: Basic Books.

S.D. Gregor and E.F. Cuskelly, 1994. "Computer mediated communication in distance education," Journal of Computer Assisted Learning, volume 10, pp. 168-181.

C.N. Gunawardena, 1992. "Changing faculty roles for audiographics and online teaching," The American Journal of Distance Education, volume 6, number 3, pp. 58-71.

S.E. Hackbarth, 1997. "Web-based learning (Special issue)," Educational Technology, volume 37, number 3.

C. Handy, 1998. "A Proper education," Change, volume 30, number 5, pp. 12-19.

D.E. Hanna, 1998. "Higher education in an era of digital competition: Emerging organizational methods," Journal of Asynchronous Learning Networks, volume 2, number 1, at http://www.aln.org/alnweb/journal/vol2_issue1/hanna.htm

N. Hara, C.J. Bonk and C. Angeli, (in press). "Content analysis of an on-line discussion in an applied educational psychology course," Instructional Science.

L.M. Harasim, 1987. "Teaching and learning on-line: Issues in computer-mediated graduate courses," Canadian Journal of Educational Communication, volume 16, number 2, pp. 117-135.

L.M. Harasim, 1990. "Online education: An environment for collaboration and intellectual amplification," In: L.M. Harasim, (editor). Online education: Perspectives on a new environment. New York: Praeger, pp. 39-64.

L.M. Harasim, 1993. "Networkd: Networks as social space," In: L.M. Harasim, (editor). Global networks: Computers and international communication. Cambridge, Mass.: MIT Press, pp. 15-34.

F. Heylighen, 1992. "Evolution, selfishness and cooperation: Selfish memes and the evolution of cooperation," Journal of Ideas, volume 2, pp. 70-84, at ftp://ftp/vub.ac.be/put/projects/Principia_Cybernetica/Papers_Heylighen/MemesandCooperation.txt

S.R. Hiltz, 1998. Teaching in a virtual classroom. Volume 2: A Virtual classroom on EIES: Final evaluation report. Newark, N.J.: New Jersey Institute of Technology.

D. Jaffee, 1998. "Institutionalized resistance to asynchronous learning networks," Journal of Asynchronous Learning Networks, volume 2, number 2, at http://www.aln.org/alnweb/journal/vol2_issue2/jaffee.htm

S.M. Johnstone and B. Krauth, 1996. "Balancing quality and access: Some principles of good practice for the virtual university," Change, volume 28, number 2, pp. 38-41.

D. Jonassen, M Davidson, M. Collins, J. Campbell and B.B. Haag, 1995. "Constructivism and computer-mediated communication in distance education," American Journal of Distance Education, volume 9, number 2, pp. 7-26.

D.H. Jonassen and B.L. Grabowski, 1993. Handbook of individual differences, learning, and instruction. Hillsdale, N.J.: Erlbaum.

I. Kang, 1988. "The Use of computer-mediated communication: Electronic collaboration and interactivity," In" C. J. Bonk and K. King (editors). Electronic collaborators: Learner-centered technologies for literacy, apprenticeship, and discourse. Mahwah, N.J.: Erlbaum, pp. 315-337.

S. Kiesler, J. Siegeland and T.W. McGuire, 1984. "Social psychological aspects of computer-mediated communication," American Psychologist, volume 39, number 10, pp. 1123-1134.

R. Kling, 1984. "Value Conflicts in Public Computing Developments," The Information Society, volume 3, number 1, pp. 1-38, and at http://www.slis.indiana.edu/TIS

R. Kling, 1994. "Reading "all about" computerization: How genre conventions shape non-fiction social analysis," The Information Society, volume 10, number 3, pp. 147-172, and at http://www-slis.lib.indiana.edu/kling/read94a.html

R. Kling, 1996. "The Ethics of fair practices for collecting social science data in cyberspace [Special issue]. The Information Society, volume 12, number 2, and at http://www.slis.indiana.edu/TIS

R. Kling, 1999. "What is Social Informatics and Why Does it Matter?" D-Lib Magazine, volume 5 number1, at http://www.dlib.org:80/dlib/january99/kling/01kling.html

R. Kling and T. Jewett, 1991. "The Dynamics of Computerization in a Social Science Research Team: A Case Study of Infrastructure, Strategies, and Skills," Social Science Computer Review, volume 9, number 2 (Summer), pp. 246-275.

R. Kling and Tillquist, 1998. "Conceiving IT-Enabled organizational change," Unpublished manuscript.

M.A. Koble and E.L. Bunler, 1997. "Trends in research and practice: An examination of The American Journal of Distance Education 1987 to 1995," American Journal of Distance Education, volume 11, number 2, pp. 19-38.

S.A. Kuehn, 1994. "Computer-mediated communication in instructional settings: A Research agenda," Communication Education, volume 43, pp. 171-183.

R. Mason and T. Kaye, 1990. "Toward a new paradigm for distance education," In: L.M. Harasim (editor). Online education: Perspectives on a new environment. New York: Praeger, pp. 15-38.

M.S. McIssac and C.N. Gunawardena, 1996. "Distance education," In: D. Johnassen (editor). Handbook of research for educational communications and technology. New York: Macmillan, pp. 403-437.

National Center for Education Statistics, 1998. "Issue brief: Distance education in higher education institutions: Incidence, audiences, and plans to expand," at http://nces.ed.gov/pubs98/98132.html

S. Nipper, 1989. "Third generation distance learning and computer conferencing," In: R. Mason and A. Kaye (editors). Mindweave: Communication, computers and distance education. Oxford: Pergamon, pp. 53-73.

D.F. Noble, 1998. "Digital diploma mills: The Automation of higher education," First Monday, volume 3, number 1 (January), http://firstmonday.org/issues/issue3_1/noble/index.html

T. Oppenheimer, 1997. "The Computer delusion," Atlantic Monthly, (July), pp. 45-62.

R.D. Owston, 1997. "The World Wide Web: A Technology to enhance teaching and learning?" Educational Researcher, volume 26, number 2, pp. 27-33.

R. Prus, 1996. Symbolic interactionism and ethnographic research: Intersubjectivity and the study of human lived experience. Albany, N.Y.: SUNY Press.

R.G. Ragsdale, 1988. Permissible computing in education: Values, assumptions, and needs. New York: Praeger.

D. Rahm and B.J. Reed, 1998. "Tangled webs in public administration: Organizational issues in distance learning," Public Administration and Management, volume 3, number 1.

A.S. Reber, 1985. Penguin dictionary of psychology. London: Penguin.

B. Reeves and C. Nass, 1996. Media equation: How people treat computers, television, and new media like real people and places. New York: Cambridge University Press.

J.M. Roberts, 1996. "The Story of distance education: A Practitioner's perspective," Journal of the American Society for Information Science, volume 47, number 11, pp. 811-816.

A. Romiszowski and R. Mason, 1996. "Computer-mediated communication," In: D. Johnassen (editor). Handbook of research for educational communications and technology. New York: Macmillan, pp. 438-456.

J.E. Roueche, 1998. "American Imperative-Essay, Virtual Library," at http://www.johnsonfdn.org/library/foundpub/amerimp/essayx.html

G. Salomon, 1985. "Information technologies: What you see is not (always) what you get," Educational Psychologist, volume 20, number 4, pp. 207-216.

M. Savard, S.M. Mitchell, P.C. Abrami and M. Corso, 1995. "Learning together at a distance," Canadian Journal of Educational Communication, volume 24, number 2, pp. 117-131.

C.C.E. Schroeder, 996. "The Student learning imperative" (Special issue), Journal of College Student Development, volume 37, number 2.

D. Silverman, 1996. Interpreting qualitative data: Methods for analysing talk, text and interaction. London: Sage.

B. Simpson, 1985. "Heading for the ha-ha," In: D. Sloan (editor). The Computer in education: A Critical perspective. New York: Teachers College Press, pp. 84-92.

D. Sloan (editor), 1985. The Computer in education: A Critical perspective. New York: Teachers College Press.

F.K. Stage, 1996. "Setting the context: Psychological theories of learning," Journal of College Student Development, volume 37, number 2, pp. 227-235.

M. Stahlman, 1996. "Prisoners to technology?" Information Week (December 9), p. 126.

R.E. Stake, 1994. "Case studies," In: N.K. Denzin and Y.S. Lincoln (editors). Handbook of qualitative research. Thousand Oaks, Calif.: Sage, pp. 236-247.

R.E. Stake, 1995. The Art of case study research. Thousand Oaks, Calif.: Sage.

R.E. Stake and L. Mabry, 1995. "Case study for a deep understanding of teaching," In: A. Ornstein (editor). Research on teaching. Boston: Allyn and Bacon, pp. 2-11.

R.M. Starr, 1997. "Delivering instruction on the World Wide Web: Overview and basic design principles," Educational Technology, volume 37, number 3, pp. 7-15.

A. Strauss and J. Corbin, 1994. "Grounded theory methodology," In: N.K. Denzin and Y.S. Lincoln (editors). Handbook of qualitative research. Thousand Oaks, Calif.: Sage, pp. 273-285.

J. Thomas, 1995. "The Ethics of Carnegie Mellon's "cyber-porn" study," at http://wwwsoci.nui.edu/~jthomas/ethics.cmu

J. Thomas, 1996. "When cyberresearch goes awry: The Ethics of the Rimm "cyberporn" study," The Information Society, volume 12, number 2, pp. 189-198, and at http://www.slis.indiana.edu/TIS

C.A. Twigg, 1997. "Is technology a silver bullet?" Educom Review (March/April), pp. 28-29.

J. Unsworth, 1997. "The Importance of failure," Journal of Electronic Publishing, volume 3, number 2, at http://www.press.umich.edu/jep/03-02/unsworth.html

R. Wegerif, 1998. "The Social dimension of asynchronous learning networks," Journal of Asynchronous Learning Networks, volume 2, number 1, at http://www.aln.org/alnweb/journal/vol2_issue1/wegerif.htm

F. Wiesenberg and S. Hutton, 1995. "Teaching a graduate program using computer mediated conferencing software," Paper presented at the Annual Meeting of the American Association for Adult and Continuing Education (November), Kansas City, Mo.

M. Windschitl, 1998. "The WWW and classroom research: What path should we take?" Educational Researcher, volume 27, number 1, pp. 28-33.

Wingspread Group on Higher Education, 1993. An American imperative: Higher expectations for higher education. Racine, Wis.: Johnson Foundation, at http://johnsonfdn.org/library/foundpub/amerimp/index.html

H.F. Wocott, 1990. Writing up qualitative research. Newbury Park, Calif.: Sage.

A.D. Yakimovicz and K.L. Murphy, 1995. Constructivism and collaboration on the Internet: Case study of a graduate class experience, Computers and Education, volume 24, number 3, pp. 203-209.


Editorial history

Paper received 12 November 1999; accepted for publication 22 November 1999; revision 28 November 1999; revised 7 December 1999; revised 9 December 1999


Contents Index

Copyright © 1999, First Monday

Students' Frustrations with a Web-Based Distance Education Course by Noriko Hara and Rob Kling
First Monday, volume 4, number 12 (December 1999),
URL: http://firstmonday.org/issues/issue4_12/hara/index.html