This paper reports on the use of focus groups in the design and development of America's Talent Bank. America's Talent Bank was designed with support from the U.S. Department of Labor as a nation-wide labor exchange system to be used on the Internet. The use of focus groups significantly enhanced the design and development of this system. The authors believe that this case is reflective of the successful use of focus groups in the development of an Internet system designed for national use.
Contents
Purpose
Trends in Information Services
Project Background
Purpose of the Focus Groups Study
Methodology
Results of the Focus Group Activities
Information Requirements
Conclusions and Implications for the Design of Service-Based Web Systems
Purpose
This paper reports on the experiences and associated benefits of using focus groups in the development of America's Talent Bank. The authors believe that the use of focus groups in the design and development of information systems can provide very useful benefits. America's Talent Bank is an online labor exchange system sponsored by the U.S. Department of Labor and supported at the state and local level. Thus, we have a national system that operates locally. The system that is now fully operational was significantly guided by the findings of the focus groups.
Trends in Information Services
Trends
There are several trends that will characterize information-intensive services in general, and government service in particular, during the next decade. These trends are: 1) greater competition for customers/clients; 2) more efficient management; 3) a "one-stop" seamless interface between agencies and services; and, 4) a closer integration at national, regional, and local levels while networking internationally ( JISC, 1995). To address these changes, organizations must be capable of responding quickly to changes in information needs. This research reports on a study conducted using focus groups to provide an active research approach for identifying information needs in the design and implementation of a national labor exchange system.
Impetus For The Use of Focus Groups
Few would argue that these new directions require fundamental rethinking about the purpose and structure of government services. The dilemma for government agencies is exacerbated, for they are experiencing relative financial contraction with increase in demand. How to delivery these services within those confines is a key issue facing these agencies. With the proliferation of information technology and the Internet, online Web-based systems are providing a vehicle for delivering those services. In approaching the design of a national labor market exchange system, the U.S. Department of Labor and the consortium states recognized the need for adequate input by stakeholders in the design of such a system.
Over a decade ago, Kirkpatrick (1985) provided a well-synthesized review of the key attributes to effecting successful change - communication and participation. In actual practice, these attributes are frequently overlooked as critical to the success of Web-based systems. Too often new Web-based systems are introduced by executive decision, through a centralist management strategy, or through ad hoc and hurried planning interventions in response to years of neglect.
A change initiative such as a Web-based system that provides a significant delivery service change, needs to be deliberated by all those who may be impacted. Bolman and Deal (1991) argue that each issue needs to be matched with an appropriate response using a multi-frame strategy and that without such an approach, the intended changes will fail or backfire. Learning how to use teams for reflective dialogue where meanings can be constructed is critical to the building of a team ( Bensimon and Neumann, 1994). This thinking together "is likely to involve listening to voices that have not traditionally been at the center of the decision process rather than favoring conventional (and dominant) views." Mintzberg's ( Mintzberg, Raisinghani, & Thoret, 1976) concept of "emergent strategy formation" may be particularly appropriate in the early stages of developing a Web-based system. In practical terms this may mean holding discussions with diverse focus groups or circulating a position paper to start the debate, that is, obtaining input from all key stakeholders, synthesizing the information, and bringing forward recommendations.
Group discussion and subsequent brainstorming should result in innovative ideas surfacing from non-traditional sources. The formative stages of Web-based system development can benefit from Mintzberg's ( Mintzberg, Raisinghani, & Thoret, 1976) concept of "emergent strategy formation." This strategy may entail conducting focus groups with participants of diverse backgrounds or initiating these discussions by distributing a position paper as a "feeder" to jump-start conversation.
Precautions in the Use of Focus Groups
If focus groups are to provide useful information it is necessary to use valid and effective methods. Selection of facilitators and selection of the focus group members are critical to ultimate success. If possible, an experienced and properly trained contractor should be selected to conduct the focus groups. Adequate planning time must be provided. Dreachelin (1998) provides some background on the theoretical underpinnings of focus groups. Morgan (1993) and Krueger (1994) both provide instructions on the use of focus groups. Neilson (1997) provides some insights on the use and misuse of focus groups. It is also important to identify all stakeholder groups so that all can be represented. Finally, it is necessary to conduct sessions with multiple focus groups.
It is also important to recognize that the use of focus groups does not change the fundamental process of systems development. Thus, a well-trained and experienced systems development team is essential to a successful system. Further, it is also critical that members of the organization familiar with the goals of the system be a part of the development team.
Project Background
The Department of Labor sponsored development of an electronic Talent Bank Network as part of America's Labor Market Information System (ALMIS). Consortium co-leaders Michigan and Missouri worked towards the Department of Labor's goal of developing the Talent Bank Network into a national electronic resume system. The U.S. Department of Labor envisioned the Talent Bank as a qualifications database that could be created, searched and screened by its users:
- Individuals seeking employment would create a document or resume listing their qualifications and experiences and "deposit" it into the Talent Bank database.
- Employers would use a keyword search to screen resumes and select qualifying resumes for further review.
Employers could benefit when using the Talent Bank's broader pool of applicants, and when utilizing the automated screening process through the application of logical criteria. For job seekers, the advantages include reaching a larger and potentially more geographically dispersed employer audience and shortening the time spent in job searching.
Foremost in the consortium's focus was integrating the Talent Bank system with existing employment service operations offered through local offices and one-stop centers to ensure a smooth implementation process.
Purpose of the Focus Group Study
The consortium's main goal was to design the Talent Bank system around user needs. To define what those user needs were, the consortium contracted with the Bureau of Labor Statistics' ( BLS) Office of Research and Evaluation (ORE), Westat, and the American Institutes for Research ( AIR) to conduct a series of focus group meetings with potential users in 10 states (Connecticut, Georgia, Maryland, Michigan, Missouri, Nebraska, North Carolina, Ohio, Puerto Rico, and Utah), summarize the results for each state, and integrate those results across the states (Latta et al., 1995).
The consortium defined three potential user groups for the study: employers, job seekers, and employment brokers. Employment brokers were identified as public and private agencies which match employers with job seekers. Qualifying groups would include temporary employment agencies, search firms, state job services, and college and university placement services.
BLS developed the focus group protocols and conducted pilot focus groups in the first two states (Michigan and Missouri). Westat and AIR conducted focus groups in the remaining eight states.
Protocol Topic Areas
Protocols covered the following main topics:
- Current activities used to recruit (employers), match jobs and candidates (brokers), or provide information about job skills/experience (job seekers).
- Use of online systems to conduct recruiting, matching, and job search activities.
- Key information that potential users want to include in the Talent Bank system.
- Feedback about a prototype resume system presented to group members (which served as a stimulus for discussing resume entry and search features and process).
- Confidentiality issues and concerns.
- Operating procedures for the Talent Bank database.
- Most important issues to consider in developing the Talent Bank.
Methodology
Study Method
The focus group method was used to seek input on a variety of system design, development, and service delivery issues. Within a focus group setting, a facilitator can use a structured protocol to facilitate group discussion and to obtain qualitative, affective information from participants. In each state, the state job service staff selected participants. These viewpoints are not necessarily representative of the general population.
Focus Group Participant Sample
The state of Michigan established guidelines to be used when selecting the participant sample. To qualify as participants within the employer and employment broker focus group sessions, participants had to be involved in personnel, and have hiring authority and policy-making responsibilities. Additionally, each state followed the following recruitment guidelines to ensure a representative mix of participants:
Employers
Employment Brokers
Job Seekers
Size: large vs. small
Public sector (One-Stop Centers, Employment Service, JTPA)
Variety of Contacts (employment agencies, training agencies, & community-based organizations)
Industry type: manufacturing, non-manufacturing, service sector, etc.
Private sector
Professional and non-professional occupations
Location: urban vs. rural
Educational institution placement services
Employment status
Placement focus: temp vs. perm
Demographics
Focus Group Participants
Twenty-four focus group meetings were conducted, with a total of 202 participants. The participant sample guidelines were adhered to during the selection process. A variety of industries were represented across employer groups (Manufacturing, health care, education, computer, technical banking, communications, and services). Employment brokers represented both public and private sectors. Job seekers were searching for employment in a variety of occupations (service, technical, managerial, professional and semi- and un-skilled jobs.)
Focus Group Procedures
Each focus group session began with a standard introduction that explained the purpose of the session. Group participants introduced themselves and identified their business area if they were employers or brokers; and job seekers stated their job title or the type of job for which they were seeking employment. Sessions were lead by co-moderators. Notes were taken and backup audiotapes were utilized. Participants addressed the first three protocol topic areas and then were shown a presentation of the resume system prototype. After discussion of the prototype, the final three protocol topic areas were discussed. Participants were asked to provide their views on the prototype and to volunteer any summary comments. Participants in the job seeker sessions were compensated for their time; other participants were not, with the exception of Utah, where participants in all three groups were compensated).
Results of Focus Group Activities
Current Job Market Experiences
The first objective was to catalogue current experience in matching job seekers and employers of the eight employer focus groups, of the eight job seeker groups, and of the eight broker groups. The goal was to elicit the experiences without reference to specific online system and to look at the current experience with an appreciation of existing online systems.
General Experiences in Job Matching
By far the most common methods of identifying and making contacts are via networking, print advertising, and state job services. Networking includes introductions and contacts via family, friends, teachers, union representatives, and others. Print advertising includes newspapers, posters, flyers, and trade journals. State job services include recruitment, screening, referral, and unemployment services.
Overwhelmingly, un-skilled, semi-skilled and skilled trades are handled on a local basis. In general, managerial and technical jobs are handled on a local basis if sufficient supply is available locally. If not, a statewide or national search is conducted. The majority of all focus group participants were interested in new methods of matching job seekers and employers.
Use and Appreciation of Online Systems
Approximately 50% of the participants reported having some knowledge of online systems. However, only 25% of all participants had used online systems. Approximately 30% of the job seekers had searched for jobs via an online system and fewer had actually entered a resume on such a system. The feedback on perceived advantages and disadvantages of online systems is shown on the following table ( Table 1). It should be noted that this is a summary of the responses received without screening or editing. No attempt was made to identify either the frequency or the depth of feeling behind the remarks. The table lists perceived advantages and disadvantages for each category of focus groups (employers, employment brokers, and job seekers) by users who have previously used online systems and by non-users.
Table 1: Advantages and Disadvantages of Online Systems
Advantages for Employers
Users
Non-Users
Advantages for Brokers
Users
Non-Users
Advantages for Job Seekers
Users
Non-Users
Increased efficiency in selection
x
x
Eases job matching process via standardized categories
x
Eeasy to use
x
Better search for specialized/technical skills
x
x
Inexpensive to use
x
Convenient and accessible
x
Search and sort with keywords
x
x
Work can be done at home
x
Information available quickly
x
Candidates can be from state or nation
x
Information is readily available
x
Greater scope in the search: other cities, states, or countries
x
x
Additional source of candidates
x
Allows use of specific search criteria
x
More employment opportunities
x
x
Convenience
x
Reduction in paperwork
x
Search can be tailored
x
x
Standardized job classifications
x
Disadvantages for Employers
Users
Non-Users
Disadvantages for Brokers
Users
Non-Users
Disadvantages for Job Seekers
Users
Non-Users
Not cost effective
x
x
Difficult to verify accuracy and authenticity
x
x
Resume builder can be inconsistent from system to system; information can be inconsistent from resume to resume
x
Time-consuming to screen info
x
x
System was impersonal, "cold" relative to hard copy resumes
x
x
Accessibility may be inconvenient and difficult
x
Difficult to identify "personal" characteristics
x
Difficult to learn
x
x
Iinformation can be falsified
x
x
Lack of blue collar, un-skilled, and semi-skilled listings and applicants
x
May not save time
x
x
Fees may be prohibitive
x
x
Information may be outdated
x
Difficult to keep resumes and postings current
x
More appropriate for national searches
x
Difficult to learn
x
x
x
Cannot run background checks
x
Information may be inadequate
x
Managers may bypass human resources
x
These findings suggest two major areas that need attention for the acceptance of a major online system. First, a system needs to be designed to overcome the disadvantages of existing systems; and second, the system must be appropriately promoted and made readily available to non-users.
The authors find the advantages cited by non-users to be insightful about some of the possibilities of online systems. They may even reflect desirable but unattained outcomes among existing online systems.
One problem seems to be that non-users may be less adventuresome and more intimidated by technology. Some of the disadvantages such as falsification cannot readily be overcome; others can be dealt with by education and direct assistance to the user.
Table 1 reflects both the common elements and differences among employers, brokers, and job seekers. Since space does not permit a full listing of the responses of employers, brokers, and job seekers in the remainder of the paper this table illustrates the kinds of responses that were found.
In summary, the system must be promoted and such promotion should reflect the "local" character of the system. Further, this promotion should "educate" potential users as well as provide access to "personal assistance" when needed.
Information Requirements
Key Information
In the next section, we will look at the emphasis on both content and confidentiality requirements. The method used to elicit responses from focus group participants is to use a series of structured questions with ample opportunity for responses to the questions as well as the opportunity to identify additional characteristics to a desired system. All focus group leaders must be experienced and preferably trained facilitators. In this way, the opportunity is provided for the elicitation of system requirements not anticipated. Six pilot focus groups (two employer, two broker, and two job seeker) were completed prior to the 24 focus groups represented here. We believe that this process provided ample opportunity for the identification of unanticipated system requirements. In addition, it is expected that focus groups will be used after the system is up and running to identify any changes that might be appropriate to the system design and implementation.
Similar information elements were mentioned across all groups. Table 2 shows the seven categories of information that were extracted from the responses of employers, brokers, and jobseekers. A review of this table shows that the seven categories are common to all types of users but that the specific information desired by all types within the categories differed.
Table 2: Desired Information in Database
Work Experience
Employers
Brokers
Job Seekers
Employment Preferences
Employers
Brokers
Job Seekers
Job title (specific)
x
x
Description of desired job environment
x
x
Dictionary of Occupational job titles
x
x
Salary preference (range)
x
x
x
General title (e.g. "technical")
x
x
Goal statement
x
Accomplishments/results
x
x
Career objective
x
x
x
Work references
x
x
x
Location preference
x
x
x
Current salary
x
Description of job preference
x
x
Chronological work history
x
x
x
Daily job duties
x
x
Availability for Job
Employers
Brokers
Job Seekers
Exempt/non-exempt status
x
Willingness to relocate
x
x
Salary history
x
Date available
x
Length of employment
x
x
Hours preferred
x
x
Reasons for leaving previous employment
x
x
Transportation (e.g. seeker has own transportation)
x
Computer experience/knowledge
x
x
Available for permanent vs. temporary work
x
x
Job levels
x
Available for contract work
x
Years in each position
x
Available for shift work
x
x
Reasons for employment gaps
x
Willingness to travel
x
Names of major employers
x
Eligibility for work in U.S.
x
x
Work history summary
x
Military experience
x
Contact Information
Employers
Brokers
Job Seekers
Name
x
x
Skills, Licenses, Certificates, & Professional Activity
Employers
Brokers
Job Seekers
Permission to conduct reference checks
x
Board certificates/licenses
x
x
Phone number
x
x
x
Board/test scores
x
Address
x
x
Skills
x
x
x
Background information
x
Foreign language skills
x
x
Supervisor's phone number
x
Computer skills and training
x
E-mail address
x
Special training
x
Professional affiliations
x
Education
Employers
Brokers
Job Seekers
Other Information
Employer
Broker
Job Seeker
Degrees (granting institutions)
x
x
x
Previous jobs
x
Academic Information/GPA
x
x
Substance abuse history
x
Awards/honors
x
x
Government clearance
x
Scholarships
x
Cover letter or "fact sheet"
x
Certificates of completion
x
Personal attributes (e.g., creativity)
x
Grades/transcripts
x
At this stage of the process no attempt was made to be judgmental about the list of preferences. However, similar responses were reduced to the same wording. Some of the information preferences may have been idiosyncratic to a small group of individuals. The responses do seem to show the orientation of the user. In some sense the job seekers seem to prefer information that is positive or prefer to avoid information that could reflect negatively (e.g. references, grades). Another issue of special importance is safety and security (e.g. not including the home address). It is interesting to observe that all parties avoided use of the social security number. Apparently, all parties are aware of the security risk associated with revealing their social security number.
Another observation of the authors is that the job seeker is more interested in the opportunity to present his/her unique skills and attributes than employers and brokers. The differences between employers, brokers, and job seekers were also present in the pilot study. Thus, the focus group facilitators were prepared to explore differences both from the standpoint of the differences within the groups and differences between different user groups (employers, brokers etc.)
For example, the issue of contact information (addresses, etc.) was discussed. Two mechanisms to avoid the widespread dissemination of such information were discussed. One possibility is a third party that would facilitate confidential contacts between employers and job seekers. A second possibility was the use of multilevel information in which access to personal information is more controlled and secure. A minority of the job seekers saw no difficulty in including such information on the electronic resume.
The majority of employers and brokers felt that "expected salary" was important, as a means of screening out those whose salary expectations would not be met. A minority of job seekers felt that expected salary should not be included.
In discussions regarding personal attributes and other non-standard information one more commonly accepted resolution was to include a "hidden" section with any information that the job seeker felt would contribute to his/her job prospects. If the job seeker's basic resume information met the requirements of the preliminary screening then the employer or broker could call up this information to obtain a more complete picture.
After discussion of the above issues, the focus groups were asked to identify, in order of importance, the "key" components of the information. All groups included six "key" elements. These elements in order of preference were: 1) work experience/employment history; 2) education level and background; 3) special skills; 4) desired salary; 5) contact information; and, 6) career goals/objectives.
Four other frequently mentioned elements were: 1) job classification/titles; 2) specific duties; 3) geographic preferences; and, 4) certificates/licenses.
At this point we can begin to see the character of the system both in terms of content and the concept of multi-level information.
Confidentiality Issues
Both the pilot focus groups and the focus groups reported here pointed out the need to directly deal with the "confidentiality" issue. Thus facilitators were prepared to ask a series of questions to prompt discussion of confidentiality issues.
There was strong agreement among all groups on types of confidentiality issues. Since the job seeker provides information, all confidentiality issues are viewed from the prospect of the job seeker. The recognized issues in order of overall importance are: 1) social security number; 2) name, address, and phone number; 3) current employer; 4) demographic information (e.g. age, gender, race); and, 5) salary (current and/or expected). However, the first two were commonly believed to be "deal breakers" in the sense that significant numbers of potential users would not use the system if they had concerns regarding the confidentiality of their social security numbers and contact information.
Almost all participants felt that the social security number could be used to find further personal information. One participant summarized the feelings by saying, "the social security number could be used by anyone in a manner you did not intend." A clear majority of participants felt that name, address, and phone number - especially when combined with other information such as salary - could be used again in ways not anticipated (e.g. sold to mailing lists or used to create a profile for scams or burglaries).
Less than a majority were concerned about the other three issues (employer, demographics, and salary). A significant minority was concerned about their employers finding out that they were job hunting. Indeed, some reported that this threat would be sufficient reason not to use the system. Some groups felt that demographic information could lead to discrimination by employers scanning the system for potential employees.
Reactions to a Prototype Resume System
At this stage, the focus groups were shown and instructed how to use a prototype resume system. The prototype was designed from the standpoint of easy use (e.g. user friendly, ease of use, clarity of use). In general, use of all of the techniques deemed appropriate for effective system design were based on the assumption that the result should be a resume that contained at least the components of a standard resume. The focus groups were then asked to indicate: 1) what they liked about the system; 2) what they disliked about the system; and, 3) any suggestions they wished to make for improving the system.
Likes and Dislikes
In general the prototype was judged quite good on issues of "look", "feel", "user friendliness", etc. However, it fared poorly on all of the confidentiality concerns expressed earlier.
The prototype had some problems with a clear and uniformly understood method of indicating skills and job titles. This was to be expected since this is a universal problem in such systems. The only relevant comment regarding skills and job content was that an additional opportunity should be made available to expand on standard job titles and skills.
The other dislikes reflect the desire for the job seeker to provide a better picture of his/her characteristics and accomplishments.
Suggestions for Improvement
Many of the suggestions for improvement reflect one or more alternative methods of overcoming the disadvantages regarding issues of: 1) confidentiality; 2) adequately presenting skills and job titles; and, 3) providing a better picture of characteristics and accomplishments. One comment was that career changes might be inadvertently discriminated against because the structure is not flexible enough to accommodate this situation. In other words, if employers searched on previous work history only, an objective reflecting a career change might be ignored.
Another group of suggestions for improvement focuses on education and assistance with the completion of the resume information. While the resume may be entered at locations where instruction and assistance can be provided, the resume may also be entered at locations (including homes) where no assistance is available. One suggestion was to include information that would tell the user where to go for personal assistance. These suggestions merit significant consideration.
In addition, some of the suggestions reflect the fact that job seekers at different job levels may react differently to using the resume system. Some felt that the system was too complex for lower level jobholders and others felt the system limited the information that could be provided for higher-level jobs.
Suggestions for improvement identified both omissions in the data that should be included, as well as, other issues not mentioned in the focus groups. For example, the education entry option did not include "Associates Degree". An issue not mentioned earlier was employer's desire for information on criminal activity.
A total of 51 different suggestions for improvements were made. Space does not permit any detail on these suggestions. However, the system developers now have a wealth of information on which to produce a system that should be extremely useful. Moreover, they have some basis for understanding the depth of feeling and an understanding of the reasons for the concerns. As with all system development activities, they must find the appropriate balance between the needs of the three distinct user groups.
Operating Procedures
The last area to be addressed by the focus groups dealt with operating aspects. This includes issues such as how the system is to be promoted and where and how access should be available. This also includes what fees, if any, should be required of users. We will not report any of the findings that do not influence the structure and design of the online system. Since no fees are anticipated we will not deal with that issue.
A second critical aspect of the system is how employers and brokers access and obtain all of the resumes and associated information. While this is very important, we will concentrate on aspects that effect resume entry procedures. Thus, aside from the security issues important to job seekers, we will not report on the employer and broker use of the system.
In the absence of fees, many groups recommended a registration procedure that would discourage browsers and others not serious about the system. Some groups felt that a fee was the only way to eliminate browsers. One suggestion for registration - and thus the ability to place a resume into the system - was to require certain fields be completed before accepting the resume. There was no consensus or viable registration method to keep out those not serious about using the system.
One other important operating procedure is maintaining and updating the resume. Across focus groups the suggested time period that resumes should be kept on the system without modification was from one to six months. Some groups felt that a job seeker should be notified before his resume was purged because it had exceeded the time allowed since entry or updating.
The last issue discussed in the focus groups was whether the job seeker should be notified when his/her resume had been accessed. Some would also like to know specifically what company had accessed their resumes so that they could follow up. This was generally supported in job seeker focus groups.
Conclusions and Implications for Design of Service-Based Web Systems
Common themes emerged from the focus group discussions across all groups in participating states. System developers, system marketers, and service delivery professionals can benefit greatly from understanding the implications of the results of these pilot studies. Specific ways for system developers and marketers to enhance the system and its benefits include: promoting the local scope of the Talent Bank; addressing concerns about privacy and confidentiality issues, promoting the benefit of future access to the Job Bank and incorporating that benefit with the use of the Talent Bank; making the system more user-friendly; and, establishing and promoting system operations according to standard procedures. These areas are detailed below:
Promote the Talent Bank and Emphasize its Local Scope
The majority of recruiting, matching, and seeking activities are conducted at the local level, therefore, the Talent Bank must be perceived as useful at the local level. A current misperception is that the Talent Bank is geared towards a national scope and towards higher-level jobs. This may discourage potential local users from using the talent bank, who will instead continue to use customary activities that work for them. To promote local usage, system marketers should:
- Attract potential users by advertising "success stories."
- Encourage employers and employment brokers to use the talent bank to recruit local applicants.
- Address and overcome the view that online systems are accessible to just the "computer elite."
- Develop and implement a marketing strategy to educate the local communities and the state about advantages of system use.
Address and Overcome Fears About Issues of Privacy and Confidentiality
At all focus group meetings, the majority of participants expressed some concerns about privacy. In fact, a few participants were adamant that they would not use the system if certain personal information was required (e.g., SSN). They wanted to protect their personal information (i.e., by putting it in resume hidden sections that are only revealed to an employer or broker who has selected their resume on the basis of its structured content), or by having a third party administrator control access to personal information. Some expressed concern that unauthorized users could illegally/unethically use personal information.
Implementation of the following recommendations would offset these concerns
- Promote procedures and features developed to protect privacy and confidentiality (e.g., registration, access code/pin, hidden screen feature, intermediary)s.
- Promote the specific advantages of using on-line systems.
- Establish clear safeguards that allow system access only to authorized users.
Make the System Easier to Use
It should be easier to use both in terms of its look and feel as well as "personal assistance". Many participants expressed the view that certain subgroups within the population, such as older workers and those who don't use computers on the job, would hesitate to use the Talent Bank. Participants suggested using features such as a mouse, touch screens, simple screen layouts, menus, help screens, and access to information resources (e.g., employment services staff members or "help line") to educate and guide users.
- Consider hardware and software features that will facilitate the resume entry, edit, and feedback processes for reluctant and computer-naive users.
- Include information to give the user access to "personal assistance".
Ensure That Both the Job Seeker's and Employer's Information Requirements Are Met
- In addition to those in the prototype give job seekers the opportunity for an added free form presentation of skills and job goals.
- Give job seekers the opportunity for free form entry of characteristics and accomplishments.
Ensure That the System Operates According to Standard Procedures
Perceived problems included that without scheduled maintenance and continuous monitoring, the system could become cluttered with outdated resumes and job postings; could contain inaccurate information, and employers could advertise "fake" job openings. Several complaints were made about existing online job posting and resume systems, such as, "system is slow" or unreliable, and listings are not current. Incorporating the following suggestions for service delivery could offset these concerns:
- Institute standard procedures to ensure the system operates efficiently; this promotes the perception that the system is orderly, reliable, and standardized.
- Ensure timely maintenance of resumes and job postings (renewals, deletions) and purge outdated entries from the system.
- Ensure that job postings are authentic and placed only by authorized employers.
Appendix: A Model Resume System
A model was developed ( Figure 1) to illustrate the general views across potential user groups about the design and functions of the Talent Bank system. The left side of Figure 1 represents the preferences of job seekers for entering and interacting with the database; the right side represents employer and job broker preferences. In this model, boxes represent system components and the arrows represent the relationships or interactions among these components.
Basic components from the job seeker side are: 1) system access locations; 2) system access procedures; 3) resume creation; and, 4) maintenance procedures. Job seekers gain access to the system at certain preferred locations (e.g., homes, employment agencies, etc.). Access procedures include registering and obtaining a PIN. Job seekers then follow preferred procedures to enter and maintain resumes, which are submitted to the Talent Bank database. The Talent Bank itself conducts specific system monitoring procedures that impact on job seekers (e.g. tracking resume expiration dates and counting the number of times resumes are searched).
Basic components from the employer/employment broker side are: 1) system access location; 2) system access procedures; 3) resume search procedures; 4) job listing procedures; and, 5) the Talent Bank database. Employers and brokers access the system at certain preferred locations (e.g., offices), and complete a process to register and obtain an access code (e.g., an Employer Identification Number (EIN)). These users then follow preferred procedures to search resumes, such as specifying broad search criteria (keywords), and narrowing the criteria. They may also choose to retrieve information by downloading and viewing resumes. At this point, employers and brokers may gain access to personal information and contact job seekers. In this model, the Talent Bank conducts monitoring procedures such as checking credentials, monitoring resume expiration dates, and recording which resumes system users view and download.
The shaded boxes labeled "Job Listing Procedures," "Job Bank Functions," and "Labor Market Information" are placeholders for those system interfaces that are proposed but not yet developed.
Summary of the Benefits of Focus Groups in the Design and Development of Online Systems
The America's Talent Bank system is functioning well; state agencies have found the system to be a success. Some state agencies have surveyed both job seekers and employers using the system. On balance, the resulting system appears to have worked well. The review in this paper of the responses of users suggests that the design and implementation of the system have been very satisfactory. If any difficulties exist they appear to be outside of the area of information system design. In a sense, the system designers are a small group of individuals who grasp computer-based needs. Consequently, the resulting system will have considered all of the stakeholders except those outside of computer-based aspects of the system.
This system is available to all Americans on the Internet. The personal support and promotion of the system must come from the 1,800 Employment Security offices in the United States. Consequently, overcoming the non-computer aspects of the system are far more complex than the design and development of just the computer interface. Nevertheless, the information and knowledge generated from the focus groups has already been extremely useful. Further, benefits of the project are still being secured thanks to a larger and more effective promotion effort as well as more effective levels of "personal support".
About the Authors
Dr. Brenda L. Killingsworth is an Associate Professor of Management Information Systems in the Department of Decision Sciences at East Carolina University.
E-mail: killingsworthb@mail.ecu.eduDr. Robert E. Schellenberger is a Professor of Management Science in the Department of Decision Sciences at East Carolina University.
E-mail: schellenbergerr@mail.ecu.eduDr. James W. Kleckley is the Associate Director of Planning and Institutional Research at East Carolina University.
E-mail: kleckleyj@mail.ecu.eduAcknowledgments
The basic information for this paper was drawn from ALMIS Talent Bank Focus Groups: Final Integrative Report. This document reported on the conduct and findings of the focus group. The authors gratefully acknowledge the support and assistance provided by Ardis Cazino, Director of the Labor Exchange Services in Michigan. Without this support this paper would not have been possible. The authors would also like to express appreciation to Glenda K. Potts, Personnel Analyst at Cherry Hospital in North Carolina. Any errors or omissions are the responsibility of the authors.
References
J. C. Anderson and J. A. Narus, 1998. "Business marketing: Understand what customers value," Harvard Business Review, volume 76, number 7 (November-December), pp. 53-64.
E. Bensimon and A. Neumann, 1994. Redesigning collegiate leadership: Teams and teamwork in higher education. Baltimore: Johns Hopkins University Press.
L. Bolman and T. Deal, 1991. Reframing organizations: Artistry, choice and leadership. San Francisco: Jossey-Bass.
J. L. Dreachelin, 1998. "Conducting Effective Focus Groups in the Context of Diversity: Theoretical Underpinnings and Practical Implications," Qualitative Health Research, Nov. 8, 1998, Vol 8, Issue 6, p813-821
D. Kirkpatrick, 1985. How to Manage Change Effectively. San Francisco: Jossey-Bass.
JISC (Joint Information Systems Committee), 1995. Exploiting Information Systems in Higher Education: An Issues Paper. Bristol: JISC.
R. Karpinski, 1999. "Really Get to Know Your Customers," Internet Week, 8 November 1999, p. 25, and at http://www.commweek.com/transform/transform110899-2.htm
R. A. Krueger, 1994. Focus Groups: A Practical Guide for Applied Research. 2nd edition. Thousand Oaks, Calif.: Sage.
M. Latta,, J. Kerwin, P. Davis, J. L. Crafts, A. M. Rose, W. A. Baughman, and B. K. Kingsley, 1995. ALMIS Talent Bank Focus Groups: Final Integrative Report. (December).
H. Mintzberg, D. Raisinghani, and A. Thoret, 1976. "The structure of unstructured decision processes," Administrative Science Quarterly, volume 21 (July), pp. 246-275.
D. L. Morgan (editor), 1993. Successful Focus Groups: Advancing the State of the Art. Newbury Park, Calif.: Sage.
J. Neilson, 1997. "The Use and Misuse of Focus Groups," IEEE Software, volume 14, number 1 (January-February), pp. 94-95.
J. Steinert-Threlkeld, 1999. "Online It's Still the Customer Stupid," Interaction Week, volume 6, number 11 (15 March), p. 22.
Editorial history
Paper received 11 May 2000; accepted 6 June 2000; revised version received 9 June 2000.
Copyright ©2000, First Monday
The Use of Focus Groups in the Design and Development of a National Labor Exchange System by Brenda L. Killingsworth, Robert E. Schellenberger, and James W. Kleckley
First Monday, volume 5, number 7 (July 2000)
URL: http://firstmonday.org/issues/issue5_7/killingsworth/index.html
