During development of the online interviewing method a small pilot study was undertaken using a group of undergraduate students to test the technology. The study consisted on a brief online interview (between 30 and 45 minutes) and self-assessment questionnaire, followed by a focus group that discussed the method used. The experiment was run over two days at the Australian National University in 1998.
Four students were randomly selected and asked to engage in an interview and focus group. The aim of the interview was described as "Observations of observed political phenomena". For the purposes of evaluating the method, the interviewees were not informed about the nature of the study until the beginning of the focus group, held on the second day of the study. This deception was seen as necessary as not to bias the study or encourage the participants to focus their attention on the computer, rather, a series of questions on current political events was used as the basis of the interview.
The interview was conducted in this manner:
At the conclusion of the interview a self-assessment questionnaire was given to all participants to gauge their demographic characteristics and assess their familiarity and attitudes to computers. This survey also contained a realistic set of attitudinal questions regarding current political events, to maintain the initial fiction of the interview and limit the chance of participants from discussing the online interview process in detail, should they meet between the interview and the focus group process.
The final part of the experiment was a focus group consisting of all four interviewees and the researcher. In this interview, the purpose of the research was revealed and the online interview was discussed. This session was video taped with the consent of the group, for later analysis and transcription of key points. The focus group was conduced as a free discussion period facilitated by the interviewer, who served to guide the discussion and follow-up on points raised by the interviewees.
The sample for the research consisted of one women and three men. Details are presented in Table 1. This information was generated from a questionnaire presented after the initial interview using the online interviewing process.
|#||Gender||Age||Edu*||Own**||Use Comp.||Use WWW||Use Email||Inter***|
|1||M||41||13yrs||yes||daily||every few days||every few days||yes|
|2||M||13yrs||71||yes||daily||every few days||rarely||no|
|3||M||51||14yrs||yes||every few days||rarely||rarely||yes|
The group can be summarised as mature, with some degree of familiarity with computers, who are engaged in study after (or during) a career in public and private sector employment. The group had mixed reactions to computers, some interested in learning more about them and some unconcerned. All, however, used computers on a relatively regular basis (due to the requirement of university students). One reported having a medical impairment that prevented prolonged use of computers.
At the completion of the interviews, the recorded logs of the sessions were analysed and the amount of output was matched to the interviewees self-reported typing speed and eyesight, as summarised in Table 2, below. Overall, the output of the sessions was modest with an average of twenty-two words per minute during the exchange.
|interviewee*||typing speed**||eyesight**||words per minute***||words per hour***|
The main limitations of this study are the size of the sample and the
limited variation of conditions in which it was conducted. Each
interviewee, within the variation of discussion topics, had a near
identical experience during the study, and no additional factors were
introduced that may have been valuable (such as the inclusion of a
non-physical contact/presence with the interviewees, use of other machines,
differential explanations of the process, etc.). As this study was used to
explore the method, these limitations were seen as acceptable.