Studying Survey Interviewers: A Call for Research and an Overview
Résumés
L'étude des intervieweurs d'enquête, un appel aux contributions et un vue générale. Pour encourager un projet de recherche en cours sur les intervieweurs d'enquête, le BMS publie ici une vue générale des nombreuses recherches sur le sujet qui sont parues dans le BMS depuis vingt ans. Tout en appelant aux contributions — particulièrement de la part de la vaste expérience en méthodologie d'enquête de nos collègues aux Pays-Bas, en Allemagne et aux Etats-Unis — nous partageons les résultats préliminaires d'une étude détaillée française qui a fait une analyse statistique des performances d'intervieweurs, puis des entretiens face-à-face avec des intervieweurs dont le "profil statistique" était loin du "standard". Ces entretiens avec les intervieweurs fournissent une description étonnante du milieu du "business" des enquêtes.
Entrées d’index
Haut de pagePlan
Haut de pageTexte intégral
Survey Data Quality & the Survey Business
1One can easily see from the documents cited below that the role and performance of interviewers, who fill out survey questionnaires, are integral aspects of the survey process and their impact on the reliability and quality of collected data has been extensively studied.
2These studies have addressed such evident questions as the training and remuneration of survey interviewers, and their direct effect on survey data quality. From the very beginning, the BMS has been interested in the question and published research results related to the question. One of the first was Johannes van der Zouwen and Wil Dijkstra's article, "Types of Inadequate Interviewer Behaviour in Survey Interviews: Their Causes and Effects" (BMS, n. 18, April 1988, pp. 5-20; see abstract below). More recently, there was Michael Lemay and Claire Durand's article, "The Effect of Interviewer Attitude on Survey Cooperation" (BMS, n. 76, October 2002, pp. 26-44), which again stressed the role of training. They also added that "the lack of a better measure of performance may also explain the weakness of the results. Future studies should examine performance measurement..."
3At the same time as survey sociologists call for more precise measurement of interviewer performance, the survey milieu business has its own unspoken "rule of thumb": at least ten percent of the data is fabricated. For conscientious methodologists, this means that no statistical result can be taken seriously unless it can withstand at least ten percent of more-or-less arbitrary data. This should be the minimum requirement for robustness.
4One one hand, we — as survey sociologists — know what can help improve interviewer performance and data quality. But, on the other, we do not often have the interviewers' appreciation of what the "survey business" is like and how — in that work context — survey data is produced and can, eventually, be improved.
The French Agoramétrie Study
5This latter point of view was the originality of a recent French study done by Agoramétrie which has produced more-or-less annual surveys of French public opinion on social conflicts since the mid-1970s. With the encouragement of Jean-Pierre Pagès at Agormétrie, a major (unnamed) French polling agency accepted to "play the game" (in French, "jouer le jeu"), and permit access to its interviewers following the 2003-2004 Agoramétrie survey. This work has produced two articles submitted for publication by the BMS: one is a thorough statistical analysis of the performance of each interviewer; the second is based on open-ended face-to-face interviews with (unnamed) selected interviewers.
6In the first article, each interviewer's use of all response variables is analyzed, along with the number of interviews per day (sometimes per hour), and correlations with known social-economic characteristics of respondents. This statistical analysis provided comparable "profiles" of interviewer performance and permitted the identification of certain interviewers who were not at all "typical" or "standard".
7With the permission of the polling agency, the interviewers were contacted in a manner which guaranteed their anonymity. All of the interviewers who were not "typical" accepted a face-to-face interview with a representative of Agoramétrie. Those interviews are the basis of the second article which provides what has been missing for so long in survey research: the interviewers' view of the survey business. Indeed, the interviews with the interviewers provide a surprising characterization of the survey business and what the social network of interviewers knows about the different companies, including how much they pay, how well or poorly constructed are the questionnaires the companies accept, what rigor — or lack of rigor — the companies demand, etc.
8The interviewers also provided a description of the "tool kit" they can use to "get the job done", depending on what questionnaire, what company, what pay, what time span... Going into a French café on Saturday morning and paying a round of drinks for everybody who fills out a questionnaire correctly will quickly get you a fair number of good questionnaires, but the data may not be very representative of the French population, and several questionnaires filled out on Christmas morning are probably even less representative...
9The BMS would like to know if other similar studies have been done in Western Europe or North America. We would also be interested in contributions which look into what has and has not been done on this topic. It's time to stop looking at survey data quality "from the top down" and start looking at it as the whole system that it has always been.
Documents Published in the BMS
10["15A" means in issue number 15 of the BMS in the section "Articles"; "19B", issue 19, section "Books"; "32J", issue 32, section "Journals", "43C", issue 43, section "Computers", etc.]
1115A, Farber (E.D.) & Joseph (J. A.), "Factors in Interview Training", Psychological Reports, 1985, 57, 3, 1021-?.
12Stuart (Eleonora W.), "Hiring and Training in House Interviews", Public Relations Journal, 1984, 40, 24-27.
13BMS, n. 18, April 1988: Johannes van der Zouwen and Wil Dijkstra, "Types of Inadequate Interviewer Behaviour in Survey Interviews: Their Causes and Effects", pp. 5-20. The authors present a model of the interaction between interviewer and interviewee during the survey interview process. On this theoretical basis, survey interviews were recorded and analysed, revealing four types of inadequate behaviour: the adaptation of questions for the interviewee by the interviewer; the interviewer's inattentiveness; the interviewer's choosing of a response "on the behalf" of the interviewee; the hinting by the interviewer to clarify the interviewee's response. These different types are thoroughly analysed, showing that it is especially hinting that distorts survey interview data.
1418A, Reese (Stephen D.), Danielson (Wayne A.) et al., "Ethnicity of interviewer effects among Mexican Americans and Anglos", Public opinion quarterly, 1986, 50, 4, 563-572. A general interpersonal deference explanation for ethnicity of interviewer effects was tested in a random telephone survey of Anglos & Mexican Americans in Texas using Anglo & Hispanic interviewers & questions related to Mexican-American culture. The theory was most clearly supported among Anglo respondents. The mixed support found among Mexican Americans suggests that deference is strongest when questions deal with the culture represented by the interviewer, rather than the respondent.
1519B, Blanchet (Alain), Ghiglione (Rodolphe), Massonnat (Jean) & Trognon (Alain), Les techniques d'enquête en sciences sociales. Observer, interviewer, questionner, Dunod (17 rue Rémy Dumoncel, 75014 Paris), 1987, 197 p, 105 FF, 2-04-016901-6. Ce livre pratique est destiné aux besoins des étudiants universitaires en sociologie et en psychologie et à tous les praticiens de l'enquête. Il est composé de quatre parties comprenant 11 chapitres, d'une bibliographie et d'un index. Partie 1, Produire des données, par A. Trognon: Observation et expérimentation (16 p). Partie 2, Observer, par J. Massonnat: Introduction (6 p); Définitions et problèmes généraux (15 p); Démarches de recherche (15 p); Pratiques d'observation et construction d'un dispositif d'études (28 p). Partie 3, Interviewer, par A. Blanchet: Limites du dispositif et de son usage (9 p); Conception interactive des niveaux de la communication (13 p); Interventions et discours (11 p); Interactions verbales (13 p). Partie 4, Questionner, par R. Ghiglione: Constructions référentielles et questionnement (33 p); Objectifs: questionnaires et méthode alternative (30 p); Conclusions (3 p). Alain Blanchet et une équipe de chercheurs ont également publié chez le même éditeur un ouvrage complémentaire: L'entretien dans les sciences sociales, 1985.
1619A, van der Drift (Joke) & Derksen (Wim), "Kritische bijdrage. Interviewstijlen en interviewresultaten" / critical review - interview style and interview results, Mens en Maatschappij, 1985, 60, 2, 187-197. Variations in interviewer style or behavior may affect the results of social scientific research. Such effects are examined through analysis of findings from a Dutch research project in which 6 interviewers adopted 3 distinct styles of interviewing: formal, empathic, & intimate. Formal styles tend to produce less full answers to questions; on the other hand, intimate styles bias respondents toward agreeing with the interviewer's sentiments. While some rapport is desirable, a certain degree of formality may be preferable to maximum rapport.
1721A, W. J. Hoag, "Der bekanntenkreis als universum: das quotenverfahren der Shell-studie" (un univers réduit au cercle de relations: la procédure d'échantillonnage par quotas de l'étude Shell), Kölner Zeitschrift für Soziologie, 1986, 38, 1, 123-132. Examen des informations apportées par l'enquête Shell de 1985 sur les pratiques de sélection, par les interviewers, de leur échantillon: pour 5% seulement des interviews, l'interviewer était inconnu de l'interviewé avant l'entretien. La comparaison de certains résultats de cette enquête avec d'autres utilisant des procédures d'échantillonnage au hasard et des statistiques officielles de vote met en évidence des biais importants. Analyse des raisons de ces biais et des problèmes de clustering quand les interviewers peuvent choisir leur échantillon.
1824A, Groves(R.M.), "Research on survey data quality", The Public Opinion Quarterly, 1987, 51, 4, 2, S156-72. Survey research has not been established as an academic discipline, and at present it lacks a unified theory of data quality. Important recognized components of survey quality include coverage error, non-response error, and sampling error. Measurement errors may arise from interviewers, respondents, questionnaires, or modes of data collection. Two other issues remain underemphasized: cost implications of error reduction and interrelationship of error sources. These various concepts provide the starting point for a theory of surveys.
1924A, Longres(J.), "The use of survey methods in researching parents of adjudicated child prostitutes", Journal of Sociology and Social Welfare, 1987, 14, 3, 65-86. In this experience in applying survey methods with difficult and hard to reach subjects, two data collection forms were used: a family contact form on which the interviewers documented their experiences in obtaining the interviews, and an interviewer experiences form on which the reactions of the interviewers to the parents were recorded. A third form was filled out by the court social worker, enabling comparisons between families interviewed and those not interviewed. Factors related to the response rate are discussed, and it is concluded that it is viable to use the survey methods with difficult to reach subjects if precautions are followed.
2025A, Jean Peneff, "The Observers Observed: French Survey Researchers at Work", Social Problems, 1988, 35, 5, 520-535. An analysis of the work of 45 regional survey research interviewers for the INSEE national French statistics institute. Data obtained via interviews and observations reveal discrepancies between researcher's norms concerning standardization, neutrality, and identical wording and how they actually ask questions and construct responses. The statistics on which quantitative sociology rests are, in an important sense, made in the field, and are thus not as separated from the qualitative procedures as is sometimes assumed.
2126A, M. L. Berk & A. B. Bernstein, "Interviewer Characteristics and Performance on a Complex Health Survey", Social Science Research, 1988, 17, 3, 239-251. The question of whether research interviewers' experience and education are related to their ability to elicit information is discussed in a study of interviewers (N=206) employed on the National Medical Care Expenditure Survey. Interviewer ability to obtain responses is examined, and the possible effects of interviewer characteristics (race, SC, age, and education level) are considered. It is concluded that conventional wisdom about survey staffing should be examined, and that age, interest, and commitment may be more important considerations than formal credentials.
2228A, Johannes van der Zouwen and Edith D. de Leeuw, "The Relationship between Mode of Administration and Quality of Data in Survey Research". Examination of a series of meta-analyses on the outcomes of dozens of "mode experiments" reveals small, but statistically significant and consistent, effects of the method of data collection (mode effects) on the quality (representativity, completeness and accuracy) of survey data. On all indicators of data quality, the face-to-face interview scores slightly better than the telephone interview, but these differences are gradually disappearing over time. Mail surveys have lower response rates and less item response than do face-to-face and telephone interviews, but result in more accurate, less biased answers, especially when sensitive or embarrassing questions are being asked. Three "mode characteristics" or intervening variables are identified to explain these mode effects: the "persuasion power" of a mode, the complexity of the task for the respondent, and the degree of control over the question-answer process (control over role-related behavior of respondents and interviewers), along with one conditional variable (the degree to which the response categories of the questions differ with respect to their social desirability). These variables are inserted in a model which has as independent variables the three modes concerned, and as dependent variables the response rate, completeness, and accuracy. Predictions about mode effects on data quality derived from this model are confronted with data from a recent mode comparison experiment conducted in the Netherlands, in which similar questions about loneliness, satisfaction with living conditions and well-being were posed in face-to-face interviews (N=300), telephone interviews (N=300) and a mail survey (N=500). This confrontation resulted in a further refinement of the explanatory model, which in turn may help survey researchers make optimal choices for the implementation of survey data collection.
2329A, M. Lueger, "Die Soziale Situation im Interview" (La situation sociale dans l'interview), Österreichische Zeitschrift für Soziologie, 1989, 14, 3, 22-36. L'auteur met l'accent sur l'importance de la relation sociale impliquée par l'interview même lorsqu'il est fortement standardisé et analyse différents facteurs influençant cette situation: attentes normatives liées à la formation antérieure de l'interviewer et aux intérêts connectés à l'interview, attitudes du sujet et mode d'établissement du contact, orientation mutuelle et structuration de l'interview dans la phase de départ, problèmes émergeant au cours de l'interview.
2429A, B. S. Mensch and D. B. Kandel, "Underreporting of Substance Use in a National Longitudinal Youth Cohort: Individual and Interviewer Effects", Public Opinion Quarterly, 1988, 52, 1, 100-124. The quality of drug data in the 1984 wave of the National Longitudinal Survey of Youth is explored. Comparisons with other national surveys indicate that underreporting of use of illicit drugs other than marijuana appears to have taken place, and that light users of these drugs are underrepresented among the self-acknowledged users. Comparison with marijuana use reported four years earlier indicates that experimental marijuana users are much less likely than extensive users to acknowledge involvement. Even after controlling for frequency of use, underreporting is more common among terminal high school dropouts and minorities. Not only individual characteristics but field conditions also contribute to underreporting. Familiarity with the interviewer, as measured by number of prior interviewing contacts, depresses drug use reporting. We speculate that interviewer familiarity increases salience of normative standards and that participants respond not only in terms of their past familiarity but also in terms of their subjective expectations regarding the probability of a future encounter with the interviewer.
2530B, D. W. Stewart and P. N. Shamdasani, Focus Groups: Theory and Practice, Sage Publications, London, 1990, bibliography, index, 153 pp., isbn 0 8039 3390 8, £10. This book includes an authors' "Preface" (2 pp.) and eight chapters: 1. "Introduction" (24 pp.); 2. "Group Dynamics and Focus Group Research" (18 pp.) which draws heavily on the literature of group dynamics and the social psychology of groups; 3. "Recruiting Focus Group Participants and Designing the Interview Guide" (18 pp.); 4. "The Focus Group Moderator" (18 pp.) considers the literature on interviewer and moderator effects; 5. "Conducting the Focus Group" (15 pp.); 6. "Analyzing Focus Group Data" (20 pp.) includes discussion of content analysis; 7. "Focus Groups in Practice" (18 pp.) gives further information on designing, conducting and interpreting the outcome of focus groups; 8. "Conclusion" (4 pp.).
2630A, XIIth World Congress of Sociology of the ISA ... J. Reinecke and P. Schmidt, "Explaining Respondent Behavior and Interviewer Effects: A Rational Choice Perspective".
2731A, J. Billiet and G. Loosveldt, "Improvement of the Quality of Responses to Factual Survey Questions by Interviewer Training", The Public Opinion Quarterly, 1988, 52, 2, pp. 190-211. In this study we analyze the effects of interviewer training on the quality of responses. Data from a field experiment reveal a number of significant differences between trained and untrained interviewers in terms on non-response and the amount of information obtained. For some questions, an interaction effect occurred between interviewer training, the use of tape recorders, and the kind of responses. The effects of training appear to be dependent on the structure of the questions. Since such effects occur primarily with questions that assume a great deal of interviewer activity, it is suggested that they are indeed due to the application of the techniques acquired by training, namely giving instructions, probing, and feedback. A brief analysis of the interviewer-respondent interaction supports this interpretation.
2832J, ZUMA Nachrichten (B2, 1, Postfach 12 21 55, D 6800 Mannheim 1; tel. 49621 180040; fax 49621 1800449; N. 28, May 1991): AchimKoch, "On the Relationship Between Interviewer Characteristics and Response Rates" (pp. 13).
2934A, Barbara A. Anderson, Brian D. Silver and Paul R. Abramson, "The Effects of Race on the Interviewer on Measures of Electoral Participation by Blacks in SRC National Election Studies", Public Opinion Quarterly, 1988, 52, 1, pp. 53-83. An investigation of the effect of race of interviewer on self-reported voting, actual voting, and political attitudes of black respondents, based on data from the SRC/CPS National Election Studies (NES) of 1964, 1976, 1978, 1980, and 1984. Over the course of the five NES studies, the proportion of black respondents who were interviewed by black interviewers declined sharply, particularly in the South. Almost all white respondents were interviewed by whites. Except for southern blacks in the pre-Voting Rights Act election of 1964, black nonvoters in the presidential election surveys of 1964, 1976, 1980, and 1984 who lived in predominantly black neighborhoods and who were interviewed by black interviewers were more likely to report falsely that they voted than black respondents interviewed by white interviewers. Black respondents in black neighborhoods who were interviewed by black interviewers were also more likely actually to vote and to endorse civic norms than blacks interviewed by whites. Hence, matching the respondent and the interviewer by race not only generates response effects-error in the answer, but also, for many black respondents, it appears to induce changes in actual behavior.
3035A, W. Axinn, "Interviewers and Data Quality in a less Developed Setting", Journal of Official Statistics, 1989, 5, pp. 265-280.
3136A, R. Schnell, "Der Einfluss gefälschter Interviews auf Survey-Ergebnisse" (L'influence des interviews truqués sur les résultats de l'enquête), Zeitschrift für Soziologie, 1991, 20, 1, pp. 25-35. Les résultats d'une enquête sur la "qualité" d'interviews (N=220) truquées par les interviewers (N=22) montrent certes une plus forte cohérence des interviews falsifiés comparativement aux autres interviews, mais pas de différence importante eu égard aux données véritables. Si ces interviews truquées n'avaient constitué qu'une faible proportion (5%) de l'échantillon, les effets de la "tricherie" des interviewers auraient été négligeables.
3236A, Sue R. Crul, Thomas L. Van Valey, Eric D. Johnson, Leticia Ayuso, Deborah Bartz, Rose Dunlap-Gist, and Tim Gallagher, "Interviewer Error: A Preliminary Analysis", American Sociological Association Meeting, 1991. Two types of interviewer error, response selection and interpretation, were measured by means of telephone monitoring for 10 computer-assisted telephone interviews (CATI) and 10 paper and pencil interviewers (n = 198 completed interviews). Analysis indicates that the paper method produced 50% as many response selection errors as did the CATI method, but the computer error rate was small (1.05%); the interpretation error was also relatively small for both methods (1.65% and 1.45%, respectively). The CATI method averaged less interview time and a higher completion rate than did the paper method. Interviewers' age and experience were negatively related to error and their perceptions about the survey were positively related to completion rates.
3336A, N. J. Molenaar, "Recent Methodological Studies on Survey Questioning", Quality and Quantity, 1991, 25, 2, pp. 167-187. Methodological studies (N = 50) on survey questioning conducted in 1986/87 are reviewed. The majority are empirical studies on validity problems, which are categorized by main source of error (interviewer, respondent, instrument, or circumstances). Methodological studies on reliability and nonresponse, and theoretical studies on survey questioning, are also included.
3436A, N. J. Molenaar, "Questioning: An Overview of Recent Methodological Studies", Kwantitatieve Methoden, 11, 1990, 33, pp. 137-159. Overview of both empirical and theoretical studies, published in 1986 and 1987 on survey questioning. Discussed are publications on the reliability and validity of survey research and the sources of systematic errors. As main sources of systematic errors is discussed literature on: interviewer effects examined by means of the correlation approach, the experimental approach, or the behavioral approach. The impact of characteristics of respondents is also considered. In view of the influence of the questionnaire on survey errors, are mentioned publications on: wording of questions, types of questions applied, and question order. Concluding sections on literature concerning the impact of the interview situation in general, and on factors that influence the cooperation of the respondents and reduce nonresponse.
3537A, William G. Axin, Thomas E. Fricke and Arland Thornton, "The Microdemographic Community-Study Approach: Improving Survey Data by Integrating the Ethnographic Method", Sociological Methods and Research, 1991, 20, 2, pp. 187-217. Survey methods have been criticized for producing unreliable, invalid data and for failing to provide contextual information to test complex causal hypotheses. Here, a technique that combines survey and ethnographic methods at every stage of the data collection process is presented to overcome these shortcomings. It is shown how the combined approach reduces coverage, nonresponse, and measurement errors arising from the interviewer, the questionnaire, and the respondent. Although the combined approach is intensive, it is flexible enough to be used in a variety of research settings.
3637A, William S. Aquilino and Leonard A. LoScuito, "Effects of Interview Mode on Self-Reported Drug Use", Public Opinion Quarterly, 1990, 54, 3, pp. 362-395. An examination of the results of random-digit-dialing (RDD) versus personal interviews in drug use surveys conducted in New Jersey in 1986/87 (N = 1,000+ respondents ages 18-34). In the personal interview, respondents recorded their drug use on self-administered answer sheets. Compared to the area probability sample in the personal interview, RDD yielded a sample of blacks higher in income and education, and more likely to be married and currently employed; white socioeconomic status was also higher in the telephone sample, but to a lesser degree than for blacks. Controlling for demographic characteristics and RDD's exclusion of nontelephone households, the telephone survey yielded significantly lower estimates of blacks' alcohol consumption, and of lifetime and recent marijuana use. Whites' alcohol consumption was slightly lower by telephone; otherwise, estimates of whites' use of the 4 substances (tobacco, alcohol, marijuana, and cocaine) were nearly identical in the 2 modes. Sample coverage, demographic characteristics, and racial matching of interviewer and respondent did not account for the significant mode differences. Characteristics of the interview situation itself, such as provision of privacy in the self-administered format, may have influenced tendencies toward socially desirable responding to a threatening topic such as drug use.
3738B, Edith Desirée de Leeuw, Data Quality in Mail, Telephone and Face to Face Surveys, 1992, TT-Publikaties (Plantage Doklaan 40, NL 1018 CN Amsterdam, Netherlands), appendices, bibliography, author and topic indexes, 168 pp., isbn 90 801073 1 X, Dfl 30. ... 2. Why Expect Differences?: Media Related Factors; Information Transmission; Interviewer Impact. This chapter gives an overview of factors that may cause mode effects. ... 4. Design of a Field Experiment: Questionnaire construction; Sampling Procedures; Procedures for Selection and Training of Interviewers; Implementation of Data Collection Procedures; Pilot Study; Field Experiment; Sample and Nonresponse. This chapter describes how a mode experiment questionnaire was designed and pre-tested.
3838B, Robert M. Groves, Survey Errors and Survey Costs, 1989, Wiley, New York, 590 pp., isbn 0 471 61171 9. On the methodology of survey research, with special attention to errors and costs in the data collection stage. Described are various sources of nonobservation errors and of measurement errors in the interviewing phase of household surveys. Distinguished are the following types of errors in sample surveys: coverage error, nonresponse error, sampling error, and measurement errors due to interviewers and respondents. Considered are measurement errors due the questionnaire applied, or to the mode of interviewing used. With an appendix on the mathematical notations of the total survey error perspective in psychometrics and econometrics. Attention is given to costs and errors in the sampling phase of survey research.
3938B, Jost Reinecke, Interviewer- und Befragtenverhalten, Theoretische Ansätze und methodische Konzepte, 1992, Studien zur Sozialwissenschaft, Bd. 106, 328 pp., isbn 3 531 12290 8. Dieses Buch behandelt die praktische Handhabung von Interviewer- und Befragtenverhalten auf der Grundlage von Theorien rationalen Handelns.
4038J, Juta Kreiselmaierand Rof Porst, Methodische Probleme bei der Durchführung telefonischer Befragungen: Stichprobenziehung und Ermittlung von Zielpersonen, Ausschöpfung und Non-Response, Qualität der Daten, 1989, ZUMA, Mannheim, 57 pp. On methodological problems of telephone interviews. After an introduction to telephone calls as a communication technique, attention is given to telephone interviews as a data collection method. Attention is given to problems dealing with the selection of respondents: sampling technique, representativeness of the sample, and nonresponse. Several aspects of data quality are discussed: item nonresponse, validity and reliability of data, and interviewer effects.
4138A, Michael J. Havice and Mark J. Banks, "Live and Automated Telephone Surveys: A Comparison of Human Interviewers and an Automated Technique", Journal of the Market Research Society, 1991, 33, 2, 91-102. Comparison of the representativeness and reliability of completely automated telephone interviews by human interviewers. Examined are differences in response due to demographic characteristics, attitudes, or behaviour. Secondary analyses of data collected by telephone interviews by means of these two techniques show differences due to age and education of the respondent. Emphasis on the importance of correct question formulation in this type of interview.
4239B, Floyd J. Fowler, Jr., Survey Research Methods, 1988, Second Edition, Sage publications, London, bibliography, index, 159 pp., isbn 8093 3294 4, £13. The eleven chapters of this book are: ... 7. Survey Interviewing with sections on Overview of Interviewer Job, Interviewer Recruitment and Selection, Training Interviewers, Supervision, Interviewing Procedures, Validation, and The Role of Data Collection in Survey Error; ... 9. Ethical Issues in Survey Research with sections on Informing Respondents, Protecting Respondents, Benefits to Respondents, and Ethical Responsibilities to Interviewers.
4339B, Jost Reinecke, Interviewer - und Befragtenverhalten: theoretische Ansätze und methodische Konzepte (Comportement de l'interviewer et des interviewés. Approches théoriques et conceptions méthodologiques), Westdeutscher Verlag, Opladen, 1991, Studien zur Sozialwissenschaft, 106, isbn 3 531 12290 8, 328 pp.; Relation enquêteur-enquêté, Interview, Biais, Aspiration, Attente.
4439A, Philippe Cibois, Alain Degenne et Jean-Paul Grémy, "Analyses bibliographiques et notes critiques: Méthodologie", Année sociologique, 1990, 40, pp. 333-355. Une série de comptes rendus de livres divisée en quatre sections. ... Alain Blanchet, Rodolphe Ghiglione, Jean Massonnat et Alain Trognon, Les techniques d'enquête en sciences sociales. Observer, interviewer, questionner, 1987, Dunod, Paris, 197 pp.
4539A, J. Reinecke, "Interviewereffekte und soziale Erwünschtheit: Theorie, Modell und empirische Ergebnisse" (Influence de l'interviewer et désirabilité sociale: théorie, modèle et résultats empiriques), Journal für Sozialforschung, 1991, 31, 3, 293-320. Interview, Interviewer Influence, Relation enquêteur-enquêté, Biais, Modèle, Modèle causal, Application.
4640B, Paul J. Lavrakas, Telephone Survey Methods: Sampling, Selection, and Supervision, 1993, Sage, London, Second Edition, Volume 7, 160 pp., isbn 8039 5307 0, £13. This book offers researchers a guide to thinking about and executing telephone surveys from generating telephone numbers to structuring the work of interviewers and supervisors. New to this edition are a focus on the total survey error concept, a comparison of CATI (computer-assisted telephone interviews) with PAPI (paper and pencil inventory) procedures, mixed-mode surveys and new telecommunication technologies. The book also covers such topics as how to control the sampling pool, how to identify appropriate respondents and how to improve on-the-job training of interviewers. It includes the following chapters: Introduction; Generalizing Telephone Survey Sampling Tools; Processing Telephone Survey Sampling Pools; Selecting Respondents and Securing Cooperation; Supervision I - Structuring Interviewer's Work; and Supervision II - Structuring Supervisory Work.
4740J, Lisette Pondman (Faculty of Social Cultural Sciences VU, Department of Social Research Methodology, Koningslaan 22-14, 1075 AD Amsterdam, Netherlands), The Influence of the Interviewer on Nonresponse due to Refusals in Telephone Survey Interviews, dissertation project under the supervision of J. van der Zouwen, W. Dijkstra and A. J. M. Dijker. The willingness to participate in an interview is decreasing. The increasing nonresponse urges an investigation of the causes, effects and possible solutions to this phenomenon. This project is focused on the influence of the interviewer on the respondent's decision whether or not to participate in a telephone interview. The project aims to answer the following questions: (1) Do interviewers vary in the way in which they try to persuade respondents to participate in a telephone interview?; (2) If so, do these difference influence the respondent's final decision to refuse or not?; and (3) Is it possible to instruct and train interviewers in such a way that nonresponse due to refusals to participate will be minimized; in other words, is it possible to teach interviewers optimum "persuasive conduct"?
4841J, J. H. Smit (AIO/IOPS, Faculty of Social Cultural Sciences VU, Department of Social Research Methodology, Koningslaan 22-24, 1075 AD Amsterdam, Netherlands), Suggestive Questioning and Probing in Interviews; Occurrence, Causes and Effects; Supervision J. van der Zouwen and W. Dijkstra; Period 1988-1993. In some types of interviews it is expected that interviewers probe for motivations and explanations for information previously given by respondents (e.g. with regard to attitudes and behaviour). Usually the formulation of the probe has to be left to the insight of the interviewer. Investigation of these formulations shows that interviewers often use phrases which are "suggestive". Insufficient knowledge exists of the occurrences, of this inadequate interviewer behaviour, the causes of this behaviour, and the effects of the behaviour on the quality of the information gathered. Therefore the formulation of the research problem of this project can be divided into four subproblems: a. How can the degree and nature of the "suggestivity" of question (probe) formulations be determined? (Development of a coding system for question-answer protocols). - b. How often do interviewers use (certain types of) suggestive formulations of questions or probes? (Analyses of the question-answer protocols with the coding system mentioned in a). - c. Which possible causes can be found for this behaviour? (Analyses of the coherence of different types of suggestive formulation of question on the one hand with certain (combinations of) distinctive characteristics of interviewers, respondents, topic(s) of conversation, interview situation and so on, on the other hand). - d. What are the effects of this behaviour on the quality of the information gathered from the respondents? (Analyses of the coherence between the "suggestivity" of question formulations and the quality of the responses). The research problem will be answered with the use of - among things - transcripts of taped question-answer exchanges; statistical analyses of coherencies of different types of suggestive formulations questions on the one hand with certain (combinations of) distinctive characteristics of interviewers, respondents, the interview and responses on the other hand and small scale experiments aimed at the determinations of the effects of suggestivity on the quality of responses.
4941A, Maria Elena Sanchez, "Effects of Questionnaire Design on the Quality of Survey Data", Public Opinion Quarterly, 1992, 56, 2, summer, 206-217. While inadequate questionnaire design is perceived as likely to contribute to measurement error, scan empirical evidence is reported in the literature to support this claim or to allow estimation of the magnitude of effects attributable to questionnaire design. Here, two surveys are examined in which, different questionnaire designs were used by field interviewers to administer the same set of survey questions to a randomly selected adult in households that comprised area probability samples of the Detroit, Michigan, metropolitan area. The later study was designed to replicate a survey completed a few months earlier. The population & sample design were identical, & survey instruments remained the same with changes restricted to differences in the layout & formatting of text on the printed questionnaire page. Both studies employed a mix of experienced & inexperienced interviewers. Empirical evidence of specific effect (questions skipped in error, unprobed answers) directly attributable to the choice of questionnaire design is presented. Further, the behaviors of experienced & inexperienced interviewers are affected in similar ways. It appears that interviewing experience does not compensate for format deficits in the design of survey instruments.
5042B, Jean Morton-Williams, Interviewer Approaches, Dartmouth, Aldershot, 1993, 256 pp., isbn 1 85521 339 7, analyzes the components of non-response in sample surveys and their causes and ways of improving response.
5143A, Jost Reinecke and Peter Schmidt, "Explaining Interviewer Effects and Respondent Behavior: Theoretical Models and Empirical Analysis", Quality and Quantity, 1993, 27, 3, 219-247. Ajzen's and Fishbein's theory of reasoned action (1980) is formalized via structural equation models. In applying this model to interviewer effects and response sets, it was assumed that actors in interview situations decide according to cost-benefit calculations. The models were empirically tested with survey data from 125 Germans on reported contact rates with foreigners, first without interviewer variables to explain respondent behavior in terms of norms, attitudes, and other determinants. Then, using the LISREL multiple group comparison technique, response behavior was analyzed by the structural variables of status and age of interviewer and respondent's need for social approval. Findings enhance knowledge of interviewer characteristics and social desirability.
5243A, J. J. Hox, "Modeling Interviewer Effects with Multilevel Models", Kwantitatieve Methoden, 1992, 13, 41, 59-74. On the application of the multilevel regression model to analyze interviewer effects. Discussed and illustrated are advantages in using the hierarchical regression model for research on the effects of specific interviewer and respondent characteristics in survey research. In an example on model selection, a controlled field experiment on mode effects is described. Respondent and interviewer effects are examined in three types of data collection methods: face-to-face interviews, telephone interviews, and computer-assisted telephone interviews.
5343C, "SEQUENCE - A program for analysing sequential data", by Wil Dijkstra (Department of Social Research Methodology, Vrije Universiteit, Koningslaan 22-24, 1075 AD Amsterdam, Netherlands). Abstract: The SEQUENCE program offers a wide range of procedures for the analysis of sequential data. In addition to usual kinds of analyses like lag-sequential analysis, the program is especially developed to answer questions with respect to the occurrence of particular patterns in sequences and the degree of similarity between different sequences. ... As mentioned before, we used the program for the analysis of life histories. Moreover, the program was successfully used to analyse interviewer-respondent interactions (Smit and Dijkstra, 1991) and newspaper analyses (van den Berg, Smit and van der Veer, 1992).
5449BJC, Magda Stouthamer-Loeber and Wilmoet Bok Van Kammen, A Practical Guide for Data Collection and Management, 1995, Sage Publications, London, 152 pp., isbn 0 8039 56576, $17, eight chapters on: Introduction; Planning and Budget; Hiring; Training of Interviewers; Participant Acquisition and Retention; Supervision of Interviewers; Data Management; and Quality Control and Research Ethics.
5549BJC, James A. Holstein and Jaber F. Gubrium, The Active Interview, 1995, Sage Publications, London, 96 pp., bibliography, isbn 0 8039 58951, $10, eight chapters on: Introduction; The Active Interview in Perspective; Assigned Competence and Respondent Selection; Narrative Resources; The Active Interviewer; Constructing Meaning Within the Interview; Multivocality and Multiple Respondents; and Rethinking Interview Procedures.
56BMS, n. 51, June 1996: Pieter van den Eeden, Johannes H. Smit, Dorly J. H. Deeg & Aartjan T. F. Beekman, "The Effects of Interviewer and Respondent Characteristics on Answer Behaviour in Survey Research: A Multilevel Approach".
5751J, The European Newsletter for Psychometrics and Sociometrics (Riek Slotboom, Faculty of Psychology UvA, Roetersstraat 15, 1018 WB Amsterdam; tel 31 20 525 6880; fax 31 20 639 0026; email iop_slotboom@macmail.psy. uva.nl) has published several interesting doctoral dissertation projects including: ... L. M. Pondman, The Influence of the Interviewer on Nonresponse due to Refusals in Telephone Survey Interviews.
5853A, Jost Reinecke, "Interviewerwahrnehmung, Befragteneinschätzung und soziale Erwünschtheit" (in Heiner Meulemann and Agnes Elting-Camus (editors), 26. Deutscher Soziologentag - Lebensverhältnesse und soziale Konflikte im neuen Europa: Sektionen, Arbeits- und Ad hoc-Gruppen, 1993, Westdeutsche Verlag, Opladen, 578-581), analyzes social desirability and interviewer influence on survey responses with latent class analysis to identify types of response behavior.
5957A, Jennifer Dykema, James M. Lepokwski and Steven Blixt, "The Effect of Interviewer and Respondent Behavior on Data Quality: Analysis of Interaction Coding in a Validation Study", in Lars Lyberg, Paul Biemer, Martin Collins, et al. (editors), Survey Measurement and Process Quality, 1997, John Wiley, New York.
6058A, Joop J. Hox, Edith D. de Leeuw and Ita G. G. Kreft, "The Effect of Interviewer and Respondent Characteristics on the Quality of Survey Data: A Multilevel Model", in Paul P. Biemer, Lars E. Lyberg, Robert M. Groves et al. (editors), Measurement Errors in Surveys (1991, John Wiley, New York), 439-461, examines the impact of interviewer socio-demographic variables, personality traits, social skills, and opinions and attitudes on responses. Response variability, attributable to the interviewers, is estimated by means of a hierarchical linear model or random component model, also called a variance component model.
6159A, Geert Loosveldt and Ann Carton, "Evaluatie van datakwaliteit in surveyonderzoek: Het gebruik van het na-interview en het rapport van de interviewer", Mens en Maatschappij, 1997, 72, 2, 166-175, on the evaluation of data quality in survey research by re-interviews of respondents and reports of interviewers which are used to construct a typology of respondents.
6260J, Edith D. de Leeuw, Joop J. Hox, Ger Snijkerset al., Interviewer Opinions, Attitudes and Strategies Regarding Survey Participation and their Effect on Responses (1998, MLS Publication, Amsterdam, 17 pp.), is a conference paper presented in September 1997 in Mannheim, Germany, at the eighth international workshop on Household Survey Nonresponse.
6360A, Pamela Campanelli, "A New Programme of Nonresponse Research at SCPR, London" (in Seppo Laaksonen (editor), International Perspectives on Nonresponse (1996, Tilastokeskus, Helsinki), pp. 30-36), describes the Social and Community Planning Research (SCPR) study of the role of interviewers in survey nonresponse. The study will isolate the effects of interviewers from other effects such as location of respondents or address and respondent characteristics. It will also analyze interviewers' patterns of making contact with potential respondents.
6460A, Colm O'Muircheartaigh and Pam Campanelli, "The Relative Impact of Interviewer Effects and Sample Design Effects on Survey Precision", Journal of the Royal Statistical Society, 1998, 161, 1, 63-77, discusses the interviewer as one of the principal sources of error in data collected from structured face-to-face interviews. The other major component of imprecision is sampling variance.
6560A, Mick P. Couper, "Survey Introductions and Data Quality", Public Opinion Quarterly, 1997, 61, 2, 317-338, used data from the 1990 National Election Study during which the interviewers completed questions after each interview concerning what was said by the sampled person and the interviewer, and other characteristics of the initial contact. Data from the interviewed cases was compared with that of nonrespondents to see how initial statements differed and how subsequent survey data quality varied.
6660A, Johannes H. Smit, Wil Dijkstra and Johannes van der Zouwen, "Suggestive Interviewer Behavior in Surveys: An Experimental Study", Journal of Official Statistics, 1997, 13, 1, 19-28, varied three kinds of suggestive interviewer behavior concerning the presentation of response alternatives following a closed question about consequences of aging. The results show that suggestive interviewing affects the quality of the data collected.
6760A, J. A. Catania, D. Binson, J. Canchola et al., "Effects of Interviewer Gender, Interviewer Choice and Item Wording on Responses to Questions Concerning Sexual Behavior", Public Opinion Quarterly, 1996, 60, 3, 345-375.
6861A, Patrick Sturgis and Pamela Campanelli, "The Scope for Reducing Refusals in Household Surveys - An Investigation Based on Transcripts of Tape-Recorded Doorstep Interactions", Journal of the Market Research Society, 1998, 40, 2, 121-139, analyzes interactions between interviewers and respondents that were tape-recorded and classified in terms of degree of reluctance expressed by the respondent and the consequent scope for the interviewer to deploy interpersonal skills and persuasion.
69BMS, n. 62, April 1999: Edith D. de Leeuw, "How Do Successful and Less Successful Interviewers Differ in Tactics for Combating Survey Nonresponse?"
7064A, Jost Reinecke, "Explanations of Social Desirability and Interviewer Effects", in Dagmar Krebs and Peter Schmidt (editors), New Directions in Attitude Measurement (1993, De Gruyter, Berlin), 315-337.
71BMS, n. 67, July 2000: Ongoing Research / Recherche En Cours, Ann Carton, "An Interviewer Network: Constructing A Procedure to Evaluate Interviewers".
7273J, Long-time RC33 member, Anuska Ferligoj, is editor of a series entitled, Metodoloski Zvezki, which publishes selected papers from the annual International Conference on Methodology and Statistics in the Social Sciences in Slovenia which Anuska Ferligoj organizes (See BMS, n. 70, p. 105). ... Volume 15 (2000, Faculty of Social Sciences, Ljubljana, isbn 961 235 039 6, 166 pp.) includes papers presented during the 1997 and the 1998 conference, and one invited paper: J. M. Lepkowski, V. Siu and J. Fisher, "Event History Analysis of Interviewer and Respondent Survey Behavior".
73BMS, n. 76, October 2002: Michael Lemay & Claire Durand, "The Effect of Interviewer Attitude on Survey Cooperation". Research to improve the quality of social surveys has been shown that methodological factors have a significant impact on respondent cooperation. The research presented here is inspired by the sociology of work and studies the impact of interviewers' sense of motivation and "self-efficacy" in convincing respondents to answer surveys. To do so, a survey of interviewers working for three private pollsters was conducted during the Canadian federal election campaign of November 2000. The results indicate some evidence of relationships between motivation, sense of self-efficacy (and associated behaviour), and interviewer performance. However, some of the results could be better explained by differences in training. In addition, the lack of a better measure of performance may also explain the weakness of the results. Future studies should examine performance measurement and try taking into account how performance changes over time.
7478J, In ZUMA Nachrichten (n. 51, November 2002): Tatjana Mika, "Wer Nimmt Teil an Panel-Befragungen? Untersuchung ueber die Bedingungen der Erfolgreichen Kontaktierung fuer Sozialwissenschaftliche Untersuchungen" (pp. 38-48), on who takes part in panel surveys, stressing the importance of interviewer selection.
7579B, Robert L. Mille and John D. Brewer (editors), The A-Z of Social Research - A Dictionary of Key Social Science Research Concepts (2003, Sage Publications, London, 345 pp.isbn 0 7169 7133 5, ú20). The editors present this book as "a 'research methods' textbook with a difference. Rather than a normal text, this book can be thought of as an encyclopaedia of social research." Indeed, it is not a sociology encyclopaedia with an exhaustive intention, but an up-to-date methods textbook which includes many entries and terms one cannot find in more classic compilations of sociological terms and concepts. This includes entries on Computer-Assisted Personal Interviewing, e-mail as a research tool, Feminist methodology, geographic information systems, Interviewer effects, and many more. A handy addition to already well-stocked sociology libraries.
7680J, Quality and Quantity, 36 (2) May 2002: "The Importance of Experimental Control in Testing the Impact of Interviewer Continuity on Panel Survey Nonresponse", Campanelli Pamela, O'Muircheartaigh Colm.
77Quality and Quantity, 36 (4) November 2002: "A Multilevel Multinomial Analysis of Interviewer Effects on Various Components of Unit Nonresponse", Pickery Jan, Loosveldt Geert.
7885J, International Journal of Social Research Methodology (Volume 7, Number 5 2004): David Gadd, "Making sense of interviewee-interviewer dynamics in narratives about violence in intimate relationships".
Pour citer cet article
Référence papier
Karl M. van Meter, « Studying Survey Interviewers: A Call for Research and an Overview », Bulletin de méthodologie sociologique, 88 | 2005, 51-71.
Référence électronique
Karl M. van Meter, « Studying Survey Interviewers: A Call for Research and an Overview », Bulletin de méthodologie sociologique [En ligne], 88 | 2005, mis en ligne le 08 juillet 2008, consulté le 13 octobre 2024. URL : http://journals.openedition.org/bms/810
Haut de pageDroits d’auteur
Le texte et les autres éléments (illustrations, fichiers annexes importés), sont « Tous droits réservés », sauf mention contraire.
Haut de page