1A series of empirical studies suggests that respondents' generalized attitudes towards surveys are important determinants for different kinds of cooperative behavior in survey contexts. Accordingly, respondents with a positive attitude towards surveys, compared with those who have a more critical position, have been found to have participated more often in surveys in the past (Goyder 1986). Evidence for the explanatory power of this factor is also found in studies where the effort necessary to obtain survey participation is compared for respondents with positive and negative attitudes towards surveys. Here, subjects who initially refused to participate, but were convinced by a second request to take part in the interview, proved to have significantly more negative attitudes than subjects who spontaneously agreed to participate (Erbslöh/Koch 1988; Stinchcombe/Jones/Sheatsley 1981). More evidence for the effect of survey attitudes on subjects' willingness to cooperate in surveys has been provided by studies where respondents with more positive attitudes have been found to follow more closely the questionnaire instruction in a mail survey, to return the completed questionnaire faster, to answer the questions more completely and to express more likely their willingness to participate in future surveys (Rogelberg/Fisher/Maynard/Hakel/Horvath 2001). In other studies, it has been found that respondents' answers about their racial prejudice are less susceptible to social desirability bias when they have a positive and, at the same time, highly accessible attitudes towards surveys in general (Stocké 2003a).
2The empirical evidence sketched in the previous section indicates that respondents' attitudes towards surveys can substantially affect the quality of data collected in survey interviews. It is therefore an important question to know which factors determine these attitudes. Studies from other areas of research have shown that the direct experience with the respective attitude object is a pivotal determinant for subjects' generalized attitudes (for instance, Hazzard 1983; Jay/Willis 1992). However, there is only little evidence available concerning the effect of interview experience on respondents' attitudes towards surveys (Porst 1998). Most available studies have tested how objective aspects of respondents' burden — as for example, the length of interviews or the number of requests for survey participation — affects the evaluation of an actual interview (Költringer 1992; Sharp/Frankel 1983). The subjective evaluation of these aspects of interview experience and their relative significance for subjects' attitudes towards surveys, in general, have not been analyzed yet.
3Our article analyzes, in a first step, whether or not respondents differentiate between different criteria when evaluating their participation in survey interviews in the past. Taking the recorded results as a starting point, a second step tests whether or not, and to what relative extent, the image of surveys in general is affected by these evaluation dimensions. In addition to these general dimensions, we also take into account more concrete aspects of respondents' interview experiences. These are the mode of administration, the sponsor of subjects' last survey interviews, the time elapsed since that interview, the total number of surveys respondents participated in the past and whether or not respondents were ever asked for an interview which turned out to be a sales pitch.
4In Germany, the percentage of respondents, who have judged their interview experience either as positive (about 50%) or as negative (about 20%), is rather stable over time (Forsa 1991, 1993a, 1993b). Despite the fact that the population in Germany, compared with other countries, has a relatively critical attitude towards surveys and the increasing number of polls intensifies respondents' burden, survey research has nevertheless a rather positive image (Forsa 1993a, 1993b; Leiblein/Klass 1990; Leiblein/Oglesby 1993). In contrast, the survey industry in the United States has been confronted with increasing numbers of more critical respondents (Kohut 1986; Schleifer 1986).
5Sjoberg's study, "A Questionnaire on Questionnaires" (1955), was the first to deal with the effects of interview participation on respondents' orientation towards surveys. Most survey researchers assume since that time, "... that the interview experience is on balance a positive rather than a burdensome one" (Sharp/Frankel 1983: 43). It is, however, well known that a substantial part of respondents regard certain questions as an invasion of privacy (see for example, Hartmann/Isaacson/Jurgell 1968). Furthermore, one has to keep in mind that respondents taking part in surveys about surveys are likely to be a selective sample with respect to their survey experience and attitudes towards surveys. Accordingly, results can not simply be generalized for the evaluation of subjects who refused survey participation.
6Generalized attitudes are undifferentiated, and independent from situational demands and behavioral dispositions towards a whole class of attitude objects which are grouped together in the respective category. In this sense, the selection of cooperative or uncooperative behavior for surveys, based on attitudes towards surveys in general, is contradictory with the assumption of optimizing decisions in traditional theory of rational (respondent) behavior. However, in the perspective of an extended version of rational choice theory, the selection of a particular attitude is a "constitutional" decision which enables bounded rational and, for single case decisions, insufficiently motivated actors to deal with a complex world (Vanberg 2002). In this theoretical framework, the adoption of an attitudinal disposition is the result of an instrumentally rational meta-decision which is corrected in the light of new information and experience with the attitude object.
7Respondents have been found to evaluate whether or not surveys are accurately implemented and capable of providing reliable information (Forsa 2000; Roper 1986; Stinchcombe/Jones/Sheatsley 1981). Furthermore, the evaluation of surveys is dependent, in particular, on the general interview topic and on the respondents' interest in survey results (Jones/Sheatsley/Stinchcombe 1979). Especially young and highly-educated subjects are found to be interested in survey results presented in the mass media (Forsa 1993a; Leiblein/Oglesby 1993). Surveys have also been found to be more positively evaluated in general when they are believed to foster respondents' private interests or perceived to be valuable for society (Bradburn 1979).
8The evaluation of surveys is, furthermore, likely to be affected by the respondents' burden and the costs subjects associate with participation in survey interviews. For instance, respondents have evaluated to what degree they judge survey questions to be too personal or whether or not they believe, in general, that too many surveys are being carried out (Roper 1986; Sharp/Frankel 1983). Another probably relevant factor for subjects' attitudes towards surveys in general is the perceived ratio of time needed to complete survey interviews and expected utility of survey results (Roper 1986).
9In the field of attitude research, it is commonly assumed that subjects utilize aspects of their direct experience with the respective attitude objects to form their attitudes (Eagly/Chaiken 1993). With respect to respondents' attitudes towards surveys, some evidence indicates that experience with survey interviews, if available, shapes these attitudes. Porst (1998) found a high degree of willingness to be reinterviewed in a sample where the majority of respondents judged the actual interview to be enjoyable and interesting. In contrast, one can expect negative survey experiences, for example, when survey interviews are abused as a sales pitch, or when long and monotone item batteries are used, and these engender more critical evaluations of survey research (Bradburn 1979; Sharp/Frankel 1983). However, it has to be emphasized that respondents' burden is not necessarily identical with objective features of the interview, but depends on the respondents' subjective reaction on these features (Bradburn 1979). Four dimensions of subjective burden have been assumed: the length of the interview, the cognitive effort necessary to answer the questions, the frequency respondents are asked to take part in surveys and the psychological stress caused by sensitive question topics (Bradburn 1979). Beside the already mentioned determinants, the following factors are probably relevant for respondents' attitudes towards surveys.
10Misuse of Surveys as a Sales Pitch: One possible factor influencing the general evaluation of survey research is the abuse of surveys as a "door opener" for the purpose of selling good and services (Baxter 1964; Schleifer 1986; Sheets/Radlinski/Kohne/Brunner 1974). Evidence for this assumption has been found in a survey with interviewers about different aspects of their work. In this case, 51 percent reported that at least once a prospective respondent has refused to take part in an survey because he or she suspected that the interview would turn out to be a sales pitch (Rugg 1971). Reports about the proportion of respondents who have been deceived in such a way vary considerably. According to Biel (1967), 60 percent of all respondents already had such an experience, whereas Schleifer (1986) has found that only 36 percent of respondents reported an abuse of surveys by salesmen. The critical question is, whether or not deceived respondents associate survey research with their negative experience. In this case, it is likely that respondents' attitudes towards surveys and their cooperation in subsequent surveys are negatively affected (McDaniel/Verille/Madden 1985; Schleifer 1986). This possibility has not been analyzed empirically.
11Interview Length: Whether the burden associated with long interviews affects respondents' general evaluation of survey research has been analyzed in a few empirical studies (see for example, Schleifer 1986; Sharp/Frankel 1983). Sharp and Frankel (1983) found in their experimental study that longer interviews were evaluated more negatively and that respondents, under this condition, are less willing to be reinterviewed. This negative evaluation of long interviews, and the fact that face-to-face interviews are typically more extensive than other types of interviews, is probably the reason why only 14 percent of respondents prefer to be interviewed face-to-face, whereas 28 percent would prefer to participate in mail surveys and 40 percent in a telephone interview (Forsa 2000). In the field of methodological research, it is commonly assumed that the time necessary to complete a survey interview leads to a negative evaluation of surveys and reduces the participants' willingness to take part in future surveys (Nederhof 1981).
12Frequency of Requests for Survey Participation: Survey researcher have been concerned about the possibility of negative effects of frequent requests for interview participation and, in particular, the consequences of this "over surveying" for respondents' attitudes towards surveys and their willingness to participate in surveys (Dillman/Carpenter/Christenson/Brooks 1974; Groves/Kahn 1979; Nederhof 1981). There are no investigations concerned explicitly with effects on the generalized attitudes towards surveys caused by frequent requests for interview participation, and there are only some which analyze direct effects on the willingness to participate in surveys. So we were forced to take the latter as an indicator for attitude differences. The results of these studies are inconsistent. Sharp and Frankel (1983) did not find a correlation between the frequency of interview participation and the readiness to participate in future surveys. In contrast, Nederhof has concluded from his data that subjects with interview experience are more likely to comply with an request for interview participation and, in each case, decide faster (Nederhof 1981, 1986). Evidence suggests that the readiness to participate in surveys increases with positive interview experiences, decreases with negative experiences and are lowest for subjects with no experience at all. The effect of neutral survey experiences is unsettled, but must presumably be located between the two extremes (Nederhof 1987). Goyder (1986) found an association between the attitudes towards surveys, in general, and the probability that the respondents had refused at least one interview in the past: this probability increases when subjects had a more critical attitude.
13The respondents in our study were a multi-stage, local probability sample of residents in the metropolitan area of Mannheim, Germany (about 300,000 inhabitants). In the first step, households were listed by project staff using a random walk procedure. The starting points for this procedure were randomly selected addresses in the sampling area. The listed addresses were assigned to the interviewer who, in a second step, selected the respondents among the adult residents in the households using the "last-birthday" method. Altogether 139 interviews were realized with 55.4 percent respondents being female and 44.6 percent male. The survey participants were on average 47.1 years old and had completed on average 10.3 years of schooling. The response rate was 40.4 percent. The social background characteristics of our sample deviated in the typical way from those of the population. Accordingly, persons with college degree (Abitur) are 11, married persons 10 and white collar employees 18 percentage points over represented. This leads to an under representation of the respective other educational, marital- and occupational status groups. With respect to the respondents' income, we find the usual "middle class bias": members of the population with low and high income are under represented in the sample. According to age and participation in the labor market, we do not find systematic deviations from the population.
14Data was collected with computer-assisted interviews at the respondents' homes where the respondents were randomly assigned to either an interviewer or self-administered mode of data collection. In the first mode, the interviewer read out the questions and recorded the answers (CAPI mode). In the second mode case, respondents read the question alone from the computer screen and typed in their answers (CASI mode). Here, the interviewer was present in order to also answer possible respondents' questions, but were instructed to keep enough distance to ensure respondents' sense of privacy. This variation of response mode was introduced to test whether or not possible associations between the evaluation of subjects' survey experience and their attitudes towards surveys are the result of only social pressure towards consistent responses. If this were the case, we should have found substantially stronger associations between both constructs under the condition of the "public" CAPI interviews.
15The questions about the respondents' evaluation of their past interview experience and their attitudes towards surveys in general where asked at the beginning of a longer interview, which was altogether 122 questions long. The other questions in the interview were on different topics, including the respondents' life satisfaction, their party preferences, attitudes towards environmental issues and racial attitudes. The interview took an average 35 minutes to be completed. In addition to the respondents' answers, the response time for these answers were recorded without subjects knowing.
16To realize the aim of our study, the respondents' generalized attitudes towards surveys, their evaluation of interviews in the past, as well as the cognitive accessibility of this evaluation, had to be measured. It was expected that cognitively more accessible interview experiences and evaluations of these experiences exert strong effects on subjects' attitudes towards surveys. In our analysis, the effect of other aspects of respondents' interview experience was tested and their relevance statistically controlled.
17Generalized attitudes towards surveys: In a first step, 31 Likert-type attitude items were collected from available studies in the literature (Dran/Hildreth 1995; Erbslöh/Koch 1988; Forsa 1993a, 1993b, 1996; Goyder 1986; Porst 1998; Stinchcombe/Jones/Sheatsley 1981). The responses on this items from a random sample in a preliminary study were used to select 16 attitude items with a maximum degree of internal consistency (see the item wording in table 1 below). The respondents from our main study indicated — on a seven point response scale — how strongly they agreed or disagreed with each of the randomly presented item contents. Responses on negatively worded items were recoded in such a way that, for all items, high scores reflects a positive attitude towards surveys. For each respondent, the average of all 16 attitude scores was computed and used as an operationalization for their attitude towards surveys in general. The resulting index varies from 1 (negative attitude towards surveys) to 7 (positive attitude towards surveys). The internal consistency of this attitude scale is 0.73, measured by Chronbach's α.
Table 1: Items to record subjects’ generalized attitudes towards surveys
18(under preparation/en préparation)
19Evaluation of the last survey interview: Respondents, who had participated in at least one survey interview in the past, were asked to evaluate different aspects of their last interview. These evaluations were recorded using a total of six Likert-type items with seven point response scales. Two of these items were related to the degree subjects judged the questions in their last interview to be confusing because they were either unable to understand the meaning, in general, or because they found the question wording ambiguous. Two other items captured respondents' judgment about how interesting or enjoyable, and therefore valuable, they found the last interview. On the remaining two items, respondents indicated the degree of burden they experienced in the last survey interview. Subjects indicated to what extend they found their last interviews too long and whether or not they felt exhausted afterwards (see the item wording in table 2 below). The results from a principle component analysis, with orthogonal rotation of factors, confirms the three-dimensional nature of subjects' evaluations (results are presented in table 7). Accordingly, respondents differentiated the burden experienced in the last interview, how valuable they found this experience and the degree to which they regarded the question wording to be unsatisfactory. After recoding the answers in a way that high scores always represent a positive evaluation of the interview experience, we computed mean evaluation scores for each respondent on the three subscales. As a result, the three indices ranges from 1 (negative evaluation) to 7 (positive evaluation).
Table 2: Evaluation of the last survey interview and response latencies when answering the evaluation questions (respondents with at least one interview experience)
20(under preparation/en préparation)
21Cognitive accessibility of subjects' evaluations: The time subjects needed to answer the questions about the evaluation of their last survey interview was used as an indicator of the cognitive accessibility of this construct. Response latencies have been found to be a valid indicator for the ease with which attitudes and judgments are available from memory and as an predictor for the actual degree of attitude-behavior consistency (Bargh 1996; Bassili 1993; Bassili/Bors 1997; Bassili/Fletcher 1991; Stocké 2002, 2003b). Response times were measured, in the case of CAPI interviews, in such a way that interviewers read the question from the computer screen and switched on the time measurement directly after the question text had been read. The measurement was switched off immediately after respondents answered the question. Then the interviewer entered the response into the laptop computer and coded, in a last step, whether or not the time recorded represents exactly the time a subject needed to answer the question.
22In the case of self-administered CASI interviews, the item text was presented together with the response scale on the computer screen. Time measurement was automatically switched on when the screen became visible for the respondent and was switched off when the respondent entered the answer. Accordingly, in the case of CASI interviews, the time recorded included, besides the time which was necessary to answer the question, the reading time and the time the respondent needed to type in the answer. The systematically longer response latencies in CASI interviews were corrected as follows. First, for each item, we computed the average response time for CASI and CAPI interviews. Second, the mean CAPI response time is subtracted from the CASI time. In the third step, the resulting mean time differences were subtracted from all response times observed in CASI interviews. This correction leaves the relative differences in the response times between respondents unaffected, but it removes all systematic differences between the administration modes.
23The measurement of response latencies in field interviews necessarily leads to a certain proportion of invalid measurements. This happens for example when respondents asks clarifying questions, have to be probed to give an appropriate answer or when subjects are distracted by external factors (Bassili 1996). Under all these conditions the time recorded includes components that do not belong to the response generation process in a narrow sense and the interviewer coded the response latencies as invalid. Whereas this was done in CAPI interviews directly in the questionnaire program, interviewer noted in CASI interviews the affected questions on a "validity sheet". Altogether for 7 percent of the answers about the evaluation of the last survey interview were no valid response times available. In order to prevent a sample selection according to the reasons of invalid time measurements, missing values were imputed using the population mean of response times for each item. The precision of response latency measurement, based on the technical restrictions of the interview software, is one-hundredth of one second.
- 1 In many surveys, the contracting and implementing organization differs. In this cases, both organi (...)
24Other aspects of subjects' experience with surveys: In addition to subjects' evaluations of their last survey interview, we included five other aspects of this experience in the analysis. First, respondents were asked about the number of survey interviews they have taken part in the past. This information was recorded separately for interviews which took part in the last twelve month and older interviews. Second, subjects with interview experience reported how long ago they participated in their last survey interview. This tests the hypothesis of whether or not the effect of survey experience on attitudes towards surveys decreases with the time elapsed since this experience. The third additional factor taken into account in our study was the type of the last interview. Accordingly, subjects reported whether their last survey interview was a face-to-face interview at their home, a telephone interview, an interview conducted with a mail questionnaire or a mall intercept. Fourth, subjects were asked if the sponsor of their last survey interview was a market research firm, a public or scientific organization and whether or not this interview was conducted by an other sponsor.1 Fifth, respondents were asked if somebody had every misused a survey interview sell products or services.
25The empirical findings are presented in three sections. In the first section, we describe the different aspects of survey experience which were reported in our sample. In the second section, we analyze whether the subjects' evaluation of their survey experience differs according to the respondents' social characteristics, the extent of their interview experience, the mode of administration and the sponsor of respondents' latest survey interview. In the third and most important part of our analysis, we demonstrate whether or not and to what relative strength different dimensions of respondents' interview experience affects their attitudes towards surveys in general.
26The following describes the respondents' generalized attitudes towards surveys, their evaluation of the latest survey interview and other potentially relevant aspects of their survey experience.
27Generalized Attitudes Towards Surveys: The overall attitude towards surveys in our sample proved to be relatively positive (see table 1). Aside from one item, the average evaluation of surveys in our sample proved to be above the midpoint of the response scale and therefore in the positive attitude domain. With an average attitude score of 5.4, surveys are most favorable evaluated with respect to their value for the core social domains of the economy, politics and science (item 1). The burden caused by too lengthy questionnaires is evaluated most critically and received an average score of only 3.9 (item 15). Although the decision to participate in a survey interview, to a certain degree, presupposes a positive attitude towards surveys (Erbslöh/Koch 1988), we observed substantial attitude heterogeneity between the respondents in our sample. The mean responses on all attitudes items varies between a slightly negative value of 2.8 and a very positive value of 6.8 (see table 1).
28Evaluation, Dimensionality and Cognitive Accessibility of Survey Experience: Results from a principle component analysis proved that the evaluation of subjects' latest interview experience is based on three independent latent dimensions (see table 7). Two of the six items at a time are found to form a common latent dimension of evaluation. The three dimensions measures the extent to which respondents found their last survey experience to be burdensome, how they evaluated the interview with respect to its intrinsic benefit and how confusing they found the question wording of the last survey interview (see table 2).
29The respondents' evaluation of their most recent survey interview on the seven point response scale has been found to be on average 4.7 across all six evaluation questions. This indicates that this interview was a rather positive experience. However, subjects' judgments differ between the three evaluation dimensions. With a mean scale value of 4.9, the experienced burden receives the most positive evaluation, whereas the "entertainment" value of the last survey interview receives on average an scale value of 4.5 and is therefore the most negative evaluated. The difference between these two dimensions is marginally significant (t=1.77, df 90, p < 0.1). Subjects' evaluation of question comprehensibility is 4.8 and therefore located between the other two dimensions. This mean does not differ significantly from one of the other dimensions. The fact that very few respondents report extremely negative experiences with survey interviews leads us to the same question as Nederhof (1987: 430): "This may either mean that very few surveys give rise to very unpleasant experiences, or that only very unpleasant experience is enough reason to turn somebody into a nonresponder for a long time". In the following analysis, respondents' factor scores on each of the three orthogonal rotated latent evaluation dimensions are utilized as an indicator of their subjective survey experience.
30The response latencies observed, when subjects answered the two questions for each evaluation dimension, are found to form a single latent factor. Here as well, the respondents' factor scores on each of the three dimensions are utilized as an indicator for the cognitive accessibility of their subjective survey experience. According to the data presented in table 2, subjects needed on average 3.09 seconds to answer the questions about how burdensome they found their last survey interview, while judgments about the question quality took respondents 3.61 seconds to answer. This indicates that the former evaluations are significantly faster than the latter (t=1.99, df 90, p < 0.05). Evaluations about how interesting and enjoyable respondents found their last survey interview are located between the other two dimensions: on an average, these judgments took 3.16 seconds to be generated. This response latency does not differ significantly from those of the other two dimensions. Therefore, the burden experienced in the last survey interview can be regarded as the cognitively most accessible aspect of subjects survey experience. This result can be taken as first evidence for the importance of this evaluation dimension with respect to subjects' generalized attitudes towards surveys.
31Amount of Experience and Time since Last Survey Interview: In our local probability sample, we found an above average prevalence of survey experience. While in the year 2000, 47 percent of the West German population and 38 percent of East Germans had participated at least once in a survey interviews (Forsa 2000), this proportion is 66 percent in our sample. Respondents with survey experience are found to have already participated, on an average, in 4.0 interviews. When we also include respondents without former survey experience, the mean number of survey interviews in the past is 2.6 (table 3). When this survey experience is differentiated, according to the time elapsed since the survey participation, 0.6 survey interviews have taken place during the last 12 months and 2.0 interviews in the period before. Among the group of respondents with survey experience, 28 percent participated in one or two survey interviews, whereas only 6.4 percent have already answered between 10 and 20 questionnaires. The average time since respondents participated in their last survey interview was 24 months.
Table 3: Amount of survey experience and time since last survey interview
32(under preparation/en préparation)
- 2 The distribution of survey sponsors and of administration modes observed in our sample is similar (...)
33Sponsor and Mode of Administration of the Last Interview Experience: With respect to the sponsor of respondents' last survey interviews, we found the expected picture. A large majority of 61.5 percent indicated that they had taken part in a survey by a commercial market research firm. Only 20.9 percent of the respondents stated that their most recent survey interview was conducted by a public or scientific organization. In most cases, the survey interviews were administered as a telephone interview (40.7 percent), followed by mall intercepts which were conducted face-to-face at public places (25.3 percent). Face-to-face interviews at the home of the respondents were reported in 16.5 percent of the cases, and 12.1 percent have filled out a mail questionnaire.2 A high proportion of 40.7 percentage of respondents reported that their willingness to participate in a survey has been at least once misused to sell products or services.
Table 4: Other aspects of respondents' survey experience
34(under preparation/en préparation)
- 3 Aside from the significant explanation factors presented in table 5, we have furthermore tested wh (...)
35Respondents' attitudes towards surveys in general were found to differ according to four socio-structural characteristics (table 5, model 1).3 First, subjects who are married or live together with a partner proved to have more positive attitudes towards surveys than other subjects. Second, religious denomination is associated with generalized attitudes towards surveys insofar as Catholic respondents have a disproportional negative evaluation compared with other denominations or those who have no denomination. The third and surprising finding is that respondents lacking at least a weak embeddedness in a religious community, as indicated by the report of not attending religious services at all, have a significantly more negative evaluation of surveys in general. This association holds even when the subjects' religious denominations are statistically controlled at the same time. Accordingly, one third of all respondents who never attends services have a more negative attitude towards surveys. However, a comparison of less extreme subgroups, with respect to "church attendance", reveals no statistically significant differences. Fourth, women were found to have a marginally more positive attitude towards surveys than men. We cannot offer a sound interpretations for the described group differences in the respondents' attitudes towards surveys. These differences are nevertheless documented here, since they may establish a varying willingness of the subgroups' cooperation in survey contexts.
Table 5: Effects of respondents' socio-structural characteristics and their quantitative survey experience on their attitudes towards surveys (OLS regression results)
36(under preparation/en préparation)
- 4 The first category of the explanatory variable represents respondents without any survey experienc (...)
37The relation between the quantitative experience with survey interviews and the generalized attitudes towards surveys turns out to be very complex (table 5; model 2). In our data, no simple linear relationship between the number of survey interviews in the past and subjects' attitudes towards surveys could be found. The same holds for the association between the number of survey interviews within the last 12 months and those subjects participated in longer ago, on the one hand, and the attitudes towards surveys. This is, however, the case when availability and continuity of survey experience is taken into account simultaneously.4 In comparison to respondents who have participated continuously in surveys in the past (reference category), all other respondent groups are found to evaluate surveys more negative. This, in particular, applies to respondents whose interview experience stems either from only the last 12 month or for subjects who have participated only in surveys longer ago. Respondents without any survey experience hold a more positive attitude towards surveys than both subgroups mentioned, but proved to be more critical towards surveys than respondents who participated continuously in empirical studies (on this issue: McDaniel/Verille/Madden 1985: 78).
38These findings can be interpreted as the combined result of two different processes. One might, on the one hand, explain positive attitudes towards surveys as the cause instead of the result of a continuous survey participation. Respondents with a positive, strong and therefore stable attitude towards surveys may have a stable disposition to support surveys and to take part in them. Another process, which may contribute to the results reported above, is that respondents who did not participate in an survey interview during the last 12 months interrupted their initial cooperative behavior in the past because of bad experiences. In fact, we found a more positive overall evaluation of the last survey interview for subjects who continuously participated in survey interviews than for those who lack this continuity. However, the average evaluation of these two groups' survey experience is 4.82 versus 4.69 and this difference proved not to be statistically significant. Because the direct experience with the attitude objects leads to an increased crystallization and therefore polarization of the survey attitude, it is comprehensible that respondents without any survey experience are found to have attitudes which are located between the two experienced groups with a different continuity in interview participation. The negative attitudes towards surveys we found for respondents, who only participated in interviews in the recent past, are more difficult to interpret. This result is however based on a non-significant regression parameter and is only based on 3 percent of our sample. Therefore, we prefer to wait for a replication of this result.
39With the regression models presented in table 6, we tested whether different aspects of respondents' qualitative characteristics affect their attitudes towards surveys. In all of these models, the socio-structural characteristics tested in table 5 are included and their effect therefore statistically controlled, but the corresponding coefficients are not reported. In regression model 1, we tested whether or not the respondents' attitudes towards surveys in general are affected by the mode and sponsor of their last survey interview. As a first result, subjects who had participated in a face-to-face interview at home (reference category) are found to have more negative attitudes towards surveys, compared with those who have experienced another type of interview. In particular, a participation in a telephone interview proved to lead to a significant more positive attitude towards surveys. The participation in a mall intercept had only a marginal significant positive effect on subjects' evaluation of surveys in general.
Table 6: Effect of different dimensions of survey experience on respondents' attitudes towards surveys in general (OLS-regression results - respondents with survey experience only)
40(under preparation/en préparation)
41This finding seemingly contradicts result according to which respondents were found to judge face-to-face interviews at their home to be most pleasant (Forsa 1991, 1993a, 1993b; see also Reuband 1998: 61). However, when asked about their most preferred mode of administration, more subjects liked to be interviewed by telephone than face-to-face (Forsa 2000). From the available evidence, we conclude that respondents' preferences for a particular interview mode are determined by two different and contradictory aspects: while telephone interviews are more comfortable for the respondents (low costs), the "entertainment value" of this mode of administration is also relatively low (low benefit). However, as anonymity of survey data collection procedures becomes more important, the perceived enjoyment associated with interviews can be expected to loose weight for the evaluation of surveys in general and therefore for the decision to participate (Leiblein/Oglesby 1993: 52). Further data analysis will provide evidence for another factor which may be responsible for the observed mode effect on the respondents' attitudes towards surveys. Accordingly, face-to-face interviews are on average longer than other interview types, which might add to respondents' burden for survey participation and explain the negative impact of this administration mode on generalized attitudes towards surveys. The sponsor of the respondents' last survey interview, however, has no significant influence on the generalized evaluation of surveys, so that commercial interviews do not impair the general image of surveys (Forsa 1996; McDaniel/Verille/Madden 1985: 78).
42The results from our regression model 2 show that the misuse of surveys as a sales pitch to sell goods and services has a negative effect on respondents' attitudes towards surveys, but this effect does not reach statistical significance (table 6). Contrary to often stated concerns about the possible negative effect of such sales pitches, respondents' obviously do not blame the survey research institution for being responsible for these negative experiences.
43As another and pivotal finding of our study, it can be stated that attitudes towards surveys in general are more positive when respondents evaluated their last interview experience as less burdensome when they found the question wording to be less confusing and have described this interview as more enjoyable (table 6, model 2). However, only respondents' subjective experienced burden exerts a statistically significant effect on the attitudes towards surveys in general. The outstanding role of this particular dimension of survey experience confirms our findings from the descriptive analysis of response latencies where the evaluation of the last survey interview, according to this criteria, turned out to be cognitively most accessible. Model 2 reveals, furthermore, that the effect of the last survey interviews' administration mode on subjects' attitudes towards survey is greatly reduced and proves not to be statistical significant anymore, when respondents' direct evaluations of this interview and, in particular, how burdensome subjects found this experience is introduced into the regression equation. This result offers support for our assumption that differences in the experienced burden between the various administration modes explains the observed mode effects. We therefore conclude that face-to-face interviews in the past exert a negative effect on the evaluations of survey interviews in general because of the it's longer duration and thus higher degree of respondents' burden.
- 5 In our analysis, the problem of high multicollinearity, when estimating multiplicative interaction (...)
44In regression model 3, in addition to respondents' evaluation of their last interview experience, we included whether or not these evaluations and the attitudes towards surveys were recorded either by interviewer or in a self-administered interview mode (table 6).5 Here, we tested whether or not the observed associations between respondents' evaluations of their survey experience and their attitudes towards surveys are due to the fact that both self reports were collected in the same interview. One could suspect that first asking about the general attitudes towards surveys and then collecting evaluations about the interview experience might exert pressure on respondents to answer both types of items consistently. If our results are indeed an artifact of subjects' motive to appear consistent, one would expect weaker associations when privacy is ensured under the condition of self-administered interviews and stronger correlations when the interviewer's ability to perceive answers in interviewer-assisted interviews increases social pressure. This possibility was tested by introducing into the regression equation an interaction term between each of the dimensions of interview experience and the actual mode of data collection. However, none of the interaction effects proved to be a significant predictor. Accordingly, the explanatory power of none of the evaluation dimensions, and particular not that of the respondents' burden, for the attitudes towards surveys are found to differ according to the privacy of the mode of data collection. The alternative explanation for the observed associations is therefore not supported.
45One could expect the effect of respondents' evaluation of their last survey interview on the attitudes towards surveys is increasingly stronger the more recent these experiences are. This hypothesis was tested by including interaction terms between each of the three evaluation dimensions and the time since the last survey interview into the regression equation (table 6, model 4). As a result, none of these interaction effects proved to be a significant predictor of respondents' attitudes towards surveys. Accordingly, the effects of the different dimensions of survey experience and, in particular, the effect of respondents' burden on the evaluation of surveys, in general, does not differ according to the time which has elapsed since the particular respondents' last survey interview.
Table 7: The dimensions of respondents' evaluation of their survey experience (results from a principle component analysis with varimax-rotation)
46(under preparation/en préparation)
47In regression model 4, we also tested whether or not the cognitive accessibility of respondents' evaluation of their survey experience predicts how strong these evaluations affect attitudes towards surveys in general (table 6). This is done by including interaction terms between each of the three evaluation dimensions and the response times observed when subjects answered the respective questions. Our results indicate, first of all, that response latencies significantly mediate the effect of experience with surveys in the past on the actual attitudes towards surveys. However, this applies only for the dimension of respondents' burden, whereas the other two evaluation criteria are found to be irrelevant, even if respondents' heterogeneity with respect to the cognitive accessibility of their evaluations is taken into account. Since in model 4 the elapsed time since the last interview experience is statistically controlled, we conclude that response latencies capture other aspects of the saliency of respondents' evaluations than the mere ability to remember the respective interview situation.
- 6 Figure 1 shows the generalized attitudes towards surveys predicted from model 4 in table 6, for di (...)
- 7 One could raise the objection that the predictive power of response latencies for how strong attit (...)
48Figure 1 represents the significant interaction effect between the respondents' evaluation of burdensome aspects of their last survey interview and the time needed to answer the respective questions.6 According to the available results, the subjectively felt burden in the last survey interview is found to have no effect on the evaluation of surveys in general for respondents with long response times and therefore low cognitive availability of interview evaluations. In contrast, subjects' evaluation of their survey experience proved to exert a strong effect on respondents' attitudes towards surveys, when these evaluations are cognitively highly accessible and the underlying items were therefore answered rather fast.7
Figure 1: Effect of respondents' subjective burden in their last survey interview on their attitudes towards surveys for high and low accessibility of evaluations
49(under preparation/en préparation)
50The analysis in the present article has shown that different aspects of respondents' survey experience in the past have partly strong effects on respondents' attitudes towards surveys in general and, therefore, potentially on their willingness to cooperate in future surveys. In a first step, it was found that subjects evaluate their survey experience using three criteria which lead to three orthogonal evaluation dimensions. These dimensions are the burden caused by long and in general exhaustive questionnaires, how enjoyable and interesting subjects found their interview participation, and the degree to which the question wording was experienced as inappropriate.
51A second result is that respondents, whose last survey interview was a face-to-face interview, are found to have more negative attitudes towards surveys than subjects who took part in other types of surveys. However, this effect disappears when the respondents' direct evaluation of this interview, with respect to the length and the necessary effort, is statistically controlled. It is therefore concluded that the initially observed negative effect of face-to-face interviews, on the generalized attitudes towards surveys, is not the result of a general rejection of this mode of administration but is likely to result from the fact that this type of interview is typically longer and therefore more burdensome.
52As a third finding, the misuse of survey interviews as a "door opener" to sell goods and services is found to have no effect on respondents' evaluation of surveys in general: subjects who have at least once agreed to take part in an interview which turned out to be a sales pitch do not report more negative attitudes towards surveys than respondents who never experienced such a deception. We have therefore reason to believe that at least that part of the population which took part in our study, and therefore can be assumed to have an attitude towards surveys which is more positive than the average, obviously differentiates clearly according to the responsibility for their negative experience and do not simply hold survey research responsible for the misuse of interviews.
53Fourth, contrary to an existing hypothesis, the type of sponsor of respondents' last survey interview does not effect their attitudes towards surveys in general. We found no differences in respondents' attitudes when this interview was either conducted by a public authority or scientific organization, on one hand, or by a commercial market research firm, on the other. The hypothesis that the topics typically covered in commercial surveys are regarded as less legitimate and that the participation in such surveys has negative effects on the respondents image of surveys in general, cannot therefore be verified with our data.
54According to the fifth and most important result of our study, the three evaluation dimensions of survey experience affect respondents' attitudes towards surveys to different degrees. On one hand, neither the judgement of how enjoyable or interesting subjects found their last interview, nor how appropriate they regarded the question wording in that interview, has an effect on the evaluation of surveys in general. However, the subjective feelings, concerning to what degree their most recent survey interview was too long and in general an exhausting experience, proved to affect how survey research is evaluated in general. This is in itself an important result, since according to other research the attitudes towards surveys affects — for instance, mediated through the probability of item-nonresponse or the general willingness to participate in surveys — the quality of survey data. We can therefore conclude that survey researchers at least partly create, through the demands they make on survey participants, the conditions for their future work. Less extensive and difficult survey instruments can help to preserve the still currently positive image of survey research in the population.
55As a sixth result, the response speed observed when subjects answered the questions about the evaluation of their previous interview experience proved to be an indicator for individual differences in the cognitive accessibility of these judgments. Response latencies were found to predict how strongly respondents' evaluation of their experienced burden in the most recent survey interview affect their attitudes towards surveys in general: the faster evaluations were made, the stronger their effect on subjects' attitudes towards surveys. It is however unclear from our analysis which factors explain these differences in the accessibility of respondents' evaluations. The only factor which we were able to test, and which proved to be irrelevant, was the length of time between the last survey experience and the present study. Accordingly, the effect of survey experience on the attitudes towards surveys does not differ according to the elapsed time. Furthermore, the explanatory power of response latencies, in this respect, is found to be unaffected when the potential moderating role of the time since the interview experience is statistically controlled. Therefore, the extinction of subjects' experience from memory does not explain the differences in the cognitive accessibility of the evaluations.
56Since our data is based on a probability sample of a metropolitan area, the observed distributions of attitudes and evaluations cannot simply be assumed representative of the national level in Germany. It is well known that market research firms and scientific research organizations concentrate on large cities, which causes a disproportionately high interview experience in this areas (McDaniel/Verille/Madden 1985: 76). It is therefore an open question whether or not, in subgroups of the population with less interview experience, the subjective burden caused by survey participation has the same degree of salience and explanatory power for respondents' attitudes towards surveys. To answer this question and to enlarge the empirical basis of our results, a larger scale and more representative replication study should be done.
57Another open question is, whether or not our results can be generalized on the group of nonresponders. This seems to be questionable in the light of available evidence which suggests that in particular subjects with negative attitudes towards surveys do not participate in survey interviews (Goyder 1986; Rogelberg/Fisher/Maynard/Hakel/Horvath 2001). More evidence in this respect can be obtained if our study could be replicated with a sample of respondents who initially refused to take part in the survey interview but were convinced in a second attempt to do so. Such a sample can be assumed to be a "known group" with a disproportional negative attitude towards surveys (Erbslöh/Koch 1988).
58It has to be emphasized that we do not claim that the evaluation dimensions which we have analyzed in our study can be regarded as exhaustive. For instance, one can expect different aspects of interviewer behavior or the topic of the respective survey interview to establish further relevant dimensions for the evaluation of survey experience and, therefore, probably additional factors which explains subjects' attitudes towards surveys. Since, at present, little is known about how survey research affects a populations' willingness to cooperate in subsequent interviews, further research in this area should be done.