Search results

In survey methodology, it is well-known that interviewers can have an impact on the registered answers. This paper aims to focus on one type of interviewer effect that arises from the differences between interviewers in the systematic effects of each interviewer on the answers. In the first case, the authors evaluate interviewer effects on the measurement of alcohol consumption in European countries. The second case is about the interviewer effects on the respondents’ tendency to (non)differentiate their responses and the consequences of this response style for the correlation between variables.

The interviewer effects are evaluated by means of interviewer variance analysis. Because respondents are nested within interviewers, we can specify a two- or three-level random intercept model to calculate the proportion of variance explained by the interviewers. Data from the seventh round of the European Social Survey are used.

The results in the first case show that the substantive conclusions about the effect of gender and education on the alcohol measures continue to hold when interviewer effects are taken into account. The results of the second case make clear that interviewer effects on attitudinal questions are considerable. There is also a significant effect of the interviewers on the degree that respondents differentiate their responses. The results also illustrate that correlations between attitudinal variables are influenced. This also implies that the results of statistical procedures using a correlation or covariance matrix can be strongly influenced by the tendency to (non)differentiate and the interviewers’ impact on this tendency.

The results clearly demonstrate that there are considerable differences between countries concerning the impact of the interviewers on substantive variables. Cross-national differences are striking and the importance and necessity to evaluate interviewer effects in a cross-national survey becomes clear.

At the invitation of the Programme for the International Assessment of Adult Competencies (PIAAC), this paper aims to examine advances in survey interviewer monitoring and make recommendations on minimizing the effect of interviewers on survey results.

The authors first provide an overview of the most recent literature on interviewer effects, quality assurance and quality control. Here, they draw upon recent publications such as the cross-cultural survey guidelines (www.ccsg.isr.umich.edu) and newly published or in-press material specifically addressing these issues in multicultural, multinational and multiregional (3MC) contexts.

The authors discuss trends and innovations in quality assurance and quality control in 3MC studies and draw upon examples from international surveys that are using cutting-edge and innovative approaches to monitor interviewer behavior and minimize interviewer effects.

With a view to continuous quality improvement, the authors conclude with concrete recommendations for PIAAC to consider for the next cycle. Many of the recommendations have general relevance for other large-scale cross-national surveys.

Low response rates in web surveys and the use of different devices in entering web survey responses are the two main challenges to response quality of web surveys. The purpose of this study is to compare the effects of using interviewers to recruit participants in computer-assisted self-administered interviews (CASI) vs computer-assisted personal interviews (CAPI) and smartphones vs computers on participation rate and web survey response quality.

Two field experiments using two similar media use studies on US college students were conducted to compare response quality in different survey modes and response devices.

Response quality of computer entry was better than smartphone entry in both studies for open-ended and closed-ended question formats. Device effect was only significant on overall completion rate when interviewers were present.

Survey researchers are given guidance how to conduct online surveys using different devices and choice of question format to maximize survey response quality. The benefits and limitations of using an interviewer to recruit participants and smartphones as web survey response devices are discussed.

It shows how computer-assisted self-interviews and smartphones can improve response quality and participation for underprivileged groups.

This is the first study to compare response quality in different question formats between CASI, e-mailed delivered online surveys and CAPI. It demonstrates the importance of human factor in creating sense of obligation to improve response quality.

The purpose of this study was to examine how interview support materials provided on a company’s website would impact potential employees’ perceptions of organizational attractiveness (OA). This study examined the effects of interviewer, interview support and employee variables on employees’ perceptions of OA.

This study obtained data from 276 working adults. The participants were asked to visualize that they were selected for an interview, then were given one of four manipulations (with interview support, without interview support; informed participant interview would be with a potential peer, or interview would be with a potential supervisor) and post-manipulation measures (OA perception, trait anxiety, demographics).

This study’s results indicated that interviewer characteristics (peer or supervisor) were not significant; however, women and individuals with a lower socio-economic (SES) status found organizations to be more attractive when provided with interview support. There was an opposite effect on men and those with a higher SES.

This study recognizes that the lack of demographic diversity and realism were limitations that must be considered; however, this study also recognizes that these findings have several practical implications (considering offering interview support, potential ideas to increase OA) and social implications (increasing demographic diversity in the workplace by also aiding historically disadvantaged groups).

Knowledge of who would be conducting the selection interview potential future peer versus a future supervisor did not affect organizational attraction OA ratings. Interview support offered on a company website had a positive impact on perceptions of OA for women and individuals with lower SES; however, the same support had a negative impact on perceptions of OA for men and individuals with higher SES.

In addition to the social and practical implications this study holds, the authors of this study are not aware of any additional studies that examine gender and SES on organizational attraction.

Based on theories of media richness and procedural justice, the authors aim to examine the influence of videoconferencing (VC) technology on applicant reactions and interviewer judgments in the employment interview, the most commonly used employee selection device.

MBA students participated in simulated VC and face-to-face (FTF) interviews. Applicant perceptions of procedural justice and interviewer characteristics were collected. Interviewers provided ratings of affect toward the applicant, perceived applicant competence, overall interview performance, as well as an overall hiring recommendation.

Applicants perceived VC interviews as offering less of a chance to perform and as yielding less selection information. They also viewed VC interviews as less job-related than FTF interviews and had significantly less favorable evaluations of their interviewer (on personableness, trustworthiness, competence, and physical appearance) in VC interviews. Finally, applicants in VC interviews received lower ratings of affect (likeability) and lower interview scores, and were less likely to be recommended for the position.

The authors' findings suggest that VC technology can adversely affect both applicant reactions and interviewer judgments. They propose several precautionary steps to help minimize the risks associated with conducting VC interviews.

The authors extend prior research concerning the use of VC interviews by directly assessing applicant perceptions of both procedural justice and of interviewer characteristics associated with the probability that job offers will be accepted. They also add to the literature in showing that VC interviews tend to result in less favorable evaluations of applicants than FTF interviews.

This paper aims to provides a brief review of the dashboard literature, an account of the development of performance dashboards for field data collection at Westat, and more specifically for the first cycle of the Programme for the International Assessment of Adult Competencies (PIAAC). A brief concluding section offers suggestions for improvements in survey dashboards for PIAAC’s next cycle.

To manage field work at Westat, the authors create views into various paradata databases and present them in a dashboard, showing key performance indicators at a glance. Users can drill down from the dashboard into underlying databases to investigate potential problems. The US PIAAC dashboard is a monitoring system that supports daily review of many activities. For example, it provides overnight alerts to the field supervisor when global positioning system (GPS) data from an interviewer’s smartphone shows the interview occurred far from the respondent’s home.

Performance dashboards may represent best practice for monitoring field activities. Paradata sources and systems vary greatly across the PIAAC countries, but a multitude of process data exists in every country and can be used to create quality indicators and a monitoring system. PIAAC can establish standards/guidelines to improve visualization of quality metrics and management data, regardless of the local survey infrastructure.

The core of the paper is a case study of the experiences on the US PIAAC implementation of dashboards to monitor survey quality, production and costs, with special attention to the issue of fabrication.

Interviewing minority ethnic older people can seem daunting. It is easy for stereotypical views of minority ethnic groups and of older people to lead to pathologising approaches being adopted unwittingly. This article attempts to illuminate several key aspects of conducting interviews with such population groups by drawing partially on the experience of working on a Growing Older project funded by the Economic and Social Research Council of Britain. This project looked at the social network and social support of older people from different ethnic groups in Britain and the relationship between these and their quality of life. The research instrument comprised both structured and unstructured components. Interviews were conducted by a multi‐ethnic team of interviewers speaking a range of community languages. By focussing attention on the interactional nature of interviewing, this article explicates the process of conducting research. Issues pertaining to the choice of language, the use of interpreters, ethnic matching of interviewers and interviewees, the use of standardised instruments and the interview itself as a tool for data‐generation are examined. This article contends that a reflexive approach to methodology can lead to a more robust approach to data by confronting the practical, methodological and ethical issues encountered in doing research with such population groups. The issues raised are not merely in terms of ‘white’ researchers working with ‘non‐white’ respondents, but relate to the improvement of research practice in general regardless of the ethnic background of researcher and respondent.

This paper seeks to provide a discourse‐based critique of the laddering interviewing technique, and to make academics as well as practitioners aware of some of the limitations in applying this particular consumer interviewing technique.

The paper first describes the laddering interviewing technique, which traditionally has been conceptualised within a cognitively‐oriented perspective, i.e. the laddering interview is seen as a cognitive task. Then a critical discussion of some of the problems inherent in this view follows. After this, an alternative conceptualisation of the laddering interview is proposed, namely, that it is a discursive event. On the basis of insights from Wittgenstein and Austin it is suggested that the laddering interview is a room for social actions where both interviewer and interviewee are “doing things with words”. An example of applying the discursive approach to a sample sequence from a laddering interview is also provided. Finally, it seeks to evaluate the laddering interviewing technique in terms of its capacity to tap into “the voices in the marketplace”.

Finds that the laddering interviewing technique has its raison d'être as a quick and structured way of tapping into the voices and institutionalised rationales of the consumers in the marketplace. However, it is also found that the laddering interviewing technique “locks” the interviewee into one particular consumer identity; it prompts only answers that are valid with perfect strangers; it prevents the interviewee from unfolding his arguments fully; and it has a constant focus on personal preferences excluding the possible dissociations from other consumers – all of this making the data less rich and varied.

The unique value of this paper is that it sums up and provides a theoretically‐based critique of the laddering interviewing technique. It is believed that this critique will lead to a more appropriate appreciation of what is going on in a laddering interview and of the utterances that the consumers make in such an interview.