Search results

1 – 10 of 27
To view the access options for this content please click here
Article
Publication date: 3 April 2018

Leyla Mohadjer and Brad Edwards

This paper aims to provides a brief review of the dashboard literature, an account of the development of performance dashboards for field data collection at Westat, and…

Abstract

Purpose

This paper aims to provides a brief review of the dashboard literature, an account of the development of performance dashboards for field data collection at Westat, and more specifically for the first cycle of the Programme for the International Assessment of Adult Competencies (PIAAC). A brief concluding section offers suggestions for improvements in survey dashboards for PIAAC’s next cycle.

Design/methodology/approach

To manage field work at Westat, the authors create views into various paradata databases and present them in a dashboard, showing key performance indicators at a glance. Users can drill down from the dashboard into underlying databases to investigate potential problems. The US PIAAC dashboard is a monitoring system that supports daily review of many activities. For example, it provides overnight alerts to the field supervisor when global positioning system (GPS) data from an interviewer’s smartphone shows the interview occurred far from the respondent’s home.

Findings

Performance dashboards may represent best practice for monitoring field activities. Paradata sources and systems vary greatly across the PIAAC countries, but a multitude of process data exists in every country and can be used to create quality indicators and a monitoring system. PIAAC can establish standards/guidelines to improve visualization of quality metrics and management data, regardless of the local survey infrastructure.

Originality/value

The core of the paper is a case study of the experiences on the US PIAAC implementation of dashboards to monitor survey quality, production and costs, with special attention to the issue of fabrication.

Details

Quality Assurance in Education, vol. 26 no. 2
Type: Research Article
ISSN: 0968-4883

Keywords

To view the access options for this content please click here
Book part
Publication date: 15 April 2020

Heng Chen, Geoffrey Dunbar and Q. Rallye Shen

The authors consider how the mode of data collection (Internet vs. paper) alters individuals’ responses to different types of survey questions, including subjective…

Abstract

The authors consider how the mode of data collection (Internet vs. paper) alters individuals’ responses to different types of survey questions, including subjective, recall, and factual questions. The authors isolate the measurement effect of the mode from the sample selection effect by exploiting predata in a convenience consumer panel. The authors propose using panelists’ reward point balance as exclusion restriction to correct for differing response probabilities by mode, because the reward point balance depends on the timing of the survey invitations and is a source of random variation in response incentive. The authors evaluate average and quantile measurement effects in a mixed-mode Web/paper survey and find statistically significant evidence of mode effects in subjective and recall questions.

Details

Essays in Honor of Cheng Hsiao
Type: Book
ISBN: 978-1-78973-958-9

Keywords

To view the access options for this content please click here
Article
Publication date: 30 September 2014

Kay Sanderson

Abstract

Details

The Electronic Library, vol. 32 no. 5
Type: Research Article
ISSN: 0264-0473

Keywords

To view the access options for this content please click here
Article
Publication date: 3 April 2018

Kristen Cibelli Hibben, Beth-Ellen Pennell and Lesli Scott

At the invitation of the Programme for the International Assessment of Adult Competencies (PIAAC), this paper aims to examine advances in survey interviewer monitoring and…

Abstract

Purpose

At the invitation of the Programme for the International Assessment of Adult Competencies (PIAAC), this paper aims to examine advances in survey interviewer monitoring and make recommendations on minimizing the effect of interviewers on survey results.

Design/methodology/approach

The authors first provide an overview of the most recent literature on interviewer effects, quality assurance and quality control. Here, they draw upon recent publications such as the cross-cultural survey guidelines (www.ccsg.isr.umich.edu) and newly published or in-press material specifically addressing these issues in multicultural, multinational and multiregional (3MC) contexts.

Findings

The authors discuss trends and innovations in quality assurance and quality control in 3MC studies and draw upon examples from international surveys that are using cutting-edge and innovative approaches to monitor interviewer behavior and minimize interviewer effects.

Originality/value

With a view to continuous quality improvement, the authors conclude with concrete recommendations for PIAAC to consider for the next cycle. Many of the recommendations have general relevance for other large-scale cross-national surveys.

Details

Quality Assurance in Education, vol. 26 no. 2
Type: Research Article
ISSN: 0968-4883

Keywords

To view the access options for this content please click here
Article
Publication date: 3 April 2018

Bryan Maddox

This paper aims to investigate small-scale, qualitative observations of interviewer–respondent interaction in the Organisation for Economic Co-operation and Development’s…

Abstract

Purpose

This paper aims to investigate small-scale, qualitative observations of interviewer–respondent interaction in the Organisation for Economic Co-operation and Development’s Programme for the International Assessment of Adult Competencies (PIAAC).

Design/methodology/approach

The paper uses video-ethnographic methods to document talk and gesture in assessment in Slovenian household settings. It presents an in-depth case study of interaction in a single testing situation.

Findings

Observing interaction in assessment captures data on assessment performance that is not available in quantitative analysis of assessment response processes. The character of interviewer–respondent interaction and rapport is shaped by the cognitive demands of assessment and the distinctive ecological setting of the household.

Research limitations/implications

Observational data on assessment response processes and interaction in real-life assessments can be integrated into and synthesized with other sources of “process data”.

Practical implications

Assessment programs such as PIAAC should consider the significance of the household setting on assessment quality and observations of interaction in assessment as a valid source of paradata.

Social implications

There is a place for small-scale observational studies of assessment to inform public understanding of assessment quality and validity.

Originality/value

The paper provides qualitative insights into the significance of interaction and “interviewer effects” in household assessment settings.

Details

Quality Assurance in Education, vol. 26 no. 2
Type: Research Article
ISSN: 0968-4883

Keywords

To view the access options for this content please click here
Article
Publication date: 6 March 2017

Søren Bøye Olsen, Jürgen Meyerhoff, Morten Raun Mørkbak and Ole Bonnichsen

Fatigue effects related to answering a sequence of choice tasks have received much scrutiny in the stated choice experiments (SCE) literature. However, decision fatigue…

Abstract

Purpose

Fatigue effects related to answering a sequence of choice tasks have received much scrutiny in the stated choice experiments (SCE) literature. However, decision fatigue related to the time of day when respondents answer questionnaires has been largely overlooked in this literature even though time of day related fatigue effects are well known in the psychology literature. The purpose of this paper is to hypothesize that variations in the time of day when respondents answer an online food choice experiment will translate into observable fatigue effects in the food choices.

Design/methodology/approach

An empirical SCE concerning food choices is conducted using a web-based questionnaire for interviews in a pre-recruited online panel of consumers. Timestamps collected during the online interviews provide knowledge about the time of day at which each respondent has answered the survey. This information is linked with knowledge from a food sociology survey on typical meal times as well as biophysical research linking food intake to blood sugar and mental energy in order to generate a proxy variable for each respondent’s level of mental energy when answering the food choice tasks in the questionnaire.

Findings

Results show evidence of a time of day effect on error variance in the stated food choices as well as the subsequently estimated market share predictions. Specifically, respondents provide less consistent answers during the afternoon than at other times of the day.

Originality/value

The results indicate that time of day can affect responses to an online survey through increased fatigue and correspondingly less choice consistency. Thus, especially online surveys might account for this in data analysis or even restrict accessibility to the online survey for certain times of day.

Details

British Food Journal, vol. 119 no. 3
Type: Research Article
ISSN: 0007-070X

Keywords

To view the access options for this content please click here
Article
Publication date: 8 January 2019

Eugene Ch’ng, Shengdan Cai, Tong Evelyn Zhang and Fui-Theng Leow

The purpose of this paper is to present the rationale for democratising the digital reproduction of cultural heritage via “mass photogrammetry”, by providing approaches to…

Abstract

Purpose

The purpose of this paper is to present the rationale for democratising the digital reproduction of cultural heritage via “mass photogrammetry”, by providing approaches to digitise objects from cultural heritage collections housed in museums or private spaces using devices and photogrammetry techniques accessible to the public. The paper is intended as a democratised approach rather than as a “scientific approach” for the purpose that mass photogrammetry can be achieved at scale.

Design/methodology/approach

The methodology aims to convert the art of photogrammetry into a more mechanical approach by overcoming common difficulties faced within exhibition spaces. This approach is replicable and allows anyone possessing inexpensive equipment with basic knowledge of photogrammetry to achieve acceptable results.

Findings

The authors present the experience of acquiring over 300 3D models through photogrammetry from over 25 priority sites and museums in East Asia. The approach covers the entire process from capturing to editing, and importing 3D models into integrated development environments for displays such as interactive 3D, Virtual Reality and Augmented Reality.

Practical implications

The simplistic approach for democratised, mass photogrammetry has implications for stirring public interests in the digital preservation of heritage objects in countries where museums and cultural institutions have little access to digital teams, provided that Intellectual Property issues are cared for. The approach to mass photogrammetry also means that personal cultural heritage objects hidden within the homes of various societies and relics in circulation in the antiques market can be made accessible globally at scale.

Originality/value

This paper focuses on the complete practical nature of photogrammetry conducted within cultural institutions. The authors provide a means for the public to conduct good photogrammetry so that all cultural heritage objects can be digitally recorded and shared globally so as to promote the cross-cultural appreciation of material cultures from the past.

Details

Journal of Cultural Heritage Management and Sustainable Development, vol. 9 no. 1
Type: Research Article
ISSN: 2044-1266

Keywords

To view the access options for this content please click here
Article
Publication date: 14 July 2020

Louisa Ha, Chenjie Zhang and Weiwei Jiang

Low response rates in web surveys and the use of different devices in entering web survey responses are the two main challenges to response quality of web surveys. The…

Abstract

Purpose

Low response rates in web surveys and the use of different devices in entering web survey responses are the two main challenges to response quality of web surveys. The purpose of this study is to compare the effects of using interviewers to recruit participants in computer-assisted self-administered interviews (CASI) vs computer-assisted personal interviews (CAPI) and smartphones vs computers on participation rate and web survey response quality.

Design/methodology/approach

Two field experiments using two similar media use studies on US college students were conducted to compare response quality in different survey modes and response devices.

Findings

Response quality of computer entry was better than smartphone entry in both studies for open-ended and closed-ended question formats. Device effect was only significant on overall completion rate when interviewers were present.

Practical implications

Survey researchers are given guidance how to conduct online surveys using different devices and choice of question format to maximize survey response quality. The benefits and limitations of using an interviewer to recruit participants and smartphones as web survey response devices are discussed.

Social implications

It shows how computer-assisted self-interviews and smartphones can improve response quality and participation for underprivileged groups.

Originality/value

This is the first study to compare response quality in different question formats between CASI, e-mailed delivered online surveys and CAPI. It demonstrates the importance of human factor in creating sense of obligation to improve response quality.

Details

Internet Research, vol. 30 no. 6
Type: Research Article
ISSN: 1066-2243

Keywords

To view the access options for this content please click here
Article
Publication date: 16 November 2018

Louisa S. Ha and Chenjie Zhang

The purpose of this paper is to examine the effect of smartphones and computers as web survey entry response devices on the quality of responses in different question…

Abstract

Purpose

The purpose of this paper is to examine the effect of smartphones and computers as web survey entry response devices on the quality of responses in different question formats and across different survey invitations delivery modes. The respondents’ preference of device and the response immediacy were also compared.

Design/methodology/approach

Two field experiments were conducted with a cluster sampling and a census of all students in a public university in the USA.

Findings

Device effect on response quality was only found when using computer-aided self-interviews, but not in e-mail delivered web surveys. Even though the computer was the preferred device, but the smartphone’s immediate response was significantly higher than the computer.

Research limitations/implications

The sample was restricted to college students who are more proficient users of smartphones and have high access to computers. But the direct comparison in the two studies using the same population increases the internal validity of the study comparing different web survey delivery modes.

Practical implications

Because of the minor differences in device on response quality, researchers can consider using more smartphones for field work such as computer-aided self-interviews to complement e-mail delivered surveys.

Originality/value

This is the first study that compares the response device effects of computer-aided self-interviews and e-mailed delivered web surveys. Because web surveys are increasingly used and various devices are being used to collect data, how respondents behave in different devices and the strengths and weaknesses of different methods of delivery survey help researchers to improve data quality and develop effective web survey delivery and participant recruitment.

Details

Online Information Review, vol. 43 no. 3
Type: Research Article
ISSN: 1468-4527

Keywords

To view the access options for this content please click here
Book part
Publication date: 15 April 2020

Abstract

Details

Essays in Honor of Cheng Hsiao
Type: Book
ISBN: 978-1-78973-958-9

1 – 10 of 27