To read this content please select one of the options below:

How effective is AI augmentation in human–AI collaboration? Evidence from a field experiment

Chengcheng Liao (Business School, Sichuan University, Chengdu, China)
Xin Wen (Business School, Sichuan University, Chengdu, China)
Shan Li (Business School, Sichuan University, Chengdu, China)
Peiyuan Du (Chengdu Research Institute of Service Industry, Chengdu, China)

Information Technology & People

ISSN: 0959-3845

Article publication date: 8 November 2024

Issue publication date: 3 December 2024

868

Abstract

Purpose

Companies increasingly leverage artificial intelligence (AI) to enhance human performance, particularly in e-commerce. However, the effectiveness of AI augmentation remains controversial. This study investigates whether, how and why AI enhances human agents’ sales through a randomized field experiment.

Design/methodology/approach

This study conducts a two-by-two factorial randomized field experiment (N = 1,090) to investigate the effects of AI augmentation on sales. The experiment compares sales outcomes handled solely by human agents with those augmented by AI, while also examining the moderating effect of agents’ experience levels and the underlying mechanisms behind agents’ responses.

Findings

The results reveal that AI augmentation leads to a significant 5.46% increase in sales. Notably, the impact of AI augmentation varies based on agents’ experience levels, with inexperienced agents benefiting nearly six times more than their experienced counterparts. Mediation analysis shows that AI augmentation improves response timeliness, accuracy and sentiment, thereby boosting sales.

Originality/value

This study highlights the role of AI augmentation in human–AI collaboration, demonstrates the varying impacts of AI augmentation based on agents’ experience levels and offers insights for organizations on how to regulate AI augmentation to enhance agent responses and drive sales.

Keywords

Acknowledgements

This paper forms part of a special section “Sharing Work with AI: Introduction to the special issue on the futures of work in the age of intelligent machines”, guest edited by Dr. Kevin Crowston, Dr. Ingrid Erickson and Dr. Jeffrey Nickerson.

The corresponding author of this paper are Peiyuan Du (986821459@qq.com) and Xin Wen (wenxin1997101@163.com). This work is supported by the Natural Science Foundation of China (Grant No. 71925003, 72172099, 72302168, 72102238 and 72372110), China Postdoctoral Science Foundation (Grant No. 2024M752286, 2023M732416 and 2021M702319), and Sichuan University (Grant No. 2023CX35, JCXK2236, 2024ZY-SX06 and 2024ZY-SX02).

Citation

Liao, C., Wen, X., Li, S. and Du, P. (2024), "How effective is AI augmentation in human–AI collaboration? Evidence from a field experiment", Information Technology & People, Vol. 37 No. 7, pp. 2357-2389. https://doi.org/10.1108/ITP-11-2022-0859

Publisher

:

Emerald Publishing Limited

Copyright © 2024, Emerald Publishing Limited

Related articles