How effective is AI augmentation in human–AI collaboration? Evidence from a field experiment
Information Technology & People
ISSN: 0959-3845
Article publication date: 8 November 2024
Issue publication date: 3 December 2024
Abstract
Purpose
Companies increasingly leverage artificial intelligence (AI) to enhance human performance, particularly in e-commerce. However, the effectiveness of AI augmentation remains controversial. This study investigates whether, how and why AI enhances human agents’ sales through a randomized field experiment.
Design/methodology/approach
This study conducts a two-by-two factorial randomized field experiment (N = 1,090) to investigate the effects of AI augmentation on sales. The experiment compares sales outcomes handled solely by human agents with those augmented by AI, while also examining the moderating effect of agents’ experience levels and the underlying mechanisms behind agents’ responses.
Findings
The results reveal that AI augmentation leads to a significant 5.46% increase in sales. Notably, the impact of AI augmentation varies based on agents’ experience levels, with inexperienced agents benefiting nearly six times more than their experienced counterparts. Mediation analysis shows that AI augmentation improves response timeliness, accuracy and sentiment, thereby boosting sales.
Originality/value
This study highlights the role of AI augmentation in human–AI collaboration, demonstrates the varying impacts of AI augmentation based on agents’ experience levels and offers insights for organizations on how to regulate AI augmentation to enhance agent responses and drive sales.
Keywords
Acknowledgements
This paper forms part of a special section “Sharing Work with AI: Introduction to the special issue on the futures of work in the age of intelligent machines”, guest edited by Dr. Kevin Crowston, Dr. Ingrid Erickson and Dr. Jeffrey Nickerson.
The corresponding author of this paper are Peiyuan Du (986821459@qq.com) and Xin Wen (wenxin1997101@163.com). This work is supported by the Natural Science Foundation of China (Grant No. 71925003, 72172099, 72302168, 72102238 and 72372110), China Postdoctoral Science Foundation (Grant No. 2024M752286, 2023M732416 and 2021M702319), and Sichuan University (Grant No. 2023CX35, JCXK2236, 2024ZY-SX06 and 2024ZY-SX02).
Citation
Liao, C., Wen, X., Li, S. and Du, P. (2024), "How effective is AI augmentation in human–AI collaboration? Evidence from a field experiment", Information Technology & People, Vol. 37 No. 7, pp. 2357-2389. https://doi.org/10.1108/ITP-11-2022-0859
Publisher
:Emerald Publishing Limited
Copyright © 2024, Emerald Publishing Limited