Designing rule-based conversational agents with behavioral programming: a study of human subjects
ISSN: 1450-2194
Article publication date: 1 February 2022
Issue publication date: 8 September 2023
Abstract
Purpose
In this work, the authors propose to harness the advantages of behavioral programming as a new technique for designing rule-based conversational agents.
Design/methodology/approach
To examine the study’s hypotheses, the authors perform a first-of-its-kind user study through which the authors examine how potential designers, both expert designers, computationally-oriented designers, and otherwise, leverage behavioral programming (BP) and dialog graphs for designing conversational agents (CAs). The authors also use two standard CA settings common in the literature: designing a CA representative for a user in an online dating service and a non-character player in a role-playing game (RPG).
Findings
The study’s results indicate that BP can be successfully utilized by computationally-oriented designers, with or without prior knowledge in CA design, and can facilitate the design of better CAs (i.e. more accurate and more robust). However, to capitalize on these potential advantages, designers may be required to devote more time to the design process and are likely to encounter higher temporal demand levels. These results suggest that BP, which was initially proposed and evaluated in the general context of software design, can constitute a valuable alternative to the classic rule-based CA design technique commonly practiced today.
Research limitations/implications
An important limitation of this study is the relatively small participant pool. While the authors do plan to extend this study in the future, the current coronavirus disease 2019 (COVID-19) situation makes it ever more complex to conduct formal user studies of this kind. It is, however, important to note that despite the low number of participants, many of the results are found to be statistically significant.
Practical implications
The authors plan to continue this line of work and conduct human studies for additional design techniques in other popular agent-based settings. Specifically, the authors seek to explore how people of different backgrounds should design agents for various tasks such as automated negotiation (e.g. how should a person design a representative agent to negotiate on her behalf?) and social choice (e.g. how should a person design a voting bot to represent her in online voting systems?).
Originality/value
People are increasingly interacting with conversational agents in various settings and for a variety of reasons, as the market size of those agents keeps on growing every year. Through a first-of-its-kind human study (N = 41), consisting of both expert designers, computationally-oriented designers, and otherwise, the authors demonstrate a few key advantages and limitations of BP in the realm of conversational agents and propose its consideration as an alternative to the classic dialog graph technique.
Keywords
Acknowledgements
Funding: Not applicable.
Availability of data and material: Links are provided in the text.
Code availability: Not applicable.
Conflict of interest: The authors declare no conflicts of interest/competing interests.
Citation
Rosenfeld, A. and Haimovich, N. (2023), "Designing rule-based conversational agents with behavioral programming: a study of human subjects", EuroMed Journal of Business, Vol. 18 No. 3, pp. 345-358. https://doi.org/10.1108/EMJB-09-2021-0144
Publisher
:Emerald Publishing Limited
Copyright © 2022, Emerald Publishing Limited