Building Intelligent Tutoring Systems for Teams: Volume 19

This chapter examines some of the challenges and emerging strategies for authoring, distributing, managing, and evaluating Intelligent Tutoring Systems (ITSs) to support computer-based adaptive instruction for teams of learners. Several concepts related to team tutoring are defined along with team processes, and fundamental tutoring concepts are provided including a description of the learning effect model (LEM), an exemplar describing interaction between learners and ITSs with the goal of realizing optimal tutor decisions. The challenges noted herein are closely related to the LEM and range from acquisition of learner data, synthesis of individual learner and team state models based on available data, and tutor decisions which center on optimizing strategies (recommendations) and tactics (actions) given the state of the learner, the team, and the conditions under which they are being instructed, the environment. Finally, we end this chapter with recommendations on how to use this book to understand and design effective ITSs for teams.

The use of teams is ubiquitous in organizations, yet teams are not always effective. Much work has been conducted to understand those factors that facilitate effective team training. While much has been learned, there is no escaping the fact that team training is a complex, resource intensive endeavor. Recent advancements in the area of intelligent tutoring may provide a way forward as one method by which to reduce some of the ongoing resource requirements involved in team training. The current chapter relies on the science of team training to describe a tool, team task analysis, that should be considered in building ITSs that move beyond the training of individual tasks to those which are team-based. In that vein, an overview of team task analysis is provided, how it differs from individual task analysis, and what it may contribute to the design of ITS for teams. In doing so, we put forth five considerations that are somewhat unique as compared with traditional task analysis as well as some corresponding guidance from the literature in light of these considerations. It is our hope that this information will not only be useful to those building team-based ITSs, but spur future thought.

Team cognition research continues to evolve as the need for understanding and improving complex problem solving itself grows. Complex problem solving requires members to engage in a number of complicated collaborative processes to generate solutions. This chapter illustrates how the Macrocognition in Teams model, developed to guide research on these processes, can be utilized to propose how intelligent tutoring systems (ITSs) could be developed to train collaborative problem solving. Metacognitive prompting, based upon macrocognitive processes, was offered as an intervention to scaffold learning these complex processes. Our objective is to provide a theoretically grounded approach for linking intelligent tutoring research and development with team cognition. In this way, team members are more likely to learn how to identify and integrate relevant knowledge, as well as plan, monitor, and reflect on their problem-solving performance as it evolves. We argue that ITSs that utilize metacognitive prompting that promotes team planning during the preparation stage, team knowledge building during the execution stage, and team reflexivity and team knowledge sharing interventions during the reflection stage can improve collaborative problem solving.

Teams focus on a common and valued goal, and effective teams are able to alter their behaviors in pursuit of this goal. When teams are viewed in the context of a dynamic environment, they must adapt to challenges in the environment in order to maintain team effectiveness. In this light, we describe various sources of team variation and how they combine with individual-level, team-level, and dynamical mechanisms for maintaining team effectiveness in a dynamic environment. The combination of these elements produces a systems view of team effectiveness. Our goals are to begin to define, both in words and in operational terms, team effectiveness from this perspective and to evaluate this definition in the context of team training using intelligent tutoring systems (team ITS). In addressing these goals, we present an example of real-time analysis of team effectiveness and some challenges for team ITS training based on a dynamical systems view of team effectiveness.

A key challenge for cost-effective Intelligent Tutoring Systems (ITSs) is the ability to create generalizable domain, learner, and pedagogical models so they can be re-used many times over. Investment in this technology will be needed to succeed in developing ITSs for team training. The purpose of this chapter is to propose an instructional framework for guiding team ITS researchers in their development of these models for reuse. We establish a foundation for the framework with three propositions. First, we propose that understanding how teams develop is needed to establish a science-based foundation for modeling. Toward this end, we conduct a detailed exploration of the Kozlowski, Watola, Jensen, Kim, and Botero (2009) theory of team development and leadership, and describe a use case example to demonstrate how team training was developed for a specific stage in their model. Next, we propose that understanding measures of learning and performance will inform learner modeling requirements for each stage of team development. We describe measures developed for the use case and how they were used to understand teamwork skill development. We then discuss effective team training strategies and explain how they were implemented in the use case to understand their implications for pedagogical modeling. From this exploration, we describe a generic instructional framework recommending effective training strategies for each stage of team development. To inform the development of reusable models, we recommend selecting different team task domains and varying team size to begin researching commonalities and differences in the instructional framework.

Unobtrusive measurement methodologies are critical to implementing intelligent tutoring systems (ITS) for teams. Such methodologies allow for continuous measurement of team states and processes while avoiding disruption of mission or training performance, and do not rely on post hoc feedback (including for the aggregation of data into measures or to develop insights from these real-time metrics). This chapter summarizes advances in unobtrusive measurement developed within Army research programs to illustrate the variety and potential that unobtrusive measurement approaches can provide for building ITS for teams. Challenges regarding the real-time aggregation of data and applications to current and future ITS for teams are also discussed.

This chapter focuses on the state-of-the-art modeling approaches used in Intelligent Tutoring Systems (ITSs) and the frameworks for researching and operationalizing individual and group models of performance, knowledge, and interaction. We adapt several ITS methodologies to model team performance as well as individuals’ performance of the team members. We briefly describe the point processes proposed by von Davier and Halpin (2013), and we also introduce the Competency Architecture for Learning in teaMs (CALM) framework, an extension of the Generalized Intelligent Framework for Tutoring (GIFT) (Sottilare, Brawner, Goldberg, & Holden, 2012) to be used for team settings.

This chapter describes a neurodynamic modeling approach which may be useful for dynamically assessing teamwork in healthcare and military situations. It begins with a description of electroencephalographic (EEG) signal acquisition and the transformation of the physical units of EEG signals into quantities of information. This transformation provides quantitative, dynamic, and generalizable neurodynamic models that are directly comparable across teams, tasks, training protocols, and team experience levels using the same measurement scale, bits of information. These bits of information can be further used to dynamically guide team performance or to provide after-action feedback that is linked to task events and team actions.

These ideas are instantiated and expanded in the second section of the chapter by showing how these data abstractions, compressions, and transformations take advantage of the natural information redundancy in biologic signals to substantially reduce the number of data dimensions, making the incorporation of neurodynamic feedback into Intelligent Tutoring Systems (ITSs) achievable.

This chapter describes how conversational computer agents have been used in collaborative problem-solving environments. These agent-based systems are designed to (a) assess the students’ knowledge, skills, actions, and various other psychological states on the basis of the students’ actions and the conversational interactions, (b) generate discourse moves that are sensitive to the psychological states and the problem states, and (c) advance a solution to the problem. We describe how this was accomplished in the Programme for International Student Assessment (PISA) for Collaborative Problem Solving (CPS) in 2015. In the PISA CPS 2015 assessment, a single human test taker (15-year-old student) interacts with one, two, or three agents that stage a series of assessment episodes. This chapter proposes that this PISA framework could be extended to accommodate more open-ended natural language interaction for those languages that have developed technologies for automated computational linguistics and discourse. Two examples support this suggestion, with associated relevant empirical support. First, there is AutoTutor, an agent that collaboratively helps the student answer difficult questions and solve problems. Second, there is CPS in the context of a multi-party simulation called Land Science in which the system tracks progress and knowledge states of small groups of 3–4 students. Human mentors or computer agents prompt them to perform actions and exchange open-ended chat in a collaborative learning and problem-solving environment.

Technology for training military teams has evolved through a convergence of advances in simulation technology for individual and collective training, methods for analyzing teamwork and designing training solutions, and intelligent tutoring technologies that adapt training to the student, to accelerate learning. A number of factors have slowed this evolution toward intelligent team tutoring systems (ITTS), including the challenges of processing communications data, which are the currency of teamwork, and the paucity of automated and generalizable measures of team work. Several systems fulfill a subset of the features required of an ITTS, namely the use of team training objectives, teamwork models, measures of teamwork, diagnostic capability, instructional strategies, and adaptation of training to team needs. We describe these systems: the Advanced Embedded Training System (AETS), Synthetic Cognition for Operational Team Training (SCOTT), the AWO Trainer, the Benchmarked Experiential System for Training (BEST), and the Cross-Platform Mission Visualization Tool. We close this chapter with recommendations for future research.

This chapter describes five disciplinary domains of research or lenses that contribute to the design of a team tutor. We focus on four significant challenges in developing Intelligent Team Tutoring Systems (ITTSs), and explore how the five lenses can offer guidance for these challenges. The four challenges arise in the design of team member interactions, performance metrics and skill development, feedback, and tutor authoring. The five lenses or research domains that we apply to these four challenges are Tutor Engineering, Learning Sciences, Science of Teams, Data Analyst, and Human–Computer Interaction. This matrix of applications from each perspective offers a framework to guide designers in creating ITTSs.

Long-duration spaceflight missions require many hours of pre-mission and inflight training to develop and maintain team skills. Current training flows rely heavily on expert instructors, while current inflight mission operations are supported by a complex series of support teams at Mission Control. However, future exploration space missions will not have real-time communications with ground-based experts at Mission Control. Portable intelligent tutoring systems may help streamline future training, reducing the burden on expert instructors and crew training time, and allowing for inflight support to mitigate negative effects of the loss of real-time communications. In this chapter, we discuss the challenges of long-duration exploration missions, and outline the myriad possibilities in which intelligent tutoring systems will enhance the crew performance and functioning.

This chapter considers the essential elements and processes in designing and building a computer-based tutor to instruct teams. In this chapter, the choices of authoring tools, the instructional context, the goal of the instruction, and the characteristics of the domain were evaluated in terms of their influence on the Intelligent Tutoring System (ITS) design in support of team learning and performance. While each team tutor may be unique in terms of its learning objectives, measures, selections of learning strategies and tutor interventions, there are some identified design decisions that need to be made. Considering the best decision for the specific tutor's design is intended to ease the authoring burden and make computer-based team tutoring more ubiquitous.