Search results

1 – 6 of 6
Book part
Publication date: 20 September 2018

Pravin Chopade, Michael Yudelson, Benjamin Deonovic and Alina A. von Davier

This chapter focuses on the state-of-the-art modeling approaches used in Intelligent Tutoring Systems (ITSs) and the frameworks for researching and operationalizing…

Abstract

This chapter focuses on the state-of-the-art modeling approaches used in Intelligent Tutoring Systems (ITSs) and the frameworks for researching and operationalizing individual and group models of performance, knowledge, and interaction. We adapt several ITS methodologies to model team performance as well as individuals’ performance of the team members. We briefly describe the point processes proposed by von Davier and Halpin (2013), and we also introduce the Competency Architecture for Learning in teaMs (CALM) framework, an extension of the Generalized Intelligent Framework for Tutoring (GIFT) (Sottilare, Brawner, Goldberg, & Holden, 2012) to be used for team settings.

Book part
Publication date: 4 August 2017

Colin Dingler, Alina A. von Davier and Jiangang Hao

Increased interest in team dynamics has resulted in new methods for measuring teamwork over time. The primary purpose of this chapter is to provide a survey of recent…

Abstract

Purpose

Increased interest in team dynamics has resulted in new methods for measuring teamwork over time. The primary purpose of this chapter is to provide a survey of recent developments in teamwork/collaboration measurement in an educational context. Key topics include conceptual frameworks, large-scale assessments, and innovative measurement techniques.

Methodology/approach

A range of methods for collecting and analyzing teamwork data are discussed, and five frameworks for measuring collaborative problem solving (CPS) over time are compared. Frameworks from Programme for International Student Assessment (PISA), Assessment and Teaching of 21st Century Skills (ATC21S) project, Educational Testing Service (ETS), ACT, and von Davier and Halpin (2013) are discussed. Results of assessments developed from these frameworks are also considered.

Social/practical implications

New techniques for measuring team dynamics over time have great potential to improve education and work outcomes. Preliminary results of the assessments developed from these frameworks show that important advances in teamwork measurement have been enabled by innovative task designs, data-mining techniques, and novel applications of stochastic models.

Originality/value

This novel overview and comparison of interdisciplinary approaches will help to indicate where progress has been made and what challenges are ahead.

Content available
Book part
Publication date: 20 September 2018

Abstract

Details

Building Intelligent Tutoring Systems for Teams
Type: Book
ISBN: 978-1-78754-474-1

Content available
Book part
Publication date: 4 August 2017

Abstract

Details

Team Dynamics Over Time
Type: Book
ISBN: 978-1-78635-403-7

Article
Publication date: 8 May 2017

Judith Zimmermann, Alina von Davier and Hans Rudolf Heinimann

Graduate admission has become a critical process for quality assurance in tertiary education. Hitherto, most research has investigated the validity of admissions…

Abstract

Purpose

Graduate admission has become a critical process for quality assurance in tertiary education. Hitherto, most research has investigated the validity of admissions instruments. However, surprisingly little work has been conducted on the overall organization of admission, which often remains “informal, ad hoc, and lacking in continuity.” The purpose of this paper is to investigate how to systematically design an admissions process for effectiveness, fairness, and the ability to continually improve, and determine how to condense and analyze the massive amount of data available from student records to obtain high-value feedback for admissions decision making.

Design/methodology/approach

An admissions process was systematically designed based on results from process management theory. Tenets of decision theory were applied to the organization of decision making and validity theory was utilized for validating admissions instruments. Performance of the implemented process was evaluated via student records covering a seven-year period.

Findings

The authors have designed a four-phase admissions process that ensures high quality through screening, scoping, selection, and evaluation/feedback. The last phase introduces closed-loop control and facilitates stabilization and continual improvement. Additionally, the authors have established a three-stage decision-making hierarchy that promotes consistency and equal treatment in admissions. The evaluations of undergraduate achievements and GRE® General Test scores indicate that both are valid admissions instruments in the European context. Finally, the evaluation of the implemented process provides evidence that decision making has effectively improved over the years and has become more consistent.

Originality/value

The systematic design of the admissions process presented generalizes well and is a significant contribution to the organization of decentralized graduate admission.

Details

International Journal of Educational Management, vol. 31 no. 4
Type: Research Article
ISSN: 0951-354X

Keywords

Article
Publication date: 28 January 2014

Jade Caines, Beatrice L. Bridglall and Madhabi Chatterji

This policy brief discusses validity and fairness issues that could arise when test-based information is used for making “high stakes” decisions at an individual level…

2465

Abstract

Purpose

This policy brief discusses validity and fairness issues that could arise when test-based information is used for making “high stakes” decisions at an individual level, such as, for the certification of teachers or other professionals, or when admitting students into higher education programs and colleges, or for making immigration-related decisions for prospective immigrants. To assist test developers, affiliated researchers and test users enhance levels of validity and fairness with these particular types of test score interpretations and uses, this policy brief summarizes an “argument-based approach” to validation given by Kane.

Design/methodology/approach

This policy brief is based on a synthesis of conference proceedings and review of selected pieces of extant literature. To that synthesis, the authors add practitioner-friendly examples with their own analysis of key issues. They conclude by offering recommendations for test developers and test users.

Findings

The authors conclude that validity is a complex and evolving construct, especially when considering issues of fairness in individual testing contexts. Kane's argument-based approach offers an accessible framework through which test makers can accumulate evidence to evaluate inferences and arguments related to decisions to be made with test scores. Perspectives of test makers, researchers, test takers and decision-makers must all be incorporated into constructing coherent “validity arguments” to guide the test development and validation processes.

Originality/value

Standardized test use for individual-level decisions is gradually spreading to various regions of the world, but understandings of validity are still uneven among key stakeholders of such testing programs. By translating complex information on test validation, validity and fairness issues with all concerned stakeholders in mind, this policy brief attempts to address the communication gaps noted to exist among these groups by Kane.

Details

Quality Assurance in Education, vol. 22 no. 1
Type: Research Article
ISSN: 0968-4883

Keywords

1 – 6 of 6