Search results

Exercises are key components of assessment centers (ACs). However, little is known about the nature and determinants of AC exercise performance. The traditional exercise paradigm primarily emphasizes the need to simulate task, social, and organizational demands in AC exercises. This chapter draws on trait activation theory in proposing a new AC exercise paradigm. First, we develop a theoretical framework that addresses the complexity of situational characteristics of AC exercises as determinants of AC performance. Second, we argue for planting multiple stimuli within exercises as a structured means of eliciting candidate behavior. Third, we show how the new paradigm also has key insights for the rating part of ACs, namely, in selecting dimensions, designing behavioral checklists, screening assessors, and training assessors. Finally, the impact of this new AC exercise paradigm is anticipated on important AC outcomes such as reliability, internal/external construct-related validity, criterion-related validity, assessee perceptions, and feedback effectiveness.

The purpose of this paper is to address the interplay of workplace exercising on self‐reported workplace performance.

A mixed methods design combined a randomised cross‐over trial with concurrent focus groups. Three workplaces (two private companies, one public service organisation) were purposefully selected for their provision of on‐site exercise facilities, size (>250 employees) and large proportion of sedentary occupations. Two mood diary questionnaires were distributed to employees exercising on‐site only. Order of questionnaire completion was randomised: self‐selected exercise‐day (ExD) or no‐exercise day (NExD) first. Exercise specifics (duration, intensity, mode) and ExD mood (pre‐/post‐exercise) were recorded. On NExD, mood was measured early and late in the working day. A 15‐item work performance grid was completed at day‐ends. Three on‐site focus groups were held concurrently to explore performance‐related topics.

Among 201 volunteer respondents (67 per cent female, mean age 38.2 years), mood improved on ExD, pre‐to‐post exercise (all p<0.01). Performance indicators were higher on ExD, versus NExD (all p<0.01), independent of exercise specifics and workload. Positive changes in performance outcomes were almost exclusively linked to changes in mood. Inductive analysis of focus groups revealed 13 (of 17) themes exhibiting positive outcomes. Employee tolerance and resilience were central to the subjective findings.

The naturalistic, dual‐paradigm study demonstrated that workday exercise can improve white‐collar workers' mood and self‐reported performance on days when they exercise at work over days when they do not. There are clear implications not only for employee wellbeing, but also for competitive advantage and motivation by increasing opportunities for exercising at work.

This is one of the few studies that addresses the acute effects of exercise in the workplace in the same people. Self‐rated productivity effects attributable to exercising during the working day were strongly mediated by changes in mood. Statistical power is amplified within the cross‐over design.

This chapter details the standard-setting methods used in the development of the NBPTS assessments. The dominant Profile Judgment Method was originally applied only to the NBPTSs Early Adolescence/English Language Arts assessment (EA/ELA). Although extremely flexible, it proved to be too complex, and increased the likelihood of false-negative errors in candidate classification. The Direct Judgment Method was found to be combatively economical; however, it was the Judgmental Policy Capturing (JPC) approach that was used since its approach was more akin to the everyday judgments we all make. The two-stage process of the JPC method will be describe in detail, using the standard-setting process used with the Early Adolescent through Young Adult/Art (EAYA/A) Assessments as an example.

This chapter, completed in 1999, provides an overview and critical analysis of the validity research agenda undertaken by the National Board for Professional Teaching Standards (NBPTS) for its assessment to certify accomplished teachers at the end of its first decade of assessment development and implementation. The review is presented in three major sections: (a) an overview of the validity criteria underlying the review; (b) a description of the National Board's research agenda presented in its own terms, focusing first on the studies that were routinely carried out for each certificate and second on the “special studies” that were not part of the routine agenda; and (c) a series of six critical observations and explanations based on the validity issues described in the first section.

In this National Board-commissioned study, we examined types of feedback that teacher candidates for certification might receive along with their score reports after completing the Board's assessment process. We designed three standardized forms of feedback and interviewed 29 teachers from the 1993–1994 Early Adolescent/Generalist cohort about their preferences for each of the feedback options and about the inferences that they drew about their performance based on each type. The three feedback formats were (a) cases – extended descriptions of actual performances, annotated with scorer notes; (b) performance syntheses – brief descriptions of the scoring criteria accompanied by a variety of excerpts from candidate materials portraying performances at each level; and (c) illustrative summaries – evaluative descriptions of various candidate responses. Teachers reported that, of the three standardized formats offered to them, they preferred the cases format with its extended descriptions of an actual performance accompanied by annotated scoring notes. In terms of learning effects, candidates drew reasonably accurate inferences about their performance based on both cases and performance syntheses. The central conclusion we reached based on these findings was that feedback needs to be clearly organized around the scoring criteria for the exercise, and that examples of actual performances illustrating the application of the scoring criteria are important. However, teachers also reported that they would have preferred individualized, customized feedback on their own performance, although this option was not offered by the Board. As well, teachers indicated they would have preferred receiving the standardized feedback as “feedfront” to use in guiding them in their teaching and in preparing their assessment materials.

The purpose of this paper is to establish the extent of general performance factors (GPF) in assessment center (AC) exercises and dimensions. The study further aims to determine if larger GPF contributes to larger ethnic group differences across exercises and dimensions that are more cognitively loaded in an emerging market context.

The authors analyzed data across three independent AC samples (Sample 1: N=172; Sample 2: N=281; Sample 3: N=428). The Schmid-Leiman solution was used to determine the extent of GPF in AC exercises and dimensions. An independent samples t-test and Cohen’s d was used to determine the size of ethnic group differences across exercises and dimensions.

The results indicate that GPF is consistently large for the in-basket exercise. Furthermore, dimensions that are more cognitively loaded, such as problem solving, strategic thinking, and business acumen, seem to produce the largest ethnic group differences. Overall, the research indicates that larger GPF is associated with larger ethnic group differences in relation to specific AC dimensions and exercises.

The authors add to the literature by investigating the prevalence of a GPF in AC ratings across AC exercises and dimensions. A novel contribution of the research attempts to link the prevalence of a GPF in AC ratings to group membership in South Africa. The study offers an alternative statistical analysis procedure to examine GPF in AC ratings.

The paper seeks to demonstrate the characteristics of group work that are required to teach the attributes of real world venture teamwork in tertiary entrepreneurship programmes.

One‐tailed Spearman correlation analysis is used to assess the associations between students' grades in four group assessment tasks involving different levels of complexity and collaboration and the attributes of teamwork.

The results indicate that students learn more of the attributes of venture teamwork, the more complex the group task, and the more collaboration required to complete it.

The results cannot be generalised due to the small sample size and restriction of the sample to one course in entrepreneurship. However, the findings are valid conceptually; thus the study makes important contributions to knowledge.

The findings should help tertiary educators to improve the design and implementation of their group assessment tasks to reflect the attributes of real world venture teamwork.

While considerable work has been carried out on entrepreneurship education at the tertiary level, limited attention has been paid to teaching venture teamwork as part of entrepreneurship programmes. Academics often claim that their courses cover the attributes of venture teamwork when students work co‐operatively rather than collaboratively on assessment tasks. The study identifies attributes of group work in the classroom that are consistent with real world venture teamwork.

This chapter discusses the reliability of the NBPTS assessments. In order to meet the challenge of ensuring reliability of a complex performance assessment, particular attention was given to the design of the assessment and scoring processes. Several models for considering and determining the reliability of the assessment are also introduced. Results from two assessment cohorts are presented. These results demonstrate that the design features of the NBPTS system support a relatively reliable set of assessments without compromising the complexity of the teaching performance it was designed to assess.

A performance management system has been implemented by Cancer Care Ontario (CCO). This system allows for the monitoring and management of 11 integrated cancer programs (ICPs) across the Province of Ontario. The system comprises of four elements: reporting frequency, reporting requirements, review meetings and accountability and continuous improvement activities. CCO and the ICPs have recently completed quarterly performance review exercises for the last two quarters of the fiscal year 2004‐2005. The purpose of this paper is to address some of the key lessons learned.

The paper provides an outline of the CCO performance management system.

These lessons included: data must be valid and reliable; performance management requires commitments from both parties in the performance review exercises; streamlining performance reporting is beneficial; technology infrastructure which allows for cohesive management of data is vital for a sustainable performance management system; performance indicators need to stand up to scrutiny by both parties; and providing comparative data across the province is valuable. Critical success factors which would help to ensure a successful performance management system include: corporate engagement from various parts of an organization in the review exercises; desire to focus on performance improvement and avoidance of blaming; and strong data management systems.

The performance management system is a practical and sustainable system that allows for performance improvement of cancer care services. It can be a vital tool to enhance accountability within the health care system.

The paper demonstrates that the performance management system supports accountability in the cancer care system for Ontario, and reflects the principles of the provincial governments commitment to continuous improvement of healthcare.

Perhaps it is inevitable that the most accurate, most realistic and most credible method of assessing a person's abilities is also the most expensive and most difficult! The method is assessment centres (see Goodge and Griffiths, for a review of the costs and benefits of different assessment methods).