Search results

The scoring system for the National Board for Professional Teaching Standards (NBPTS) assessments was a groundbreaking undertaking that brought with it a host of unanticipated challenges. These, in turn, generated a complete revision of the approach to scoring and the design underwent a number of changes during the first decade. Beginning with an analytical model which was so ambitious that it was entirely too cumbersome and complex to be undertaken within a reasonable timeframe, assessment developers had to systematically redesign a scoring system that would be at once reliable, valid, and operationally feasible.

To appreciate the distance the National Board for Professional Teaching Standards (NBPTS) certification program has come, and the speed with which it has traveled that distance, a glance at its first decade from the perspective of assessment development is essential. The particulars of the history of the NBPTS's assessment strategies and designs have determined in many ways the current assessment architecture: the evolution of the assessment's design reveals the growth in our knowledge of innovative assessment strategies and formats and their uses. In this chapter, I will briefly summarize the history of the NBPTS assessment program, then describe and analyze the earliest assessment designs, some intermediate approaches, and then the current iteration (commonly referred to as the next generation certificates). I will finally detail the current assessment architecture, connecting that architecture to both the history and the lessons learned from the initial assessments.

The purpose of this paper is to compare two years of results for one state’s performance-based assessments for principal licensure Performance Assessment for Leaders (PAL). This includes the field trial (2014–2015) and first year of statewide implementation (2015–2016) when passing score requirements and fees were added. Survey results on candidates’ career aspirations provide concurrent validation.

Two years of PAL submissions (n=569 candidates) were scored by trained, certified scorers. Task and total score results were compared by year, preparation pathway and gender. Online feedback survey results on career aspirations for (n=146 candidates) were compared by year.

The results show that PAL assessments measure independent dimensions of leadership, differentiate candidates on leadership knowledge and skills, and confirm PAL’s internal validity. Implementation year scores were higher than field trial scores, and preparation program candidates scored better than non-program candidates did. Candidate career aspirations were stronger in the implementation year than during the field trial.

The study is limited to one state’s candidates, but findings are generalizable based on the wide range among candidates’ districts (demographically and economically).

The results are promising for the leadership preparation and assessment field, demonstrating the effectiveness of performance assessment for authentic evaluation of leadership candidates’ knowledge and skill and overall readiness for initial leadership work.

This is the first large scale performance assessment for aspiring leaders designed for state licensure decisions. It is being replicated in another state and shown promise for both formative and summative leadership assessment.

Although standardized testing of K-12 student knowledge and understanding garners considerable attention, few observers profess satisfaction with the assessments in place. In this exploratory paper, we report on the data gathered from an open-ended email survey of small, convenience samples of teachers and researchers. Although no clear consensus about alternative assessments of students’ historical knowing and understanding emerged, we argue that the potential for a consensus exists. Any emergent consensus, however, must be negotiated with several issues in mind.

Peninsula Technikon has for the past few years engaged in a review of their current qualifications with a view to transforming these curricula into Outcomes Based Education (OBE) format with learning specified in terms of broad, exit level outcomes. In turn the subjects making up these programmes were further broken down into enabling and sub‐enabling learning outcomes. The institution is currently engaged in designing tasks and assessment criteria which attempt to piece together these learning outcomes into projects which integrate student learning and introduce reflective assessment practices. The process underlying the design of these integrated projects has necessitated collaborative planning and assessment among academic staff at various levels of each programme. Through this process staff have grappled with and puzzled over the realities of implementing OBE at classroom level. This paper focuses on an integrated project designed for the exit level of the Civil Engineering diploma programme and explores the assessment practices from lecturer perspectives through unstructured focus group discussions, as well as individual, structured questionnaires and interviews. The outcomes of this research are presented as work in progress and tentative recommendations will be made regarding the implementation of integrated projects.

Using Bonner's model, this paper aims to examine the effects of skill level and the two task complexity dimensions – clarity and quantity of information – on subjects' internal control risk assessments.

The research expands the literature by isolating the individual components of task complexity and examining how skill interacts with either component in affecting decision making. It uses a 2×4 mixed factors laboratory experiment. This design allows the effects of task complexity and skill between and within subjects to be examined. The mixed factors design includes two levels (high and low) for each dimension of task complexity (clarity and quantity) on a between‐subjects basis, with four separate cases on a within‐subjects basis. Skill level is measured as the subject's task‐related knowledge. The subjects are senior‐level students in auditing courses at three Midwest universities.

It is found that subjects assess control risk too high, consistent with the conservatism principle. Skill level mediates this finding: high‐skill subjects make more accurate risk assessments; low‐skill subjects consistently assess control risk too high. Over repetitions of complex tasks, high‐skill subjects make more accurate assessments, while low‐skill subjects initially overstate, then improve. For repetitive simple tasks, both skill levels get worse, increasingly overstating their assessments. These findings support current practice, indicating that experienced auditors make complex risk assessments, where repetitive performance of complex tasks improves risk assessments. However, repetitive simple tasks may result in assessing control risk too high, resulting in excessive testing.

Consistent with prior research, the results suggest task complexity and subject skill are important considerations in experimental research designs.

This chapter will explore how assessment might look in next generation learning spaces where we have the potential to merge physical and virtual activities. Students now have ready access to a world of resources within their classroom and this fundamentally changes the nature of learning and assessment. The trend toward gamification of learning and assessment will be examined and the issue of assessment in new educational environments such as MOOCs will be explored. The impact of the semantic web (Web 3.0), where web objects and their context are all linked and objects have memory of how an individual student used them on previous occasions, will be discussed.

Next generation learning spaces encapsulate the affordances of both physical and virtual spaces and yet many assessment tasks are still designed as if students occupied only one of these spaces. Teachers will need to design more authentic, meaningful tasks that will engage students in using the full range of their capabilities and available resources, both physical and virtual. Students come together physically to engage in the social construction of their knowledge and can use the virtual spaces to broaden the social dimension of their learning environment.

Gamification of learning and assessment will require new approaches to defining tasks as teachers will need to decide how to incorporate diagnostic, formative, and summative assessment components within a more holistic educational environment. Game theory will be blended with learning theory in curriculum design and will result in the redesign of learning and assessment activities that are based on engagement (flow), user needs, and an evidence-centered design approach.

Using the UK's recent disability legislation as a trigger, the paper explores issues in evidencing competence in the current context of using assessment as a tool for learning.

A modified version of the pragmatic method is used, in which assessment theory is used to explore the tension in assessment which is both legislatively compliant and theoretically coherent.

In the current context of requiring assessment to be edumetrically sound, the legislation of competence standards is problematic in four respects. Task validity is now a much more diffuse concept. Scoring validity has to contend with many possible accommodations. Assessment generalisability has to consider both the relevance and representativeness of the assessment task. Consequential validity is essentially concerned with formative assessment, which is considered pedagogically important but, politically, is of less significance than summative assessment.

The analysis here is in terms of legislation within the UK and therefore may not refer to nuances of difference in the anti‐discrimination legislation in other jurisdictions.

The study offers academics and administrators a framework within which to review the edumetric soundness of their assessment practices and policies. In so, doing possible difficulties in equitable assessment can be made explicit. This, in turn, has implications for staff development.

The paper uses significant pedagogical theory to illuminate a legislative requirement and thereby contextualise difficulties in implementing equitable assessment.

Increased interest in team dynamics has resulted in new methods for measuring teamwork over time. The primary purpose of this chapter is to provide a survey of recent developments in teamwork/collaboration measurement in an educational context. Key topics include conceptual frameworks, large-scale assessments, and innovative measurement techniques.

A range of methods for collecting and analyzing teamwork data are discussed, and five frameworks for measuring collaborative problem solving (CPS) over time are compared. Frameworks from Programme for International Student Assessment (PISA), Assessment and Teaching of 21st Century Skills (ATC21S) project, Educational Testing Service (ETS), ACT, and von Davier and Halpin (2013) are discussed. Results of assessments developed from these frameworks are also considered.

New techniques for measuring team dynamics over time have great potential to improve education and work outcomes. Preliminary results of the assessments developed from these frameworks show that important advances in teamwork measurement have been enabled by innovative task designs, data-mining techniques, and novel applications of stochastic models.

This novel overview and comparison of interdisciplinary approaches will help to indicate where progress has been made and what challenges are ahead.