Search results

A meta-evaluation is an assessment of evaluation practices. Meta-evaluations include assessments of validity and usefulness of two or more studies that focus on the same issues. Every performance audit is grounded explicitly or implicitly in one or more theories of program evaluation. A deep understanding of alternative theories of program evaluation is helpful to gain clarity about sound auditing practices. We present a review of several theories of program evaluation.

This study includes a meta-evaluation of seven government audits on the efficiency and effectiveness of tourism departments and programs. The seven tourism-marketing performance audits are program evaluations for: Missouri, North Carolina, Tennessee, Minnesota, Australia, and two for Hawaii. The majority of these audits are negative performance assessments. Similarly, although these audits are more useful than none at all, the central conclusion of the meta-evaluation is that most of these audit reports are inadequate assessments. These audits are too limited in the issues examined; not sufficiently grounded in relevant evaluation theory and practice; and fail to include recommendations, that if implemented, would result in substantial increases in performance.

This chapter maps out and provides a rapid assessment of, the successes and shortcomings of key national interventions on youth employment in South Africa. It focuses on programmes intended for young people aged 15–34 years of age, considering the specific needs of 15–24 year olds as compared 25–34 year olds and the particular needs of young women, youth with different educational qualifications, youth from the rural areas, youth with disabilities as well as youth from varied socio-economic backgrounds and social issues. The conceptual framework underpinning this chapter informs the way in which the data has been categorised and analysed. The framework is underpinned by a core assumption, that, while unemployment is a national challenge in South Africa, there are particular challenges that specifically affect youth. This requires a youth employment strategy that addresses the needs of different cohorts of young people and specifically addresses the myriad of ways in which young people transition into the labour market. The chapter further proposes that we need to understand that enabling successful Labour Market Transitions necessitate a strategy that can overcome ‘failures’ with respect to both supply and demand as well as ‘failures’ of alignment between supply and demand.

Increased emphasis on offering quality education underscores the need for developing a rigorous process for assessing academic programs in higher education. In this chapter, we develop a practical and rigorous framework for comprehensive assessment of academic programs. This framework generates in-depth communication between the academic departments and the university administration. It provides a useful tool for advancing the university mission, setting priorities, allocating resources, and identifying future areas of potential growth. This data-driven framework covers a wide range of qualitative and quantitative variables. To ensure a smooth and efficient implementation of the assessment process we present the critical stages in the development of a successful program assessment framework − from determining the assessment criteria, establishing the organizational climate, appointing the assessment committee, preparing program self-studies, to collecting and analyzing data. We present real examples from the author’s home institution to illustrate and support the reader’s understanding of the framework.

Evidence-based practices have been shown to meaningfully improve learner outcomes by bodies of high-quality research studies and should therefore be prioritized for use in schools, especially with struggling learners such as students with learning disabilities. Although many resources are available on the internet with information about evidence-based practices, the magnitude and technical nature of the websites are often overwhelming to practitioners and are therefore not frequently used as part of the instructional decision-making process. In this chapter, we aim to provide a “one stop shopping experience” for readers interested in evidence-based practices for students with learning disabilities by reviewing five relevant website. Specifically, for each website we review (a) the procedures used to classify the evidence-based status of practices, (b) the classification scheme used to indicate the level of research support for practices, and (c) the practices reviewed for students with learning disabilities and their evidence-based classification. We conclude with a discussion of issues related to interpreting and applying information on evidence-based practices from these websites.

Many youth with a disability would like to work but encounter challenges finding employment. Vocational interventions can help youth with disabilities gain employment skills and jobs. In this chapter, we assess: (1) how vocational programs for youth with physical disabilities influence employment-related skills and outcomes; and (2) the common components of vocational programs for these youth.

Our research team conducted a systematic review of peer-reviewed literature with six major databases: Medline, PsychInfo, Scopus, Web of Science, CINAHL, and Embase. Publications selected for inclusion met the following criteria: (1) peer-reviewed journal article, dissertation, or conference paper, published between 1990 and January 2014; (2) addresses vocational program or intervention for youth with physical disabilities; and (3) sample includes at least 50% youth (aged 15–25) with an acquired or congenital physical disability.

Of the 4,588 studies identified in our search, 8 met the inclusion criteria. In six of the studies, the majority of participants gained paid or unpaid employment after participating in a vocational program. Five studies showed improved knowledge and perceptions of employment. Most studies showed improvements in at least one vocational outcome such as knowledge about job searching, job interviews, advocating for workplace adaptations, and how to access services and supports. Common intervention components included: experiential learning, mentorship, and family involvement. Most programs took place in the community or rehabilitation centers that varied in length and were delivered by a variety of professionals. Most programs had a combination of group and individual components.

There is some evidence to suggest that vocational programs can influence employment outcomes for youth with physical disabilities. However, further research is needed with more rigorous and longitudinal designs.

Purpose – To develop a framework for studying financial incentive program implementation mechanisms, the means by which physician practices and physicians translate incentive program goals into their specific office setting. Understanding how new financial incentives fit with the structure of physician practices and individual providers’ work may shed some insight on the variable effects of physician incentives documented in numerous reviews and meta-analyses.

Design/Methodology/Approach – Reviewing select articles on pay-for-performance evaluations to identify and characterize the presence of implementation mechanisms for designing, communicating, implementing, and maintaining financial incentive programs as well as recognizing participants’ success and effects on patient care.

Findings – Although uncommonly included in evaluations, evidence from 26 articles reveals financial incentive program sponsors and participants utilized a variety of strategies to facilitate communication about program goals and intentions, to provide feedback about participants’ progress, and to assist practices in providing recommended services. Despite diversity in programs’ geographic locations, clinical targets, scope, and market context, sponsors and participants deployed common strategies. While these methods largely pertained to communication between program sponsors and participants and the provision of information about performance through reports and registries, they also included other activities such as efforts to engage patients and ways to change staff roles.

Limitations – This review covers a limited body of research to develop a conceptual framework for future research; it did not exhaustively search for new articles and cannot definitively link particular implementation mechanisms to outcomes.

Practical Implications – Our results underscore the effects implementation mechanisms may have on how practices incorporate new programs into existing systems of care which implicates both the potential rewards from small changes as well as the resources which may be required to obtain buy-in and support.

Originality/Value – We identify gaps in previous research regarding actual changes occurring in physician practices in response to physician incentive programs. We offer suggestions for future evaluation by proposing a framework for understanding implementation. Our model will assist future scholars in translating site-specific experiences with incentive programs into more broadly relevant guidance for practices by facilitating comparisons across seemingly disparate programs.

Television critics play a central role in the interpretation of cultural forms, objects, and productions. In contrast to critics in elite art worlds, the role and status of television critics are less institutionalized and less well understood. One indicator of the degree and status of the institutionalization of critics’ roles is the codification of evaluative criteria and critical practices. Our research examines whether critics in television draw upon a recognizable set of evaluation criteria, and if so, whether that repertoire of aesthetic concepts increasingly parallels criteria employed by critics in elite art worlds. Using multidimensional scaling to delineate television criticism over the last two decades, a period of considerable transformation in the industry, we find that television criticism attends to a core set of conventional criteria. These include appraisal of formal aesthetic elements, signaling increased attention to television as an art form, while retaining consideration of factors such as entertainment value that are of interest to audiences and business constituencies alike.

The involvement of stakeholders such as employers, alumni, and students has always been considered a key element in improving the higher education (HE) system. While considering stakeholders as key players in serving the market and in improving HE instruction, a two-sided collaborative involvement should aim at satisfying the mutual interests and overcoming existing barriers. Quality assurance systems have always supported crossing these barriers to link with the external stakeholders. However, many of the external quality assurance agencies (EQA) in the Middle East and North Africa (MENA) region specify a group of external holders, limiting the various types of key stakeholders needed to enhance the academic programs. On the other hand, there are encountered risks in involving stakeholders if left with no objective guidance, especially that quality agencies are formidably urging the universities to consider the external stakeholders’ inputs to satisfy the quality assurance standards. The main objective of this chapter is to investigate the types of stakeholders’ and their levels of involvement within the local higher education institutions (HEIs). The chapter aims to provide an insight to invest in this involvement and utilize it to further improve the programs and their graduate attributes and suggests actions that would proficiently and truly enhance the involvement of external stakeholders. The outcomes of this chapter are expected to guide the EQAs and the HEIs to develop new practices in involving stakeholders, such as curriculum input, collegiate internships, aligning graduate attributes to market needs, financial support through endowments, professional development, and partnerships in service-level agreements.

In 2017, the CPA exam added task-based simulations that require candidates to demonstrate analytical, critical thinking, and problem-solving skills. In addition, there has been an evolution in training and learning from face-to-face, synchronistic, group study, to online, asynchronistic, and independent study. These changes suggest the need to reexamine the effectiveness of CPA exam candidates’ study methods. In this study, the authors analyze the degree of use and effectiveness of various study methods (e.g., instructor-led study programs, self-study review courses, flashcards, study notes, and practice problems). Based on a survey of candidates taking the CPA exam in 2018 and 2019, the authors find that the most effective method of study is practicing problems. Opposite the results of some extant research, using an instructor-led study program does not appear to increase the likelihood of passing. Because this is the only study to examine a variety of CPA exam study methods since the 2017 exam changes, its results serve as a benchmark for researchers interested in investigating future changes in study habits.

Although a large body of research has focused on young children with learning disabilities (LD) and behavioral disorders (BD) in preschool and elementary school settings, there is considerably less information about this population during adolescence. Recent work suggests that youth with these disabilities experience challenges in areas such as social skills, increased depressive symptoms, and involvement in the juvenile justice system. In addition, for a small percentage of the population, negative outcomes experienced during early childhood appear to persist in adolescence and early adulthood suggesting the need for additional interventions. Two primary aims guide the current chapter. First, we review key domains of adolescent development (social, emotional, and behavioral) and highlight ways in which development differs for students with LD and BD. Second, we introduce the field of social and emotional learning (SEL) and the accumulating body of research that suggests that this approach could have numerous benefits for this population. We describe the results of recent meta-analytic reviews of SEL programs to indicate the current state of the field, highlight a few evidence-based universal and indicated SEL programs for secondary school settings, and describe important areas for future research.