Search results

Calls for the development and dissemination of evidence-based programs to support children and families have been increasing for decades, but progress has been slow. This paper aims to argue that a singular focus on evaluation has limited the ways in which science and research is incorporated into program development, and advocate instead for the use of a new concept, “scientific accompaniment,” to expand and guide program development and testing.

A heuristic is provided to guide research–practice teams in assessing the program’s developmental stage and level of evidence.

In an idealized pathway, scientific accompaniment begins early in program development, with ongoing input from both practitioners and researchers, resulting in programs that are both effective and scalable. The heuristic also provides guidance for how to “catch up” on evidence when program development and science utilization are out of sync.

While implementation models provide ideas on improving the use of evidence-based practices, social service programs suffer from a significant lack of research and evaluation. Evaluation resources are typically not used by social service program developers and collaboration with researchers happens late in program development, if at all. There are few resources or models that encourage and guide the use of science and evaluation across program development.

This paper aims to examine the relationship between advocacy on behalf of libraries and impact evaluation in a national public library development context in which the boundaries between these two roles are likely to be blurred, creating ethical issues for all concerned.

The authors draw upon their broad experience of helping various international organisations to develop strategic approaches to impact planning and evaluation for public libraries, as well as their work in helping to develop library practice at national level, in order to focus on and examine the creative tensions between impact evaluation and advocacy.

There are particular issues for all key participants (international programme funders, policy shapers, service managers, evaluators and advocates) in planning, supporting and delivering impact evaluation programmes. Most of these can be addressed directly but some (such as balancing programme requirements against local priorities, or achieving a balance between collecting evidence based on predetermined impact indicators and progressive focusing) entail management of the tensions between conflicting pressures.

Specific ways forward are offered to encourage ethical advocacy and impact evaluation at national library development level. These principles are equally applicable to education and health library development and to public library development at regional or local levels.

The particular issues of advocacy and impact evaluation in the national public library development context have not previously been recognized in the international development literature addressing advocacy and evaluation or in the library and information services research literature.

This study is an attempt to explore Spanish auditors' sensitivity towards financial evidence and the implications of the so‐called “recency effect” in a context of an ambiguous standard.

Using Hogarth and Einhorn's belief revision model, we designed a lab experiment to examine how presentation order affects the going concern judgments and the audit decisions of auditors, and the level of skepticism showed by auditors towards the signs of evidence. Two important factors, which may affect the decision‐making process and auditors' attitude to the evidence were also manipulated: the hypothesis frame and audit experience. The sample of subjects participating in this study comprises 81 Spanish auditors and 104 auditing postgraduate students.

It was found an order effect, whereby those auditors who received favorable evidence at the end of a series showed greater confidence in their client's continuity and thus issued less severe audit reports than in the case of when negative evidence was processed last. Further, it was did not found that experience or framing reduced the recency bias. Moreover, the estimation of auditor sensitivity to the evidence suggests a lack of professional skepticism in the evaluation of the client's going concern status.

First, the structure of auditing standards might contribute to auditors' reluctance to issue qualified audit reports. Second, the absence of skepticism might be contributing to the profound debate and notable feeling of distrust regarding the social function that the auditor profession should fulfill.

Recent major financial scandals give us a reason to question whether auditors are skeptical in the going concern task. This study expands previous research of Bamber et al. by investigating the auditors' attitude towards the evidence in the going concern evaluation.

I synthesize the extant experimental literature examining auditor evaluation of others’ credibility published in six top accounting journals over the last three-and-a-half decades. I adapt the original definition of credibility by Hovland, Janis, and Kelley (1953): the extent of perceiving someone as competent and trustworthy. Audit guidance requires auditors to consider credibility of management, internal auditors, and staff, yet the research literature on auditor evaluation of others’ credibility is fragmented and scarce, limiting our understanding of determinants and consequences of auditor evaluations. I develop a framework for analysis of research on auditor evaluation of others’ credibility and review extant literature by types of examined effects (determinants of credibility vs. consequences of credibility) and by examined credibility components (competence, trustworthiness, or both). Throughout the literature review I suggest areas for future research.

Many higher education institutions (HEIs) have constructed their internal evaluation systems to secure teaching quality. This paper aims to analyze teaching quality, HEIs use students to evaluate their professors as they have direct contact with the professors during the whole semester. The authorities hope to receive valuable information, which can be used for many administrative purposes. The bias in the evaluation toward professors’ gender and attractiveness has already been proven. However, there is only limited evidence whether students give higher value to teaching quality over the professors’ personality.

In this paper, the authors go further in the gender-attractiveness evidence and put the evaluation in contrast with professors’ experience, age, etc.

The results indicate that the effect of experience predominates the effect of gender and in some areas also the effect of age. What is more, a semester in which a course is taken also influences the evaluation as different professors’ abilities are required in teaching in a different semester. On the other hand, the results do not fully confirm the effect of gender on the evaluation.

The results reveal that it is important to consider the course structure to assign professors to the right courses.

This research seeks to examine whether two relevant characteristics, source objectivity and internal control effectiveness, influence how auditors evaluate evidence items supporting accounting estimates.

A controlled experiment approach with a sample of 24 auditors from one large international firm.

Results indicate that effective internal controls reduce the impact of relying on internal as opposed to external evidence items. Results also suggest that auditors place reliance on internal control effectiveness when they evaluate external evidence items.

Recent professional trends, such as the demand for faster financial reporting, put pressure on auditors to rely on internal rather than more persuasive external evidence items. Relying on less persuasive evidence items reduces audit effectiveness. Auditors may respond by examining a second evidence characteristic; US audit standards suggest evaluating internal control effectiveness if evidence was generated from internal (i.e. client) sources. Thus, this study explores whether internal control effectiveness reduces the impact of relying on evidence items with lower source objectivity.

Prior research has concentrated on examining the impact of a change in one evidence characteristic on audit judgment; this study expands the understanding of the evidence evaluation process by exploring how auditors evaluate multiple evidence characteristics. Furthermore, as suggested by Bonner, this research identifies an audit judgment deficiency (i.e. reliance on less persuasive internal evidence due to the demand for faster financial reporting) and examines one potential remedy (i.e. consideration of internal control effectiveness).

This paper is designed to critically review and analyze the body of research on a popular gang reduction strategy, implemented widely in the United States and a number of other countries, to: (1) assess whether researchers designed their evaluations to align with the theorized causal mechanisms that bring about reductions in violence; and (2) discuss how evidence on gang programs is generated and consumed. That review and assessment is then used to frame a research agenda for studying gang interventions.

A case study design is used to generate a multi-faceted understanding of the possible avenues for evaluation research on the law enforcement-based strategy known as the Group Violence Intervention. The paper discusses questions that remain to be answered about the strategy, such as “what type of deterrence is operating?” and if the model actually works by the threat of deterrence, and not by removing high-risk offenders and shootings from the street, what activities are needed to maintain the effect?

Across roughly two dozen impact evaluations of GVI, none have examined the likely cause and effect components of this multi-partner strategy in reducing the violence. Furthermore, there are many issues related to the production and generation of criminal justice evaluation research that have adversely pushed the balance of evidence on what works in gang reduction toward law enforcement programming. However, there are many strategies that researchers can use to think broadly about appropriate and holistic research and evaluation on gangs and gang programming.

The recommendations for research, if implemented, can help build a body of knowledge to move toward community-based and restorative models of gang violence reduction.

This original piece is one of the first essays to contextualize and discuss how aspects of the production of social science research on gangs may directly impact what programs and strategies are implemented on the ground.

Training in research and evaluation skills is a frequently expressed need among health promotion practitioners. Research conducted in Scotland among health promotion specialists and their managers showed that training in research on its own would be an insufficient response. In this paper, it is argued that there is a need to develop a broader strategy which seeks to strengthen research capacity within health promotion practice settings, rather than simply offering training to improve practitioners’ research skills. This will help to improve the quality of research conducted in practice settings and contribute to building an evidence base for health promotion. A broader professional development strategy for health promotion research in Scotland is proposed which utilizes a range of learning routes and delivery mechanisms. This will be backed up by the establishment of a broad strategic research partnership which brings together practitioners, researchers and policy‐makers so as to develop a better understanding of what evaluation evidence is needed and who is contributing what.

Signs of Safety (SoS) is a strengths-based approach to child protection casework that has been widely adopted in countries across the world. The purpose of this paper is to report on a study that aimed to synthesise the many evaluations of SoS that have been conducted to assess their strengths and limitations. The intention is to identify the aspects which should be explored further and those that remain unexamined to inform future evaluations.

The study collated and then examined many of the evaluations that are in the public domain as well as some of those that were conducted within agencies and have not been made publicly available.

At the present time (early 2019), the evidence base for SoS is limited. Independent, robust research needs to be undertaken over time to build on the studies that have been conducted. New research must be designed to fill gaps and be capable of producing the evidence required and it must address its own limitations.

This study is the most comprehensive contemporary review of the evaluations of SoS that have been conducted to the best of the authors’ knowledge.

Numerous information system evaluation methodologies have been proposed in the literature. However, most of them suffer from a number of inherited disabilities originating in the nature of their fundamental principles as well as the nature of information systems evaluation. Uses evidence from the evaluation literature and two case studies to address the multiple dimensions for evaluation practices. We focus on the context, content and process of information systems evaluation as a source of organisational change. We found a noticeable gap between the recent theoretical work on IS evaluation and the practices within the case study organisations. This was not due to any lack of knowledge but was attributed to contextual variables such as the organisational culture and the power of important stakeholder groups.