Search results

The purpose of this paper is to suggest indicators for evaluating Iran national nanotechnology plan at the policy and program levels. Governments are required to evaluate their policies to be accountable to their audience and to improve their policies. The evaluation of nanotechnology policies both in academic and governmental institutions has been mainly confined to measuring the research and development outputs, while explicit models are required for evaluating national nanotechnology policies and programs. In other words, although policies are translated into several activities, it does not mean that the evaluation of activities results in holistic policy evaluation.

The present research uses a qualitative method to explore the process of developing a model for assessing nanotechnology policymaking in Iran. Expert opinions have been collected and analyzed by using local and international surveys to validate different elements of the model.

The model includes the evaluation time-frame, criteria and indicators for policy and program levels. Assessment at program and policy levels, as distinct from activity evaluation, measures the ultimate objectives of a policy intervention, based on which the outcome and impact indicators should be developed. This paper argues that the Iran national nanotechnology policy should be evaluated and revised triennially based on the model in which 46 outcome and impact indicators are included under three general criteria of appropriateness, efficiency and effectiveness. A number of these indicators should be evaluated annually or biannually.

The evaluation of nanotechnology plans has been mainly limited to assessing the research and development outputs that are related to the activity level. The main value of this work is developing outcome and impact indicators that are related to policy and program levels.

The purpose of this paper is to discuss different approaches to evaluation taken in the case of two regeneration programmes in England and Wales and identify how assumptions about the nature of policy and programme implementation affect evaluation and considerations of its use in the public sector.

Using an analysis of evaluation reports relating to two regeneration programmes, a distinction is drawn between two approaches to evaluation. This distinction is then articulated further with reference to differing perspectives in policy implementation studies. Related issues of policy and programme success and failure are discussed, in addition to the potential for policy learning.

There is a difference between the two approaches to evaluation employed in the two cases, which is likely to reflect contrasting beliefs about the nature and process of implementation. Implementation research indicates that while a process of evaluating progress against a set of specific objectives clearly has legitimacy in a public sector environment, it is likely to be inconclusive or even counterproductive. This is due to the inevitability of policy evolution, the multiple reformations of policy at the micro level, and the constraints engendered by norms of appropriacy in the wider policy environment.

This article suggests how ongoing debates in implementation studies can inform and illuminate decisions made about approaches to evaluation. The dynamic between conceptions of implementation and the purpose of evaluation is illustrated through a comparison of the use of evaluation in two regeneration programmes.

The purpose of this paper is to describe how elementary school principals adjust their leadership practice in response to Washington’s new teacher evaluation policy.

The authors conducted a modified content analysis of open-ended survey responses collected from elementary school principals in Washington State. In all, the survey included responses from 354 elementary school principals representing 25.0 percent of the state’s elementary school principal population. ATLAS.ti supported data analysis and assisted in the derivation of three key findings.

Elementary school principals changed their instructional leadership practice in response to the new teacher evaluation policy in three significant ways. First, principals adjusted their approach to classroom observation to complete more intentional, in-depth observational activities. Second, principals redistributed non-instructional responsibilities to clerical staff members to allow themselves and other administrators more time for classroom observation. Third, principals adopted a learning stance to the new policy and thus sought external support, especially coaching, to assist them with the implementation of new evaluation practices.

The study faced three limitations. First, the sample of respondents included in this study cannot be generalized to the state as participants were not randomly selected. Second, the survey did not utilize a longitudinal design, and thus its findings only relate to the first year of the policy’s implementation. Third, the study does not include school-based evidence to triangulate principals’ survey responses.

The study contributes to the instructional leadership literature. Specifically, the study offers further insights into the adjustments principals make in their leadership to accommodate expectations found in new teacher evaluation policy.

This chapter will focus on the use of evaluative research in higher education policy analysis. The approach will be illustrated by reference to higher education policy in Scottish higher education, with particular reference to the longitudinal evaluative research of support of teaching and learning (T&L) (the Quality Enhancement Framework or QEF). The chapter will discuss the features of the research process which are shaped by evaluation theory. We adopt a theoretical position on policy research which foregrounds the situated experience of policy as a core research focus. Policy is depicted as being underscored by an implicit theory of change which is used to structure and orientate the research focus. The design of the research is characterised by the involvement of potential users of the research output, with implications on the way in which findings are articulated, presented and ultimately used, along with aspects of the evaluative research design. The case study of the QEF will be contextualised, and the intersection between the design features and theoretical approaches, and the use and usability of research outputs, will be established.

This article explores the applicability of participatory action research (PAR) on two levels: on the one hand, as a participatory evaluation method for community engagement and community development; on the other hand, as a tool to link ex ante and ex-post evaluation that situated at various stages of the policy cycle.

Through a practice-based case study of the community engagement process of Valletta Design Cluster, this paper aims to illustrate how PAR can offer collaborative and continuous evaluation by facilitating social action through a practical, situative, context-bounded, responsive and transformative framework.

The study explores how PAR can contribute to cultural sustainability by linking community development with participatory evaluation, and it offers new perspectives on the applicability of PAR as a tool to link ex ante and ex-post evaluation, situated at various stages of the policy cycle.

Although based on a single case, the paper demonstrates that the method has the potential to be applied in various contexts, as it helps to foster local ownership and to develop future cultural strategies, thus providing a base for cultural sustainability.

The novelty of this study is to link evaluation – a majorly top-down and ex ante approach – with participatory planning. PAR-E offers a continuous participatory framework for the whole European Capitals of Culture (ECoC) cycle, as well as serving as a tool for empowerment and community development.

Information and communications technology (ICT) offers enormous opportunities for individuals, businesses and society. The application of ICT is equally important to economic and non-economic activities. Researchers have increasingly focused on the adoption and use of ICT by small and medium enterprises (SMEs) as the economic development of a country is largely dependent on them. Following the success of ICT utilisation in SMEs in developed countries, many developing countries are looking to utilise the potential of the technology to develop SMEs. Past studies have shown that the contribution of ICT to the performance of SMEs is not clear and certain. Thus, it is crucial to determine the effectiveness of ICT in generating firm performance since this has implications for SMEs’ expenditure on the technology. This research examines the diffusion of ICT among SMEs with respect to the typical stages from innovation adoption to post-adoption, by analysing the actual usage of ICT and value creation. The mediating effects of integration and utilisation on SME performance are also studied. Grounded in the innovation diffusion literature, institutional theory and resource-based theory, this study has developed a comprehensive integrated research model focused on the research objectives. Following a positivist research paradigm, this study employs a mixed-method research approach. A preliminary conceptual framework is developed through an extensive literature review and is refined by results from an in-depth field study. During the field study, a total of 11 SME owners or decision-makers were interviewed. The recorded interviews were transcribed and analysed using NVivo 10 to refine the model to develop the research hypotheses. The final research model is composed of 30 first-order and five higher-order constructs which involve both reflective and formative measures. Partial least squares-based structural equation modelling (PLS-SEM) is employed to test the theoretical model with a cross-sectional data set of 282 SMEs in Bangladesh. Survey data were collected using a structured questionnaire issued to SMEs selected by applying a stratified random sampling technique. The structural equation modelling utilises a two-step procedure of data analysis. Prior to estimating the structural model, the measurement model is examined for construct validity of the study variables (i.e. convergent and discriminant validity).

The estimates show cognitive evaluation as an important antecedent for expectation which is shaped primarily by the entrepreneurs’ beliefs (perception) and also influenced by the owners’ innovativeness and culture. Culture further influences expectation. The study finds that facilitating condition, environmental pressure and country readiness are important antecedents of expectation and ICT use. The results also reveal that integration and the degree of ICT utilisation significantly affect SMEs’ performance. Surprisingly, the findings do not reveal any significant impact of ICT usage on performance which apparently suggests the possibility of the ICT productivity paradox. However, the analysis finally proves the non-existence of the paradox by demonstrating the mediating role of ICT integration and degree of utilisation explain the influence of information technology (IT) usage on firm performance which is consistent with the resource-based theory. The results suggest that the use of ICT can enhance SMEs’ performance if the technology is integrated and properly utilised. SME owners or managers, interested stakeholders and policy makers may follow the study’s outcomes and focus on ICT integration and degree of utilisation with a view to attaining superior organisational performance.

This study urges concerned business enterprises and government to look at the environmental and cultural factors with a view to achieving ICT usage success in terms of enhanced firm performance. In particular, improving organisational practices and procedures by eliminating the traditional power distance inside organisations and implementing necessary rules and regulations are important actions for managing environmental and cultural uncertainties. The application of a Bengali user interface may help to ensure the productivity of ICT use by SMEs in Bangladesh. Establishing a favourable national technology infrastructure and legal environment may contribute positively to improving the overall situation. This study also suggests some changes and modifications in the country’s existing policies and strategies. The government and policy makers should undertake mass promotional programs to disseminate information about the various uses of computers and their contribution in developing better organisational performance. Organising specialised training programs for SME capacity building may succeed in attaining the motivation for SMEs to use ICT. Ensuring easy access to the technology by providing loans, grants and subsidies is important. Various stakeholders, partners and related organisations should come forward to support government policies and priorities in order to ensure the productive use of ICT among SMEs which finally will help to foster Bangladesh’s economic development.

Federal and state policymakers in the USA have sought to better differentiate the performance of K-12 teachers by enacting more rigorous evaluation policies. The purpose of this paper is to investigate whether these policies are working as intended and explore whether district stressors such as funding, enrollment, and governance are associated with outcomes.

The authors examined teacher evaluation ratings from 687 districts in Michigan to identify the relationship between district stressors and two outcomes of interest to policymakers: frequency of high ratings and variation of ratings within districts. A qualitative index of variation was used to measure variation of the categorical rating variable.

About 97 percent of teachers in Michigan are rated effective or highly effective, and variation measures indicate overwhelming use of only two ratings. Charter school districts have fewer teachers rated highly than traditional districts, and districts with higher fund balances have more teachers rated highly. Districts with increasing fund balances have higher variation.

The findings suggest that district stressors presumably unrelated to teacher performance may influence teacher evaluation ratings. State teacher evaluation reforms that give districts considerable discretion in designing their teacher evaluation models may not be sufficient for differentiating the performance of teachers.

This research is important as policymakers refine state systems of support for teacher evaluation and provides new evidence that current enactment of teacher evaluation reform may be limiting the value of evaluation ratings for use in personnel decisions.

The purpose of this paper is to study the changing relationship between auditing and evaluation. Over the past several years, supreme auditing institutions (SAIs) in a number of advanced democracies have evolved from conventional auditing institutions to becoming increasingly concerned with assisting policy change and administrative reform in the public sector; tasks that are traditionally associated with evaluation. The paper discusses the potential consequences of this development for the SAIs themselves as well as for the audited and reforming institutions and for policy-making.

The paper uses qualitative method and draws on the extensive literature on auditing and evaluation. The analysis has also benefitted from the authors’ recent comparative research on SAIs.

The findings, summarized in six points, are that the growth of auditing in areas previously assigned to evaluators, has led to a shortened time perspective; stronger emphasis on the administration of policies; increased focus on efficiency of the audited entity; greater independence from the evaluated organizations; a shift in receiver of information toward the legislature and/or the public; and improved communication.

Evaluation as a professional and scholarly field has developed theories and advanced methods to assess the effectiveness of public programs. The growth of auditing may thus change the focus and quality of policy evaluation.

The paper speaks to both scholars and practitioners. To the best of the knowledge a similar analysis has not been done before.

The purpose of this paper is to examine how US school leaders and teachers make sense of multiple accountability policies, including the Common Core State Standards and teacher evaluation, and how this process relates to school priorities and classroom practice.

This study uses a comparative case study approach to understand how principals and teachers in three high-poverty urban schools respond to policy.

Although principals and teachers viewed academic standards and the teacher evaluation framework as complementary, two of the three schools focused on meeting the expectations of the teacher evaluation framework at the expense of attention to academic standards. Without attention to the connections among policies and school priorities, the introduction of new policies may detract from rather than reinforce attention to academic standards.

Principals who are stronger instructional leaders may be better able to “craft coherence” among multiple standards-based policies and school priorities for instruction and student learning. Although their experiences are not generalizable, findings suggest that attending to standards for student and teacher performance without connecting to the implications for content and students’ learning may lead to superficial integration of accountability policies with school priorities.

Findings provide further evidence that principals play an essential role in responding to policy and suggest that districts and external support providers can assist their efforts by creating opportunities for professional learning about the connections among multiple policies and their implications for practice.

This paper extends Honig and Hatch’s conceptualization of “crafting coherence” to the work of teachers and the implications for classroom practice.

Brokering and buffering represent two ways in which principals may respond to hyperrational elements of policy demands in the current era of accountability. The purpose of this paper is to examine how some principals broker more efficient, measurable, and predictable evaluation practices for teachers and others buffer their teachers from inefficient, immeasurable, and unpredictable aspects of policy.

Qualitative data were obtained from 37 school principals and 363 teachers across 12 districts participating in a new teacher evaluation policy in one state of the USA. Principal interviews and teacher focus groups were conducted at the beginning, middle, and end of 2012-2013. Transcripts were coded to identify hyperrational elements of the policy and principals’ brokering and buffering practices.

All principals described elements of the new evaluation policy as inefficient, incalculable, or unpredictable – hallmarks of hyperrationality. Principals brokered efficiency by designing schoolwide parent goals and centralizing procedures; brokered transparency of calculation methods and focused teacher attention on measuring effort, rather than outcomes; and encouraged collective sensemaking to facilitate predictable procedures and outcomes. Principals buffered teachers by de-emphasizing the parent-based component; minimizing the quantitative nature of the ratings; ceding responsibility over calculations to district leaders; and lowering expectations to make ratings controllable.

The paper provides new understanding of principals’ strategic leadership practices, which represented rational responses to hyperrational policy demands. Therefore, the paper includes recommendations for principal preparation, district support for policy implementation, and further research on principal practice.