The Impact of International Achievement Studies on National Education Policymaking: Volume 13

The rapid expansion of national participation in international achievement studies has been a hallmark of educational accountability and planning in countries around the world for the past 25 years. Since the IEA's first international studies on mathematics and science achievement in the late 1960s, the availability and use of international achievement studies for national education policy has exploded (DeBoer, “Why the Fireworks?: Theoretical Perspectives on the Explosion in International Assessments”; Smith & Baker, 2001; Wiseman & Baker, 2005). The most widely adopted studies are now administered on regular cycles and include participating countries from every region and level of development around the world.

The main aim of this chapter is to argue that a sound conceptualization and methodology for measuring the quality of education is a necessary, but not a sufficient, condition for establishing a link between research and policy to improve the quality of education. The following elements have been provided to support this argument: (1) a literature review of the different concepts and methods of measuring the quality of education that are in place internationally, as well as their importance; (2) a UNESCO desk review of 35 developing countries to compare the way educational quality is featured and monitored in National Education Sector Plans (NESPs); and (3) case studies of two developing countries focusing on the implementation of research to measure the quality of education, its impact, and the link between research and policy. It was found that the quality of education is recognized as an important factor in most NESPs, but it has not been defined, measured, or interpreted in a consistent way. Furthermore, while sophisticated and innovative methodologies have already been developed to measure the quality of education, the processes of linking research results with policy still seem to be at a developmental stage. This is a challenge not only for researchers and policy makers, but also for development partners to ensure that (i) policy and planning become more firmly grounded in objectively verifiable scientific evidence and (ii) through its impact on policy and planning, research leads to improvements in the quality of education.

Participation in cross-national assessment is becoming a global phenomenon. While there were only 43 countries that participated in the Programme for International Student Assessment (PISA) in 2000, the number of participating countries/economies has increased to 65 in 2009. To understand this global trend, this chapter seeks to answer the following research questions: What are the real incentives for developing countries to participate in cross-national assessments? What do they gain from actual participation in cross-national assessments, given that there are many constraints and barriers associated with test participation? It employs country-level fixed effects to test the hypothesis that there is a positive association between participation in cross-national assessments and foreign aid to education. This study shows that countries that participate in major cross-national assessments receive, on average, 37 percent more foreign aid to education than countries that do not participate in major cross-national assessments, while holding all other variables constant. Although further research is necessary to make a causal warrant of the association between participation in cross-national assessment and education aid, the results of this study have great implications for developing countries that are considering participating in cross-national assessments.

In this chapter, we seek to contribute to a line of international and comparative research that began with Heyneman and Loxley's 1983 study examining the importance of schools across national contexts. In their influential paper, Heyneman and Loxley found that in lower-income societies, schools (rather than families) constitute the predominant influence in explaining student achievement. Similar studies followed, often with results challenging Heyneman and Loxley's original findings. We argue that one reason for inconsistencies among these studies is the failure to account for the distribution of income. Until recently, few studies had examined whether school effects vary across countries with different levels of income inequality. Yet emerging evidence suggests that inequality plays an important role in determining the extent to which schools “matter” for student learning. In this study, we employ hierarchical linear modeling and two related yet distinct measures of inequality to examine how inequality relates to within- and between-country variations in student performance. We also explore whether, in certain countries, schools are differently able to help children from higher- and lower-Socio Economic Status (SES) groups. To capture sufficient variation in country context, we use data from nine diverse countries participating in the fourth grade application of the 2003 Trends in International Mathematics and Science Study (TIMSS). Our findings indicate that schools are important in their own right, and especially important in unequal countries. However, schools may affect SES-based achievement gaps only in countries with high income and resource inequality, accompanied by heterogeneous classrooms in terms of SES composition.

Our first goal is to discuss new information for national policymaking which may arise from the analyses of international achievement study data. The second is to illustrate this potential by exploring determinants of students' career plans in a cross-national perspective. Using neo-institutionalism as our theoretical framework, we propose that the influence of a global educational ideology encourages high levels of occupational ambitions among students. This is particularly the case in countries where the transfer of this ideology is supported by the reception of aid for education, where economic prosperity is at modest levels but the service sector employment is expanding. To explore this proposition, we analyze students' occupational expectations using the 2006 PISA surveys from 49 countries. We account for a broad range of possible determinants by estimating three-level hierarchical models in which students are clustered in schools and schools within countries. We find that at individual and school levels, ambition is positively correlated with economic and noneconomic resources. In contrast, students in poorer countries, where secondary education is not yet universally accessible, tend to be more ambitious. The global educational ideology, indicated by the reception of education-related aid, is associated with student career optimism, while students in affluent nations with less economic inequality have modest occupational plans. In addition, the rate of service sector expansion is positively related to high levels of ambition. These patterns hold even after we control for cross-national variation in the extent to which PISA respondents represent populations of 15-year-olds in their countries.

Improving Turkey's low level of education quality and achieving equity in quality education across its seven regions continue to be a monumental challenge. The purpose of this study was to document the extent of Turkey's regional differences in science achievement at the eighth grade and to investigate factors associated with these differences. Identifying the factors influencing Turkey's regional inequalities in student learning is crucial for establishing policies that will help raise the educational performance at the national level as well as close regional gaps. A series of hierarchical linear modeling (HLM) analyses were performed at two levels (the school/class level and student level) using Turkey's nationally representative data from the Trends in International Mathematics and Science Study (TIMSS) 2007. Findings suggested that attempts to increase Turkish students' achievement and close the achievement gaps between regions should target the students in the undeveloped regions, particularly in Southeastern and Eastern Anatolia. Designing interventions to improve competency in Turkish and to compensate for the shortcomings of insufficient parental education, limited home educational resources, poor school climate for academic achievement, and inadequate instructional equipment and facilities might be expected to close the regional achievement gaps as well as raise the overall achievement level in Turkey. Using TIMSS data, this study provided an example of a methodology that may be employed to describe how student achievement is distributed among various subpopulations of national interest and to investigate the factors that contribute to differences in the achievement distribution.

In 2006, Kyrgyzstan entered the Program for International Student Assessment (PISA) competition and the results were very poor, with it securing the last position among all participating countries. However, to date, there are no in-depth studies examining the results and the impact of the PISA test on the quality of secondary education in Kyrgyzstan. This chapter attempts to fill this gap. The study was conducted in post-Soviet Central Asian education context where standardized tests are only emerging and what their far-reaching implications are not yet known. The data were collected using semistructured interviews and document analysis. Respondents to semistructured interviews included representatives of government, education officials, specialists from the independent testing center, representatives of international development organizations, university professors, school administrators and teachers, community members, and students. The study showed that the poor results of PISA 2006 awakened many policymakers, education officials, and educators about the current state of the country's education. However, the findings of the study also showed that the lessons and implications were not analyzed systematically and, as a result, rather fragmented and less coordinated efforts and initiatives were undertaken.

The chapter discusses the introduction of standardized tests as they move education system from equity of access to quality of instruction and learning. The aim of the chapter is to analyze the role of international education projects like PISA, TIMSS, and PIRLS in shaping national education policies and in helping them to tackle such issues as limited access to education, corruption, illegal practices, quality manipulations in academia, and achievement gaps. The main method used is cross-national comparative analysis. The theoretical scope of the chapter covers major scholarly works on international testing, achievement gaps, and corruption in education. The research finds that the arrows of influence operate in both directions, implying that while setting global standards, international projects base their judgments on identified local challenges in education systems of individual countries. Besides, internationalization of standards has spillover effects on curbing corruption and illegal actions that often cause widening of achievement gaps. The findings of the research could be used in designing education policies both on national and international levels to make education systems more transparent and comparable to international standards. The chapter sets forth a novel idea that the international projects like PISA, TIMSS, and PIRLS could serve not only as means of setting and checking quality standards in education, but as mediators in closing achievement gaps and even curbing corruption. This novelty presents a value to the local education policymakers, and more importantly, to the public.

This chapter undertakes a comparative analysis of discourses on equity found in OECD and Norwegian policy documents. This is an interesting area to study as the OECD is found to be an important agenda setter for many countries' educational policies. A comparative analysis of OECD and Norwegian educational policies is especially interesting because the OECD is often found to be pressing for a neo-liberal agenda, while Norway has a socialist-alliance government. Combining Basil Bernstein's theoretical framework with key principles from Critical Discourse Analysis, the author investigates power relations within OECD and Norwegian educational policy documents. Two equity models serve as analytical tools: equity through equality and equity through diversity, which can be described along the three dimensions: de-/centralization, de-/standardization and de-/specialization. Using the analysis of two key documents on equity in education from the OECD and Norway, the author points out the similarities and differences in two documents. Both the OECD and Norwegian approaches to equity in education can be related to a centralized decentralization or a conservative modernization of education. However, there are also important differences between the two documents. For example, the Norwegian ministry has more emphasis on equity through equality and is less influenced by neo-liberalism and authoritarian populism than the OECD. In conclusion, the author argues that neither of the two described approaches appears to improve the inequities in education. A different way of targeting these inequities could be based on critical theory and research.

This chapter discusses the influence of international educational studies on knowledge in a general sense. In a theoretical framework, a split between realistic and constructivist theories of knowledge with special regards to global and local knowledge is discussed.

Since Slovenia is a country that is included in a number of different international comparative educational studies and assessments, even more so, it has been participating in these studies continually for the last two decades, the focus is on Slovenian educational policymaking (PM). The chapter for the first time analyzes the impacts on national PM of different international studies and predicts the future Slovenian participation in these studies; therefore, the chapter could be interesting for national and international audiences involved in comparative education research. For the estimation of existing impacts on national PM, semi-structured interviews were used. The findings suggest that international results represent an argumentation for some directly and indirectly curricular and syllabus changes over the years. Furthermore, some of the argumentations for changing the national educational system regarding the international findings are still more declarative; irrespective of experts and policymakers estimations of how great impacts these studies have in Slovenia. Future research on the topic for Slovenian PM in education will need more secondary analysis of collected data from both national and international assessments.

Finland's performance in PISA has created considerable interest in the country's education system, to ascertain what has made Finland so successful in the survey. In reference to the phenomenon, this chapter discusses cross-national attraction, policy borrowing, the effect of Finland in PISA, and its influence on education policy. This chapter explores at length the theoretical background of cross-national attraction and policy borrowing, also investigating cases that have already occurred. It discusses Finland's role as the new object of cross-national attraction and eventual policy borrowing. The chapter incorporates research into the reasons for Finland's success in PISA, the possibilities of policy transfer from Finland, and delves into the likelihood of policy implications as a result of Finland in PISA. This cross-national attraction denotes the first stage in policy borrowing; however, comparative educationalists, for years, have warned about the uncritical transfer of education policy. Research in Finland has revealed many reasons for the country's PISA success stem from contextual factors: those related to historical, cultural, societal, and political features of Finland. Therefore, policy borrowing from Finland needs to heed warnings of past comparativists. The new phenomenon of Finland in PISA has generated much curiosity from those in education, educational policy, and politics. Policymakers are keen to incorporate Finland's educational features into their education systems. PISA and Finland's performance in the survey influence educational policy. This illustrates the importance the warnings of past and present comparative educationalists in order to prevent uncritical policy borrowing.

There has been a notable growth in the number, participants, and frequency of international assessments of student academic performance over the past 50 years. This chapter provides a structure for the perspectives that could be used to analyze this rise. This chapter highlights case study examples of specific countries' choices to participate in particular assessments. It further describes the utility of three analytic frameworks in understanding the decision factors, diffusion mechanisms, and environmental dynamics that relate to international testing. Factors such as the cost of testing, the cultural connections between nations participating, and the temporal relevance of testing to today's focus on accountability arise in illustrations of the transmission mechanism for international achievement tests. This chapter organizes large and diverse amounts of important testing sampling frame information in a unique way. The questions we ask are driven by the framework we begin analyzing with. Organizations conducting these tests can better understand the touchpoints for nations deciding whether or not to participate. Concerns about developing country participation, for example, can be better addressed.

This chapter discusses some of the criticisms of standardized assessments by doing a document analysis of mainly Mexico's and Argentina's ministries of education's web sites and exploring the theoretical work of diverse authors, mainly critical pedagogues and culturalists. This chapter argues that the process of assessment using standardized tests is a highly political and even commercial process, but the challenge to compete globally, still perform locally, collaborate in solidarity, and decide collectively whose knowledge is of most worth is still before us. As exemplified in Mexico's test ENLACE, standardized tests tend to show a negative bias against minorities and tendency to highlight certain values and knowledge. Countries should seek for as many partnership opportunities with teachers and communities to be able to assess learning collectively and even consider not adopting policies passively, as opposed to having an international organization or policy dictating what is worth knowing and testing. This way, assessment will still help countries compete globally, still perform locally, and collaborate in solidarity.