1TIMSS (Trends in International Mathematics and Science Study) and PIRLS (Progress in International Reading Literacy Study) are core international assessments of the International Association for the Evaluation of Educational Achievement (IEA). As such, they regularly provide participating countries with reliable data on students’ academic achievement in the fundamental subjects of mathematics, science, and reading together with a rich array of information about the country contexts for teaching and learning in these curriculum areas. Nearly 70 countries, representing a range of developed and developing economies and various different educational systems, participate in TIMSS, PIRLS, or both. As might be anticipated, these countries use the assessment results in a wide variety of ways.
2This article describes the ways countries, including the various stakeholders in education such as policy makers, teacher educators, and researchers are using TIMSS and PIRLS materials, procedures, and results. Many countries report that TIMSS and PIRLS are important for their role in spurring educational reform and monitoring the effects of changes in their educational systems. Because TIMSS and PIRLS are designed to provide detailed, high quality data on student achievement together with information on crucial aspects of the educational context (e.g., the curriculum, school environments, and instructional practices), the assessment results often provide an impetus for educational improvement. For example, in his Keynote speech opening IEA’s 3rd International Research Conference in Taipei, Fou-Lai Lin, former head of Taiwan’s National Science Council, made the following observation: “The results of IEA studies are policy initiators. The identified phenomenon is like a starter of an engine for education reform policy” (Lin, 2008).
3Professor Lin continued to explain that, although Taiwan routinely is a top-performing country in TIMSS assessments, between TIMSS 1999 and TIMSS 2003 Taiwan had an increased percentage of low-achieving students in mathematics. These results prompted the Ministry of Education to set specific improvement goals for TIMSS 2011 and launch an ambitious program to tutor these students. The first goal was to reduce the group of low-achievers from 15 to 10 percent, and the second was to increase the percentage of students liking mathematics and science to 60 percent. To help reach these goals, in 2006 the Ministry of Education established a program of after-school support in Mandarin, English, mathematics, science, and social studies. Cadres of tutors, often substitute teachers, are responsible for working with low-performing students on a one-to-one basis to ensure completion of homework at a high level and on time.
4TIMSS and PIRLS are projects of the IEA, which is an independent international cooperative of national research institutions and government agencies. IEA pioneered the idea of conducting studies of cross-national achievement as a way for countries to learn about effective educational approaches from each other and has been conducting international comparative studies in a wide range of subjects since 1959. TIMSS and PIRLS are directed by the TIMSS & PIRLS International Study Center at Boston College.
5Briefly, TIMSS measures trends in mathematics and science achievement at the fourth and eighth grades. It has been conducted on a regular 4-year cycle since 1995, so that TIMSS 2011 provides the fifth assessment of trends, and plans are underway for TIMSS 2015. TIMSS Advanced, which measures trends in advanced mathematics and physics for students in their final year of secondary school, was conducted in 1995 and 2008, and is scheduled for 2015 (with TIMSS). PIRLS measures trends in reading comprehension at the fourth grade and has been on a regular 5-year cycle since first assessed in 2001. Most recently, PIRLS was expanded in 2011 to include prePIRLS, which is a less difficult version of PIRLS. Plans are underway to assess both PIRLS and prePIRLS in 2016. Importantly, the cycles of both TIMSS and PIRLS came into alignment in 2011; more than 30 countries took this opportunity to assess the same fourth grade students in all three curricular areas (reading, mathematics, and science). Since PIRLS has a parent questionnaire about students’ home environment and supports for learning, this information also is available for analyses with TIMSS data. Countries can compare the relative effectiveness of primary schooling across the three subjects and study the relationships among home, school, and classroom variables, controlling for extraneous factors.
6Because TIMSS and PIRLS are trend studies monitoring changes in achievement at regular intervals, the assessments are effective vehicles for examining whether increases or decreases in achievement occur after implementing new or updated educational policies. Once a phenomenon or issue has been identified and steps have been taken to remedy the situation, subsequent assessments can be used to help determine whether particular reform efforts are having an impact.
7This article provides a number of examples of how countries participating in TIMSS and PIRLS use the assessment results to initiate and monitor educational change, depending on their priorities. Primarily, countries use the assessment results for system-wide monitoring in an international context. In addition, because TIMSS and PIRLS emphasize a rigorous curriculum and strong curricular implementation as keys to high achievement, many countries use the results for improving curriculum and instruction in specific content areas (e.g., biology) or topic areas (e.g., fractions). Also, for many countries, participation in TIMSS and PIRLS plays a central role in building capacity in assessment methods. Because TIMSS and PIRLS are organized to encourage participation by subject matter specialists from the participating countries at the various stages of implementation (framework review, item development, data analysis, etc.), these specialists begin incorporating the best international practices into their work and often become national policy advisors as well as trainers of other educators. Finally, TIMSS and PIRLS are used for a number of other purposes, including research into teaching and learning, conducting additional related studies, training researchers and teachers in assessment and evaluation, and improving national assessments.
8This article summarizes information gathered via several avenues, primarily the TIMSS 2011 Encyclopedia (Mullis, Martin, Arora, Stanco, Centurino & Minnich, 2012) and the PIRLS 2011 Encyclopedia (Mullis, Martin, Drucker & Minnich, 2012). Because one of IEA’s central missions is enabling policy-makers and researchers to learn from their colleagues around the world, TIMSS and PIRLS routinely prepare Encyclopedias in which participating countries each provide a chapter describing the structure of its education systems, policies, and curriculum. The Encyclopedias are an important resource for helping to understand the context for teaching and learning around the world. The encyclopedia chapters are written by experts from ministries of education, research institutes, or institutes of higher education who have extensive knowledge about their country’s education system. As such, they are well positioned to write authoritatively about how TIMSS and PIRLS are being used in their countries. However, because the authors often include the individuals responsible for implementing TIMSS and PIRLS in their countries, the chapters understandably focus on the positive aspects of participation.
9The article also relies on information disseminated by the countries at various IEA meetings, including the annual meetings of IEA’s General Assembly and IEA’s biennial International Research Conferences. Countries regularly describe the dissemination of IEA study results and the impact of these results on educational reforms at these meetings. Altogether, the article provides examples from 24 countries that have participated in several iterations of TIMSS and/or PIRLS : Austria, Bulgaria, Botswana, Chinese Taipei, Czech Republic, Finland, France, Germany, Hong Kong, Hungary, Iran, Japan, Korea, Norway, New Zealand, Oman, the Russian Federation, Singapore, Slovenia, Spain, South Africa, Sweden, Syria, and the United States.
10TIMSS and PIRLS are based on assessment frameworks developed collaboratively with the participating countries. The frameworks specify in some detail the knowledge, skills, and understandings to be assessed. In mathematics, TIMSS 2011 assesses three content domains at fourth grade-number, geometric shapes and measures, and data display-and four content domains at eighth grade-number, geometry, algebra, and data and chance (Mullis, Martin, Ruddock, O’Sullivan & Preuschoff, 2009). TIMSS also assesses three content domains in science at fourth grade-life science, physical science, and earth science-and four content domains at eighth grade-biology, chemistry, physics, and earth science. The cognitive domains that TIMSS assesses are knowing, applying, and reasoning for mathematics and science at both grades. PIRLS 2011 assesses a range of four reading comprehension processes-retrieving, inferencing, interpreting/integrating, and examining/evaluating-within two major reading purposes-literary experience and obtaining information (Mullis, Martin, Kennedy, Trong & Sainsbury, 2009). To measure achievement as specified by the comprehensive frameworks, TIMSS includes nearly 800 items (approximately 200 per subject at each grade). PIRLS includes 10 reading passages and approximately 130 items. In all assessments, about half of the items are multiple-choice and half require students to write their own answers.
11The primary approach to reporting the TIMSS and PIRLS achievement data is based on item response theory (IRT) scaling methods. The PIRLS reading achievement scale and the TIMSS mathematics and science achievement scales are used to summarize overall results. To enable countries to interpret students’ competencies at different scale levels, a scale anchoring process is used to describe achievement in terms of students’ demonstrated knowledge, skills, and thinking at four International Benchmarks on the scales-Advanced, High, Intermediate, and Low. In addition, separate achievement scales are produced for each content and cognitive domain for TIMSS, and process and purpose for PIRLS. Thus, for each curriculum area, countries receive data about achievement and trends in achievement summarized for the subject overall, at the International Benchmarks, and for specific areas of interest (e.g., algebra, biology, etc.). Studying the item-by-item results in international and national contexts can provide detailed insights. Also, combining item-by-item results into topic areas enables evaluation of achievement in these topic areas across countries and for relative strengths and weaknesses within countries.
12In addition to data on student achievement, a major purpose of IEA studies is to support educational improvement efforts by identifying promising instructional practices from around the world. TIMSS and PIRLS use the curriculum, broadly defined, as the major organizing concept in considering how educational opportunities are provided to students. IEA’s curriculum model has three aspects: the intended curriculum officially specified by countries, the curriculum actually taught, and what students have learned. The countries summarize their official curricula in the TIMSS and PIRLS Encyclopedias, teachers report on the topics they have taught the assessed students, and the results on the TIMSS and PIRLS tests provide an indication of what students have learned.
13Besides collecting extensive data about curriculum and curriculum implementation, TIMSS and PIRLS have contextual frameworks to guide questionnaire development (similar to the subject area assessment frameworks). For example, in accordance with both the TIMSS and PIRLS contextual frameworks, the students that are assessed provide information about their home and classroom experiences, and their teachers and school principals provide information about instructional practices, school resources, and school climate for learning. In addition, the PIRLS contextual framework specifies that students’ parents provide information about home resources and students’ early learning experiences.
14The context questionnaire data are reported in relation to the achievement data. Countries learn about the educational systems and practices used in the high performing countries and gain information about the relationship between educational practices and achievement in their own countries. The data are published in a series of International Reports and made available to the public for further research and use. After each assessment, the TIMSS & PIRLS International Study Center prepares a well-documented international database containing the achievement and questionnaire data from each country, and makes it available on the TIMSS & PIRLS International Study website (timss.bc.edu or pirls.bc.edu).
15TIMSS and PIRLS expend enormous energy to ensure reliability, validity, and comparability of data through careful planning and documentation, cooperation among participating countries, standardized procedures, and rigorous attention to quality control throughout. In carrying out the assessments, the TIMSS & PIRLS International Study Center works closely with the IEA Secretariat in Amsterdam, the IEA Data Processing and Research Center in Hamburg, Statistics Canada in Ottawa, and Educational Testing Service in Princeton, New Jersey. After each assessment, the Technical Report provides technical documentation about the development and implementation of the study. For 2011, the TIMSS and PIRLS technical documentation is currently accessible via the “Methods and Procedures” section of the website.
16At the minimum, countries take note of their international standing and levels of achievement to gauge their overall educational health in a global context. Typically, once the media disseminates the results, some degree of discussion occurs among policy makers and the public, often followed by some form of action. As illustrated by the following examples, a number of countries, especially those with trend data across multiple assessment cycles, rely on international assessment results as an integral part of the country’s educational improvement strategy. In some cases, TIMSS and PIRLS countries also participate in other ongoing IEA studies (such as civics and computer literacy) and OECD’s international assessment of 15-year-olds (PISA).
17Since 2000, participation in international studies has been an important part of the Norwegian strategy for quality improvement in schools, according to IEA’s Norwegian representative from the Norwegian Directorate for Education and Training, (Kavli, 2008). This has led to the following: a stronger focus on learning outcomes ; emphasis on basic skills in reading, mathematics, and science ; increased education for teachers ; and the National Quality Assessment System. Norway has participated in all PIRLS and TIMSS assessments, including TIMSS Advanced. TIMSS and PIRLS results, as well as those from other IEA and OECD studies, are used to evaluate the skills and competence of Norwegian students in relation to other countries, to develop benchmarks, and to set national policy. For example, the PIRLS 2006 results led to an earlier start for reading instruction, early interventions for weak learners, and continued emphasis on reading through the entire primary level.
18As explained in The Development of National Educational Standards: An Expertise (Klieme et al., 2004), a report prepared jointly by the Ministry of Education and Research and the German Institute for International Educational Research, the quality of the German education system became the subject of increasing concern after the publication of the TIMSS 1995 results and the subsequent PISA 2000 results. The report presented a plan for educational improvement based on three components : educational goals, competency models, and assessments and led to the German educational monitoring strategy adopted in 2006. The strategy includes compulsory participation through 2016 in TIMSS and PIRLS at the primary education level and PISA at the secondary level. It also includes state and school comparisons based on the national standards, all culminating in joint state-federal reporting of educational progress.
19Slovenia has engaged in large-scale reforms during the past 10 years (e.g., transitioning from an 8- to 9-year elementary school system to improve students’ literacy skills) and the Ministry supports and widely uses the TIMSS and PIRLS results to monitor the impact of the reforms in a global context (Pavešić & Svelik, 2012 ; Doupana-Horvat, 2012). Most recently (spring of 2011) the Ministry of Education and Sport released a White Book of Education (Bela Knjiga) presenting new national goals for education, including Slovenia’s placement among the first third of countries in TIMSS and PIRLS. Also, based on decreased achievement on TIMSS Advanced 2008, the White Book argues that advanced students do not get the opportunity to learn the most demanding mathematics because the courses are required for all pre-university students and have become too general. The White Book thus proposes two difficulty levels in gymnasia (Grades 10–13 pre-university students), with the current level being the basic level and a new higher level for more advanced students. Additional hours of mathematics instruction will be provided for students who have difficulties reaching the standards. Also, based on TIMSS results, the White Book proposes curriculum reform in the lower levels of elementary school mathematics. Positive results are used in policy making, too, in support of current practice. Slovenia’s high achievement in TIMSS science assessments has reinforced educators’ position science continue to be taught as separate subjects from Grades 8 to 13. Science teachers and schools feel encouraged to keep working hard because the results indicate that the curriculum is well-organized. It is expected that in Slovenia’s future TIMSS will remain significant by providing international comparisons as well as opportunities to link international assessments to national studies.
20A key feature of New Zealand’s system-level evaluation framework is participation in the TIMSS, PIRLS, and PISA international assessments (Chamberlain, 2012). The information is used to monitor students’ achievement in the international context and alongside other national data to assist in decision-making for policy and practice. In particular, TIMSS and PIRLS are used as indicators of current and past achievement, and to set aspirational goals for future achievement (Caygill, 2012). Goals for the New Zealand educational system are set out in the Ministry of Education Statement of Intent, and one of the six priorities in the most recent Statement is that “every child achieves literacy and numeracy levels that enable their success.” TIMSS and PIRLS results will help measure the nation’s success at achieving this goal. More specifically, the Ministry of Education’s Iterative Best Evidence Synthesis (BES) program has also used TIMSS and PIRLS to improve educational outcomes. BES evaluates and synthesizes New Zealand and international research to inform educational policy, teacher practice, and suggest educational development approaches. Prior to TIMSS 2007, much discussion focused on mathematics achievement, but TIMSS results together with results from the National Education Monitoring Project brought attention to science education. The Royal Society of New Zealand, the New Zealand Association of Primary Science Educators, and the Education Review Office (ERO) have taken action to address concerns, with ERO writing a report to exemplify good practices in primary science teaching. The quality of science teaching in upper primary schools became a focus area for 2011.
21The Russian Federation has participated in a number of IEA studies over the last 20 years and considers participation in TIMSS and PIRLS important in evaluating the quality of education in the country (Kovaleva, 2011). Russian Academy of Education member Dr. Kovaleva explained that, although Russia has been involved in TIMSS, TIMSS Advanced, and PIRLS since their inception, data use has intensified in the past five years, spanning policy areas as well as academic research. For example, the TIMSS and PIRLS data have played a role in capacity building and developing educational standards. Specialists involved in standards development analyzed the TIMSS and PIRLS frameworks and on this basis developed recommendations for new state education primary school standards, introduced in 2011. Since 2005, TIMSS and PIRLS participation has been included in the Federal Program of Education Development adopted by the State Duma and financed through the country budget (Kovaleva, personal communication, August 15, 2011).
22Swedish trend data from IEA studies have prompted concern about changes in educational policy (Gustafsson & Rosen, personal communication, August 8, 2011). In the early 1990s, Sweden transformed its highly centralized educational system into one of the most decentralized and deregulated educational systems in the world. A major change was that municipalities were to employ teachers and school personnel, but even companies could run schools. Also, the curriculum became much less specific about content and methods. During this time, Sweden’s TIMSS 1995 results showed high levels of achievement, but the 2001 reading assessments showed mixed results. In PIRLS 2001, the Swedish fourth grade students had the highest participation of all students, but the 10-Year Reading Literacy Trend Study (1991-2001) showed declines at the third grade from among the top-performing countries in 1991 to average levels in 2001 (Martin, Mullis, Gonzalez, & Kennedy, 2003). TIMSS 2003 showed dramatic declines for mathematics and science, and PIRLS 2006 and TIMSS 2007 showed further declines in reading, mathematics, and science. Performance on TIMSS Advanced plummeted between 1995 and 2008. The pattern of results from PISA 2003, 2006, and 2009 supported the pattern of declining achievement found in the IEA studies. In attempts to change the negative achievement trends, since 2006 the Swedish government has emphasized the follow-up of individual students, national tests in more subjects and grades, new curricula, implementation of a large school inspectorate, and new teacher education. However, TIMSS and PIRLS 2011 may be too soon to conclusively evaluate the impact of these policy actions.
23The South African Ministry of Education output goals (Government Gazette, August 2, 2010) include performance on TIMSS as a way of determining whether reform interventions have been successful (Reddy & Rogers, 2012). TIMSS data inform the Department of Education about areas of weakness deserving attention, and TIMSS results are used as a benchmark to measure school effectiveness. TIMSS performance in 1999 and 2003 informed a number of reform initiatives throughout the system and served to direct resources for science and mathematics education. For example, after TIMSS 1999, the Dinaledi initiative focused on improving conditions in 500 schools around the country to increase quality outputs from the schooling system.
24As described by IEA’s representative from the French Ministry of Education, PIRLS results have contributed to a renewed emphasis on fundamental competencies in reading and mathematics in France, and to providing stronger support for pupils with trouble in reading and mathematics (Sauvageot, 2008). However, at the same time, there has been a decrease in the number of annual hours of primary education due to the elimination of Saturday morning courses.
25Spain has launched a national goal to promote reading and the use of libraries in response to results from PIRLS, PISA reading literacy, and national reading assessments (Núñez-Arenas, 2010). The Ministry of Education has directed that some time be dedicated to reading in the teaching of all subjects. Project Leer promotes reading among children and offers activities and resources for teachers of all subjects and grades, including reading in traditional and digital formats. School libraries have a major role in the Reading Promotion Plan to help students acquire and develop strong reading habits.
26Botswana’s TIMSS 2007 report invites “all actors, partners and stakeholders in education and training to move into an action mode” (Botswana Examinations Council, 2007). Botswana started participating in TIMSS in 2003, motivated by the national aspiration for a standard of education that is internationally competitive. In part because of TIMSS 2003 results, the Ministry of Education and Skills Development implemented a program known as SMASSE (Strengthening of Mathematics and Science in Secondary Education) in 2006 to strengthen the quality of teaching and learning of mathematics and science in secondary schools (Kesamang, Mudongo, Keatimilwe & Bolaane, 2012).
27As reported by representatives of Korea’s Institute for Curriculum and Evaluation in their TIMSS 2011 Encyclopedia chapter, Korea continuously benchmarks and references TIMSS results and analyses for educational policymaking at the national level. However, as a top-performing country there have only been moderate reforms (Cho, Kim, Kim & Rim, 2012). Policymakers in the United States consider students’ comparatively low achievement on TIMSS and PISA grounds for improving education in the areas of science, technology, engineering, and mathematics (STEM). A recent report to the U.S. Congress used the results as justification for promoting further investments in STEM educational programs (Kuenzi, 2008).
28Hong Kong has a complex language policy that supports bi-literacy in Cantonese and English as well as tri-literacy in Cantonese, Putonghua, and English. Beginning in 1994 there were serious efforts to improve language education guided by the Standing Committee for Language Education and Research (Leung, 2008). A language education fund of HK$ 300 million (at the time equivalent to US$ 38.5 million) was established and later increased to HK$ 500 million (US$ 64 million). Reading goals were established in 2000 emphasizing comprehension for learning and reading for pleasure. A “whole school” approach was adopted to build a culture of reading across all subjects, and Hong Kong looked forward to a good result on PIRLS 2001. However, in contrast to its top-performance in mathematics and science in TIMSS 1995 and 1999, the PIRLS 2001 reading results were closer to average, internationally. The government and Legislative Council discussed the disappointing findings, which also were widely reported in the press. Subsequently, the PIRLS results were widely disseminated to schools together with information on concepts and theories of reading through an ambitious series of talks and workshops, and schools began incorporating PIRLS reading skills into their curriculum. Also talks and workshops were given to more than 5,000 parents on how to create an effective home reading environment. In 2004, the Territory-wide School Assessment adopted the PIRLS framework for the Chinese reading comprehension examination. In PIRLS 2006, Hong Kong was a top-performing country and confident that reforms in language education were moving in the right direction.
29Many countries, including industrialized countries as well as countries with developing economies, have used the TIMSS and PIRLS assessments for curriculum reform. The TIMSS and PIRLS attention to curriculum coverage, implementation, and delivery make the results useful powerful vehicle for comparing a given country’s curriculum to that of the high-performing countries. Also, because the assessment items are carefully aligned with curriculum topics, item-by-item analysis of the results, together with teachers’ reports about whether those topics were taught, can be very effective in identifying areas of strengths and weakness not only in achievement, but in curriculum coverage both in national guidelines/standards and in instructional emphasis.
30In Japan, IEA studies have made important contributions to Japanese education, especially for curriculum reform, as reported by the National Institute for Educational Policy Research (Watanabe, 2009). The courses of study have been revised seven times since their introduction in 1947 to keep up with societal changes (Senuma & Saruta, 2008). Influenced by TIMSS and PISA results, the seventh curricular revision implemented in 2011 places more emphasis on mathematics and science education in elementary and lower-secondary schools. Both the amount of content and number of class periods spent on mathematics and science have been increased, because although Japan is one of the top-level countries in the world in mathematics and science, it is not at the very top. Also, students have comparatively less positive attitudes towards mathematics and science.
31Researchers in the Czech Republic conducted an item-by-item analysis of the TIMSS 2007 data to identify students’ strengths and weaknesses in mathematics and science (Martinec, 2010). In particular, they found that Czech students were weak in fractions and decimals. In the short term, a series of teacher manuals was developed that contain activities and tasks covering the Czech students’ most common misconceptions and errors. In the long term, there will be a revision of the national curriculum.
32TIMSS 2003 and 2007 findings were used in evaluating Slovenia’s mathematics curriculum developed in 1998 (Pavešić & Svelik, 2012). A new, improved curriculum was implemented in 2011 with many of the changes justified by TIMSS results, including beginning the study of algebra earlier in Grade 6 and, once again, emphasizing abstract thinking.
33In the United States, the Common Core State Standards Initiative is a state-led effort to develop clear and consistent frameworks to guide teaching practices across states and prepare children for college and the workforce. The mathematics standards define the knowledge and skills students should have within their K-12 education careers so that they will graduate from high school able to succeed in entry‑level college courses and in workforce training programs. Experts who guided the development of the Common Core State Standards relied on TIMSS results, especially focusing on the curriculum of high-performing countries such Singapore and have prepared guidelines for states to benchmark their curricula to Singapore’s (http://www.achieve.org).
34In Finland, the results of TIMSS 1999 were released when development of the new core curriculum for basic education was just beginning (Kupari, 2012). Several published articles and reports explored the strengths and weaknesses of the Finnish education system in light of the TIMSS results, and TIMSS mathematics and science researchers actively participated in national meetings and symposia organized by the Finnish National Board of Education.
35According to the Institute for Education Research in Iran’s Ministry of Education, there have been a number of TIMSS-related changes in the curriculum and textbooks (Karimi & Bakshalizadeh, 2012a). After TIMSS 1995 and 1999, the Earth science curriculum was updated, and following TIMSS 2003, mathematics curriculum planners added more emphasis on fractions and visual thinking in primary school and more emphasis on algebra and statistics in the middle school. Also, the teachers’ editions of textbooks now include the TIMSS cognitive levels (knowing, applying, and reasoning). PIRLS resulted in adding informational texts to the primary school language textbooks, and in 2003 the teachers’ editions of these textbooks incorporated the PIRLS Framework (Karimi & Bakshalizadeh, 2012b). Currently, Iran is reviewing its mathematics and science curricula for both primary and middle school, and the TIMSS Frameworks are a main resource.
36After the Sultanate of Oman’s participation in TIMSS 2007, it formed two teams-one for mathematics and one for science-of curriculum officers, regional supervisors, and assessment officers to submit recommendations to the Ministry of Education (Al Maskari, Noorani & Al Kharousi, 2012). The direct impact was apparent in curriculum and assessment, where the scope and sequence for Grades 1 through 10 of both the mathematics and science curricula were completely revised, taking the topics covered by TIMSS into consideration. Some learning outcomes were rearranged among the grades, and new outcomes were introduced at some grades to align with international scope and sequence as well as local experience.
37In Botswana, the Curriculum & Evaluation Department reviewed the curricula for upper primary and junior secondary school in 2006–2007 using the TIMSS mathematics and science frameworks (Kesamang, Mudongo, Keatimilwe & Bolaane, 2012). Because TIMSS 2003 and 2007 showed learners were not able to apply higher order thinking skills such as critical thinking and problem solving, the review of the primary and secondary education curricula placed emphasis on the acquisition of these skills across all subject areas, not just mathematics and science. The Botswana Examinations Council also enhanced assessment objectives to include higher order thinking skills in 2008 to target the primary school leaving examination and junior certificate examinations that assessed the revised curriculum.
38The TIMSS 2011 Encyclopedia chapter provided by the Syrian Arab Republic’s Ministry of Education reports that participation in several TIMSS assessments has improved methods of teaching, learning, and testing (Abouawn, 2012). Syria’s TIMSS experience has had a positive influence on educational policies and reforms, and continues to inspire further development. For example, the new curricula, in general, and the mathematics, physics, and chemistry curricula, in particular, are based on current teaching and learning methods and an improved approach to testing includes assessing higher-order cognitive skills instead of only questions dependent on memorization.
39Although curriculum reform, teacher education initiatives, and preparation or revision of instructional materials and textbooks can occur separately, these activities are often closely related because it is important to provide teachers with the information and instructional strategies necessary to address the requirements of the new curriculum. In some countries, however, disseminating information about TIMSS and PIRLS can be the primary vehicle for teacher education and updating the classroom curriculum.
40In New Zealand, TIMSS and PIRLS have led to teacher education initiatives in mathematics, science, and reading. The TIMSS 1995 results, in combination with teachers’ difficulties in implementing a new curriculum, led to the Numeracy Development Project, a national teacher professional development program (Chamberlain, 2012). As a first step, in 1997 the Minister of Education established the Mathematics and Science Taskforce recommending, among other things, the development of a comprehensive array of new and revised resource material for students and teachers. Then in 1998, because the number content domain was identified as a particular weakness at the primary level, work began on developing a comprehensive strategy and numeracy policy leading to the Numeracy Development Project. The national evaluation of science achievement after TIMSS 2007, in combination with national assessment results, also has resulted in further teacher education and professional development related to science in primary schools. In addition, New Zealand began a national discussion of the PIRLS 2006 results via a national symposium, followed by a series of teacher focus groups in regions throughout the country. Teachers learned about the findings, studied released passages and items, and considered literacy experts’ priorities for literacy learning in light of the PIRLS results.
41Singapore’s Ministry of Education uses TIMSS and PIRLS data to identify specific strengths and weaknesses in various domains of learning for different groups of students (Poon, Personal communication, June 30, 2011). The item-by-item results are especially useful in identifying common misconceptions, which the Ministry of Education communicates to the Heads of Science and Mathematics Departments. These head teachers then work together to develop teaching approaches that can improve student learning.
42Norway analyzed the TIMSS trend data from primary, middle, and upper secondary school to develop a comprehensive picture of mathematics and science education (Gronmo, 2012). The Ministry Directorate in Norway supported regional seminars around the country to disseminate the results to teachers, school administrators, and school researchers from all levels in the school system. It was the first time schools across the country had experienced such seminars, and they were very well received. In general, the TIMSS results provided a better understanding of how one level of education influences the next. Also, the results showed that, compared to other countries, Norwegian students have more individual and fewer class activities, less emphasis on discussion and argumentation, less emphasis on automation of skills, and more emphasis on using calculators. In addition, the teachers’ role as the person in charge of learning was emphasized.
43As a result of TIMSS and PIRLS findings, the Ministries of Education in the German Länder have increased their focus on teacher education and professional development, according to the Co-Director of IEA’s Data Processing and Research Center in Hamburg (Sibberns, personal communication, August 19, 2011). In particular, teachers and prospective teachers are receiving additional training in standards-based instruction, classroom management, and strategies for addressing diversity in classrooms and meeting the needs of students with different ethnic backgrounds.
44Iran conducts in-service training sessions for teachers in order to introduce them to the goals and objectives in the TIMSS and PIRLS Frameworks (Karimi & Bakshalizadeh, 2012a, 2012b). Iran also has published and disseminated the released items from TIMSS at fourth grade, eighth grade, and TIMSS Advanced.
45Following a national report of PIRLS 2006, a consensus was reached in Chinese Taipei that no true reading instruction was being provided in classrooms (Ko & Yu, 2012). As a result, the Ministry of Education reading researchers have initiated projects to encourage university professors and primary school teachers to work together to develop reading strategy instruction, and have published the Reading Comprehension Strategies Instructional Manual for teachers’ reference. The regional government has also sponsored many reading-related workshops for teachers, focusing on such topics as the international definitions and criteria of reading literacy. Members of Austria’s BIFIE Institute (the Federal Institute for Education Research) reported that they give presentations and lectures to schools, teacher training colleges, and school supervising authorities, especially about the PIRLS data (Wallner-Paschon & Suchań, 2012).
46In Slovenia, TIMSS items, scoring guides for constructed-response items, and examples of student answers to these items are used regularly in training future teachers and for in-service teacher training sessions (Pavešić & Svelik, 2012). Because international assessment results showed the need for improving students’ reading and writing skills, the Ministry supported a series of in-service teacher programs in all Slovene regions with one aim being to improve the reading literacy needed in science and mathematics subjects. The main teaching materials for these courses were TIMSS science open-ended items and student answers to them.
47A number of countries have developed ambitious research programs to analyze TIMSS and PIRLS databases, with findings disseminated through workshops, speeches, and journals. In addition, many countries have availed of the training opportunities in assessment methods and data analysis provided by IEA’s Data Processing and Research Center to strengthen their research capacity. During the past five years, the Russian Federation has increased the number of presentations and articles published about TIMSS and PIRLS results in educational newspapers and magazines (Kovaleva, 2011). The expanded role of secondary research is exemplified by the Unexpected Victory volume published after Russia’s high achievement on PIRLS 2006, reporting on five secondary analysis projects covering a variety of topics (e.g., school practices and textbooks). Also, a new study was conducted in 2010 at Grades 4, 6, and 9 using PIRLS items, PISA items, and national assessment items to explore contradictory findings between PIRLS 2006 and PISA. As part of the secondary research conducted after TIMSS 2007, the READ Russia small grant program supported Russian specialists working jointly with Tajikistan mathematicians to analyze TIMSS 2007 data and develop recommendations to improve mathematics learning in the primary and basic schools. To facilitate research using the TIMSS and PIRLS databases, specialists from the national center for IEA studies conducted a series of regional seminars to train the regional coordinators and specialists in educational measurement and data analysis. In 2009, two Moscow Universities opened new Masters’ degree programs in educational measurement and evaluation, which include courses on international assessments.
48In Hungary, an ambitious TIMSS 2007 dissemination program included extensive media coverage, widespread distribution of free copies of the report, regional workshops for schools, and a two-day workshop for educational researchers and sociologists to use the international databases and the IEA International Database (IDB) Analyzer program (especially designed by IEA to handle complexities in the TIMSS and PIRLS data) (Brassói, 2009). A series of articles was published in the 2009/1 issue of New Pedagogic Review entitled “The Teaching of Mathematics and Science-TIMSS 2007.” Because eighth-grade performance increased from 1995 to 1999 but declined steadily since, university researchers and policy makers turned to the earlier data to search for reasons for the decrease in achievement. For example, the Ministry researched the TIMSS trend data to identify problems in instruction, which involved teacher associations and led to significantly more attention to pedagogy and updating classroom teaching. With the support of the Ministry and National Education Authority, hundreds of teachers have graduated from two-year university courses in assessment and evaluation and the public education sector is now more receptive to data and evidence-based school improvement.
49The research connected with Bulgarian students’ achievement in PIRLS 2006 has been presented at conferences and symposia and was implemented in the International Project EUCIM (European Core Curriculum for Mainstream Second Language Learning) (Mavrodieva & Angelova, 2012). In addition, Bulgarian participation in PIRLS has led to more than 64 publications, including articles, workshops, and media commentary.
50In Austria, after the initial release and press conferences for PIRLS 2006 and TIMSS 2007, the BIFIE Institute followed-up a year later by publishing a “National Expert Report” and holding a second press conference (Wallner-Paschon & Suchań, 2012 ; Suchań, Grafendorfer & Wallner-Paschon, 2012). To produce the report, BIFIE asked national scientists and experts to analyze the data regarding specific and relevant themes in Austria. In addition, BIFIE conducts research and publishes the results in academic journals.
51In 2005, Taiwan’s National Science Council initiated an annual call for proposals to conduct “Profound Studies” using data from IEA and OECD international student assessments (Lin, 2008). The research projects are expected to study trends and international comparisons in mathematics and science achievement, and are encouraged to examine national competence and to explore related variables to determine how to improve achievement.
52The Slovenian Ministry of Education and Sport supports and requests additional secondary analyses of TIMSS and PIRLS data to explain and study national educational concerns (Pavešić & Svelik, 2012). Ministry and subject matter or curriculum specialists and policy makers from the National Education Institute discuss these findings. The results are regularly disseminated to school principals and teachers through ongoing activities and in-service training.
53Iran is providing research opportunities and topics to be investigated based on TIMSS and PIRLS results to master’s and doctoral students majoring in mathematics education and curriculum planning (Karimi & Bakshalizadeh, 2012a, 2012b).
54South African researchers conduct analyses of the data to develop recommendations for educational reform (Reddy & Rogers, 2012). For example, researchers are able to use the background data to examine the performance of population subgroups and provide information about equity concerns. They also have provided helpful suggestions for improving negative perceptions about education and encouraging parents to be more supportive of their children and to ensure that further learning occurs in the home.
55To support the German effort to reach the new educational goals, the Federal Ministry of Education funded a large program of empirical educational research to identify ways to improve educational achievement (Sibberns, personal communication, August 19, 2011).
56In 2008, following public awareness that Swedish achievement was falling dramatically, the National Agency for Education initiated a review of literature and secondary analysis on factors influencing student achievement (Gustafsson & Rosen, personal communication, August 8, 2011). The resulting 2009 report included chapters on the effects on achievement (including levels and equity) of different categories of factors such as resources and societal influences (National Agency for Education, 2009). The report generated considerable interest and discussion.
57An additional benefit of TIMSS and PIRLS participation is the opportunity to coordinate a national study with the international data collection effort. For example, Slovenia uses TIMSS and PIRLS to monitor national equity goals (Pavešić & Svelik, 2012). Despite a highly centralized school system with uniform goals for all students, large differences were found among regions in PIRLS 2006, TIMSS 2007, and TIMSS Advanced 2008. Educational resources have been increased substantially in the lowest-performing region, and TIMSS and PIRLS 2011 sample sizes were increased to permit monitoring achievement trends at the regional level.
58Results of this study are expected to receive considerable attention from policy makers and educators. In a similar vein, in New Zealand, PIRLS has contributed to a greater understanding of achievement and inequitable outcomes for two groups of New Zealand’s student populations-Maori and Pasifika students (Chamberlain, 2012).
59As another example of additional data collection for national purposes, some countries assess students at more than one grade. Because Norwegian national policy used to designate the first grade as a compulsory preschool year, Norwegian fourth grade students were among the youngest in TIMSS and PIRLS and had less instruction. Thus, Norway routinely assesses fifth grade students (a smaller sample) as a national option. In 2011, several countries participated in TIMSS or PIRLS/prePIRLS at more than one grade to gather information about improvement from Grade 4 to Grade 6. Also, to obtain comprehensive information about students’ reading achievement at the end of the primary cycle, Colombia participated in both PIRLS and prePIRLS at the fourth grade.
60Some countries use the TIMSS and PIRLS background data in conjunction with the achievement results, and also on their own. Botswana’s TIMSS 2003 and 2007 reports identified similar background variables for ministry intervention, such as availability of resources, teacher effectiveness and absenteeism, student motivation and aspirations, access to preschool, and parental involvement in children’s learning (Kesamang, Mudongo, Keatimilwe, & Bolaane, 2012). Similarly, in Oman, most TIMSS 2007 recommendations to the ministry were related to the variables included in the questionnaires in relation to student achievement (Al Maskari, Noorani, & Al Kharousi, 2012). Interestingly, several countries have used the background data to change educational policy. For example, Hungary reported using information about the demographics of its teachers, students’ attitudes toward mathematics and science (anxiety and confidence), instructional approaches in mathematics and science classrooms, and the amount of yearly instructional time (Brassoi, 2009). New Zealand has used information about school administration and leadership, student safety, and classroom grouping practices (Chamberlain, 2012).
61Several countries also reported that politicians, including those in Austria, South Africa, and New Zealand, have mentioned the results in the media to support particular plans. For example, the Minister of Basic Education in South Africa uses TIMSS and PIRLS results to point to difficulties in the quality of literacy and numeracy. However, New Zealand has observed that information can be misquoted or used selectively.
62Finally, besides the increased assessment emphasis on higher-order thinking skills by Oman and Syria previously mentioned, France, Slovenia, and the Russian Federation have used TIMSS and PIRLS results and procedures to improve the quality of national assessments. In South Africa, TIMSS and PIRLS assessment procedures and methodology are used as a basis for ensuring high quality data collection in other surveys.
63It is clear that countries participate in TIMSS and PIRLS for the primary purposes for which these assessments have been designed. That is, policymakers are using the results for system-level monitoring in a global context to varying degrees, from routinely benchmarking their international standing to implementing comprehensive national monitoring programs that include participation in one or multiple international assessments in concert with national and regional assessments. Many countries are using the data as a basis for establishing achievement goals and standards, and then implementing educational reforms to support meeting the goals and standards.
64A range of developed and developing countries have TIMSS and PIRLS achievement results, Frameworks, and assessment items to implement curricular reform in mathematics, science, and reading. Such reform has encompassed substantial reconfigurations across grades, additional concentration on particular content areas (e.g., algebra, earth science, or informational reading), and concentrating on specific topics such as fractions. A number of countries mentioned increased emphasis on higher-order thinking skills. Particularly in developing countries where there may not have been exposure to broad and rigorous curricula or the resources to develop and implement such curricula, participation in TIMSS and PIRLS provide an impetus for improvement. The assessment frameworks encompass an international view of necessary competencies for success in a global environment and provide a stepping stone for countries to develop their own national curricula.
65The burgeoning use of international assessment results for evidence-based educational policy setting has led to increased interest in analysis of international assessment data. Participation in international assessments such as TIMSS and PIRLS has increased many countries’ capacity for conducting evaluations, improving national assessments, and analyzing data. Researchers around the world are analyzing TIMSS and PIRLS data, and conducting ancillary studies to identify instructional practices, resources, and materials that can improve teaching and learning in mathematics, science, and reading. Internationally, numerous papers, reports, and symposia have served to disseminate TIMSS and PIRLS results to policymakers, educators, and the general public. Researchers in some countries even are working with specialists in nonparticipating countries to help those countries initiate reforms in curriculum and teacher education.
66In conclusion, TIMSS and PIRLS achievement results and background data on educational contexts for teaching and learning are primarily used for monitoring educational achievement and, equally importantly, to implement educational reform especially in the mathematics, science, and reading curricular areas. However, the benefits of participation reach beyond the assessment results to using the Frameworks and assessments items ; developing capacity in assessment procedures ; training researchers and teachers in evaluation, assessment, and data analyses ; facilitating the conduct of national studies as part of the data collection efforts ; and conducting analyses of international assessment data. In any country, educating all of the nation’s children is an extremely challenging and complicated enterprise. Participation in international assessments provides access to a host of ideas and procedures for educational improvement. IEA assessments were founded on the premise that every country can learn from the experience of other countries, and most countries participate to gain valuable insights into successful approaches to teaching and learning. Important and valuable though these contributions are, it is essential to acknowledge the differences between education systems and to recognize that a notion such as 'good practice' is not transferable, in any simplistic way, form one to another. Equally, research exploring the relationships between the outcomes of TIMSS and PIRLS and subsequent changes in curriculum and pedagogy will need to take due account of any other factors that may be involved. Subject to these considerations, TIMSS and PIRLS will continue to make a major contribution to the development of evidence-based policymaking in the field of education and to the raising of standards of achievement.
Flux RSS
