Using the TIMSS and PIRLS 2011 Fourth Grade Combined International Database for Secondary Analysis

International large-scale assessments (LSA) collect an enormous amount of data, which result in datasets containing a wealth of information for secondary analysts. In addition to achievement data from students, they also collect data from teachers, school principals and parents in the hope that this information will assist in our collective understanding of education systems and their improvement. Unfortunately, much of LSA data are under- or poorly utilized by policy makers as well as researchers, in part due to the fact that proper analysis requires the use of analysis methods that complex and generally unfamiliar to users of the data (e.g. plausible values, replicate and sampling weights). The intent of this course is to provide researchers with a set of tools to assist them in unraveling the complexities of one of the recent largest and most important LSA study: the joint Trends in International Mathematics and Science Study (TIMSS) and Progress in International Reading Literacy Study (PIRLS) 2011 conducted with the 4th grade students in 37 countries around the world.

This workshop has three main objectives.

The first one is to provide an overview of the objectives and theoretical frameworks of one of the latest LSAs conducted by the International Association for the Evaluation of Educational Achievement (IEA). This is the joint PIRLS and TIMSS, conducted in 2011 at the 4th grade and in 37 countries around the world, testing the same students in reading literacy, mathematics and science. The first part of the workshop will briefly review the documentation provided in the International Report, Technical Documentation, User Guide, and Theoretical Frameworks.

The second objective of the workshop is to review the studies’ methodological complexities (survey and sample design), implications for analysis and the appropriate analysis methods. An overview of the combined TIMSS and PIRLS 2011 database will be provided. This database will permit researchers to compare the relative effectiveness of schooling across reading literacy, math and science for 4th grade students. Detailed discussion on the complexities of this combined dataset and the value of the analyses using it will take place.

The third objective of this course is to train the participants in analyzing the data from the combined TIMSS and PIRLS 2011 dataset taking into account the statistical complexities and design issues providing hands-on training on analyzing the data using software (provided for free by the course organizers) that handles all issues related with the analysis of large-scale assessment data. The features of the software will be presented, introducing both improvements and new functionalities. Among these new functionalities is the use of plausible values achievement scores as explanatory variables.

Experts in the field of international LSAs will conduct the course and will be on-hand for questions about the study as well as general questions concerning the administration of LSAs and analysis of their data.

This course is intended for graduate students, emerging researchers and researchers at all levels that are interested in issues concerning international LSAs, as well as secondary data analysis in reading, mathematics, and science instruction. Participants are required to bring laptops (Windows PC) with SPSS 15 or higher and have a basic working knowledge in inferential statistics.

The training has several instructional goals. After completing the course, the participants will be able to:

The syllabus/planned workshop activities are as follow:
Hour 1 – Introduction to the combined PIRLS and TIMSS 2011study, questionnaires and data files
Hour 2 – Introduction to statistical complexities of using the combined PIRLS and TIMSS data
Hour 3 – Overview of the IEA IDB Analyzer
Hour 4 – Work through guided practice exercise, analysis, Q & A

Note: While a considerable amount of information will be reviewed in a short period of time, the instructors for this course have adopted a set of successful techniques that they have employed in similar trainings. The last hour will be dedicated to a structured data analysis assignment. During this time participants are encouraged to ask specific questions they may have on the topics covered.

Instructional Staff/Organizers

Plamen Vladkov Mirazchiyski, IEA Data Processing and Research Center, Germany
Andres Sandoval-Hernandez, International Association for the Evaluation of Educational Achievement
Eugenio Gonzalez, Educational Testing Services (ETS)