By Andreas Schleicher
Director, Directorate of Education and Skills
As we are putting the finishing touches on the analysis of the data from the latest PISA round, due to be published on 3 December, people from all over the world have started calling in to find out what the major surprises will be. But there are few surprises in international comparisons like PISA. Quality and equity in education do not come about by surprise; they are always the result of deliberate, carefully designed and systematically implemented policies and practices. Perhaps the most surprising finding from PISA is that, despite their many differences, high-performing schools and education systems share so many features that transcend cultural, national and linguistic borders.
We should ask ourselves: What can we learn from the world’s best-performing school systems? How can their experiences help students, teachers and school leaders in other countries? How can policy makers draw upon lessons from countries facing similar challenges and make better-informed decisions? Even when there are international examples, why has it often proved difficult to learn from them and stop repeating the same mistakes? That’s why it is worthwhile studying education from a global perspective and that’s what PISA is about.
Up to the end of the 1990s, the OECD’s comparisons of education outcomes were mainly based on measures of years of schooling, which are not reliable indicators of what people actually know and can do. PISA changed this. The idea behind PISA lay in testing the knowledge and skills of students directly, through a metric that was internationally agreed upon; linking that with data from students, teachers, schools and systems to understand performance differences; and then harnessing the power of collaboration to act on the data, both by creating shared points of reference and by leveraging peer pressure.
The aim with PISA was not to create another layer of top-down accountability, but to help schools and policy makers shift from looking upward within the education system towards looking outward to the next teacher, the next school, the next country. In essence, PISA counts what counts and makes that information available to educators and policy makers so they can make more informed decisions.
…high-performing schools and education systems share so many features that transcend cultural, national and linguistic borders.
The OECD countries that initiated PISA tried to make PISA different from traditional assessments in other ways too. In a world that rewards individuals increasingly not just for what they know, but for what they can do with what they know, PISA goes beyond assessing whether students can reproduce what they have learned in school. To do well in PISA, students have to be able to extrapolate from what they know, think across the boundaries of subject-matter disciplines, apply their knowledge creatively in novel situations and demonstrate effective learning strategies. If all we do is teach our children what we know, they might remember enough to follow in our footsteps; but if they learn how to learn, they can go anywhere they want.
Some people argued that the PISA tests are unfair, because they may confront students with problems they have not encountered in school. But life is unfair, because the real test in life is not whether we can remember what we learned at school, but whether we will be able to solve problems that we can’t possibly anticipate today.
But the greatest strength of PISA lies in its working methods. Most assessments are centrally planned and then contracted to engineers who build them. That’s how tests are created that are owned by a company – but not by the people who are needed to change education. PISA turned that on its head. The idea of PISA attracted the world’s best thinkers and mobilised hundreds of experts, educators and scientists from the participating countries to build a global assessment. Today, we would call that crowdsourcing; but whatever we call it, it created the ownership that was so critical for its success.
In a nutshell, PISA owes its success to a collaborative effort between the participating education systems, the national and international experts and institutions working within the framework of the PISA Consortium, and the OECD. Subject-matter experts, practitioners and policy makers from the participating countries worked tirelessly to build agreement on which learning outcomes are important to measure and how to measure them best; to design and validate assessment tasks that can reflect those measures adequately and accurately across countries and cultures; and to find ways to compare the results meaningfully and reliably. Our task at the OECD has been to co-ordinate this effort and to work with countries to make sense of the results.
PISA owes its success to a collaborative effort between the participating education systems, the national and international experts and institutions working within the framework of the PISA Consortium, and the OECD.
Since 2000, PISA has shown that education systems can provide both high-quality instruction and equitable learning opportunities for all, and that they can support academic excellence in an environment that also nurtures students’ well-being. At its 18th birthday, PISA has become the world’s premier yardstick for comparing quality, equity and efficiency in learning outcomes across countries, and an influential force for education reform. It has helped policy makers lower the cost of political action by backing difficult decisions with evidence – but it has also raised the political cost of inaction by exposing areas where policy and practice have been unsatisfactory.
But turning 18 is just the beginning of adult life, and PISA needs to assume ever greater responsibility in guiding policy decisions towards education opportunities that are more relevant, of higher quality and more equitable.
Released 3 December, 2019
Some argue that if PISA is to assess progress and change in education, then we cannot change its measures. But if we do not continually develop the PISA measures, we will wind up evaluating students by what was considered important at some point in our past, rather than measuring students against what they will need to thrive in their future. The use of computer-delivered assessments for PISA means that a wider range of knowledge and skills can now be tested. The PISA 2012 assessment of creative problem-solving skills, the PISA 2015 assessment of collaborative problem-solving skills, and the PISA 2018 assessment of global competencies are good examples of this. Some of the findings about social and emotional outcomes belong to the most interesting aspects of this latest PISA round. Further broadening PISA beyond what is easy to measure to what is most important for student success will remain a priority.
But there are other ways in which PISA might look different when it reaches full adulthood. As of today, most tests do not allow students to connect to the Internet, based on the fear that students may look up the answers to the test questions. The challenge for future assessments is whether they can encourage students to go on line to connect with the world’s most advanced knowledge without jeopardising the validity and reliability of results.
Similarly, one of the worst offences in test taking is to consult with another student. But given that innovation is now more often based on sharing knowledge, future tests should not disqualify students for collaborating with other test-takers, but find ways that they can do so. The PISA assessment of collaborative problem-solving skills showed clearly that proficiency in individual problem solving only partially predicts the ability to work with others to solve problems.
Not least, when designing assessments, we often trade gains in validity for gains in efficiency, and relevance for reliability. We do that because it makes results seemingly more objective and thus reduces the risk that they will be contested. Some education ministers have lost their job because of disputes around examination results, few have been challenged for poor validity and relevance in tests and exams that stifle education reform and innovation. We need to turn that around.
And perhaps one day, we can bridge the gap between summative and formative assessments, where assessment becomes such an integral part of the learning process, and big data replace the additional collection of data through surveys. Future tests might provide a window into students’ thinking and understanding, and reveal the strategies a student uses to solve a problem. They may provide productive feedback in real time, at appropriate levels of detail, to fuel improvement decisions, telling students how to develop their learning, and teachers and schools to create better education opportunities. Everyone will then no longer see testing as separate from instruction, taking away valuable time from learning, but rather see it as the key to improved learning. Maybe that’s the time when tests as we know them today will be retired.
Find out more:
