ChatGPT and PISA: Ensuring Our Education Systems Keep Up

By Andreas Schleicher

OECD Education and Skills Directorate

ChatGPT and the likes remind us that many of things that are easiest to teach are now also easy to digitise and automate, and that too often we educate students for our past, rather than for their future. At the OECD, we have been tracking how well systems like ChatGPT fare on tasks from the Programme for International Student Assessment (PISA), the global yardstick of educational success that over 80 countries use to assess the knowledge and skills of 15-year-olds. This comparison shows rapid advances in the capabilities of artificial intelligence when compared with humans. In March 2022, ChatGPT could answer 28% of a set of PISA mathematics tasks, in March 2023 GPT answered 46% of the tasks successfully. In science, the corresponding percentages were 65% and 85%.

So, in a world with artificial intelligence, education is no longer just about teaching students, it is also about helping them to develop a reliable compass and the tools needed to navigate with confidence through an increasingly complex, volatile and uncertain world. Education needs to shift from teaching learners to reproduce the established wisdom of our times to questioning and extending it.

My academic background is in science, and one of the things that makes my heart sink these days when I watch science classes is to see how we teach science like religion. We make students believe in some scientific theory, then we give them lots of exercises to practice that and at the end, we test whether they have learned the right answers. That has so little to do with science, which is about understanding the ways of thinking and knowing of scientists and finding the right questions rather than knowing the answers. When it comes to science education, we need to place much greater weight on the intellectual methodologies of science that have allowed us to progress as humans, rather than just teaching the surface of scientific content. We also need to have people engage with probabilistic thinking which frames so many of the challenges that humans face, whether it is climate change or the evolution of a pandemic. We need to learn to find a signal in the noise, understand relative likelihoods and think in terms of alternative futures.

Even a construct as basic as literacy has fundamentally changed. In the 20th century, literacy was about extracting and processing pre-coded knowledge; in the 21st century, it is about constructing and validating knowledge. In the past, teachers could tell students to look up information in an encyclopaedia, and to rely on that information as accurate and true. Nowadays, Google presents them with millions of answers, and nobody tells them what’s right or wrong, nortrue or untrue. As technology allows us to search and access more knowledge, it is important we have a deep understanding and capacity to navigate ambiguity; to triangulate viewpoints, and to make sense of content. Contrast that with the findings from the last PISA assessment of reading literacy where, on average across OECD countries, just 47% of 15-year-old students were able to distinguish facts from opinions.

The fact that advancements in literacy skills have fallen sharply behind the evolution of the nature of information has profound consequences in a world where virality seems sometimes privileged over quality in the distribution of information. In the “post-truth” climate in which we now find ourselves, assertions that “feel right” but have no basis in fact, become accepted as so. Algorithms sort us into groups of like-minded individuals, create social media echo chambers that amplify our views, and leave us insulated from opposing arguments that may alter our beliefs. These virtual bubbles homogenise opinions and polarise our societies; and they can have a significant – and adverse – impact on democratic processes. Those algorithms are not a design flaw; they are how social media work. There is a scarcity of attention, but an abundance of information. We are living in this digital bazaar where anything that is not built for the network age is cracking apart under pressure.

The conventional approach in school is often to break problems down into manageable pieces and to then teach students how to solve them. However, modern societies create value by synthesising different fields of knowledge, making connections between ideas that previously seemed unrelated, and connecting the dots where the next innovation will come from.

In the past, schools were technological islands, with technology often limited to supporting and conserving existing practices, and students outpacing schools in their adoption of technology. Now schools need to use the potential of technologies to liberate learning from past conventions and connect learners in new and powerful ways; with sources of knowledge, with innovative applications and with one another.

The past was also divided – with teachers and content divided by subjects and students separated by expectations of their future career prospects; with schools designed to keep students inside, and the rest of the world outside; with a lack of engagement with families and a reluctance to partner with other schools. The future needs to be integrated – with an emphasis on the interrelation of subjects and the integration of students.

In today’s schools, students typically learn individually and at the end of the school year, we certify their individual achievements. But the more interdependent the world becomes, the more we need great collaborators and orchestrators. We could see during this pandemic how the well-being of countries can depend on people’s capacity to take collective action. Schools need to help students learn to be autonomous in their thinking and develop an identity that is aware of the pluralism of modern living. This is important. At work, at home and in the community, people will need a broad understanding of how others live, in different cultures and traditions, and how others think, whether as scientists or as artists.

The foundations for this do not  come naturally. We are all born with “bonding social capital”, a sense of belonging to our family or other people with shared experiences, common purposes or pursuits. This requires deliberate and continuous efforts to create the kind of “bridging social capital” through which we can all share experiences, ideas and innovations with others, and increase our radius of trust to strangers and institutions. The bottom line is that we need to become better at anticipating the capabilities of technology and ensure that we educate humans in ways that complement, and not substitute the artificial intelligence we have created.