by Barbara Ischinger, Director, OECD Directorate for Education and Skills
Michael Ward, Senior Policy Analyst, PISA for Development
and Alejandro Gomez Palma, Policy Analyst, PISA for Development
In our previous blog about PISA for Development, we were pleased to announce Ecuador’s agreement to participate in this new pilot project. We’ve just returned from the Zambian capital of Lusaka and are delighted to report that Zambia has also agreed to participate – the first sub-Saharan African nation ever to take part in any PISA survey.
You might well ask: how can we compare the performance of students in highly developed countries – such as Japan and Germany – with that of students in low- and middle-income countries in Africa? And how do we assess the competencies of the tens of millions of 15-year-olds in developing countries who aren’t enrolled in school? These were precisely some of the challenges we put to ourselves when the idea of PISA for Development emerged from discussions with countries and OECD partners.
To make the assessments of 15-year-olds more relevant to a wider range of countries, including Zambia, we knew that we would have to adapt both the tests and the contextual information we collect through PISA’s student, school and parent questionnaires. For the assessment of reading, mathematics and science, we will tap into our large stock of questions that were already successfully used in previous PISA cycles and, in collaboration with our partners, select, review and adapt and then field trial those that we find are best suited to assess the level of skills we identify in the participating countries. The PISA for Development assessments will be better targeted to describe the performance at the middle and low end of the proficiency spectrum, while also measuring the higher levels and maintaining comparability with the international PISA scales. The range of questions included in the assessment will give us a fine-grained picture of performance at these lower levels that will, in turn, provide more valuable diagnostic information for countries.
The questions we select will be field tested in Zambia and the other participating countries to see how well students respond to them, then reviewed and adapted, as necessary, with the assistance of local and regional experts, to ensure they are relevant across countries and cultures. This adaptation may involve making the questions more familiar to students by changing choices of words and references, for example, and, where necessary, translating the questions into the languages of the participating countries. The translation and verification process is well established in the main PISA assessment and it also allows for some adaptations to accommodate local contexts. There are, for example, different ways of asking the same question depending on whether the student taking the test lives in Australia or in the United Kingdom; and Scotland also adapts some of the terminology of some of the PISA test questions. But regardless of these contextual changes, the nature of the questions, and the knowledge and skills assessed, must remain true to the original intent of the question, so that the assessments still measure what they were originally designed to measure and results are comparable with those of other countries.
The context questionnaires, distributed to students, school directors and parents, will be similar to those we already use in the main PISA assessment, but will be developed to include questions that may be more relevant to countries like Zambia. So, for example, where the current PISA questionnaire to school directors asks about school facilities and resources, this would need to be adapted to include such questions as: Does the school have separate sanitation facilities for girls and boys? Does the student have electricity at home? Running water? A separate kitchen? And we will have to be prepared for very different answers to some basic questions – like: who is the head of your household? – as households in developing countries are sometimes headed by children or grandparents. An important point to remember is that we have to keep some questions common to all so that we can assess all countries on the same scale.
Field trials of the PISA for Development assessment will begin in the second half of 2015; we’re expecting that more than 25,000 students in 4 or 5 countries – including Ecuador and Zambia – will participate. The main assessment will be conducted during 2016 and early 2017. During this time, we’ll also continue working with our partners, including the World Bank, UNICEF and UNESCO, to try to identify 15-year-olds who are not enrolled in school and develop ways of reaching them, too. It’s an ambitious agenda, certainly. But we’re convinced that, as the world moves beyond the Millennium Development Goals towards a new post-2015 education agenda that is focused on improving learning outcomes worldwide, we must be sure that children in all countries not only have access to school, but are acquiring both fundamental and 21st century skills when they get there.
Links:
Expanding PISA’s circle of influence
PISA for Development
Millennium Development Goals
Photo Credit: Drawing of Zambia on Blackboard, drawn in chalk / @Shutterstock
