In approximately six weeks' time, policy makers, administrators, researchers and educators will meet at the OECD PISA (Program for International Student Assessment) conference in Tokyo to study the educational strategies of countries which excel in the global standardized test. Many of these strategies will be previewed over the next few weeks in The Global Search for Education.
The performance of U.S. students in the 2009 PISA was poor, especially when you consider that the U.S. has the biggest global economy and is among the top 10 countries in standards of living. Despite the fact that every global thought leader interviewed in The Global Search for Education agreed that standardized testing is not a comprehensive measure of student abilities, no one in the U.S. can feel comfortable that of 65 countries who took PISA, our students ranked 17th in reading, 23rd in science, and 31st in math. While these scores were higher than our 2006 results, they were still far behind the highest scoring countries -- South Korea, Finland, China, and Canada.
So what is the basis of the validity of PISA? Perhaps more important, given the global economic, social and demographic changes taking place, how is PISA planning to ensure that its test will continue to measure the competencies that are most relevant to the future success of students? I discussed this with Andreas Schleicher, the widely esteemed Director of PISA.
Andreas, which countries around the world participate in the PISA test?
PISA covers 87 percent of the world economy currently. In China we have only covered 12 provinces so far because it is a very complex place to administer the test. If you look at the last PISA assessment, the province of Shanghai came up Number 1 in all three areas. The PISA test is very demanding. It's not a simple multiple choice test, so you require a complex marking process. You must have reliability and consistency. It takes time and effort to build up participation.
How do you take into account the demographic differences among countries?
In the U.S. for example, PISA is applied in public and private schools alike. We have test results broken down demographically: urban, rural, parental income, and other significant measures. Countries must provide a sample that accurately represents the different demographics. You need that demographic data to understand how well a country is really doing. When you compare Mexico with Japan and take into account the different economic situations, you gain greater insight.
The U.S. is favorably evaluated and fares reasonably well overall partly because it is a wealthier country that has spent more on education. The rankings overestimate the overall position of the U.S., which is something you will find quite well explained in Volume 2 of our report.
For example: If you don't take the demographic context into consideration in Chile, you are evaluating the results unfairly. Rich countries have an unfair advantage.
What kinds of content make up the test?
There is a mix of questions reflecting a wide range of tasks around the PISA subjects which are Math, Science and Reading. There are some multiple choice questions but there are also many questions that require students to construct answers -- some involving extended responses.
Andreas Schleicher, Director of PISA (Photo courtesy of OECD PISA)
How does PISA assess critical thinking/creative thinking/interpersonal skills/language knowledge?
Foreign language testing is not a part of PISA testing. Critical thinking and creative thinking have been a very important part of PISA testing from the start. If you can't creatively apply knowledge, if you can't think critically across different fields of knowledge, you're not going to get to the highest levels of PISA performance.
In some areas we do well, although in other areas we can do better. For example, one of our objectives for the future is that we want the test to include interpersonal skills. So we are trying to build an assessment of collaborative problem solving skills, because in the world today, countries do not just work by themselves. The test of skills is how well can you exchange ideas, collaborate with others, and work across socioeconomic heterogeneous groups.
How well are you assessing artistic and creative abilities? Artistic and cultural achievements have an impact on a society's quality of life, would you agree?
I very much agree with you. The Arts are a very important subject area. We spoke earlier about the importance of creativity in critical thinking and reasoning. I will say that in PISA, we have not done very well on this because these subjects are very much culturally bound. We are not equipped to assess these capabilities in terms of international comparisons, but I do agree that this is an important agenda for the future, and that it is an area we all should be doing better in.
Are there any expansion plans related to the existing test for the future?
Yes, we are moving towards computer delivered assessments which will allow more interaction in terms of answering questions. It allows us to play with concepts like space and time. That's the major area of development as it will allow us to cater to a wider range of student abilities. We are not equipped for the technology revolution in current educational curriculums. We are hoping that assessments will evolve, and we are trying to reflect that as well as we can. I think we are able to assess things that we couldn't assess in the past. I am fairly optimistic that a lot of effort is being made to extend the range of constants that can legitimately extend assessment.
The PISA test assesses student capabilities more broadly than the current U.S. national standardized tests. We perform poorly. What do we need to do with our educational system to excel in this global test? What do we need to do with the education of our students to help them lead the way in the 21st century? These questions will be addressed in The Global Search for Education.