American students are barely above the average of 44 countries and economies in problem-solving skills, far behind teens in Asia, according to the first international test of that attribute in 15-year-olds.
U.S. teens on average earned a score of 508 on the Programme for International Student Assessment -- Creative Problem Solving test, between top-ranked Singapore's 562 and bottom-ranked Colombia's 399. The PISA results, released Tuesday, put U.S. students in the middle of the pack, hardly supporting the American workforce reputation for creativity.
PISA is an international test for 15-year-olds that has been administered in reading, math, and science since 2000. The Organisation for Economic Co-Operation and Development -- the Paris-based group behind the test -- administered the computer-based problem-solving test for the first time in 2012 in response to a job market that increasingly demands what the group called "non-routine analytic" and "non-routine personal tasks."
"This is the reason why the countries that participated in PISA decided this was an important area," said Pablo Zoido, an OECD analyst.
The U.S. performed higher than 28 countries. The U.S. ranked from 12th to 21st place -- the rank is imprecise because of the lack of statistical significance separating mid-pack performers -- with results close to Austria, the Czech Republic, Estonia, France, Germany, the United Kingdom and Italy. Singapore, Korea and Japan came out on top, followed by China, Finland, Canada and Australia. Colombia and Bulgaria were lowest.
American students fared particularly well on "interactive" questions, which "require students to uncover useful information by exploring the problem situation and gathering feedback on the effect of their actions," according to the test results. That means U.S. students can "tolerate doubt and uncertainty, and dare to use intuitions to initiate a solution," according to the results.
The top-performing Asian countries are known as "pressure-cooker countries," according to Amanda Ripley, whose recent book, The Smartest Kids In The World, followed American high school students in exchange programs in Finland, Korea and Poland to better understand how those countries' education systems fare so well on PISA. Korea in particular has an intense culture around tutoring.
"The big criticism of these systems -- the one you hear most often from people within these countries -- is that they aren't teaching kids to think creatively and problem-solve," Ripley said. "Well, now we have a test that gets closer to measuring those skills than any other -- and they are killing it. Again. So what does this mean?"
Ripley suggested two likely factors. First, critics may be "underestimating just how effective their system is in helping kids think for themselves." Second, while the exam is probably among the best tools for measuring problem-solving skills, "it probably still is not measuring all the many, many ways in which humans can be creative," Ripley said.
The test was taken by 85,000 students who also took the main PISA. They answered questions on computers, solving problems such as finding the most convenient route on a map for three friends who wanted to meet up, or choosing the cheapest train ticket for a certain destination. Across the board, boys were more likely to be higher performers, though average scores did not differ by gender.
The OECD used scores on the main PISA test to estimate how students in various countries would fare. U.S. students beat the estimate by 10 points. According to the report, the higher performance "may indicate" that students' learning opportunities prepare them for dealing with real-life challenges.
"We did 10 points better than we were expected to do -- but it's 10 points out of 508, not a whole lot better than we were expected to do," said Mark Schneider, a vice president for the American Institutes of Research who previously led the U.S. Education Department's research arm. "It's statistically significant, but they don't talk about the degree to which it's substantively significant."
OECD noted that U.S. students "perform significantly better in problem solving, on average, than students in other countries who show similar performance in mathematics, reading and science." Still, 18.2 percent of U.S. students -- or one in six -- performed below the baseline level.
Results for the general PISA test reported in late 2013 showed the U.S. performed mid-pack.
To some extent, the scores reflected socioeconomic status, with higher-income students besting their lower-income peers. That pattern held true internationally. According to a Huffington Post analysis, even America's top 10 percent of students by wealth generally didn't outperform wealthy peers in other countries. According to OECD, socioeconomic status played less of a role in determining competence in problem-solving skills than math chops.
"The impact of socioeconomic background on problem-solving skills is smaller than what we observed in mathematics and reading or in science," Zoido said. "We see socioeconomic status playing less of a role in problem-solving than in other domains."