Newsletter LLECE: The limits of test score comparisons. Henry M. Levin, Teachers College, Columbia University
Almost all international comparisons of education among countries and domestic comparisons within countries are based heavily on test scores. Of particular importance are the test results for 15 year olds in reading, mathematics, and science published every three years by the OECD under PISA and those published every four years in mathematics and science at fourth grade and eighth grade by the IEA for TIMSS as well as the regional studies by UNESCO. These test results are often used as an indicator of school quality and rankings. Such use can be challenged in many ways. For example, a top performer, Korea, spends as much on private tutoring, 3 percent of Gross Domestic Product, as its government spends on education, raising issues on whether schools are producing the results. Achievement tests are useful indicators of what students know, but must interpreted carefully as to their meaning and importance.
One reason that test results are taken so seriously is that it is widely assumed that they are strong predictors of future labor force productivity and prosperity. Yet, surprisingly, the statistical evidence linking test scores to productivity shows only a modest relation. For example, only about 10 percent of variance in earnings and only 5 percent of variance in supervisory ratings of workers is explained by test scores. This leaves most of productivity differences to be explained by other worker characteristics. In the last decade, considerable attention has been focused on non-achievement outcomes of what schools are expected to do and their impacts on worker productivity. Of course schools are expected to contribute to full human development in many ways for not only productive employment, but also for civic participation, and meaningful social and parental relations. These aspects have been termed non-cognitive or socio-emotional outcomes of education, and recent studies show that they are important determinants of adult success.
In addition to knowledge and cognitive skills as reflected in test scores, well-functioning individuals need both effective intrapersonal and interpersonal skills. Intrapersonal refers to such internal human capabilities as listening, reflection, control of impulses, patience, persistence, empathy, curiosity, and the like. Interpersonal refers to how one relates to others in a social setting and includes such aspects as collaboration, respect, friendship, leadership, communication, and sharing, among many other traits. Both intrapersonal and interpersonal skills interact with cognitive skills to fulfill such important workplace demands as decision-making, planning, coordination, and problem-solving.
We are far from a full understanding of socio-emotional learning and a specification of all of the important intrapersonal and interpersonal characteristics that are important for productive functioning of individuals and what schools can contribute to their development. The research on this topic is relatively recent, but showing highly promising results. For example, a recent World Bank study of the economic returns to cognitive and non-cognitive skills in urban Peru found that the non-cognitive measures were at least as important as the test scores in explaining earnings differences. Further, the independent impact of more schooling showed a powerful contribution beyond test scores, suggesting the unmeasured impacts of school beyond what is presently measured by test results.
These recent and important developments in understanding the productive outcomes of education beyond test scores suggest that we should broaden the measures of educational results obtained in international assessments of education. Test scores are not the only dimensions of importance, and we may be distorting educational policy by not taking account of a fuller range of important impacts of schooling, particularly their performance in developing key intrapersonal and interpersonal skills. Future international comparisons of educational effectiveness must build on the evidence from current research that shows that productive education is more than just test scores.
<- Back to: UNESCO Office in Santiago