“A university’s worth is more than the criteria used in ranking.”
Writer, social activist, gender equality advocate, and Vice-Chancellor of Malaysia’s National University, Sharifah Hapsah S.H. Shahabudin likes to quote Einstein’s maxim: “Not all that counts can be counted”. She developed the quality assurance framework for all qualifications in Malaysia, among other achievements. EduInfo spoke to Professor Hapsah on the occasion of UNESCO’s Global Forum on University Rankings and Accountability in Higher Education (UNESCO, Paris 16-17 May) .
Where did the obsession with rankings begin?
Apparently the original impetus was a British Government report recommending international benchmarks to measure the strength of British universities in relation to American universities. The process is now international. However, very few indicators of quality in higher education translate reliably across borders.
What, for you, is the genuine test of a university?
I would say it is how a university continuously anticipates and leads change through innovations that create new value as well as offering social, environmental and financial returns for the university, the nation and region.
Ideally, what should indicators measure?
They should assess a university’s impact on business innovation, socio-cultural promotion and the environmental development of a region. But these indicators have yet to be devised and perfected. As Einstein once said, “Not all that counts can be counted”. Not all that is counted, counts!
How do current world ranking measures work?
Current methods provide a snapshot of the institution based on simple measures - two qualitative data (employer and peer review) are selected together with four quantitative data to reflect strength in teaching, research impact and international reputation. The greatest influence is exerted by those in the best position to judge the academics. Finally, a single number is mathematically derived to give the position of an institution relative to others with regards to the different aspects of its quality.
Despite all the criticisms leveled against them, world university rankings are still viewed as international benchmarks for comparing the strength of institutions.
Should rankings be used to help decide educational policy and resource allocation?
Yes, but only if rankings are used to help allocate resources for capacity building (for example, providing sufficient funding for research and teaching), or to formulate policies granting greater institutional autonomy and academic freedom, or to restructure higher education for national competitiveness in the global economy. In such cases, ranking becomes a valuable tool for driving excellence.
The Malaysian government created an Accelerated Program for Excellence or APEX university, in addition to existing research-intensive universities. They looked not so much for a “star” university as for one with potential and perceived readiness to transform and manage change.
Is the public perception of rankings a problem?
Public expectations are high and they want quick results in terms of a rise in rankings. People, especially politicians, do not understand the nature of rankings and equate that to standards or the university’s worth. This is truly problematic when you try to predict potential to perform on something that is complex and time-demanding, but use uni-dimensional ranking as an outcome measure. Thus a drop in ranking is interpreted as declining standards and wasted investment and the political opposition in particular, sees it as an opportunity to bash the government and the universities.
Can rankings be used to make judgments about who is best?
No, our National University (UKM), our work is not only confined to producing leaders, research output and science that is expected of a “world class” university but also in nation-building and promoting Malay as a scientific language, as befitting our mandate as a national university.
Rankings are not a measure of a university’s actual worth. They fail to address the needs of students who are looking for specific information to help them select an institution. Also, the values inculcated by universities are not translated into ranking indicators and cannot be measured by mere staff-student ratios. Teaching quality must be measured by students’ learning experience.
What do you think about the choice of indicators?
This is a never-ending debate. Why depend on employers’ perceptions when universities now stress self-employment as a measure of graduate success? In addition, using international students as an indicator disadvantages universities in many developing countries who are expected to fulfil unmet local demands.
There is heavy reliance on qualitative peer reviews and recruiter surveys which comprise 50% of the scores. We know that such judgment is influenced by factors such as tradition which may confer advantages to older institutions. Top-ranking universities are on average about 200 years old, have about 2500 academic faculty, nearly 24,000 students, are highly selective and thus attract and retain top personnel, have endowments of about US$1 billion and an US$2 billion annual budget.
Are you concerned about the commercial interest in ranking ?
Yes - especially the tendency for universities to “play the ranking game”. Recently academic 'experts' whom ranking organizations use for the peer review have been tracked down and offered “incentives” by institutions eager to push up their ranking. In the rush to enhance international reputation some universities recruit international staff and students with scant regard to their qualification. Such an approach is not only shortsighted and counterproductive for institutional capacity-building, but may jeopardize the nurturing of a true academic culture and endanger the mission of the university itself.
So, to sum up, you still believe international ranking provides useful data…
Yes - but it would be even more useful if ranking methodology evaluated universities at a deeper contextual level. This would enable forward planning for institutional changes to ensure a genuine and sustainable improvement in the quality of universities. A university’s worth is more than the criteria used in ranking.