The parting of the ways between Times Higher Education and QS, its international league table number-cruncher for the past seven years, was bound to cause ripples when it was announced late last year. The two former partners are now vying with each other to capture hearts and minds for their diverging methodologies as they gear up for the 2010 rankings cycle.QS, or Quacquarelli Symonds, the research and information specialists behind the QS World University Rankings, begins work this week on its academic and employer surveys for the 2010 rankings. It also continues a partnership with US News and World Report to reproduce the league tables alongside the magazine's domestic rankings with the publication late last month of a mid-year update.
That there are now to be two rival northern hemisphere English-language rankings to spar with the Academic Ranking of World Universities compiled by the Shanghai Jiao Tong University will be bound to reinforce criticisms that international league tables favour universities in the European and North American mould and discriminate against institutions elsewhere, especially where academics tend to publish in languages other than English.
The THE fired the first shot with an unequivocal statement from Phil Baty, editor of the international rankings, that the controversial reputational surveys of academics that formed a key part in the early years would be retained after all.
Baty wrote in The Australian last month: "Perhaps the most embarrassing aspect [of the THE rankings] was the so-called 'peer review' score. Forty per cent of a university's overall ranking score was based on the results of a 'peer review' exercise: in fact a simple opinion survey of academics, asking them which institutions they rated most highly.
"Some critics object in principle to the use of such subjective measures in rankings on the grounds they reflect past, not current, performance, that they are based on stereotype or even ignorance, and that a good or bad reputation may be mindlessly replicated. "We believe there can be a useful role for reputation surveys, and their use was tentatively endorsed in a recent Thomson Reuters survey."
Baty posted on the university rankings blog rankingwatch.blogspot.com late last year that the idea behind the use of Ipsos MORI was to "inject some proper social science" into the peer review element of the rankings.
"We are clear the response rate to the peer review element of the old THE-QS rankings was far too low and we intend to improve on that with the new 2010 THE world university rankings," he said.
"But we are also clear that surveying is not simply a numbers game. Ipsos will ensure we have a properly targeted and correctly sampled response, to truly reflect the demographics of world higher education."
Richard Holmes, the blogger who has devoted time and energy to analysing the fluctuations in the THErankings, welcomed another Baty admission that rankings had the "deleterious" effect of pressing staff to publish in English-language journals, "which may lift an institution's profile but may not best serve its local community".
Holmes commented: "This is true but it should be noted that THE has shifted from using Scopus data to Thomson Reuters whose database has been criticised for its overwhelmingly English language content."
Niels Weertman, Director of Scopus, said last week: "Scopus is the largest and most comprehensive abstract and citation database of peer-reviewed literature."
In fact it covers almost 18,000 titles from more than 5,000 international publishers, results from 435 million scientific web pages, 23 million patent records from five patent offices and 'articles-in-press' from over 3,000 journals.
Thomson Reuters' Web of Science platform, to be used by The THE, draws on citation databases covering 12,000 of the highest-impact academic journals and more than 110,000 conference proceedings. It spans the sciences, social sciences, arts and humanities, including retrospective coverage dating back to 1900.
The survey cited by Baty was conducted by Thomson Reuters and Ipsos Insight Corporation to inform the development of the 2010 THE rankings. It found that published rankings did change the behaviour and even the strategy of institutions "not to become more effective, but to perform well against arbitrary ranking criteria".
The report on the survey findings says: "Some [institutions] would even manipulate their data to move up in the rankings. This is of great concern and warns against any reliance on indicators that could be manipulated without creating a real underlying improvement."
The survey highlighted a specific concern from Asia: that all the current analyses tend to favour English-speaking nations.
"This is an important reminder that while English remains the international language for academic discourse its pervasiveness may obscure the changing geography of academic activity," it says.
Seventy per cent or more of the 350 respondents said that rankings led some institutions to manipulate their data to move up in the rankings, made institutions focus on numerical comparisons rather than on educating students and lacked transparent and/or reproducible methodologies and data uses.
While most respondents said that league tables favoured research-oriented institutions, they also felt that research metrics and institutional characteristics were the more important measures to use in comparison systems - over 90% rated faculty output and impact as 'nice to have' or 'must have' - the highest ratings for any indicators - clearly showing how much people value the information that comes from data on research publications and citations.
QS fielded Dr Kevin Downing of the City University Hong Kong, a specialist in global university rankings, who said: "All rankings are controversial. Nevertheless, the QS World University Rankings provide a well-accepted measure of the current activity and quality of universities around the world.
"As such they are proving extremely useful to university leaders seeking to benchmark their performance at an international level. Many senior managers of leading universities around the world now include the QS rankings in their strategic plans, because they provide useful, practical targets."
It is evident the league table compilers are falling over themselves to be seen to be responsive to their critics and to the broader academic community.
Nunzio Quacquarelli, Managing Director of QS says, "QS welcomes any new initiatives which provide more information to people making key decisions about universities... QS will also be announcing new initiatives to provide unprecedented access to our data through new research outputs over the next two years.
"These initiatives will fuel the debate and provide an objective basis for comparison, providing academics, candidates and the world at large with an ever-increasing set of essential measures with which to make informed decisions."
David Jobbins
No comments:
Post a Comment