Thursday, August 19, 2010

More from THE

Phil Baty in Times Higher Education gives us some clues about what the forthcoming THE World University Rankings will contain.


"While all the self-reported material bears the imprimatur of the supplying
institutions (and our tables include only those that have cooperated with our exercise) and it has been vetted for quality, the consultation had some concerns
about its consistency and robustness - especially in this inaugural year. For example, not all institutions could provide a clear or internationally comparable figure for their research income from industry.

For maximum robustness, we plan to give extra weighting to data that have been sourced independently of the institutions themselves and are globally consistent.

Citations data, for example, which are widely accepted as a strong proxy for research quality, will have a high weighting - perhaps about 30 per cent of the total ranking score.

We also have high confidence in the validity and independence of the results of our reputation survey. Although we may yet adjust its weighting, this subjective measure will not be weighted as highly as it was in our old methodology (2004-09), where reputation was worth 40 per cent."

It looks as though citations per paper, a measure of its influence throughout a research community, will count for a lot in the forthcoming rankings. It is questionable whether such a high weighting for a single component is justified. At the very least it could be combined with other measures of quality such as the h-index which is, in effect, a measure of both productivity and impact.

The reluctance to place too much emphasis on research income and perhaps other types of income, is understandable but perhaps unfortunate. This indicator would give the new rankings a distinctive feature and might also allow us to see whether institutions are giving value for money.

It is inevitable that the reputational survey would never be given the same weight that its predecessors received in the THE-QS rankings. Whether its results are really valid -- we still do not know the response rate -- remains to be seen.

No comments: