Click here
I had sent you email to inquire about the ranking performance of Taiwan's universities.
I am waiting for your reply.
Please share your opinion.
Thanks!
best regards,
yuching
Discussion and analysis of international university rankings and topics related to the quality of higher education. Anyone wishing to contact Richard Holmes without worrying about ending up in comments can go to rjholmes2000@yahoo.com
I appreciate that measuring the research impact of an institution is difficult. Unfortunately, the THE seems to have got it quite badly wrong this time. The best evidence for this is not that Warwick doesn't make it into the top 200 (even though this hurts; yes I am faculty at Warwick), but the fact that the University of Alexandria makes it number 4, based on the output of one single person.
1. Do not count self-citations. Even better, it is common sense that a citation by someone further from your field should count more than a citation by your former PhD student. Of course, these things are difficult to figure out in an automated way. But one could for example use the collaboration distance (coauthors have distance one, coauthors of coauthors have distance two, etc) to weight citations (with a cap at 4, say).
2. Quality of research is paramount. As already pointed out, it is easy to get many citations for mediocre research if there are sufficiently many other mediocre researchers working in the same area. Again, this is vastly more common than you may think, for the simple reason that it is easier to perform mediocre research than world-class research. Furthermore, you get more recognition as counted by basic citation count, so why not doing it?
One way of taking this into account is to give higher weight to citations coming from articles published in highly respected journals. (Similarly, when measuring "research output", higher weight should be given to articles published in journals with high reputation.)
However, please *DO NOT* use the impact factor as a measure of the quality of a journal, as it can be (and is!) very easily manipulated, as the story of the journal "Chaos, Solitons and Fractals" shows. Instead, the only reliable way of assessing the quality of a journal within a given field is to ask researchers in that field to provide their own rankings. Yes, this seems subjective, but unfortunately that's all you are ever going to get, and I can assure you that you will get a very consistent picture within each area. The fact that the "Annals of Mathematics" is the most respected journal in mathematics simply cannot be measured in terms of impact factor.
3. Count current citations to older work. If someone's article turns out to spawn an entire new field of research five years later, it will not show up at all in the current metric. This makes simply no sense. Of course, this doesn't happen all that often, but the reason why top institutions have a reputation is precisely because of those instances in which it happens. Furthermore, there are areas of research (like mathematics) in which the "lifespan" of a good article is measured in decades, which goes way beyond the two to five years that you use as a rule. Counting current citations to older articles would be one small but absolutely essential step to correct this.
4. Measure the total impact of the institution in a field, and not its "average" impact. The only way I could see that the output of one single person can count so much is that this person somehow has an abnormally high weight, probably due to the fact that there is very little research output from the U. of Alexandria. If this suspicion is indeed correct (I hope that I am wrong on this one), then this would effectively mean that universities are penalised by having large (and influential!) departments and should rather strive to have only very few but quite prolific researchers on their payroll.
There is probably more, but I am getting hungry now ;-) I very much hope that you will take these comments to heart. Best wishes,
Martin
"“I believe we are well deserving of being on the list,” professor of economics
Amr Hussein told Bikya Masr. “We have worked hard to improve our system of
education and it is showing that we are succeeding in doing so.”...
Hend Hanafi, President of Alexandria University, told local media that she is proud
of the ranking and hopes the university will continue to make efforts to
consistently improve the quality of education at the university."
"Haaretz has learned that most Israeli universities were not on the list because they failed to respond to repeated requests for information, including on faculty and students, which is necessary for the listing.There are other surprising omissions such as all the Indian Institutes of Technology, the University at Buffalo: SUNY and the Catholic University of Louvain (the French one -- the Dutch one is there at 120)
TAU and the Hebrew University say that they never received such a request from THE. According to THE, only the Technion and Bar-Ilan University responded with information, but they were ranked 221 and 354 respectively.
As for the other universities, the editor of the ranking, Phil Baty, told Haaretz that although it is upsetting for Israel, he hoped that the Israeli universities would recognize the amount of serious work invested in creating the ranking and the degree to which the methodology was transparent, and would participate in the initiative, like other universities have done. He also expressed certainty that next year "they will be included."
Didn't do their homework
THE says that more than 6,000 universities participated in the ranking and most provided the necessary information.
A spokesperson for the Hebrew University responded that contrary to the claim of
the survey's editors, "following an examination we did not find any such request
[for information]. When we asked for the correspondence to the university on the
subject, they could not provide it. The university is saddened by the fact that
the editors of the ranking did not carry out their work responsibly, and thus
harmed the university." "
My bet is that TR uses the Leiden "Crown indicator" since this is what is embodied in their product InCites.
To cut it short, each paper is linked to a subdiscipline, a type of publication (letter, review, ...) and a year of publication. With this data for the whole world, it is easy to calculate the expected number of citations for a paper of a given type, in a given discipline, in a given year.
For a set of papers (e.g. all the papers of Alexandria university), the indicator is calculated as Sum(received citations)/Sum(expected citations).
This number can become very high if you have a small number of paper or if you look only at recent papers (if, on average you expect 0.1 citations for a recent paper in math, a single citation will give you a score of 10 for this paper!)
Note that Leiden as recently decided to change its favorite indicator for a mean(citations received/citations expected) which gives less weight for a few highly cited papers in a set. But it seems that TR has not implemented yet this new indicator.
Note also that, in order to avoid the overweight given to few papers in a small set, Leiden publish its own ranking of universities with thresholds on the total number of papers published.
This is not very convincing. Does Alexandria produce strong research? Overall, No. It is ranked 1014 in the world for total papers over a ten year period by SCImago.“Alexandria University is Egypt's only representative in the global top 200, in joint 147th place. Its position, rubbing shoulders with the world's elite, is down to an exceptional score of 99.8 in the "research-influence" category, which is virtually on a par with Harvard University.
Alexandria, which counts Ahmed H. Zewail, winner of the 1999 Nobel Prize for
Chemistry, among its alumni, clearly produces some strong research. But it is a
cluster of highly cited papers in theoretical physics and mathematics - and more
controversially, the high output from one scholar in one journal - that gives it
such a high score.Mr Pratt said: "The citation rates for papers in these fields may not appear exceptional when looking at unmodified citation counts; however, they are as high as 40 times the benchmark for similar papers. "The effect of this is particularly strong given the relatively low number of papers the university publishes overall."
Quacquarelli Symonds has continued to produce those rankings, now called the QS World University Rankings, and is partnering with U.S. News and World Report for their publication in the United States.
The relationship between the former collaborators has deteriorated into
barely veiled animosity. QS has accused Times Higher Education of unfairly
disparaging the tables they once published together. This week the company
threatened legal action against the magazine over what Simona Bizzozero, a QS
spokeswoman, described as "factually inaccurate" and misleading statements by
representatives of Times Higher Education. She said THE's role in the
collaboration was limited to publishing the rankings based on a methodology that
QS had developed. "What they're producing now is a brand-new exercise. A totally
brand-new exercise, with absolutely no links whatsoever to what QS produced and
is producing," she said. "So when they refer to their old methodology, that is
not correct."
Phil Baty, editor of the rankings for Times Higher Education, declined to respond to QS's complaints: "We are now looking forward, not looking backward."
"Really, nothing has changed," said Ellen Hazelkorn, executive director of the Higher Education Policy Research Unit at the Dublin Institute of Technology, whose book "Rankings and the Battle for Worldclass Excellence: The Reshaping of Higher Education" is due to be published in March.
Despite Times Higher Education's assurances that the new tables represent a much more rigorous and reliable guide than the previous rankings, the indicators on which the new rankings are based are as problematic in their own way, she believes. The heavily weighted measure of teaching, which she described as subjective and based on reputation, introduces a new element of
unreliability.
Gauging research impact through a subjective, reputation-based measure is troublesome enough, and "the reputational aspect is even more problematic once you extend it to teaching," she said.
Ms. Hazelkorn is also troubled by the role Thomson Reuters is playing through its
Global Institutional Profiles Project, to which institutions provide the data
used in the tables. She dislikes the fact that institutions are going to great
effort and expense to compile data that the company could then sell in various
ways.
"This is the monetarization of university data, like Bloomberg
made money out of financial data," she said.
Thomson Reuters is a private global information services
firm, and a highly respected one at that. Apart from ‘deep pockets’, they have
knowledgeable staff, and a not insignificant number of them. For example, on 14
September Phil Baty, of Times Higher Education sent out this fact via their
Twitter feed:
2 days to #THEWUR. Fact: Thomson Reuters involved more
than 100 staff members in its global profiles project, which fuels the rankings
The incorporation of Thomson Reuters into the rankings games by Times
Higher Education was a strategically smart move for this media company for it
arguably (a) enhances their capacity (in principle) to improve ranking
methodology and implementation, and (b) improves the respect the ranking
exercise is likely to get in many quarters. Thomson Reuters is, thus, an
analytical-cum-legitimacy vehicle of sorts.
Malaysia Star Online
Nine Taiwan universities listed among the world's top 500 Radio Taiwan International Israeli universities drop in international rankingsTrinity and UCD slip down rankings of top universities |
I will let readers decide how representative or accurate such a survey can be.
Incidentally, QS should be given credit for the detailed description of the methodology of this criterion.
A new broad category, called "Teaching - the learning environment", will be
given a weighting of 30 per cent.
Using five separate indicators, this category will use data on an institution's income, staff-student ratios and undergraduate-postgraduate mix, as well as the results of the first-ever global academic reputation survey examining the quality of teaching.
A further 30 per cent of the final rankings score will be based on another new indicator, "Research - volume, income and reputation".
This category will use four separate indicators, including data on research income, research output (measured by publications in leading peer-reviewed journals) and the results of the academic reputation survey relating to research.
The highest-weighted category is "Citations - research influence".
This category will examine a university's research influence, measured by the number of times its published work is cited in other academics' papers.
Based on the 12,000 journals indexed by Thomson Reuters' Web of Science, and taken over a five-year period, the citations data will be normalised to take account
of different volumes of citations between disciplines.
Reflecting the high levels of correlation between citations data and research excellence, this category will be given a weighting of 32.5 per cent.
A fourth category, "International mix - staff and students", will use data on the proportion of international staff and students on campus. This indicator will be given a 5 per cent weighting.
Knowledge transfer activities will be reflected in "Industry income - innovation", a new category worth 2.5 per cent of the total rankings score. This will be based on just one measure in 2010 - research income from industry.
One of the things that I have been keen to do as editor of the Times Higher
Education World University Rankings is to engage as much as possible with our
harshest critics.
Our editorial board was trenchant in its criticism of our old rankings. In particular, Ian Diamond, principal of the University of Aberdeen and former chief executive of the Economic and Social Research Council, was scathing about our use of research citations.
The old system failed to normalise data to take account of the dramatically different citation volumes between different disciplines, he said - unfairly hitting strong work in fields with lower average figures. We listened, learned and have corrected this
weakness for the 2010 rankings.
Another strong critic is blogger Richard Holmes, an academic at the Universiti Teknologi MARA in Malaysia. Through his University Ranking Watch blog, he has perhaps done more than anyone to highlight the weaknesses in existing systems: indeed, he highlighted many of the problems that helped convince us to develop a new methodology with a new data provider, Thomson Reuters.
He has given us many helpful suggestions as we develop our improved methodology. For example, he advised that we should reduce the weighting given to the proportion of international students on campus, and we agreed. He added that we should increase the weighting given to our new teaching indicators, and again we concurred.
Of course, there are many elements that he and others will continue to disagree with us on, and we welcome that. We are not seeking anyone's endorsement. We simply ask for open engagement - including criticism - and we expect that process will continue long after the new tables are published.
"While all the self-reported material bears the imprimatur of the supplying
institutions (and our tables include only those that have cooperated with our exercise) and it has been vetted for quality, the consultation had some concerns
about its consistency and robustness - especially in this inaugural year. For example, not all institutions could provide a clear or internationally comparable figure for their research income from industry.
For maximum robustness, we plan to give extra weighting to data that have been sourced independently of the institutions themselves and are globally consistent.
Citations data, for example, which are widely accepted as a strong proxy for research quality, will have a high weighting - perhaps about 30 per cent of the total ranking score.
We also have high confidence in the validity and independence of the results of our reputation survey. Although we may yet adjust its weighting, this subjective measure will not be weighted as highly as it was in our old methodology (2004-09), where reputation was worth 40 per cent."
"It is no secret that the Indian students who cannot get into any of their universities and colleges are the ones that are willing to come to the UK. Their intention is the 20 hour/week work allowed and assume rightly once they use the university route to get into UK they can stay in the country to work. In my university which recruits these students, the drop out rates for such students is high as they work more than 20 weeks to meet their expenses. Their attendance drops down after a few months. I have yet to come across a single non-EU student who comes with enough funds to complete a 3 UG degree. They are told by agents that they can work in the UK to meet part of their fees and all the living expenses. The students coming in to do MSc are poorly equipped and struggle to pass their modules and write project proposals."
"We can expect some big-name institutions to take a hit in the new World
University Rankings.
Why? Because the rankings we will publish this autumn will be based less on subjective opinion and more on objective evidence".
..........................................................................................................
"Under the initial proposals for our methodology, currently being refined in line with responses from the global academy, reputational measures are worth no more than 20 per cent of overall scores.
I have also set a cap to ensure that subjective elements are never again anywhere near the 50 per cent used in our previous methodology. This means that big names with big reputations that lack world-class research output and influence to match will suffer in comparison with previous exercises. Conversely, unsung heroes have a better chance of recognition".
An epic effort by our world university rankings data supplier, Thomson Reuters, to collect information from hundreds of universities around the world concluded successfully last week.I am not sure whether "epic" is the right word. The number of universities in the database does not seem much higher than that for which QS has collected information. The data does apparently include some information that QS has ignored such as institutional income and research income but has not included items counted by QS such as total student numbers or the number of postgraduate students other than doctoral candidates. Meanwhile, the number of respondents to the opinion survey has fallen far short of the original target of 25,000, even with a bit of topping up, like QS, from the Mardev mailing lists.
"Consistent with other studies, though in much greater detail, Espenshade and Radford show the substantial admissions boost, particularly at the private colleges in their study, which Hispanic students get over whites, and the enormous advantage over whites given to blacks. They also show how Asians must do substantially better than whites in order to reap the same probabilities of acceptance to these same highly competitive private colleges. On an "other things equal basis," where adjustments are made for a variety of background factors, being Hispanic conferred an admissions boost over being white (for those who applied in 1997) equivalent to 130 SAT points (out of 1600), while being black rather than white conferred a 310 SAT point advantage. Asians, however, suffered an admissions penalty compared to whites equivalent to 140 SAT points.
The box students checked off on the racial question on their application was thus shown to have an extraordinary effect on a student's chances of gaining admission to the highly competitive private schools in the NSCE database. To have the same chances of gaining admission as a black student with an SAT score of 1100, an Hispanic student otherwise equally matched in background characteristics would have to have a 1230, a white student a 1410, and an Asian student a 1550. Here the Espenshade/Radford results are consistent with other studies, including those of William Bowen and Derek Bok in their book The Shape of the River, though they go beyond this influential study in showing both the substantial Hispanic admissions advantage and the huge admissions penalty suffered by Asian applicants. Although all highly competitive colleges and universities will deny that they have racial quotas -- either minimum quotas or ceiling quotas -- the huge boosts they give to the lower-achieving black and Hispanic applicants, and the admissions penalties they extract from their higher-achieving Asian applicants, clearly suggest otherwise."
The advantage accorded to Non-Asian minority students, even those whose claim to moral reparation for generations of slavery or dispossession is questionable, is well known. What is surprising about Espenshade and Radford's study is the extent of the discrimination against poor, rural and working class whites.
In part, this is a consequence of the indicators used by American ranking organizations. Selective colleges are apparently reluctant to offer places to students who might not take up an offer for financial reasons since this would push down their acceptance rates and yield scores.
But there is more. Espenshade and Radford found that less affluent whites were dramatically less likely to be offered a place in a competitive private college even when SAT scores, a reasonable proxy for general intelligence, and high school grades were controlled for. In addition, they found evidence of serious discrimination against students who were involved in incorrect activities such as ROTC and Future Farmers of America, especially those holding leadership positions. Apparently "feeding the homeless" will boost one's chances of getting into a top private college if it means doling out soup in between starring in the school play and AP English classes but not if means showing an interest in growing the stuff that the homeless eat.
As cognitive skills become increasingly irrelevant to admission into America's best schools, it seems almost certain that US higher education will be less and less able to compete with those countries that continue to recruit those students most capable of demanding college-level work.
Will a redemption narrative help Times Higher earn credibility for its new rankings? Perhaps. It should certainly be applauded for its openness to criticism, and for all it is doing to inform the public about its next moves in what its editor characterizes, with appropriate caution, as “a decent first step” at improvement. But ultimately, debating tactics notwithstanding, the global league tables will be judged on their merits. As the wars over league tables continue, the next rankings season should be well worth watching.
While brilliant and progressive research continues apace here and there, the amount of redundant, inconsequential, and outright poor research has swelled in recent decades, filling countless pages in journals and monographs. Consider this tally from Science two decades ago: Only 45 percent of the articles published in the 4,500 top scientific journals were cited within the first five years after publication. In recent years, the figure seems to have dropped further. In a 2009 article in Online Information Review, Péter Jacsó found that 40.6 percent of the articles published in the top science and social-science journals (the figures do not include the humanities) were cited in the period 2002 to 2006.
As a result, instead of contributing to knowledge in various disciplines, the increasing number of low-cited publications only adds to the bulk of words and numbers to be reviewed. Even if read, many articles that are not cited by anyone would seem to contain little useful information. The avalanche of ignored research has a profoundly damaging effect on the enterprise as a whole. Not only does the uncited work itself require years of field and library or laboratory research. It also requires colleagues to read it and provide feedback, as well as reviewers to evaluate it formally for publication. Then, once it is published, it joins the multitudes of other, related publications that researchers must read and evaluate for relevance to their own work. Reviewer time and energy requirements multiply by the year. The impact strikes at the heart of academe.
I was disturbed by an email that dropped into my in-box late last
month.
No, it was not another offer of cheap Viagra, or an announcement that I
had won an overseas lottery. It was more unsettling than that.
"Dear academic," it began. The greeting alone was a surprise, given
that I am a journalist with little more than a bachelor's degree by way of
academic credentials.
But my unease grew with each line of the message. The email was from a
major education information company inviting me to take part in an online survey
that would be used to create a university ranking.
It said that my role as a leading educationalist combined with my
subject focus made my opinion very important. It even offered to enter me into a
prize draw if I passed on my great wisdom and spent 10 minutes filling in the
form.
It would be amusing if the implications were not so serious. As the
email claimed, the audience for the company's annual exercise is in the
millions, and it is clear that university league tables in various forms have
become a very big business with wide influence.
Any organisation, such as Times Higher Education, that seeks to create
rankings must accept its responsibility to conduct thorough research and to
employ sound data.
There is a responsibility on companies doing such surveys that
academics are selected carefully by discipline, and by country and continent if
appropriate. If compilers want universities and students to see their league
table as robust the onus is on them to take a rigorous approach. When rankings can make or break a university's reputation, or influence multimillion-pound strategic decisions, anything less will simply not do.