Are "publication metrics" also used heavily in China by the bureaucracy ?
I know for a fact that the number of fake-journals exploded once the Govt. of India decided to use this for promotions.
It's a bit sad really: in the classical world both these countries spent inordinate amount of time on the questions of epistemology (India esp.). Now reduced to mimicking some silly thing that vaguely tracks knowledge-production even in the best case in the West.
Sabine Hossenfelder has been on about this topic in the field of physics publishing for quite some time now.
It really is a terrible thing, though I can understand how some researchers feel trapped in a system that gives them little if any alternative if they wish to be employed the next year. Not just one thing needs to be changed to fix it.
Things like citation brokers (paid to cite papers), abuse of power, paper mills, and blackmail (pg. 10) is appalling to me. I have to question how we ended up here. Academia seems very focused on results and output, and this is used as a metric to measure a researcher's worth or value.
Has this always been an issue in academia, or is this an increasing or new phenomenon? It seems as if there is a widespread need to take shortcuts and boost your h-index. Is there a better way to determine the impact of research and to encourage researchers to not feel so pressed to output and boost their citations? Why is it like this today?
Academic mathematics, from what I've seen, seems incredibly competitive and stressful (to be fair, so does competition math from a young age), perhaps because the only career for many mathematicians (outside a topics with applications such as but not limited to number theory, probability, and combinatorics) is academia. Does this play into what this article talks about?
In my time in academia (~20 years) I have seen the demands and competition increase quite significantly, however talking to older researchers the this really started in the 90s the demands to demonstrate measurable outcomes increased dramatically and funding moved to be primarily through competitive grants (compared to significant base funding for researchers previously). The issue is that while previously it was common for academics to have funding for 1-2 PhD students to look into new research areas, now many researchers are required to bring in competitive grants for even covering part of their salary.
What that means is that researchers become much more risk averse, and and stay in their research area even if they believe it is not the most interesting/imapactfull. You just can't afford to not publish for several years, to e.g. investigate a novel research direction, because without the publications it becomes much much harder to secure funding in the future.
> Has this always been an issue in academia, or is this an increasing or new phenomenon?
The introduction of this article [1] gives an insight on the metric used in the Middle Ages. Essentially, to keep his position in a university, a researcher could win public debates by solving problems nobody else could solve. This led researchers to keep their work secret. Some researchers even got angry about having their work published, even with proper credit.
The issue in all fields became significantly worse as developing countries decided their universities needed to become world class and demanded more international publications for promotion. Look at the universities in the table in the paper and you can see which countries are clearly gaming the system. If your local bureaucrats can’t tell which journals are good and which are fake, the fake journals become the most efficient strategy. Even worse, publishers figured out that if you can attract a few high-citation papers, your impact factor will go way up (it’s an arithmetic mean) and your fake journal becomes “high quality” according to the published citation metrics!
Math is particularly susceptible to this because there are few legitimate publications and citation counts are low. If you are a medical researcher you can publish fake medical papers but more easily become “high impact” on leaderboards (scaled by subject) by adding math topics to your subjects/keywords.
Abuse of power is definitely not new. Professors have historically overworked their grad students and withheld support for their progress towards a PhD or a paper unless they get something out of it. For women it’s extra bad because they can use their power in other ways.
I've seen similar stuff in a couple of other places, including IT back in the 1990s (back when it wasn't nearly as glamorous as it is today).
I think some of this has to do with... resentment? You're this incredibly smart person, you worked really hard, and no one values you. No one wants to pay you big bucks, no one outside a tiny group knows your name even if you make important contributions to the field. Meanwhile, all the dumb people are getting ahead. It's easy to get depressed, and equally easy to decide that if life is unfair, it's OK to cheat to win.
Add to this the academic culture where, frankly, there are fewer incentives to address misbehavior and where many jobs are for life... and the nature of the field, which makes cheating is easy (as outlined in the article)... and you have an explosive mix.
This article does not seem to quite convey the experience of a pure mathematician. Yes, citation fraud is happening on an apalling scale, but no it is not a serious issue for mathematicians.
The problem of AI generated papers is much more serious, although not happening on the same scale (yet!).
TLDR: The publication culture of mathematics (with relatively few papers per researcher, few authors per paper, and few citations per paper) makes abuse of bibliometrics easier. The evidence suggests widespread abuse.
My take: I’ve published in well-regarded mathematical journals and the culture is definitely hard to explain to people outside of math. For example, it took more than two years to get my key graduate paper published in Foundations of Computational Mathematics, a highly regarded journal. The paper currently has over 100 citations, which (last I checked) is a couple times higher than the average citation count for the journal. In short, it’s a great, impactful work for a graduate student. But in a field like cell biology, this would be considered a pretty weak showing.
Given the long timelines and low citation counts, it’s not surprising that it’s so easy to manipulate the numbers. It is kinda ironic that mathematicians have this issue with numbers though.
Pure math has a far greater vulnerability to this than applied math. Top journals have impact factors of around 5.0. Respectable but tiny specialist journals can have impact factors less than 1.0 (like, 0.4). Meanwhile, MDPI Mathematics is a Q1 journal with an impact factor over 2.0.
The now-standard bibliometrics were not designed by statisticians :-)
I know for a fact that the number of fake-journals exploded once the Govt. of India decided to use this for promotions.
It's a bit sad really: in the classical world both these countries spent inordinate amount of time on the questions of epistemology (India esp.). Now reduced to mimicking some silly thing that vaguely tracks knowledge-production even in the best case in the West.
It really is a terrible thing, though I can understand how some researchers feel trapped in a system that gives them little if any alternative if they wish to be employed the next year. Not just one thing needs to be changed to fix it.
Has this always been an issue in academia, or is this an increasing or new phenomenon? It seems as if there is a widespread need to take shortcuts and boost your h-index. Is there a better way to determine the impact of research and to encourage researchers to not feel so pressed to output and boost their citations? Why is it like this today?
Academic mathematics, from what I've seen, seems incredibly competitive and stressful (to be fair, so does competition math from a young age), perhaps because the only career for many mathematicians (outside a topics with applications such as but not limited to number theory, probability, and combinatorics) is academia. Does this play into what this article talks about?
What that means is that researchers become much more risk averse, and and stay in their research area even if they believe it is not the most interesting/imapactfull. You just can't afford to not publish for several years, to e.g. investigate a novel research direction, because without the publications it becomes much much harder to secure funding in the future.
The introduction of this article [1] gives an insight on the metric used in the Middle Ages. Essentially, to keep his position in a university, a researcher could win public debates by solving problems nobody else could solve. This led researchers to keep their work secret. Some researchers even got angry about having their work published, even with proper credit.
[1]: https://www.jstor.org/stable/27956338
Math is particularly susceptible to this because there are few legitimate publications and citation counts are low. If you are a medical researcher you can publish fake medical papers but more easily become “high impact” on leaderboards (scaled by subject) by adding math topics to your subjects/keywords.
I think some of this has to do with... resentment? You're this incredibly smart person, you worked really hard, and no one values you. No one wants to pay you big bucks, no one outside a tiny group knows your name even if you make important contributions to the field. Meanwhile, all the dumb people are getting ahead. It's easy to get depressed, and equally easy to decide that if life is unfair, it's OK to cheat to win.
Add to this the academic culture where, frankly, there are fewer incentives to address misbehavior and where many jobs are for life... and the nature of the field, which makes cheating is easy (as outlined in the article)... and you have an explosive mix.
The problem of AI generated papers is much more serious, although not happening on the same scale (yet!).
My take: I’ve published in well-regarded mathematical journals and the culture is definitely hard to explain to people outside of math. For example, it took more than two years to get my key graduate paper published in Foundations of Computational Mathematics, a highly regarded journal. The paper currently has over 100 citations, which (last I checked) is a couple times higher than the average citation count for the journal. In short, it’s a great, impactful work for a graduate student. But in a field like cell biology, this would be considered a pretty weak showing.
Given the long timelines and low citation counts, it’s not surprising that it’s so easy to manipulate the numbers. It is kinda ironic that mathematicians have this issue with numbers though.
The now-standard bibliometrics were not designed by statisticians :-)