data-mining.philippe-fournier-viger.comThe Data Blog | A blog about data mining, data science, machine learning and big data, by Philippe F

ournier-Viger The Data Blog A blog about data mining, data science, machine learning and big data, by Philippe Fournier-Viger Home About the author ← Older posts How to deal with unethical reviewers? The good example of the EAAI journal Posted on 2024-05-13 by Philippe Fournier-Viger Today, I will talk about a problem that has been plaguing many academic journals in recent years. It is that several unethical reviewers are asking authors to cite several of their papers to boost their citation count . I previous wrote a few blog posts about this: post1 , post2 , post3 but today, I want to talk about solutions . I see this problem happening very often in academic journals, especially as an author. In some case, I for example, received reviews for a journal paper where the reviewer was asking to cite 10 papers (!!!) by the same author, which were very remotely relevant to the topic. In this case, it is obvious that the reviewer is just trying to boost his citation count to game the system, and perhaps obtain a promotion or something else. Usually, when this happen, I will complain to the editor because this behavior is unethical, and I dont want add useless citations in my paper, which would degrade the quality of the paper. But I know several authors that are afraid of such unethical reviewers and that will just do what the reviewers ask and add the citations rather than reporting them for their unethical behavior. But I also understand the authors who do not want to report this problem because the reviewers are in a position of power. So it is hard to fight this problem, and sometimes the editor will not even pay attention to those reports. So how to handle this problem? I think that a good solution is to follow the approach of the EAAI journal , and this is the topic of this blog for today. I want to praise that journal for the form that they provide to reviewers. The first question of that form is as follows: In that question , the reviewer is required to clearly indicate all the references that they are asking the authors to cite and to provide the full author list and DOI of each reference. This is important because many unethical reviewers will ask to cite papers but remove their names from the author list, hoping that the editor would not detect the conflict of interest. The second question of the review form ask the reviewer to clearly indicate any citations from themselves and the reason for asking to cite such papers. There is also a clear warning that this is generally considered unethical. These two questions in the review form are very simple, but they show that the journal cares and this form tells the reviewers to stay away from this type of unethical behavior. It can certainly have a dissuasive effect. I think that this is a good approach and other journals should take this as an example and do the same. I know that some other journals do this already but still many journals do not, and this problem is still very common in academia. And also, a major issue is that several editors just don’t pay attention to this problem as they are sometimes handling hundreds of papers. Thus, some editor will just read the reviews from reviewers very quickly and will not see the problem. And it also happens in some cases that the editor is the problem itself. I have ever seen some suspicious behavior where reviewers where asking me to cite papers from the handling editor. In this case, it is rather obvious that the editor is editing the reviews from reviewers to boost his citation count or the impact factor of his own journal! So that is all for this post. I just wanted to praise the approach adopted by the EAAI journal as a good example of how to deal with unethical reviewers by adding questions to the review form. Do you have any other suggestions? You may share them in the comment section below. — Philippe Fournier-Viger is a computer science professor and founder of the SPMF open-source data mining library , which offers more than 170 algorithms for analyzing data, implemented in Java. Related posts: My paper is rejected. Can I appeal the editor’s decision? Unethical reviewers in academia (part 3) How to become a well-known researcher? Posted in Academia | Tagged academia , bad reviewer , cheating , citation , citation count , editor , journal , reviewer , unethical editor , unethical reviewer | Leave a comment CSRankings: still a biased ranking Posted on 2024-05-13 by Philippe Fournier-Viger Today, I will write about a ranking called CSRankings , which is used by some people to rank computer science programs from different universities. That ranking has some interesting aspects such that its code and data is open to the public, but it is in my opinion biased for several reasons. I previously wrote a blog post about the shortcomings of CSRankings . Some shortcomings are that it only considers conference papers , while journal papers are more important in several countries, and that CSRankings is US-centric in its methodology. But a bigger problem, as I explained last time, is that CSRankings is biased towards some subfields of computer science . For example, that ranking has a category for robotics with some conferences such as ICRA that have a very high acceptance rate (~49%), while some other categories have conferences with a much lower acceptance rate (~10%), and some subfields of computer science are basically ignored such as data mining . About data mining , in my previous blog post, I pointed out that the KDD conference , which is arguably the #1 conference in data mining with an acceptance rate of around 10%, was deactivated by default in CSRankings. Here is a screenshot: This is quite baffling given that KDD is highly regarded, even with top companies and universities regularly publishing in this conference. Other data mining conferences are also omitted from that ranking. So basically, we could say that the field of data mining does not exist in that ranking, and that would be somewhat true. There is a category called database” but it also excludes ICDE by default. On the Github page of CSRankings, several people have complained that KDD has been removed from CSRankings, as obviously several people are unhappy with this (source: github.com/emeryberger/CSrankings/issues/5397): Some other supporting messages are as follows: There was a message from one of the CSRankings owner announcing that this might change: However, more than a year has passed and nothing has changed. I wrote this blog post to raise awareness about this i ssue with CSRankings . So how to fix it? In my opinion, the best solution would be to add adata mining ” category with KDD and ICDM at least. Then, it would be more fair for the data mining community, which has been thriving for almost three decades. Data mining should not be wiped-out just like that from a ranking. Or another solution would be that someone clone the code and data of CSRankings (since it is open-source) to start a more fair ranking that would fix these problems because CSRankings might not fix them. Of course, no ranking is perfect, but I think that some obvious improvements could be done in this case. Who is willing to do it? That is all for today. By the way, note that the content of this blog post represents my personal opinion. Related posts: How to write a research grant proposal? A Tool to Generate Reviews of Academic Papers What are the milestones in the career of an academic researcher? Posted in Academia , General | Tagged academia , computer science , computer science ranking , csrankings | Leave a comment The story of the most influential paper award of PAKDD 2024 Posted on 2024-05-12 by Philippe Fournier-Viger Recently, I have attended the PAKDD 2024 conference, where I was happy to receive the most influential paper award with my co-authors. This award is a test of time type of award that is given to the paper from PAKDD 2014 that received the largest number of citations or had the...