Researcher blogs

Sometimes, I’m a reviewer.

As a researcher-in-training, I’ve learned about the flaws of the peer-reviewed-publication-impact-factor system almost at the same time I’ve learned of its existence. Submission and reviewing process can end up being more detrimental than beneficial for the quality of research. At the end of my PhD I became convinced, and I still am, that the sadly holy impact factor and publication rate are terribly biased measures of “success”.

The reviewing process itself can be a gigantic loss of time. You know what I’m talking about: rejection because a specific condition is missing, even though this condition actually was included in the study. What about the “it is not interesting” statement, without any scientific arguments, just not interesting, how do you respond to that? And don’t you like when Reviewer 1 loves your introduction but reviewer 2 thinks it’s the worst you could have done (always remember to appreciate the nuanced comments). I could mention the “why haven’t you looked at this question instead”, well maybe because it was not my starting point? And let’s not even talk about the tone of some reviewers, the delicious “I know everything, you know nothing”. How is this making any research better?

So, peer review = bad?

Yet, I’ve been spoiled with awesome supervisors, collaborators, and mentors (some of them were even women, hi Nature Communication…). People for whom the ‘scientific community’ was not a vain word. Pedagogues for whom supervising a PhD student is a real mission involving transfer of knowledge, skills, and good practices, not only about a specific scientific topic, but also about the research world. Colleagues who will not count their hours to help and even train other researchers without anything in return. Brilliant scientists that opened my eyes on professional, ethical, political, and social questions in our line of work, never giving dogmatic answers, always sharing their thoughts. Simply researchers I look up to. Having these people and other like them assessing my work could only improve said work. And it did! I received reviews that objectively made my work better, as it should do. Discussion points that I had overlooked were now discussed, suggested references brought new insights to my studies, limitations and strong elements of my different papers are better defined, my work is simply more readable and reliable after an outside eye looked at it. 

So, peer review = good?

These reflections on what it means to be part of a publication-based research community continued for me as a postdoc when speaking with my colleagues, inside my lab and out. With time, I am becoming more and more experienced with the manuscript submission path through hell process. Naturally, I also review more and more papers. And I feel terrible as a reviewer. Impostor syndrome, lack of time, and the archaic “I’ve been through this so everyone should too” tendency we all share make a terrible mix. Every time I am full of low-esteem related questions (did I understand right? Should I read all the references used in the manuscript to be sure I’m on the same page as the authors? Do my points even make sense? Is it because I’m stupid that I don’t agree with their point?) and some too much ego, misplaced pride questions (why did they not quote my work? Their results are not newer or stronger than mine, why should they publish higher than me?). Yeah, I’m not a good person. But I still want to be a good colleague, a good peer.

So how do I combine all of this? That is, my strong belief in a welcoming community of researchers to help produce better work, with my desire to see such an unfair system collapse. Be the change you want to see, they say, but what about when you are an early career researcher, good enough but not a shining star, not influential enough, probably too idealistic and rather lazy? I don’t want to be all mouth no guts, but I could do without torpedoing my career. I’m not naïve enough to imagine I can change the whole system on my own, but where do I start? Well, baby steps. I’ve recently decided to follow some principles based on my own instinct, my experience as an author and mostly some great insights collected here and there among researchers I respect and admire. Easy to implement, they make me feel okay with myself as a reviewer, even if sometimes it’s hard and scary to follow them.

1. Reviewing is opening a discussion.

When I receive a paper to review, I want to have the same mind set as when a labmate ask for my advice on a project or a study. How can I use what I know to make this study better? I want to point out what I think are the main findings as well as the biggest limitations. If I disagree with a statement or an analysis, I will always consider that I may have misunderstood. If it is the case, I may not be the only one so how about rephrasing? If I understood right, it is my responsibility to justify why I think it is wrong and to propose a correction. I will always leave the possibility to the authors to respond and justify their choices, if they disagree with me. This means that unless I see a big problem in the way data have been acquired, I will never suggest rejection. What is the point of rejection if the paper is just sent to another journal with none of the improvements I think would have been important? Did I “help science” then? I will make myself available to go to the end of the reviewing process, to contribute as much as I can to the improvement of the manuscript before publication. Of course, I hope the authors will understand my point of view and respond to it. I cannot control what other people do, I can control how I behave. After a few reviews following this principle, I realized that an unexpected consequence is that I spend more time on a review. I think it is necessary and I also feel more comfortable with refusing to review.

2. I am responsible for my comments.

This mainly means that I sign my reviews. Even when it is expected that I am anonymous. Even if I am terrified that the authors will judge me and think I am stupid and did not understand their work. Even when among the authors, there may be influential people in my field. I spend time on my review, I respect the authors, I try to be mindful of how I say things in addition of what I say. I am doing my best because I am not an enemy, I am a temporary collaborator trying to check if everything looks legit and solid. I can only wish this is visible in my comments, so I sign them. Wish me luck!

3. I am not taking into account the journal.

Editors be warned, I don’t care for your impact factor. I do not believe impact factor is a good way to assess individual research, so I’m not going to judge if a study is worthy of something I don’t recognize (also, you don’t pay me, so I do what I want). This is controversial I imagine, because I can see some counter arguments. Journals cannot publish everything, of course, but that’s an editor’s problem. As a reviewer, my main concern is “is this good science?”. We are often asked to assess the novelty. Yet, more and more concerns are raised about the replication crisis. I choose to consider novelty as a good point but not above a solid method or a strong replication. Of course, not every work provides the same contribution to its field, not every finding is ground breaking, but that’s the beauty of it! We all know examples of awesome studies that are published “low” (check out mine obviously!) and we are aware of factors that can help publish “high” that have nothing to do with rigorous scientific work (gender of the first authors, number of “high” references, the list can go on forever). How is it helpful to maintain a system I don’t believe in? I also like this principle because it keeps at bay the little voice that makes me feel envious or just sad every time I review a paper in a journal much higher than my usual publication level. I am really not proud of this, but I’m trying to be better and this principle helps.

Other people choose to stop reviewing, to publish only in open access journals, to ask for compensation when reviewing. There’s probably a lot of other initiatives or self-applied rules that we don’t know about. I’m sure lots of people take a stand at their level. I don’t think my way is better, I’m just sharing my point of view. My little principles are insignificant in regards to the global situation and issues with publication, grant and funding, positions availabilities, and other challenges. A whole new system needs to be proposed, but who’s got time for this when your contract/PhD/grant (delete where applicable) finishes at the end of the year?

Meanwhile, here are the principles I choose to follow. I’m sure they will evolve as I keep meeting incredible people along the way, and I am looking forward for this.