Wednesday, February 24, 2016

Rejection and how to deal with it.


Rejection is a part of life – a big part. In science, it is a huge part.

For many scientists, publications are really the only criterion on which we – and our careers and contributions – are judged. We offer up our papers to journals and ask for them to be published. Often, the reviewers and editors say no. Such rejection can be a soul-crushing experience for new scientists, feeding the “imposter syndrome” and sometimes chasing them from science altogether. On this background, the current post will make several points. First, even established scientists are frequently rejected, and so a rejection should not be considered an indictment of your paper or your research ability or your future in science. Second, rejection sometimes leads to a better paper published in a better journal. Third, strategies exist to reduce the sting of rejection – hopefully to the point where it becomes only a minor annoyance.

In 2004, Cassey and Blackburn published a paper in BioScience titled “Publication and rejection among successful ecologists.” They surveyed 155 authors who had published at least 10 papers in core ecology journals (their criterion for “successful”), of whom 61 respond to a questionnaire. Of the 2907 papers published by those authors, 450 (15.5%) had been rejected at least once by a different journal and 224 (7.7%) had been rejected at least twice. In short, even the most successful ecologists have to deal with at least some rejection. And what of today, when rejection is – seemingly – so much higher, at least at journals that are not open access. Since no more recent study has repeated the survey of Cassey and Blackburn, I will have delve into my own experience (which might or might not be representative).

From Cassey and Blackburn (2004 - Bioscience).
I have submitted a total of 308 papers to peer-reviewed journals. This tally includes multiple submission of the same paper to different journals but not re-submissions to the same journal following “reject with the possibility of resubmission” or “reject without prejudice”. Of  those 308 total submissions, 144 (47%) were ultimately rejected from the journal without the possibility of further resubmission. If you count only the first submission of a given paper (to any journal), 71 out of 165 (43%) were rejected without the possibility of resubmission – a higher number than the above 15.5% experienced by “successful ecologists” in the 1990s. Perhaps my rejection rate is higher because the 2000s have seen higher overall rejection rates than did the 1990s (indeed, my first 6 papers in the 1990s were all accepted at the journal to which they were first submitted), perhaps I am submitting to journals with higher rejection rates (more about Nature/Science later), perhaps my work is less good than that of “successful ecologists”, perhaps rejection rates are higher for evolutionary biologists than ecologists, perhaps …

However, considering all 308 submissions might be misleading because some were notes/comments/responses, some were invited by editors, and some were submitted to special issues where I was an editor. Rejection was presumably less likely in such cases, so let’s remove them and recalculate. This removal leaves 251 total submissions of which 143 (57%) were rejected and of which 70 (60%) were rejected on their first submission. No matter what happens next, it is clear that a lot of my papers are rejected!

But wait, my coauthors and I often submit to general journals with very high rejection rates, such as Nature (~92%), Science (~93%), PNAS (~82%), Current Biology, and PLosBiol. Indeed, I have made 55 total submissions to the big five. All 6 reviews/comments/responses/News&Views/Dispatches/etc. were accepted, whereas only 1 of the 48 “real” submissions was accepted. It was my first such submission, subsequently followed by 47 straight rejections – and counting. Of course, I didn’t actually submit 48 separate papers to these journals, instead it was more like 15 or so papers submitted to multiple journals within the big five.

Rejection rates are generally higher at "high impact" journals. (From Aarsen et al. 2008. Open Ecology Journal.)
We might call these general journals the “big five” and assume they shouldn’t really count to the “normal” rejection rate for journals in my discipline. Removing them from the above already-winnowed list leaves 203 total submissions with 96 rejections (47%), 54% on their first submission. In short, I still have lots of rejections even within disciplinary journals in my field.

But, actually, the journal with the highest rejection rate (100%) also has the lowest impact factor (undefined).
Some other interesting tidbits come out of my rejection list. One paper was rejected from 8 journals (none of them in the big 5) before finally being published. It has since been cited a respectable 27 times. A number of others were rejected from 4 journals before we simply gave up or, more accurately, moved on –  and published them in PLoS ONE or Ecology and Evolution or the like. Interestingly, several papers were rejected from low-impact journals only to be later published in high-impact journals, including Ecology Letters.

Although the publishing journal is usually "lower impact" than the first submission, you can sometimes move up the scale. From Colcagno et al. (2012 - Science).
OK, so lots and lots and lots of rejection no matter how you slice it. What should we take from this outcome: that my research sucks or is at least – more charitably – not very important? I suggest not, for two reasons. First, my work (including several repeatedly rejected papers) is actually cited quite frequently and seems to have had some influence on the field. Second, having served as an editor for a number of journals over a number of years, it is clear that the primary reason for rejection is simply “fit.” For instance, the paper isn’t thought to be of general enough interest to the readership of a journal, or the editor doesn’t think it can “compete with other papers for limited space in the journal.” Papers are almost never rejected simply because they are scientifically “bad” somehow, which is reflected in the much lower rejection rate of “pay-to-publish” open access journals like PLoS ONE (~30%) or Ecology and Evolution.

Some of my frequently rejected highly cited papers.
So, rejection is just a part of life – like stop signs and traffic lights –  that we just need to “get over” and get on with life. Easy enough to say but the reality is that rejection can hurt, especially for a young scientist. So can one find a way to deal with rejection so it becomes a mosquito buzzing in the background rather than a wasp stinging us?

My own way of dealing with rejection is to be pre-emptive: I merely assume even BEFORE submission that the paper will be rejected wherever I am submitting it to. Thus, I make a short list of the sequence of journals to which I will submit the paper. That way, when the paper comes back with a rejection, I am just on the cusp of submitting to the next journal I had targeted. Of course, if the paper isn’t rejected, then all is well and I don’t have move down my list. In addition, I don’t read the reviews of a paper (rejected or otherwise) until I am actually sitting down ready to make revisions. Otherwise, I can stew over the “idiot reviewers” and capriciousness of the editors for weeks (see Butch Brodie's examples here) before I can get around to doing something about it. Of course, this is only the approach that works for me and I am sure that other approaches will work better for others.

Everybody's doing it - re-submission patterns among journals. From Calcagno et al. (2012 - Science).


Summary and closing thoughts

Rejection is something that all scientists have to deal in the currently scientific enterprise.

Rejection is not an indictment of the quality of your research or yourself as a scientist. By far the most frequently cause of rejection is simply that the paper isn’t considered a good fit for the journal but that it is publishable elsewhere in a “more specialized” journal. Indeed, the low rejection rates at “pay-to-publish” open access journals prove this point.

Frequent rejection means that you are not under-selling your work. Journals with the highest rejection rate are often the highest profile (highest impact – for better or worse)  journals.

Rejection rates tend to higher for scientists who publish more papers, which can be viewed as bad (minimum publishable units) or good (no publication bias).

Rejection doesn’t mean your paper is bad and that it won’t be cited. Indeed, many papers rejected from one journal go on to be highly cited at some other journal.

Rejection can be used to improve a paper, especially if you revise according to (useful) reviewer suggestions. This is generally a good idea anyway as it might well go to the same reviewer at the next journal. Indeed, this is common. And, once a paper is revised according to good suggestions, it can often be traded-up to an even better journal. I know of several people rejected from disciplinary journals who entirely revamped their paper and had it published at Nature/Science.

Importantly, rejections such as "reject with the possibility of resubmission" or "reject without prejudice" or "reject in present form" ARE NOT REJECTIONS. They are simply a way for the journal to reset the clock on you paper making it seems as though they have a shorter time from submission to publication. In fact, I would say that 90% of the papers that I have had with the above decisions on an initial submission were ultimately published in that journal. Thus, these apparent rejections should be considered "reconsider after revision" and a very careful revision should be made - here is my advice on responding to reviewers.

Rejection isn’t fun – but neither need it be paralyzing or depressing. It is simply a part of life and, like all such parts, can be made less painful and (sometimes) useful. Good luck.

Or you could just submit to my new journal


8 comments:

  1. Andrew, what's your record for rejections of a single manuscript before publication? I have one that was rejected by SIX different journals... which makes me both dumb and stubborn, I think!

    ReplyDelete
    Replies
    1. It is buried in the post - 8 rejections of one MS before publication - now cited 27 times! If you care that much about it to persevere through that many rejections, then you are probably right about it.

      Delete
  2. Everyone might enjoy this URL from Butch Brode:

    http://www.faculty.virginia.edu/brodie/ghosts.html

    I think may hold the record for rejections without review. I wrote a paper on success rates and on death rates of Everest mountaineers as a function of age and gender. I tried Nature, Science, PNAS, JAMA, PLoS, & Brit Med Journal. Again, all 6 rejected WITHOUT review.

    At that point I wrote to Brian Charlesworth (Editor of Biol Letters) and told him the full story. I was tired of reformatting, and decided just to be completely straightforward with him about the history of submissions. I wrote a careful justification of why I thought the paper was of general interest, even though the topic was obviously "off the wall." And asked whether he thought it would be suitable for Biology Letters.

    Brian wrote back and said he would give it a try. This time the ms. got two positive reviews and was published.

    Lessons learned: 1 ) be persistent if you think your paper is worth publishing, and 2) be straightforward with editors and write a good justification for the paper.

    ReplyDelete
  3. My record is 6 rejections before acceptance. Admittedly, when a paper is having a tough time I will go out of my way to find the next journal(s) with similar format to minimize time wasted with formatting. I have to admit that I am also loving the "your paper, your way" from Elsevier - it reduces wasted time if you have to offer the paper to several journals. Check it out... https://www.elsevier.com/authors/journal-authors/your-paper-your-way

    I don't think I "overshoot" very often but that is because I despise the Vancouver (numbered) reference format (which seems to be the norm with many top-tier journals) and will go out of my way to avoid journals that have that format. I also dislike reading papers with the Vancouver ref format because I am forced to constantly flip back and forth between text and ref list.

    ReplyDelete
  4. Completely agree on everything...just curious about why you put PLoS One and Ecology and Evolution in the same category. To my understanding, PLoS One has an explicit policy of only considering technical soundness but not "interest" while Ecology and Evolution doesn't have such policy (at least not explicitly, or I couldn't find it)...

    ReplyDelete
    Replies
    1. Yeah, surprising (disheartening) to think publishing in Ecology or Evolution is "giving up".

      Delete
    2. Not "Ecology" or "Evolution", but "Ecology and Evolution"

      Delete
  5. I enjoyed your list of ‘frequently rejected highly cited papers’. Here are some examples from my own experience as a (co)author:

    # Tylianakis et al 2008. Published in: Ecology Letters. Previously rejected from: Nature, Science, Ecology Letters (rejected without review; successfully appealed). Citations: 859.
    # De Bello et al 2010. Published in: Biodiversity and Conservation. Previously rejected from: Frontiers Ecol Env, Ecosystems, Functional Ecology. Citations: 260.
    # Wardle et al 2003. Published in: Science. Previously rejected from: Global Change Biology. Citations: 185.

    ReplyDelete

A 25-year quest for the Holy Grail of evolutionary biology

When I started my postdoc in 1998, I think it is safe to say that the Holy Grail (or maybe Rosetta Stone) for many evolutionary biologists w...