My year in peer review 2016

It’s been another year of trying to be a useful member of the scientific community.

For the most part of this year, I handled less manuscripts than I did last year. Part of the reason could be that the editorial board of Biodiversity & Conservation underwent a little bit of restructuring. Dirk Schmeller used to be the only Managing Editor helping out Editor-in-Chief David Hawksworth, but now there are six “Senior Editors” including Dirk. All the Senior Editors will sift through received manuscripts according to their areas of expertise before rejecting them directly, handling them on their own or distributing them to the Associate Editors.

Or it could be just that there are periodic lulls in the submission volume. It certainly seems like it now, with increasing frequency of manuscripts coming my way and multiple requests for reviews from other journals.

So far I have completed the handling of four, two of which were rejected before and two after review, so there is no midwifery to crow about yet. It was particularly hard to find reviewers for one of them: on and off the record I approached at least 11 people, probably 13. I can’t remember exactly how many, but it included some contacts with the relevant expertise that I texted or emailed but were too busy (or humble!) to take it up. Finally and suddenly two kind souls agreed and I had three reviewers for the manuscript.

I also signed up for a webinar by the British Ecological Society (BES) on “Becoming an Associate Editor”, not because I was hungry for another role, but because I was curious about the experiences that would be shared by the panel of speakers, who are Associate Editors of BES journals: Luca Borger from Journal of Animal Ecology, Jane Catford of Journal of Ecology, and Stephanie, whose last name I didn’t catch but I am guessing is Januchowski-Hartley because she mentioned she recently wrote an Editor’s Choice and I searched for it, of Journal of Applied Ecology. I am, relatively speaking, a noob and there’s not much opportunity for training, so hearing from others would be as good as it gets.

It was comforting to hear that my 11+ reviewer declines is not that extreme: Jane said that (I think it was) just the week before that she had been turned down some 17 times before getting her second reviewer for a manuscript. Well, not really comforting, because it really tells us how much strain the peer review community is under to process the ever-increasing volumes of manuscripts submitted.

Another interesting takeaway was that manuscript submissions peak twice a year, once just before the northern hemisphere summer, and one just before the year-end festive season. This arises from authors in Europe and North America trying to clear their desk before going away for their holidays. It’s also the most difficult time for editors to get reviewers because the reviewers disappear for holidays too! Sure explains the uptick I’m experiencing now. So if you want your manuscript to be reviewed quicker, try to avoid submitting them during these times.

During the webinar, I asked the panel a question of my own through the chatbox: “One of the functions of peer review is also to provide feedback to authors. In cases of rejecting without going out to review, would you [provide] authors provide some in-depth comments beyond misfit with scope or novelty, etc.?”

I asked this question because I do so myself. If I am disposed to reject a manuscript without review, I provide my own comments from my reading of the manuscript. That way, the authors sort-of get at least one review, without which how else would they know what concrete steps can they take next to get it published elsewhere? Luca replied that he, too, tries to give some comments when he decides a manuscript can’t go out for review. It’s good to know.

It was my first webinar. I set an alarm to wake up at midnight for it, and it was worthwhile.

I still have two manuscripts on hand being processed, but the final decision will almost certainly be in 2017. An interesting thing happened with one of them. As usual, I searched for possible reviewers, found some, did some checks to make sure they didn’t co-author anything within 5 years (my personal guideline), edited the template invitation letter, and sent them out. One declined almost immediately with the following message:

“The last time I submitted something to you, it wasn’t even sent out for review, so I am not inclined to help you out.”

Oooh. That was harsh.

Some snooping around in the editorial system revealed that, indeed, two manuscripts submitted by this person in the past were rejected without review. We’ve all been on the receiving end of such editorial decisions, so I can sympathise, but there was obvious lack of grace on this persons’ part here.

It’s tough. The strain caused by multiple rounds of reviews in different journals is the key reason for rejections without review. If an editor finds that a manuscript probably wouldn’t survive peer reviewer comments, i.e., it would most probably get rejected, then there’s no reason to burden reviewers with it, is there? Like the one that I had problem finding reviewers for: when the reviews came back and I read the manuscript in detail, I kinda regretted sending it out for review and “wasting” the time of three reviewers–if I had examined its Tables in detail before sending it out, I would have realized the fatal flaws immediately. Although again, part of the service of peer-review is to provide feedback and in this sense all reviewer comments were delivered to authors with my own so there is no waste to speak of.

As an author myself, it’s been almost a full year of rejections as well, right until the last one month when finally two manuscripts came through with good news. (More on them next time.)

The idea was to take advantage of this Fellowship to write papers like a madman, and for the first few months it almost worked like that.

Then the rejections started coming in, and every rejection set you back at least a few weeks to deal with reformatting. If there were reviews, you had to deal with the comments the best you could before resubmitting. Every round of reformatting consumed precious pockets of your best productivity time-slots and knocks your next manuscript back.

It’s not just my own first-authored manuscripts, but also all those that I was co-author of. I’ve had to console/encourage a graduate student from the old lab who led two manuscripts with multiple rejections, with and without review.

I’ve been rejected twice by PLoS ONE without review! Whoever gets rejected by PLoS ONE without review??

That said, I’m now glad I’ve broken my two year drought of having no ISI-listed first-author papers. The ones just accepted won’t make it to a print issue until next year, so 2015 and 2016 are the drought.

But I’m optimistic for 2017. A bird in the hand is worth two in the bush; an accepted paper is worth ten that’s “in review”, but a manuscript in review is worth ten that’s “in prep”, and a draft in prep is worth ten that’s not. As long as there’s a draft, there’s hope; eventually it will get published somewhere.

So here’s to more drafts in prep!

My year in peer review

It’s been just over a year since I have been serving as an Associate Editor for Biodiversity & Conservation. It is a great opportunity that I am guessing very few people so junior in our career path would have had–Editor-in-Chief David Hawksworth remarked the same when I sent in my CV–and the opportunity came not because I’m a star scientist or anything close to that. More later.

Being involved in the editorial processes for the first time, as with being asked to review an article for the first time, is an eye-opener (an Achievement unlocked! kind of feeling). Aside from the professional duty and honour of being able to contribute to the peer review process that is so crucial in science, being a reviewer and then later an editor makes our own work better, because you get to see the assessment of the quality of scientific work from a different viewpoint–a sort of bird’s eye viewpoint.

So much so that the earlier a student/practitioner of science gets involved in each subsequent stage of the process, the better it is for his/her own development, I feel.

How many B.Sc’s graduate without ever being aware of the pivotal role of peer review in science? And because these graduates go on into jobs like in the public sector, it is no wonder that government agencies that fund and use science are unempathetic of the scientific process. So undergraduate science courses should start incorporating some kind of peer review into their curriculum, at least at the advanced undergraduate level, and teach that this is part of how science is made. For example, the late Navjot Sodhi made us peer-review each other’s individual behavioural ecology essay, with himself as a mock editor; Pete Todd did the same for his marine biology module, although it was group work. Both were 4th-year undergraduate courses. I think this is about second or third in terms of importance+lacking in our undergraduate curriculum, next to generating research questions and designing experiments.

At senior postgraduate level, students should start getting professionally involved as peer reviewers, especially since they would be among the most up-to-date in the literature in their chosen subfield. The gatekeepers to this step are their advisors and department faculty, many of whom are likely to be on editorial boards or would regularly receive review requests. The earlier these gatekeepers rope postgraduate students into the process, the better it is for developing/motivating them as science workers.

The trade-off with involving science workers as early and as broadly as possible in scientific peer review is the requisite for an approriate level of knowledge and experience/ability to critically assess a piece of work. That’s why we have gatekeepers.

An observation that Giam Xingli shared with me, and that I think is true, is that early career researchers like postgraduate students and post-docs tend to be far more critical and pedantic than old hands in the field. So that’s another trade-off to bear in mind.

There have been recent articles calling for the same thing about graduate students as reviewers and how to do your first peer review, so I’m not going to repeat all that. But a perspective of the interface with higher up in the editorial process is provided less often.

Here at CEED where I’m visiting as a post-doc until 2017, there are regular seminars organised by and for early career researchers, and in November there was one on peer review, just right after I was musing about a blog post to review my first ever year in an editorial role. I only had time to write about it now, during the year end holidays.

CEED is a lucky place for me to be in: for the seminar we had Michael Bode (Journal of Applied Ecology), Eddie Game (Conservation Letters), and Andrew Knight (Conservation Letters) to comment on best practices and observations of peer review from their positions on various journal editorial boards, and Liz Law sharing from an early career researcher’s point of view as a peer reviewer. Katrina Davies helped Liz to facilitate and organise it. Liz wrote a blog post before the discussion [and I believe Liz and Katrina would write up something from the seminar for CEED’s Decision Point newsletter soon enough]. My own notes from the seminar:

  1. New reviewers shouldn’t be afraid to ask for feedback from the handling editor that invited them. I would vouch that I would be happy to give such feedback–I once solicited a review from a close former colleague who should be considered an expert in that subject but he was wary of making mistakes because it was his first time so I walked him through it with our resources I’ve found before (see here and here), although I didn’t provide any comments specific to the manuscript.
  2. Too-long and too-short reviews are both problematic. It should be like Goldilock’s: just right, just enough. But while the panel suggested that reviewers shouldn’t go through the manuscript with a grammatical tooth comb and should leave copy-editing to the publishers or return it to the authors, I think they might be speaking from the position of very prestigious journals. Many mid- to lower-end journals do not have the luxury of doing so, and so the peer reviewer who can help out with improving communicability in detail is doing them a great service.
  3. The handling (or in some places called the associate) editor role is the most pivotal.
  4. Reviewers get better by doing more. I think this applies to anything else in life.
  5. “Every review is a job application.” I like this one. Peer reviews should be done conscientiously. I think that’s how I bagged the opportunity to serve for Biodivers & Conserv. I took every peer review assignment I got very seriously (maybe a bit too much–when you’re spending a whole day writing a review, which is usually the case for newbie reviewers, you tend to turn in too-long reviews), doing it in the spirit that I want it to help the authors in the same way I want reviewers to help me in my submitted manuscripts. I turned in two Biodivers & Conserv reviews for Wong Khoon Meng. I think the conscientiousness of my reviews was why Khoon Meng was so kind to ask David Hawksworth if he would consider letting me help out on the editorial board. I never verified this, though, but judging from what the panel were saying it is indeed one way that they look for new handling/associate editors. So if you want a higher chance of helping out in that most pivotal part of the process sooner, do your peer reviews well!
  6. Different people in the editorial process are concerned with different things:
    • The top guys, the chief editors, identify issues such as fit within the journal’s remit. That’s why the cover letter, the abstract, and things that can be assessed at a glance, like figures/tables, must communicate this fit. For the really competitive journals, the abstract and figures must therefore be “compelling”.
    • The handling editors “identify bullshit”, so-to-speak. Unless the chief editors happen to be also familiar with the topic that particular manuscript is addressing, this is why handling/associate editors are playing a pivotal role. But the handling editors also rely on the advice of the reviews they are trying to solicit.
    • The main role of the reviewers is two-fold: to advise the handling editors on whether the paper is technically sound, while at the same time providing details to the authors on how to improve the paper. This is the whole point of peer review: more a conversation that helps to improve the science, less to judge the publishable value of the work. While the reviewers can provide their opinion, usually discreetly to the editors, gatekeeping is the role of the editors.

So how have I done since September 2014?

As of today, I have handled six submissions. Apparently this is less than what their average associate editor handles in a year, but I have not much control over that. I only turned down one request to help out in a review article, but that was really out of my knowledge domain for me to assist meaningfully. On the other hand, I’ve seen a few other articles newly published that I would have liked to handle, but were given to other associate editors. Dang!

I recommended that two for rejection without sending to review; one was recommended for rejection after review. Two eventually were accepted. I recommended one for consideration after major revisions, but the higher-ups decided to phrase it as a reject with a possibility of resubmission.

I haven’t really asked David, or the Managing Editor Dirk Schmeller for permission to blog about this, so I hope the above is alright. I will definitely not give any details about those that I rejected. But the two that were accepted have already been published, and you can see a line saying “Communicated by [name of associate editor]” on the first page of the articles, so this is in the open domain. Unlike Nature/Science/PNAS, our humble Biodiversity & Conservation doesn’t have commentaries accompanying papers to help boost their accessibility. Having helped in their birthing process, I’m a proud midwife, so present them here briefly I will:

  • Marc-Oliver Adams & Konrad Fiedler. 2015. The value of targeted reforestations for local insect diversity: a case study from the Ecuadorian Andes. Biodiversity & Conservation 24:2709.

This assignment came after I wrapped up a project on the use of native plants in urban landscaping. One of the benefits of using native plant species versus non-native species that is emerging to be quite prominent thanks partly to the efforts to communicate it in accessible science by people like Douglas Tallamy is (1) how many insect herbivores are specialists, depending on native species with which they have coevolved for food and (2) most birds rely on insect protein, even if they are normally considered frugivores or nectarivore, etc., at least during the breeding season. Adams & Fiedler investigated if diversity and turnover of groups of insect herbivores still depended on the surrounding landscape/habitat mattered when you reforest with native plant species.

  • Chistos Mammides et al. 2015. The indirect effects of habitat disturbance on the bird communities in a tropical African forest. Biodiversity & Conservation 24:3083.

A lot of correlative work on human impacts on biodiversity assumes direct effects between the correlates/measures of human impacts and the response. However, if we think about it mechanistically, many of these human impacts work through their effects on intermediates. If we intervene/make decisions without considering if the causal pathways were indirect, we may be surprised when things don’t turn out the way we expected them too. Here Mammides et al. used Structural Equation Modeling to derive a more explicitly mechanistic process of how proxy measures of “human disturbance” affects bird abundance and richness through sensible intermediaries such as vegetation composition and structure.

What have been my key priorities when I tried to assess submissions? Obviously, Biodivers & Conserv while reputable is not Nature or Science. Therefore things such as “novelty” that frustrate many of us when we get rejection letters of our own from editors is definitely not a factor. In fact, journals like Biodivers & Conserv have an important role to play in the ecosystem of scientific publishing: to publish “good work” that otherwise might never see the light if only hypercompetitive journals exist.

So what is “good work”? I think good work is (1) scientifically rigorous  that (2) is written clearly so that other people can understand its results easily and replicate the work. So with the help of the reviews that come in, I concentrate on whether the Methods have provided sufficient details, the Results have reported everything that is necessary and appropriately for use by the larger community, whether the discussions and conclusions are overreaching relative to the shortcomings of the study. My recommended rejections are eventually because the paper has failed to achieve its written objectives, and need to be rewritten completely. But as long as it has some potential to reach publication without complete re-writing, I try my best to give it another chance, supplementing it with a detailed read-through and comments of my own that I hope will improve it further. However, I will not let one go through if I think it has substandard analyses or reporting; if necessary I will wear the authors down through repeated requests for revisions, even while I continue to believe in the manuscripts’ potential value.

Perhaps that’s foolish of me, because it consumes my time more than it would be to just recommend a rejection, e.g., for an author that doesn’t really seem to get it in the revisions. But we’re all authors ourselves, and we know how we want others to give us a chance for the work we’ve put effort into.

It’s been a great and humbling learning experience as promised by David, and I look forward to doing more. Meanwhile, it would be great if back in Singapore/NUS/our department we have a similar seminar to help acquaint more junior researchers with the publishing process, just as CEED did. I’m sure it would help with our group’s overall research output.

More on peer review

I was browsing through the Bulletin of the Ecological Society of America (ESA)–one publication that I’ve almost never looked at because it’s not an outlet for the peer-reviewed research that would count towards our KPIs–when I read a couple of valuable articles:

1. About another “golden rule” for peer review:

If you wouldn’t say it in person, don’t say it in an anonymous review.

additional to that suggested in a 2009 editorial in the American Naturalist:

Review for others as you would have others review for you.

2. Whether rejections without review are justified. This article gather evidence to argue that editors making such rejections were not good judges of mismatch in scope or poor scientific quality, because most of the manuscripts rejected this way were eventually published in journals of similar subject category and academic repute.

This article argues that such rejections, which they call “evading reviews”, therefore do not serve the scientific community. For one thing, they felt that evading reviews were a missed opportunity for the authors to learn how to improve.

But I think that is not difficult to solve. The associate/subject editor should give at least a quick review, if not a full review, by themselves with comments to the authors when they choose to reject a manuscript without sending out for review. I think most do the quick review/comments. That way, the authors would always get to learn what are the shortcomings before revising and re-submitting elsewhere. Of course, stipulating such a rule, especially for a full review, would increase the burden on editors, but then it also serves as a “cost” that would make the editor think carefully before rejecting without review.

Update: A robust response from the editors at ESA’s family of journals.

More (free) books for science workers

The editors at the British Ecological Society put out two booklet “guides” a few weeks ago:

A Guide to Peer Review in Ecology and Evolution

A Guide to Data Management in Ecology and Evolution

At a time of increased workload on the peer review community and rapid growth in big, complex datasets, these are much needed.

Books for Christmas for the science worker

The publisher Wiley just advertised for Christmas about “essential titles no scientists should live without”.

“Scientists”, so must be very general topics. I clicked on the link and found titles that were really very interesting and in fact quite timely for me (as an early career researcher).

Two that are available from Wiley’s electronic library and NUS’s libraries have subscribed to them:

  • Prosanta Chakrabarty. 2012. A Guide to Academia: Getting into and Surviving Grad School, Postdocs, and a Research Job.
  • Irene Hames. 2007. Peer Review and Manuscript Management in Scientific Journals: Guidelines for Good Practice.

One more from the electronic library that NUS’s libraries haven’t subscribed to:

  • Sundar A. Christopher. 2011. Navigating Graduate School and Beyond: A Career Guide for Graduate Students and a Must Read for Every Advisor.

A book available as a physical copy in the NUS libraries:

  • Margaret Cargill, Patrick O’Connor. 2013. Writing Scientific Research Articles: Strategy and Steps.

And more that I would like to leaf through one day:

  • Alaina G. Levine. 2015. Networking for Nerds: Find, Access and Land Hidden Game-Changing Career Opportunities Everywhere.
  • Laura Bowater, Kay Yeoman. 2012. Science Communication – A Practical Guide for Scientists.
  • C. Philip Wheater, James R. Bell, Penny A. Cook. 2011. Practical Field Ecology: A Project Guide.