My year in peer review

It’s been just over a year since I have been serving as an Associate Editor for Biodiversity & Conservation. It is a great opportunity that I am guessing very few people so junior in our career path would have had–Editor-in-Chief David Hawksworth remarked the same when I sent in my CV–and the opportunity came not because I’m a star scientist or anything close to that. More later.

Being involved in the editorial processes for the first time, as with being asked to review an article for the first time, is an eye-opener (an Achievement unlocked! kind of feeling). Aside from the professional duty and honour of being able to contribute to the peer review process that is so crucial in science, being a reviewer and then later an editor makes our own work better, because you get to see the assessment of the quality of scientific work from a different viewpoint–a sort of bird’s eye viewpoint.

So much so that the earlier a student/practitioner of science gets involved in each subsequent stage of the process, the better it is for his/her own development, I feel.

How many B.Sc’s graduate without ever being aware of the pivotal role of peer review in science? And because these graduates go on into jobs like in the public sector, it is no wonder that government agencies that fund and use science are unempathetic of the scientific process. So undergraduate science courses should start incorporating some kind of peer review into their curriculum, at least at the advanced undergraduate level, and teach that this is part of how science is made. For example, the late Navjot Sodhi made us peer-review each other’s individual behavioural ecology essay, with himself as a mock editor; Pete Todd did the same for his marine biology module, although it was group work. Both were 4th-year undergraduate courses. I think this is about second or third in terms of importance+lacking in our undergraduate curriculum, next to generating research questions and designing experiments.

At senior postgraduate level, students should start getting professionally involved as peer reviewers, especially since they would be among the most up-to-date in the literature in their chosen subfield. The gatekeepers to this step are their advisors and department faculty, many of whom are likely to be on editorial boards or would regularly receive review requests. The earlier these gatekeepers rope postgraduate students into the process, the better it is for developing/motivating them as science workers.

The trade-off with involving science workers as early and as broadly as possible in scientific peer review is the requisite for an approriate level of knowledge and experience/ability to critically assess a piece of work. That’s why we have gatekeepers.

An observation that Giam Xingli shared with me, and that I think is true, is that early career researchers like postgraduate students and post-docs tend to be far more critical and pedantic than old hands in the field. So that’s another trade-off to bear in mind.

There have been recent articles calling for the same thing about graduate students as reviewers and how to do your first peer review, so I’m not going to repeat all that. But a perspective of the interface with higher up in the editorial process is provided less often.

Here at CEED where I’m visiting as a post-doc until 2017, there are regular seminars organised by and for early career researchers, and in November there was one on peer review, just right after I was musing about a blog post to review my first ever year in an editorial role. I only had time to write about it now, during the year end holidays.

CEED is a lucky place for me to be in: for the seminar we had Michael Bode (Journal of Applied Ecology), Eddie Game (Conservation Letters), and Andrew Knight (Conservation Letters) to comment on best practices and observations of peer review from their positions on various journal editorial boards, and Liz Law sharing from an early career researcher’s point of view as a peer reviewer. Katrina Davies helped Liz to facilitate and organise it. Liz wrote a blog post before the discussion [and I believe Liz and Katrina would write up something from the seminar for CEED’s Decision Point newsletter soon enough]. My own notes from the seminar:

  1. New reviewers shouldn’t be afraid to ask for feedback from the handling editor that invited them. I would vouch that I would be happy to give such feedback–I once solicited a review from a close former colleague who should be considered an expert in that subject but he was wary of making mistakes because it was his first time so I walked him through it with our resources I’ve found before (see here and here), although I didn’t provide any comments specific to the manuscript.
  2. Too-long and too-short reviews are both problematic. It should be like Goldilock’s: just right, just enough. But while the panel suggested that reviewers shouldn’t go through the manuscript with a grammatical tooth comb and should leave copy-editing to the publishers or return it to the authors, I think they might be speaking from the position of very prestigious journals. Many mid- to lower-end journals do not have the luxury of doing so, and so the peer reviewer who can help out with improving communicability in detail is doing them a great service.
  3. The handling (or in some places called the associate) editor role is the most pivotal.
  4. Reviewers get better by doing more. I think this applies to anything else in life.
  5. “Every review is a job application.” I like this one. Peer reviews should be done conscientiously. I think that’s how I bagged the opportunity to serve for Biodivers & Conserv. I took every peer review assignment I got very seriously (maybe a bit too much–when you’re spending a whole day writing a review, which is usually the case for newbie reviewers, you tend to turn in too-long reviews), doing it in the spirit that I want it to help the authors in the same way I want reviewers to help me in my submitted manuscripts. I turned in two Biodivers & Conserv reviews for Wong Khoon Meng. I think the conscientiousness of my reviews was why Khoon Meng was so kind to ask David Hawksworth if he would consider letting me help out on the editorial board. I never verified this, though, but judging from what the panel were saying it is indeed one way that they look for new handling/associate editors. So if you want a higher chance of helping out in that most pivotal part of the process sooner, do your peer reviews well!
  6. Different people in the editorial process are concerned with different things:
    • The top guys, the chief editors, identify issues such as fit within the journal’s remit. That’s why the cover letter, the abstract, and things that can be assessed at a glance, like figures/tables, must communicate this fit. For the really competitive journals, the abstract and figures must therefore be “compelling”.
    • The handling editors “identify bullshit”, so-to-speak. Unless the chief editors happen to be also familiar with the topic that particular manuscript is addressing, this is why handling/associate editors are playing a pivotal role. But the handling editors also rely on the advice of the reviews they are trying to solicit.
    • The main role of the reviewers is two-fold: to advise the handling editors on whether the paper is technically sound, while at the same time providing details to the authors on how to improve the paper. This is the whole point of peer review: more a conversation that helps to improve the science, less to judge the publishable value of the work. While the reviewers can provide their opinion, usually discreetly to the editors, gatekeeping is the role of the editors.

So how have I done since September 2014?

As of today, I have handled six submissions. Apparently this is less than what their average associate editor handles in a year, but I have not much control over that. I only turned down one request to help out in a review article, but that was really out of my knowledge domain for me to assist meaningfully. On the other hand, I’ve seen a few other articles newly published that I would have liked to handle, but were given to other associate editors. Dang!

I recommended that two for rejection without sending to review; one was recommended for rejection after review. Two eventually were accepted. I recommended one for consideration after major revisions, but the higher-ups decided to phrase it as a reject with a possibility of resubmission.

I haven’t really asked David, or the Managing Editor Dirk Schmeller for permission to blog about this, so I hope the above is alright. I will definitely not give any details about those that I rejected. But the two that were accepted have already been published, and you can see a line saying “Communicated by [name of associate editor]” on the first page of the articles, so this is in the open domain. Unlike Nature/Science/PNAS, our humble Biodiversity & Conservation doesn’t have commentaries accompanying papers to help boost their accessibility. Having helped in their birthing process, I’m a proud midwife, so present them here briefly I will:

  • Marc-Oliver Adams & Konrad Fiedler. 2015. The value of targeted reforestations for local insect diversity: a case study from the Ecuadorian Andes. Biodiversity & Conservation 24:2709.

This assignment came after I wrapped up a project on the use of native plants in urban landscaping. One of the benefits of using native plant species versus non-native species that is emerging to be quite prominent thanks partly to the efforts to communicate it in accessible science by people like Douglas Tallamy is (1) how many insect herbivores are specialists, depending on native species with which they have coevolved for food and (2) most birds rely on insect protein, even if they are normally considered frugivores or nectarivore, etc., at least during the breeding season. Adams & Fiedler investigated if diversity and turnover of groups of insect herbivores still depended on the surrounding landscape/habitat mattered when you reforest with native plant species.

  • Chistos Mammides et al. 2015. The indirect effects of habitat disturbance on the bird communities in a tropical African forest. Biodiversity & Conservation 24:3083.

A lot of correlative work on human impacts on biodiversity assumes direct effects between the correlates/measures of human impacts and the response. However, if we think about it mechanistically, many of these human impacts work through their effects on intermediates. If we intervene/make decisions without considering if the causal pathways were indirect, we may be surprised when things don’t turn out the way we expected them too. Here Mammides et al. used Structural Equation Modeling to derive a more explicitly mechanistic process of how proxy measures of “human disturbance” affects bird abundance and richness through sensible intermediaries such as vegetation composition and structure.

What have been my key priorities when I tried to assess submissions? Obviously, Biodivers & Conserv while reputable is not Nature or Science. Therefore things such as “novelty” that frustrate many of us when we get rejection letters of our own from editors is definitely not a factor. In fact, journals like Biodivers & Conserv have an important role to play in the ecosystem of scientific publishing: to publish “good work” that otherwise might never see the light if only hypercompetitive journals exist.

So what is “good work”? I think good work is (1) scientifically rigorous  that (2) is written clearly so that other people can understand its results easily and replicate the work. So with the help of the reviews that come in, I concentrate on whether the Methods have provided sufficient details, the Results have reported everything that is necessary and appropriately for use by the larger community, whether the discussions and conclusions are overreaching relative to the shortcomings of the study. My recommended rejections are eventually because the paper has failed to achieve its written objectives, and need to be rewritten completely. But as long as it has some potential to reach publication without complete re-writing, I try my best to give it another chance, supplementing it with a detailed read-through and comments of my own that I hope will improve it further. However, I will not let one go through if I think it has substandard analyses or reporting; if necessary I will wear the authors down through repeated requests for revisions, even while I continue to believe in the manuscripts’ potential value.

Perhaps that’s foolish of me, because it consumes my time more than it would be to just recommend a rejection, e.g., for an author that doesn’t really seem to get it in the revisions. But we’re all authors ourselves, and we know how we want others to give us a chance for the work we’ve put effort into.

It’s been a great and humbling learning experience as promised by David, and I look forward to doing more. Meanwhile, it would be great if back in Singapore/NUS/our department we have a similar seminar to help acquaint more junior researchers with the publishing process, just as CEED did. I’m sure it would help with our group’s overall research output.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: