[3dem] sharing em maps during peer-review process

Sat May 2 16:11:56 PDT 2015

Hi,
   I agree with many of Niko's points, and thank him for his efforts in 
this regard that have been noticed by the community. But all journals 
published by professional societies (such as Biophysical Journal) are 
edited by active scientists, and not professional editors, so eLife is 
not unique in that regard. I have found that eLife's claim to 
transparency is farcical. In my recent eLife paper on Helical 
Ambiguities, the reviewer's comments that are posted are very far from 
what I received. What I received contained many statements from the 
reviewers that I thought were wrong, I responded to these, and this was 
all removed from what appeared online!
Regards,
Ed

On 5/2/15 12:39 PM, Nikolaus Grigorieff wrote:
> I have been a supporter of open sharing of data for many years, and I am
> very happy that deposition of maps and, more recently also entire
> datasets has become the standard in our field. I appreciate the forceful
> statements made by Ed and others and, of course, I support the spirit in
> which they are made. Nevertheless, even though it is important to remain
> positive, I feel compelled to add to this discussion that there is also
> abuse of the anonymous reviewing system. While this fact is not disputed
> by anybody, and it does not automatically mean that there is a different
> system we should adopt, it validates the worries that some have voiced.
> I also would like to point out that there is some importance in ensuring
> fair and unbiased reviews. It is true that for many of us one of the
> main reasons for doing science is the enjoyment we get from our work.
> However, I think that there are also a number of other reasons for being
> in science including, for example, a job with a career, financial gain
> (especially in scientific publishing) and curing a disease. Whether or
> not a paper is reviewed while observing high ethical standards may
> therefore have a real-life impact on people’s trajectories, both in
> science and in areas benefiting from it. For those of us already
> established this may be less of a worry but those trying to get in, a
> lot could be riding even on a single publication.
>
> What I find lacking a bit in our current system is the accountability of
> the reviewers. I agree that trust is part of the reviewing process,
> ideally in both directions, but I find it a bit single-sided to see the
> need for accountability only on the side of the scientific contributors.
> If social media has taught us one thing, it is that people who do not
> need to worry about accountability sometimes also feel free to abandon
> ethical standards. For the reviewing system to work, we also have to
> think of ways to hold reviewers and editors accountable. I mention
> editors because they play an important role as the stewards of the
> reviewing process. For obvious reasons their interests may not always
> align with the noble pursuit of science but focus more on making money
> for the journal they are working for. In this respect I have come to
> appreciate the system adopted by eLife. I agree with Sjors that it is
> not always helpful to reveal your identity as a reviewer, especially if
> this may prevent you from giving your honest and unbiased opinion.
> However, at eLife, even if a reviewer chooses not to reveal their name
> they are known during the review process to the other reviewers.
> Furthermore, the editor of a paper is a practicing scientist with a
> general understanding of the subject of the paper under review, and
> without an obvious conflict of interest that an editor at a for-profit
> journal may face. Finally, besides being open-access, eLife also
> publishes the exchange between reviewers and authors for papers that are
> ultimately published. I therefore think that eLife has made a
> substantial step towards a more transparent and fairer system. In this
> regard, a recent blog post by Science Careers may also be interesting to
> read:
>
> http://sciencecareers.sciencemag.org/career_magazine/previous_issues/articles/2015_01_27/caredit.a1500025
>
> Regarding the sharing of 3D maps with the reviewers, one way to reduce
> the potential risk of abuse might be to share only boxed-out regions of
> the map that are needed to support the claims made in the publication.
> This would allow reviewers to inspect these parts of the map while the
> map fragments could not so easily be used as a reference structure for
> another project. This is still not foolproof, of course, but it may help
> balance a little more the positions of authors and reviewers.
>
>       Niko
>
>
> On 5/2/2015 7:49 AM, Edward Egelman wrote:
>> I agree strongly with Sjors, and think that more openness and more data
>> availability can only strengthen science. It was not too long ago where
>> some people refused to make reconstruction volumes available AFTER
>> publication, since there was no journal policy dictating submission to a
>> repository. As for double-blind reviewing, in a recent NSMB there is an
>> editorial about the new double blind submission possibility. They
>> provide a link to a rather equivocal editorial evaluation of the system
>> that appeared in one of the first Nature journals to institute this,
>> Nature Nanotechnology:
>>
>> http://www.nature.com/nnano/journal/v9/n11/full/nnano.2014.265.html
>>
>> In that editorial, they state:
>>
>> "However, the/American Economic Review/ dropped double-blind review in
>> 2011, citing the role of search engines in limiting their capacity to
>> maintain author anonymity as a major reason for the decision."
>>
>> In our field (as opposed to economics) it would be even harder to have
>> anonymous authors, as one needs to describe the microscope and camera,
>> cite previous publications, etc. One could not say "the protein was
>> prepared as previously described" and give a citation.
>> They also state that although 80% of respondents to some survey
>> supported double-blind, less than 20% of submitted manuscripts chose
>> that option (what they actually state is that is that the monthly
>> percentage has never risen above 20% for the each of the two journals,
>> so this is the maximum of something that is fluctuating and the overall
>> percentage may be 10% or less!).
>> Regards,
>> Ed
>>
>> On 5/2/15 7:10 AM, Sjors Scheres wrote:
>>> Dear all,
>>>
>>> I think moving towards more openness will be a good thing. As a reviewer
>>> it is often hard to assess the quality of a map from 2D still pictures
>>> alone, and having a look at the map will surely help to assess its
>>> quality. Trust already is at the foundation of the review process. Whether
>>> the reviewer also gets access to the map and the model does not change
>>> much in that respect.
>>>
>>> Reversed anonymity in the review process as suggested by Matthijn is
>>> problematic because the identity of the authors is usually very easy to
>>> distill from the paper itself. A useful comment on this (in the context of
>>> double-blind review) is in Nature this week. Going the other way, with
>>> open identities of both authors and reviewers, is attractive in some ways,
>>> but also has its difficulties. I personally tend to tick the option to
>>> reveal my identity as a reviewer in eLife, and have also had my papers
>>> reviewed by people who revealed their identity. Last year I wrote one
>>> negative review for a journal that did not have this option, and later
>>> revealed myself to one of the authors and had an open, face-to-face
>>> discussion with the intention to be helpful. However, revealing one's
>>> identity as a reviewer also has a potential danger of bias towards being
>>> less critical (perhaps especially for less established reviewers?), or the
>>> danger of a culture where positive reviews are expected to be 'paid back'
>>> later. Again, I think trust is the key word, and recognizing the
>>> difficulties of every option is a good start.
>>>
>>> Perhaps I'm naive, but I think most of us are in this for the fun of it,
>>> and to ultimately change the world for the better. More openness will
>>> probably help in speeding up the scientific process. In that context I
>>> also agree with Grant that discussing options like (bio)-arxiv may be
>>> useful.
>>>
>>> Sjors
>>>
>>>> dear colleagues,
>>>>
>>>> i would be interested in experiences / suggestions / views of others in
>>>> the
>>>> field on  the following issue that may be of interest to many of us:
>>>> the editor of our manuscript forwarded the request of a peer-reviewer to
>>>> access the cryo-em map of our beloved complex. this has never happened to
>>>> us, but to our surprise the editor did not consider the request to be
>>>> unusual.
>>>> of course, we share the point that the map would be of great help in
>>>> judging the interpretation of the data. however, we also feel very
>>>> uncomfortable sending the condensed result of lengthy research to an
>>>> anonymous colleague, who could theoretically make considerable misuse of
>>>> it. nevertheless, the policy of the journal seems to let us little
>>>> choice: "Supporting
>>>> data must be made available to editors and peer-reviewers at the time of
>>>> submission for the purposes of evaluating the manuscript. Peer-reviewers
>>>> may be asked to comment on the terms of access to materials, methods
>>>> and/or
>>>> data sets".
>>>> in any case we would be curious whether others indeed got similar requests
>>>> and how they dealt with it. a good solution for (paranoid?) people like us
>>>> could be a good web-based viewer that lets others view our map, but i
>>>> would
>>>> not know of such a tool.
>>>>
>>>> Thanks
>>>>
>>>> Friedrich
>>>>
>>>> --
>>>> Dr. Friedrich Foerster
>>>> Max-Planck Institut fuer Biochemie
>>>> Am Klopferspitz 18
>>>> D-82152 Martinsried
>>>>
>>>> Tel: +49 89 8578 2632
>>>> Fax: +49 89 8578 2641
>>>>
>>>> www.biochem.mpg.de/foerster
>>>> _______________________________________________
>>>> 3dem mailing list
>>>> 3dem at ncmir.ucsd.edu
>>>> https://mail.ncmir.ucsd.edu/mailman/listinfo/3dem
>>>>
>> -- 
>> Edward H. Egelman, Ph.D.
>> Professor
>> Dept. of Biochemistry and Molecular Genetics
>> University of Virginia
>>
>> President
>> Biophysical Society
>>
>> phone: 434-924-8210
>> fax: 434-924-5069
>> egelman at virginia.edu
>> http://www.people.virginia.edu/~ehe2n
>>
>>
>>
>> _______________________________________________
>> 3dem mailing list
>> 3dem at ncmir.ucsd.edu
>> https://mail.ncmir.ucsd.edu/mailman/listinfo/3dem
>>
> _______________________________________________
> 3dem mailing list
> 3dem at ncmir.ucsd.edu
> https://mail.ncmir.ucsd.edu/mailman/listinfo/3dem

-- 
Edward H. Egelman, Ph.D.
Professor
Dept. of Biochemistry and Molecular Genetics
University of Virginia

President
Biophysical Society

phone: 434-924-8210
fax: 434-924-5069
egelman at virginia.edu
http://www.people.virginia.edu/~ehe2n