[3dem] sharing em maps during peer-review process

Sat May 2 12:09:51 PDT 2015

My first comment to 3DEM, let's start the British way:
Thanks, Nikolaus Grigorieff, for that plea for mutual trust 
(supporting, in consequence, a double-open approach).
It is all an ongoing discussion (see, for instance, 
http://occamstypewriter.org/scurry/2015/04/29/amend-laws-scholarly-publication/), 
and there are (since 2001) working models up in the air 
(http://journal.frontiersin.org/article/10.3389/fncom.2012.00033/full; 
see also 
http://www.laborjournal-archiv.de/epaper/LJ_15_04/#14 - 
unfortunately only in German). This ACP journal has even an 
in-built pre-publish archive with its "discussion papers".
And thanks to Friedrich for kindling this discussion also here.

Christoph

On 02/05/2015 17:39, Nikolaus Grigorieff wrote:
> I have been a supporter of open sharing of data for many years, and I am
> very happy that deposition of maps and, more recently also entire
> datasets has become the standard in our field. I appreciate the forceful
> statements made by Ed and others and, of course, I support the spirit in
> which they are made. Nevertheless, even though it is important to remain
> positive, I feel compelled to add to this discussion that there is also
> abuse of the anonymous reviewing system. While this fact is not disputed
> by anybody, and it does not automatically mean that there is a different
> system we should adopt, it validates the worries that some have voiced.
> I also would like to point out that there is some importance in ensuring
> fair and unbiased reviews. It is true that for many of us one of the
> main reasons for doing science is the enjoyment we get from our work.
> However, I think that there are also a number of other reasons for being
> in science including, for example, a job with a career, financial gain
> (especially in scientific publishing) and curing a disease. Whether or
> not a paper is reviewed while observing high ethical standards may
> therefore have a real-life impact on people’s trajectories, both in
> science and in areas benefiting from it. For those of us already
> established this may be less of a worry but those trying to get in, a
> lot could be riding even on a single publication.
>
> What I find lacking a bit in our current system is the accountability of
> the reviewers. I agree that trust is part of the reviewing process,
> ideally in both directions, but I find it a bit single-sided to see the
> need for accountability only on the side of the scientific contributors.
> If social media has taught us one thing, it is that people who do not
> need to worry about accountability sometimes also feel free to abandon
> ethical standards. For the reviewing system to work, we also have to
> think of ways to hold reviewers and editors accountable. I mention
> editors because they play an important role as the stewards of the
> reviewing process. For obvious reasons their interests may not always
> align with the noble pursuit of science but focus more on making money
> for the journal they are working for. In this respect I have come to
> appreciate the system adopted by eLife. I agree with Sjors that it is
> not always helpful to reveal your identity as a reviewer, especially if
> this may prevent you from giving your honest and unbiased opinion.
> However, at eLife, even if a reviewer chooses not to reveal their name
> they are known during the review process to the other reviewers.
> Furthermore, the editor of a paper is a practicing scientist with a
> general understanding of the subject of the paper under review, and
> without an obvious conflict of interest that an editor at a for-profit
> journal may face. Finally, besides being open-access, eLife also
> publishes the exchange between reviewers and authors for papers that are
> ultimately published. I therefore think that eLife has made a
> substantial step towards a more transparent and fairer system. In this
> regard, a recent blog post by Science Careers may also be interesting to
> read:
>
> http://sciencecareers.sciencemag.org/career_magazine/previous_issues/articles/2015_01_27/caredit.a1500025
>
> Regarding the sharing of 3D maps with the reviewers, one way to reduce
> the potential risk of abuse might be to share only boxed-out regions of
> the map that are needed to support the claims made in the publication.
> This would allow reviewers to inspect these parts of the map while the
> map fragments could not so easily be used as a reference structure for
> another project. This is still not foolproof, of course, but it may help
> balance a little more the positions of authors and reviewers.
>
>       Niko
>
>
> On 5/2/2015 7:49 AM, Edward Egelman wrote:
>> I agree strongly with Sjors, and think that more openness and more data
>> availability can only strengthen science. It was not too long ago where
>> some people refused to make reconstruction volumes available AFTER
>> publication, since there was no journal policy dictating submission to a
>> repository. As for double-blind reviewing, in a recent NSMB there is an
>> editorial about the new double blind submission possibility. They
>> provide a link to a rather equivocal editorial evaluation of the system
>> that appeared in one of the first Nature journals to institute this,
>> Nature Nanotechnology:
>>
>> http://www.nature.com/nnano/journal/v9/n11/full/nnano.2014.265.html
>>
>> In that editorial, they state:
>>
>> "However, the/American Economic Review/ dropped double-blind review in
>> 2011, citing the role of search engines in limiting their capacity to
>> maintain author anonymity as a major reason for the decision."
>>
>> In our field (as opposed to economics) it would be even harder to have
>> anonymous authors, as one needs to describe the microscope and camera,
>> cite previous publications, etc. One could not say "the protein was
>> prepared as previously described" and give a citation.
>> They also state that although 80% of respondents to some survey
>> supported double-blind, less than 20% of submitted manuscripts chose
>> that option (what they actually state is that is that the monthly
>> percentage has never risen above 20% for the each of the two journals,
>> so this is the maximum of something that is fluctuating and the overall
>> percentage may be 10% or less!).
>> Regards,
>> Ed
>>
>> On 5/2/15 7:10 AM, Sjors Scheres wrote:
>>> Dear all,
>>>
>>> I think moving towards more openness will be a good thing. As a reviewer
>>> it is often hard to assess the quality of a map from 2D still pictures
>>> alone, and having a look at the map will surely help to assess its
>>> quality. Trust already is at the foundation of the review process. Whether
>>> the reviewer also gets access to the map and the model does not change
>>> much in that respect.
>>>
>>> Reversed anonymity in the review process as suggested by Matthijn is
>>> problematic because the identity of the authors is usually very easy to
>>> distill from the paper itself. A useful comment on this (in the context of
>>> double-blind review) is in Nature this week. Going the other way, with
>>> open identities of both authors and reviewers, is attractive in some ways,
>>> but also has its difficulties. I personally tend to tick the option to
>>> reveal my identity as a reviewer in eLife, and have also had my papers
>>> reviewed by people who revealed their identity. Last year I wrote one
>>> negative review for a journal that did not have this option, and later
>>> revealed myself to one of the authors and had an open, face-to-face
>>> discussion with the intention to be helpful. However, revealing one's
>>> identity as a reviewer also has a potential danger of bias towards being
>>> less critical (perhaps especially for less established reviewers?), or the
>>> danger of a culture where positive reviews are expected to be 'paid back'
>>> later. Again, I think trust is the key word, and recognizing the
>>> difficulties of every option is a good start.
>>>
>>> Perhaps I'm naive, but I think most of us are in this for the fun of it,
>>> and to ultimately change the world for the better. More openness will
>>> probably help in speeding up the scientific process. In that context I
>>> also agree with Grant that discussing options like (bio)-arxiv may be
>>> useful.
>>>
>>> Sjors
>>>
>>>> dear colleagues,
>>>>
>>>> i would be interested in experiences / suggestions / views of others in
>>>> the
>>>> field on  the following issue that may be of interest to many of us:
>>>> the editor of our manuscript forwarded the request of a peer-reviewer to
>>>> access the cryo-em map of our beloved complex. this has never happened to
>>>> us, but to our surprise the editor did not consider the request to be
>>>> unusual.
>>>> of course, we share the point that the map would be of great help in
>>>> judging the interpretation of the data. however, we also feel very
>>>> uncomfortable sending the condensed result of lengthy research to an
>>>> anonymous colleague, who could theoretically make considerable misuse of
>>>> it. nevertheless, the policy of the journal seems to let us little
>>>> choice: "Supporting
>>>> data must be made available to editors and peer-reviewers at the time of
>>>> submission for the purposes of evaluating the manuscript. Peer-reviewers
>>>> may be asked to comment on the terms of access to materials, methods
>>>> and/or
>>>> data sets".
>>>> in any case we would be curious whether others indeed got similar requests
>>>> and how they dealt with it. a good solution for (paranoid?) people like us
>>>> could be a good web-based viewer that lets others view our map, but i
>>>> would
>>>> not know of such a tool.
>>>>
>>>> Thanks
>>>>
>>>> Friedrich
>>>>
>>>> --
>>>> Dr. Friedrich Foerster
>>>> Max-Planck Institut fuer Biochemie
>>>> Am Klopferspitz 18
>>>> D-82152 Martinsried
>>>>
>>>> Tel: +49 89 8578 2632
>>>> Fax: +49 89 8578 2641
>>>>
>>>> www.biochem.mpg.de/foerster
>>>> _______________________________________________
>>>> 3dem mailing list
>>>> 3dem at ncmir.ucsd.edu
>>>> https://mail.ncmir.ucsd.edu/mailman/listinfo/3dem
>>>>
>> -- 
>> Edward H. Egelman, Ph.D.
>> Professor
>> Dept. of Biochemistry and Molecular Genetics
>> University of Virginia
>>
>> President
>> Biophysical Society
>>
>> phone: 434-924-8210
>> fax: 434-924-5069
>> egelman at virginia.edu
>> http://www.people.virginia.edu/~ehe2n
>>
>>
>>
>> _______________________________________________
>> 3dem mailing list
>> 3dem at ncmir.ucsd.edu
>> https://mail.ncmir.ucsd.edu/mailman/listinfo/3dem
>>
> _______________________________________________
> 3dem mailing list
> 3dem at ncmir.ucsd.edu
> https://mail.ncmir.ucsd.edu/mailman/listinfo/3dem
>

___________________________________________________________
Christoph Hagen, Dr. rer. nat. habil.
OPIC-Oxford Particle Imaging Centre
Division of Structural Biology
Wellcome Trust Centre for Human Genetics
Henry Wellcome Building for Genomic Medicine
Roosevelt Drive
Headington
Oxford
OX3 7BN
United Kingdom

Tel. +44(0)1865 287815
Fax  +44(0)1865 287553
Email christoph at strubi.ox.ac.uk
Web:
https://www.strubi.ox.ac.uk/research/kay-grunewald
http://www.ebi.ac.uk/pdbe/emdb/searchResults.html/?q=(grunewald%20OR%20gruenewald%20OR%20grünewald)
http://www.researcherid.com/rid/B-5025-2010