[3dem] Don't blame your thermometer... (the story continues)

Marin van Heel marin.vanheel at googlemail.com
Fri Oct 11 03:53:42 PDT 2013

The saga continues...


          Protein Data Bank in Europe (PDBe)

          Controversy continues over the validity of the cryo-electron
          microscopy structures of the HIV-1 envelope glycoprotein
          complex http://dx.doi.org/doi:10.1073/pnas.1307382110

          Richard Henderson's article in PNAS (http://bit.ly/17pHW2q) is
          accompanied by letters from Subramaniam
          (http://bit.ly/1c2Qfq4), van Heel (http://bit.ly/1ac3xNA) and
          a response from Mao (http://bit.ly/1hE0p0Z

(Yes Pawel, you read me right, that is what was behind my original 


On 28-Aug-13 3:29 PM, Penczek, Pawel A wrote:
> Hi,
> "cheating" is a strong a loaded term.  Most mistakes are made by overzealous and sometimes not fully knowledgeable users.
> Given a tool that is not entirely correctly advertised as a panacea against mistake the dangers paradoxically increase.
> I am not aware of any statements that would specify what resolution a reference should be in "golden standard" procedure.
> Regards,
> Pawel.
> On Aug 28, 2013, at 9:20 AM, Steven Ludtke <sludtke at bcm.edu> wrote:
>> Ouch, now I can see a real 'discussion' arising.
>> One minor correction to your statement. They did, indeed, claim to do a 'gold standard' refinement in that paper, however, they used the same (high resolution) reference for both of the 'independent' refinements. They claimed a final resolution of ~6 Å which is higher than the '11 Å reference'. Then again, I suspect that the '11 Å resolution' reference was not rigorously filtered to 11 Å. So, they used the word 'gold standard' without following the procedure it is supposed to embody.
>> You are correct, of course, even so that there are many many ways to 'cheat' any standard. Nonetheless, it isn't a bad idea to develop standards which at least make it more difficult to do so, and also dramatically reduce the possibility of _unintentional_ cheating. There is very little anyone can do about someone who's determined to cheat. (That isn't intended as a comment about the PNAS manuscript, which I suspect could only be resolved at this point if they decided to make their raw data publicly available).
>> On Aug 28, 2013, at 8:56 AM, "Penczek, Pawel A" <Pawel.A.Penczek at uth.tmc.edu> wrote:
>>> My reading of Marin's parabole is that he is trying to clarify issues surrounding some of the exaggerated claims that surfaced
>>> in few recent years and espoused in the papers listed by Steve.
>>> While by themselves they are innocuous the danger is that for some they can provide a false sense of self-confidence: I used
>>> the well-documented "prevention" method therefore my structure is beyond doubt and reproach.
>>> Regrettably, it is not quite so and while additional validation never does any harm by itself it is not sufficient to prove the
>>> results are correct.  A determined person can alway squeeze out helices and phosphate bonds from the data.
>>> A very good illustration of perils associated with the uncritical use of "golden standard" is provided by Richard Henderson interviewed in Science:
>>>         Is High-Tech View of HIV Too Good to Be True?
>>> *   Jon Cohen
>>> *   Science 2 August 2013: 443-444.
>>> Regards,
>>> Pawel A. Penczek, Ph.D.
>>> ====================================================
>>> On Aug 28, 2013, at 6:27 AM, Marin van Heel <marin.vanheel at googlemail.com<mailto:marin.vanheel at googlemail.com>> wrote:
>>> Thought of the day:
>>> If you overheat your house, do you blame your thermometer?
>>> It is rather your own responsibly to change your thermostat settings!
>>> If you over-fit your 3D reconstruction, do you blame the FSC resolution measure?
>>> It is rather your own responsibility to avoid overfitting by reference bias.
>>> There is no such thing as a biased FSC (Fourier Shell Correlation). Neither is there a “gold standard” FSC!
>>> Yes, you CAN do deviously biased data processing to try to prove you are better than everybody else.
>>> But what ever you do, please don’t blame your thermometer.
>>> Cheers,
>>> Marin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://mail.ncmir.ucsd.edu/mailman/private/3dem/attachments/20131011/6c8356e1/attachment.html

More information about the 3dem mailing list