[3dem] Don't blame your thermometer...

[Sjors Scheres]

Hi Marin,

I tried to stay out of this, but now that my name has been mentioned...

First of all thank you for bringing gold-standard procedures for 
refinement again into the spotlight. This is an important issue, and as 
you rightly pointed out, this problem has been well known for many 
years. Still, many in the field have been slow to change their 
refinement procedures accordingly...

I don't think that the use of "gold-standard FSC" leaves much room for 
confusion. It is for example equivalent to the use of "free R-factor" in 
protein crystallography: there is only one mathematical formula to 
calculate the measure (so no room for confusion), and the adjective 
refers to the procedure in which it is being used (the R-factor itself 
is not "free"). I'm afraid that the confusion and lack of understanding 
that you observe with some scientists is unlikely to be resolved by 
changing semantics. Good-old training will have to solve that and teach 
responsible use of the available procedures.

Groeten,
Sjors



On 08/28/2013 03:28 PM, Marin van Heel wrote:
> "Sorry officer I did not know I was speeding, I don't have a gold 
> standard speedometer!"
>
> Steven:
>
> (Trolling? Wikipedia: In Internet slang, a troll (/ˈtroʊl/, /ˈtrɒl/) 
> is a person who sows discord on the Internet by starting arguments or 
> upsetting people...)
>
> It is not just Sjors in his recent paper who refers to bad processing 
> by adding adjectives to the FSC; but it is certainly the clearest 
> case. I do notice as a referee that a new generation of scientists are 
> confused by the matter and some use it more as a buzz-word without any 
> understanding of the underlying issues. Therefore I do think the 
> semantics are important here.
>
> Few people take the trouble to look at the older papers on reference bias:
> [
> Boekema EJ, Berden JA, van Heel M: *Structure of mitochondrial 
> F1-ATPase studied by electron microscopy and image processing*. 
> /Bioch. //Biophys. Acta/*851* (1986) 353-360 ]
> and to the general risk of introducing artificial similarities between 
> the two 3D structures being compared (note that this is the FSC paper):
> [ Harauz G, van Heel M : *Exact filters for general geometry three 
> dimensional reconstruction*, /Optik/ *73* (1986) 146-156 ].
>
> Or a bit more modern/recent (only 10 years old) on the importance of 
> independent 3D refinements:
> [Yang S, Yu X, Galkin VE, Egelman EH: *Issues of resolution and 
> polymorphism in single-particle reconstruction.* /J. of Structural 
> Biology/ *144* (2003) 162–171. ]
> [ Stewart A, Grigorieff N: *Noise bias in the refinement of structures 
> derived from single particles.* /Ultramicroscopy/ *102* (2004) 67-84 ].
>
> Cheers,
>
>
> On 28-Aug-13 9:54 AM, Steven Ludtke wrote:
>>> Ahh, but does your thermostat have a wet bulb or a dry bulb 
>>> thermometer ;^)
>>>
>>> First I thought, "No, what an obvious trolling attempt", then I 
>>> decided there may be some actual confusion, so perhaps we should 
>>> resolve the semantics. So I will succumb to the urge to reply this 
>>> one time (and almost certainly regret doing so later).
>>>
>>> True, a "gold standard FSC" means "gold standard refinement, 
>>> followed by a normal FSC", since that is too long to say, people 
>>> have started saying "gold standard FSC". Neither does cryoEM refer 
>>> to microscopy with cold electrons.
>>>
>>> For anyone who is confused, "gold standard refinement" simply refers 
>>> to the process of splitting ones data in half at the very beginning, 
>>> and performing two completely independent refinements (with 
>>> independent starting models). It is, of course, not the only way of 
>>> preventing resolution exaggeration due to model/noise bias, but 
>>> (barring the use of other artifact inducers like hard spherical 
>>> masks), it is certainly one robust technique for doing so. If you 
>>> follow this technique honestly, then you can be confident that your 
>>> resolution is not over-estimated. However, most people apply some 
>>> sort of mask to reduce noise on the final structure before FSC 
>>> computation (otherwise small box sizes lead to better resolutions 
>>> but worse structures). This can raise concerns about exaggeration 
>>> again, if masking isn't done with an appropriate bias-minimizing mask.
>>>
>>> * Scheres, S. H. & Chen, S. (2012) Prevention of overfitting in 
>>> cryo-EM structure determination. Nat. Methods. 9, 853-854.
>>>
>>> * Murray, S. C., Flanagan, J., Popova, O. B., Chiu, W., Ludtke, S. 
>>> J. & Serysheva, I. I. (2013) Validation of Cryo-EM Structure of 
>>> IP3R1 Channel. Structure. 21, 1-10. PMC3696195
>>>
>>> So, if you have any doubts, or are using some other method which you 
>>> believe avoids bias, Richard's recent suggestion to randomize the 
>>> phases in your raw data, then rerefine to prove that your algorithm 
>>> doesn't claim to have achieved resolutions beyond this point can be 
>>> used to really test your process.
>>>
>>> * Chen, S., McMullan, G., Faruqi, A. R., Murshudov, G. N., Short, J. 
>>> M., Scheres, S. H. & Henderson, R. (2013) High-resolution noise 
>>> substitution to measure overfitting and validate resolution in 3D 
>>> structure determination by single particle electron cryomicroscopy. 
>>> Ultramicroscopy.
>>>
>>>
>
>
>
> _______________________________________________
> 3dem mailing list
> 3dem at ncmir.ucsd.edu
> https://mail.ncmir.ucsd.edu/mailman/listinfo/3dem

-- 
Sjors Scheres
MRC Laboratory of Molecular Biology
Francis Crick Avenue, Cambridge Biomedical Campus
Cambridge CB2 0QH, U.K.
tel: +44 (0)1223 267061
http://www2.mrc-lmb.cam.ac.uk/groups/scheres