[3dem] Don't blame your thermometer...

[Marin van Heel]

Hi Sjors,

I beg to differ, since I really believe a terminology such as: "unbiased 
particle picking" and "unbiased structural refinement" is more 
appropriate/correct/helpfull than flashy but out-of-context terms like 
"gold standard FSC", that confuses everybody and especially those new to 
the field.

However, if you insist on using "gold standard" thermometers I can 
suggest you a sound piece of good old Dutch research: 
http://www.jwatch.org/pa201009290000001/2010/09/29/how-accurate-are-tympanic-and-infrared-skin.

Groeten, :)

Marin

===============================================================

On 28-Aug-13 1:00 PM, Sjors Scheres wrote:
> Hi Marin,
>
> I tried to stay out of this, but now that my name has been mentioned...
>
> First of all thank you for bringing gold-standard procedures for 
> refinement again into the spotlight. This is an important issue, and 
> as you rightly pointed out, this problem has been well known for many 
> years. Still, many in the field have been slow to change their 
> refinement procedures accordingly...
>
> I don't think that the use of "gold-standard FSC" leaves much room for 
> confusion. It is for example equivalent to the use of "free R-factor" 
> in protein crystallography: there is only one mathematical formula to 
> calculate the measure (so no room for confusion), and the adjective 
> refers to the procedure in which it is being used (the R-factor itself 
> is not "free"). I'm afraid that the confusion and lack of 
> understanding that you observe with some scientists is unlikely to be 
> resolved by changing semantics. Good-old training will have to solve 
> that and teach responsible use of the available procedures.
>
> Groeten,
> Sjors
>
>
>
> On 08/28/2013 03:28 PM, Marin van Heel wrote:
>> "Sorry officer I did not know I was speeding, I don't have a gold 
>> standard speedometer!"
>>
>> Steven:
>>
>> (Trolling? Wikipedia: In Internet slang, a troll (/ˈtroʊl/, /ˈtrɒl/) 
>> is a person who sows discord on the Internet by starting arguments or 
>> upsetting people...)
>>
>> It is not just Sjors in his recent paper who refers to bad processing 
>> by adding adjectives to the FSC; but it is certainly the clearest 
>> case. I do notice as a referee that a new generation of scientists 
>> are confused by the matter and some use it more as a buzz-word 
>> without any understanding of the underlying issues. Therefore I do 
>> think the semantics are important here.
>>
>> Few people take the trouble to look at the older papers on reference 
>> bias:
>> [
>> Boekema EJ, Berden JA, van Heel M: *Structure of mitochondrial 
>> F1-ATPase studied by electron microscopy and image processing*. 
>> /Bioch. //Biophys. Acta/*851* (1986) 353-360 ]
>> and to the general risk of introducing artificial similarities 
>> between the two 3D structures being compared (note that this is the 
>> FSC paper):
>> [ Harauz G, van Heel M : *Exact filters for general geometry three 
>> dimensional reconstruction*, /Optik/ *73* (1986) 146-156 ].
>>
>> Or a bit more modern/recent (only 10 years old) on the importance of 
>> independent 3D refinements:
>> [Yang S, Yu X, Galkin VE, Egelman EH: *Issues of resolution and 
>> polymorphism in single-particle reconstruction.* /J. of Structural 
>> Biology/ *144* (2003) 162–171. ]
>> [ Stewart A, Grigorieff N: *Noise bias in the refinement of 
>> structures derived from single particles.* /Ultramicroscopy/ *102* 
>> (2004) 67-84 ].
>>
>> Cheers,
>>
>>
>> On 28-Aug-13 9:54 AM, Steven Ludtke wrote:
>>>> Ahh, but does your thermostat have a wet bulb or a dry bulb 
>>>> thermometer ;^)
>>>>
>>>> First I thought, "No, what an obvious trolling attempt", then I 
>>>> decided there may be some actual confusion, so perhaps we should 
>>>> resolve the semantics. So I will succumb to the urge to reply this 
>>>> one time (and almost certainly regret doing so later).
>>>>
>>>> True, a "gold standard FSC" means "gold standard refinement, 
>>>> followed by a normal FSC", since that is too long to say, people 
>>>> have started saying "gold standard FSC". Neither does cryoEM refer 
>>>> to microscopy with cold electrons.
>>>>
>>>> For anyone who is confused, "gold standard refinement" simply 
>>>> refers to the process of splitting ones data in half at the very 
>>>> beginning, and performing two completely independent refinements 
>>>> (with independent starting models). It is, of course, not the only 
>>>> way of preventing resolution exaggeration due to model/noise bias, 
>>>> but (barring the use of other artifact inducers like hard spherical 
>>>> masks), it is certainly one robust technique for doing so. If you 
>>>> follow this technique honestly, then you can be confident that your 
>>>> resolution is not over-estimated. However, most people apply some 
>>>> sort of mask to reduce noise on the final structure before FSC 
>>>> computation (otherwise small box sizes lead to better resolutions 
>>>> but worse structures). This can raise concerns about exaggeration 
>>>> again, if masking isn't done with an appropriate bias-minimizing mask.
>>>>
>>>> * Scheres, S. H. & Chen, S. (2012) Prevention of overfitting in 
>>>> cryo-EM structure determination. Nat. Methods. 9, 853-854.
>>>>
>>>> * Murray, S. C., Flanagan, J., Popova, O. B., Chiu, W., Ludtke, S. 
>>>> J. & Serysheva, I. I. (2013) Validation of Cryo-EM Structure of 
>>>> IP3R1 Channel. Structure. 21, 1-10. PMC3696195
>>>>
>>>> So, if you have any doubts, or are using some other method which 
>>>> you believe avoids bias, Richard's recent suggestion to randomize 
>>>> the phases in your raw data, then rerefine to prove that your 
>>>> algorithm doesn't claim to have achieved resolutions beyond this 
>>>> point can be used to really test your process.
>>>>
>>>> * Chen, S., McMullan, G., Faruqi, A. R., Murshudov, G. N., Short, 
>>>> J. M., Scheres, S. H. & Henderson, R. (2013) High-resolution noise 
>>>> substitution to measure overfitting and validate resolution in 3D 
>>>> structure determination by single particle electron cryomicroscopy. 
>>>> Ultramicroscopy.
>>>>
>>>>
>>
>>
>>
>> _______________________________________________
>> 3dem mailing list
>> 3dem at ncmir.ucsd.edu
>> https://mail.ncmir.ucsd.edu/mailman/listinfo/3dem
>