[3dem] [ccpem] on FSC curve (A can of worms...)

Marin van Heel marin.vanheel at googlemail.com
Wed Aug 12 05:19:44 PDT 2015


Dear Smith Liu,

You have hit upon a can of worms here… Although the FRC/FSC metrics we 
introduced in 1982/1986 [1, 2] are now considered the "gold standard" 
cryo-EM resolution criterion, these resolution issues continue to be 
heavily debated [3]. Many FSC add-ons/variants and tangential issues 
such as “reference bias” have been inserted into the resolution 
criterion discussion. These discussions unfortunately confuse even 
established researchers (referees of major journals…), let alone 
newcomers to the field. Many believe the resolution issue is better 
resolved in X-crystallography. In fact, the FSC is arguably a better 
metric than the R-factor, the generally accepted resolution metric in 
X-ray crystallography [4]. Fortunately, FRC/FSC criteria are now slowly 
also becoming the standard in optical microscopy, X-ray microscopy, 
X-ray crystallography, and other fields of 2D/3D imaging.

The most controversial part of the FSC discussion is the FSC threshold 
value to serve as a resolution criterion (such as the FSC 0.5 value you 
mention). It took more than a decade to remove the mathematically flawed 
DPR (Differential Phase Residual) from the literature, after I 
explicitly discussed its shortcomings and proposed a corrected phase 
residual in 1987 [3]. The discussion in the field was then deviated 
towards the FSC threshold at which one defines the average resolution of 
a 3D structure. The “0.5” “criterion” was just postulated ad hoc, 
without any scientific justification. Ten years ago, we argued that all 
fixed-valued FSC threshold criteria (such as: “0.5” and “0.143”) are 
based on flawed statistics [5]. Virtually all more formal justifications 
for resolution criteria start off referring to the old formula “SNR = 
(CCC/(1-CCC))” by Frank & Al-Ali  1975 [6]. Unfortunately this formula 
is also mathematically incorrect as was discussed previously [5].

Here is another very simple argument to illustrate its flawed 
definition: the normalised CCC (or FSC) has values in the range: 
-1<=CCC<=+1, whereas the SNR (=S2/N2) is, per definition, positive. Now 
insert the value CCC= -1, the case of perfectly anti-correlated data, 
into the formula. This yields: SNR = “-0.5”, a rampant violation of the 
SNR definition range. The formula could be valid for the limiting case 
of CCC is close to unity, but such high correlation values are not 
relevant in the resolution-threshold context. For uncorrelated 
signals/noise the CCC oscillates around the zero mark and, through the 
flawed Frank & Al-Ali formula, produces as many positive as it does 
erroneous negative SNR values.

Unfortunately, virtually all (~100?) papers on resolution criteria and 
validation tests in cryo-EM (from friends and foes) are based on this 
formula and are thus based on “flawed statistics” to say the least. With 
the great recent success of cryo-EM, everybody appears to have stopped 
thinking about the basics, and merrily continue to refer to incorrect 
stuff while focusing on “my resolution is better than yours”. After 
decades of funny jokes and verbal FSC controversies at GRC meetings, I 
don’t find it so funny anymore: it is time to clean up the mess. I have 
lost the patience to discuss these issues with referees who continue to 
consider the subject as debatable. Questionable actions are sometimes 
hidden behind this controversy such as in Mao & Sodrosky [7], who 
cynically accuse us - their critics - of not knowing how to interpret 
the FSC: “FSC estimates of resolution are known to be quite sensitive to 
statistical bias …” etc. etc.  As I said, this whole issue is no longer 
amusing; it has become a matter of the debatable scientific culture 
(integrity?) in the field of the cryo-EM field.

Oh, by the way, Smith Liu, what I really was going to say when I started 
typing an answer to your question is that if you are new to the field it 
is a good idea to read some basic literature in Fourier Optics. Maybe my 
lecture notes can help [8]. The horizontal axis in the FSC is 
1/spatial-frequency (we are in Fourier space) and the FSC values in the 
curve indicate the cross-correlation level at that level of resolution 
(= inside that specific 3D Fourier shell).

Hope this helps,

Marin

[1] Van Heel M, Keegstra W, Schutter W, van Bruggen EFJ: Arthropod 
hemocyanin structures studied by image analysis 
http://singleparticles.org/methodology/MvH_FRC_Leeds_1982.pdf
[2] Harauz G & van Heel M: Exact filters for general geometry three 
dimensional reconstruction, Optik 73 (1986) 146-156
[3]Van Heel M: Similarity measures between images. Ultramicroscopy 21 
(1987) 95-100.]. [4] Van Heel: Unveiling ribosomal structures: the final 
phases. Current Opinions in Structural Biology 10 (2000) 259-264.
[5] Van Heel M & Schatz M:  Fourier Shell Correlation Threshold 
Criteria, J. Struct. Biol. 151 (2005) 250-262
[6] Frank J & Al-Ali L:  Signal-to-noise ratio of electron micrographs 
obtained by cross correlation. Nature (1975)
[7] Mao Y, Castillo-Menendeza LR, Sodroski JG: Reply to Subramaniam, van 
Heel, and Henderson: Validity of the cryo-electron microscopy structures 
of the HIV-1 envelope glycoprotein complex. PNAS 2013 
www.pnas.org/cgi/doi/10.1073/pnas.1316666110
[8] Van Heel:  Principles of Phase Contrast (Electron) Microscopy. 
http://www.single-particles.org/methodology/MvH_Phase_Contrast.pdf

===========================================



On 08/08/2015 07:45, Smith Liu wrote:
> Dear All,
>
> I know the x-axis of the FSC curve is on the reverse of the 
> resolution, and the value in the x-axis corresponding FSC 0.5 is 
> usually regarded as the reverse of the resolution of the whole EM map.
>
> Here I do not know the meaning of the resolution in the X-axis. The 
> Whole map has only one resolution corresponding FSC 0.5, then why 
> the x-axis is on different resolutions (for example the x-axis is from 
> resolution 0 to 20 A, or the reverse of that scope)? Is it because 
> different parts of the map have different resolutions (caused by 
> different parts of map  have different quality), or it is because the 
> X-axis of the FSC curve has some relation with Fourier shell? If the 
> X-axis of the FSC is on the property related to Fourier shell, then 
> what is in the relation of resolution (or the reverse of it) in the 
> x-axis with Fourier shell (in addition, what is the Fourier shell)?
>
> Best regards.
>
> Smith
>
>
>


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ncmir.ucsd.edu/pipermail/3dem/attachments/20150812/563a909c/attachment.html>


More information about the 3dem mailing list