[3dem] [ccpem] on FSC curve (A can of worms...)

[Frank, Joachim]

IT DOESN'T.

Joachim.

________________________________
From: 3dem [3dem-bounces at ncmir.ucsd.edu] on behalf of Marin van Heel [marin.vanheel at googlemail.com]
Sent: Wednesday, August 12, 2015 8:19 AM
To: Smith Liu; CCPEM at JISCMAIL.AC.UK
Cc: 3dem at ncmir.ucsd.edu
Subject: Re: [3dem] [ccpem] on FSC curve (A can of worms...)


Dear Smith Liu,

You have hit upon a can of worms here… Although the FRC/FSC metrics we introduced in 1982/1986 [1, 2] are now considered the "gold standard" cryo-EM resolution criterion, these resolution issues continue to be heavily debated [3]. Many FSC add-ons/variants and tangential issues such as “reference bias” have been inserted into the resolution criterion discussion. These discussions unfortunately confuse even established researchers (referees of major journals…), let alone newcomers to the field. Many believe the resolution issue is better resolved in X-crystallography. In fact, the FSC is arguably a better metric than the R-factor, the generally accepted resolution metric in X-ray crystallography [4]. Fortunately, FRC/FSC criteria are now slowly also becoming the standard in optical microscopy, X-ray microscopy, X-ray crystallography, and other fields of 2D/3D imaging.

The most controversial part of the FSC discussion is the FSC threshold value to serve as a resolution criterion (such as the FSC 0.5 value you mention). It took more than a decade to remove the mathematically flawed DPR (Differential Phase Residual) from the literature, after I explicitly discussed its shortcomings and proposed a corrected phase residual in 1987 [3]. The discussion in the field was then deviated towards the FSC threshold at which one defines the average resolution of a 3D structure. The “0.5” “criterion” was just postulated ad hoc, without any scientific justification. Ten years ago, we argued that all fixed-valued FSC threshold criteria (such as: “0.5” and “0.143”) are based on flawed statistics [5]. Virtually all more formal justifications for resolution criteria start off referring to the old formula “SNR = (CCC/(1-CCC))” by Frank & Al-Ali  1975 [6]. Unfortunately this formula is also mathematically incorrect as was discussed previously [5].

Here is another very simple argument to illustrate its flawed definition: the normalised CCC (or FSC) has values in the range:  -1<=CCC<=+1, whereas the SNR (=S2/N2) is, per definition, positive. Now insert the value CCC= -1, the case of perfectly anti-correlated data, into the formula. This yields: SNR = “-0.5”, a rampant violation of the SNR definition range. The formula could be valid for the limiting case of CCC is close to unity, but such high correlation values are not relevant in the resolution-threshold context. For uncorrelated signals/noise the CCC oscillates around the zero mark and, through the flawed Frank & Al-Ali formula, produces as many positive as it does erroneous negative SNR values.

Unfortunately, virtually all (~100?) papers on resolution criteria and validation tests in cryo-EM (from friends and foes) are based on this formula and are thus based on “flawed statistics” to say the least. With the great recent success of cryo-EM, everybody appears to have stopped thinking about the basics, and merrily continue to refer to incorrect stuff while focusing on “my resolution is better than yours”. After decades of funny jokes and verbal FSC controversies at GRC meetings, I don’t find it so funny anymore: it is time to clean up the mess. I have lost the patience to discuss these issues with referees who continue to consider the subject as debatable. Questionable actions are sometimes hidden behind this controversy such as in Mao & Sodrosky [7], who cynically accuse us - their critics - of not knowing how to interpret the FSC: “FSC estimates of resolution are known to be quite sensitive to statistical bias …” etc. etc.  As I said, this whole issue is no longer amusing; it has become a matter of the debatable scientific culture (integrity?) in the field of the cryo-EM field.

Oh, by the way, Smith Liu, what I really was going to say when I started typing an answer to your question is that if you are new to the field it is a good idea to read some basic literature in Fourier Optics. Maybe my lecture notes can help [8]. The horizontal axis in the FSC is 1/spatial-frequency (we are in Fourier space) and the FSC values in the curve indicate the cross-correlation level at that level of resolution (= inside that specific 3D Fourier shell).

Hope this helps,

Marin

[1] Van Heel M, Keegstra W, Schutter W, van Bruggen EFJ: Arthropod hemocyanin structures studied by image analysis http://singleparticles.org/methodology/MvH_FRC_Leeds_1982.pdf
[2] Harauz G & van Heel M: Exact filters for general geometry three dimensional reconstruction, Optik 73 (1986) 146-156
[3]Van Heel M: Similarity measures between images. Ultramicroscopy 21 (1987) 95-100.]. [4] Van Heel: Unveiling ribosomal structures: the final phases. Current Opinions in Structural Biology 10 (2000) 259-264.
[5] Van Heel M & Schatz M:  Fourier Shell Correlation Threshold Criteria, J. Struct. Biol. 151 (2005) 250-262
[6] Frank J & Al-Ali L:  Signal-to-noise ratio of electron micrographs obtained by cross correlation. Nature (1975)
[7] Mao Y, Castillo-Menendeza LR, Sodroski JG: Reply to Subramaniam, van Heel, and Henderson: Validity of the cryo-electron microscopy structures of the HIV-1 envelope glycoprotein complex. PNAS 2013 www.pnas.org/cgi/doi/10.1073/pnas.1316666110<http://www.pnas.org/cgi/doi/10.1073/pnas.1316666110>
[8] Van Heel:  Principles of Phase Contrast (Electron) Microscopy. http://www.single-particles.org/methodology/MvH_Phase_Contrast.pdf

===========================================



On 08/08/2015 07:45, Smith Liu wrote:
Dear All,

I know the x-axis of the FSC curve is on the reverse of the resolution, and the value in the x-axis corresponding FSC 0.5 is usually regarded as the reverse of the resolution of the whole EM map.

Here I do not know the meaning of the resolution in the X-axis. The Whole map has only one resolution corresponding FSC 0.5, then why the x-axis is on different resolutions (for example the x-axis is from resolution 0 to 20 A, or the reverse of that scope)? Is it because different parts of the map have different resolutions (caused by different parts of map  have different quality), or it is because the X-axis of the FSC curve has some relation with Fourier shell? If the X-axis of the FSC is on the property related to Fourier shell, then what is in the relation of resolution (or the reverse of it) in the x-axis with Fourier shell (in addition, what is the Fourier shell)?

Best regards.

Smith






-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ncmir.ucsd.edu/pipermail/3dem/attachments/20150812/c3f3edeb/attachment-0001.html>