[3dem] [ccpem] GroEL best resolution map. (FSC RESOLUTION ??)

Tue Dec 25 15:55:16 PST 2018

Dear Gerard,

Sorry for this late reaction but this discussion erupted at a time I was
fully occupied with the Brazil school (www.brazil-school.org).

 Reacting to your posting:

“*My final conclusion (the key step in reaching it being illustrated by
Figure 6) was that unless the said symmetry was of very high order, it was
necessary to use a sampling interval of 1/6 or 1/5 of the "resolution",
i.e. at 3.0 or 2.5 times the Nyquist frequency, to keep interpolation
errors within acceptable limits.” I used this sampling rate in my own
phasing calculations on the Tobacco Mosaic Virus coat protein disk and the
Tomato Bushy Stunt Virus. A competing group who at the time was both
algorithmically and CPU-resource-wise challenged resisted this conclusion,
claiming that such a sampling rate was unnecessarily high and that the
"traditional" (one third of the resolution) rate was enough. A few years
later, when computing resources were no longer so limiting, their landmark
Nature paper on another virus structure simply stated that "All averaging
calculations were performed at a sampling interval of one fifth of the
resolution" - considering it as obvious.*”

What we suggested many years ago in EM is to (at least) use what you call
the *"traditional" (one third of the resolution) sampling rate.*  What the
abusers of this #twothirdsNyquist rule are suggesting is that you can use
up #fullNyquist and still get 'good results'.  I agree with you that the
traditional 2/3 Nyquist sampling frequency rule may in many cases even not
be sufficient and that realisation is supported by our calculations (paper
in preparation). (This may even go beyond the '4 pixels' ('1/2 Nyquist')
that Steve Ludke was discussing.)  The argument that the #fullNyquist
proponents are using for interpreting the FSC beyond its validity range is
that the results ‘look good’. Good looking results are in the eye of the
beholder but the objective 1982/1986 FRC/FSC metrics should not be used
beyond their validity range (
https://en.wikipedia.org/wiki/Fourier_shell_correlation). Those who use
such metrics beyond their definition range are in fact creating a new
metric that they themselves must justify. I suggest the name "My Shell
Correlation" (MSC) for such validity extensions (he-he).

“*In any case, this is just to point out that the damage done in Fourier
space by linear interpolation in real space should never be overlooked. It
has caused confusion that this "damage" has been much underestimated by
computing a correlation coefficient between *maps* and concluding that the
sampling rate mattered little. However, this is because map correlation
coefficients are dominated by the large terms at low spatial frequencies -
since, by Parseval's theorem, they are equal to a FSC computed in a single
shell containing all the Fourier data - and are therefore quite insensitive
to the degradation of the smaller high-frequency terms that is caused by
linear interpolation. I hope this connection to another instance of a
similar problem may be helpful*.”

The connection is helpful and I fully agree that “correlation coefficients
are dominated by the large terms at low spatial frequencies”.  We even
wrote a separate paper about that topic 😉 (“Correlation function
revisited”) https://doi.org/10.1016/0304-3991(92)90021-B  . The
considerations that all spatial frequencies should be weighted correctly
were at the base of our earlier FRC/FSC proposals and the normalisation we
chose for them. (Details in
https://www.biorxiv.org/content/early/2017/11/24/224402)

Merry Xmas

Marin

On Mon, Sep 3, 2018 at 2:23 PM Gerard Bricogne <gb10 at globalphasing.com>
wrote:

> Dear all,
>
>      This is my first posting to CCPEM :-) .
>
>      It sounds as if the matters discussed bear some relationhip to
> the investigation of errors introduced into the Fourier spectrum of a
> band-limited function by linear interpolation from the values of that
> function on a grid. I had the opportunity to look into this question
> in the Appendix at the end of the following paper:
>
> https://journals.iucr.org/a/issues/1976/05/00/a12981/a12981.pdf
>
> when implementing phase improvement by non-crystallographic symmetry
> through iterative map averaging. My final conclusion (the key step in
> reaching it being illustrated by Figure 6) was that unless the said
> symmetry was of very high order, it was necessary to use a sampling
> interval of 1/6 or 1/5 of the "resolution", i.e. at 3.0 or 2.5 times
> the Nyquist frequency, to keep interpolation errors within acceptable
> limits. I used this sampling rate in my own phasing calculations on
> the Tobacco Mosaic Virus coat protein disk and the Tomato Bushy Stunt
> Virus. A competing group who at the time was both algorithmically and
> CPU-resource-wise challenged resisted this conclusion, claiming that
> such a sampling rate was unnecessarily high and that the "traditional"
> (one third of the resolution) rate was enough. A few years later, when
> computing resources were no longer so limiting, their landmark Nature
> paper on another virus structure simply stated that "All averaging
> calculations were performed at a sampling interval of one fifth of the
> resolution" - considering it as obvious.
>
>      Steven underlines the fact that it takes 4 points to completely
> and unambiguously specify a sinusoidal oscillation, so one might
> wonder why one would need 5 or 6 points. The fact is that an input
> sinusoid liearly sampled between 4 points, then Fourier transformed,
> would come back with an attenuation factor; and that the rms power
> that is shaved off that term in this attenuation reappears as noise
> through the "side-bands" as shown in Figure 6.
>
>      In any case, this is just to point out that the damage done in
> Fourier space by linear interpolation in real space should never be
> overlooked. It has caused confusion that this "damage" has been much
> underestimated by computing a correlation coefficient between *maps*
> and concluding that the sampling rate mattered little. However this is
> because map correlation coefficients are dominated by the large terms
> at low spatial frequencies - since, by Parseval's theorem, they are
> equal to a FSC computed in a single shell containing all the Fourier
> data - and are therefore quite insensitive to the degradation of the
> smaller high-frequency terms that is caused by linear interpolation.
>
>      I hope this connection to another instance of a similar problem
> may be helpful.
>
>
>      With best wishes,
>
>           Gerard.
>
> --
> On Mon, Sep 03, 2018 at 04:37:37PM +0000, Ludtke, Steven J wrote:
> > Again, I am NOT arguing that FFTs are inconsistent in some way, or that
> aliasing will prevent you from achieving FSC curves giving you "resolution"
> past 2/3 Nyquist. It is absolutely possible to perform iterative
> refinements which extend beyond 2/3 Nyquist, and always has been (ie - this
> isn't something new with "modern software").
> >
> > The point is that the real space representation of signal between 1/2
> Nyquist and Nyquist has significant artifacts because the sine waves with
> frequencies in this range do not have complete information in the original
> image. In real-space, you must have 4 pixels (1/2 Nyquist), not 2 pixels
> (Nyquist) to completely and unambiguously specify a sinusoidal oscillation.
> Between 2 pixels and 4 pixels you have partial information. This does not
> mean you cannot achieve Fourier space reconstructions which are self
> consistent to Nyquist, it means that there are artifacts in the real-space
> representation.
> >
> > When you do X-ray crystallography you are sampling directly in Fourier
> space, and as Pawel said (assuming you have the right phases), you can
> oversample the results in real-space as much as you like to produce nice
> smooth densities, the details of which will be limited by the highest order
> reflection you use.
> >
> > In CryoEM, we are making measurements in real-space, meaning the
> information between 1/2 Nyquist and Nyquist is incomplete at the time the
> data is measured. I used the +1,-1,+1,-1 example because it is the easiest
> case for people to picture. That is, it is clear that if you try to measure
> a pattern with exactly Nyquist periodicity, if you see a signal with some
> amplitude, you cannot tell if the amplitude you observe is correct, with
> zero phase, or if it is a sampling of a phase-shifted signal with much
> higher amplitude. This ambiguity extends partially all the way to 1/2
> Nyquist, with odd spatially localized patterns. At 1/2 Nyquist periodicity,
> full information is present.
> >
> > So, the argument is that beyond 1/2 Nyquist, you will have real-space
> artifacts which can lead to misinterpretation when doing model building and
> other tasks, but that to ~2/3 Nyquist the effect is pretty minimal.
> >
> >
> --------------------------------------------------------------------------------------
> > Steven Ludtke, Ph.D. <sludtke at bcm.edu<mailto:sludtke at bcm.edu>>
>             Baylor College of Medicine
> > Charles C. Bell Jr., Professor of Structural Biology
> > Dept. of Biochemistry and Molecular Biology                      (
> www.bcm.edu/biochem<http://www.bcm.edu/biochem>)
> > Academic Director, CryoEM Core                                        (
> cryoem.bcm.edu<http://cryoem.bcm.edu>)
> > Co-Director CIBR Center                                    (
> www.bcm.edu/research/cibr<http://www.bcm.edu/research/cibr>)
> >
> >
> >
> > On Sep 3, 2018, at 10:46 AM, Dimitry Tegunov <tegunov at GMAIL.COM<mailto:
> tegunov at GMAIL.COM>> wrote:
> >
> > Dear Steven,
> >
> > thank you for the examples.
> >
> > However, I'm not sure the Nyquist sine wave is the best example of
> aliasing. It is one extreme case valid only for the FFT of even-sized,
> real-valued signals. To circumvent this behavior of the FFT without
> breaking any of your initial conditions, please consider this experiment:
> Fourier-pad the signal by a factor of 2 to make space for the original
> Nyquist frequency component's Friedel buddy, shift back and forth by 0.5*2,
> Fourier-crop back to original size, find no changes in the original
> pattern. For the opposite, fill an even-sized window with noise, shift back
> and forth by a non-integer value, find the Nyquist frequency component
> corrupted. FFT-based non-integer shifts in even-sized windows are lossless
> up to, but not including, Nyquist.
> >
> > The PDB example, indeed, illustrates the aliasing in a single
> under-sampled observation. Now let's consider a pipeline where the only
> under-sampled observation of the signal in real space is made at the image
> acquisition stage. All subsequent resampling is performed in Fourier space
> with sufficient padding in real space. The result is an average of many
> independently aliased observations of the underlying non band-limited
> signal. Sure, the aliasing corrupts each initial observation (and not only
> its Nyquist frequency), but this noise will be independent between the
> half-maps and thus won't artificially increase the FSC. As far as I can
> tell, it will also be 0-mean – resulting in perfectly fine maps beyond 2/3
> Nyquist. Am I missing something?
> >
> > Cheers,
> > Dimitry
> >
> > ########################################################################
> >
> > To unsubscribe from the CCPEM list, click the following link:
> >
> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.jiscmail.ac.uk_cgi-2Dbin_webadmin-3FSUBED1-3DCCPEM-26A-3D1&d=DwIFaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=GWA2IF6nkq8sZMXHpp1Xpg&m=dzG4INJqk9Niq7zSShsu9mZb3Q1kqybUVPwN7eRH234&s=ii43w7MbFj1V2eYlRIviZzRoDp_YB4aH_eXMFpympEw&e=
> >
> >
> > ########################################################################
> >
> > To unsubscribe from the CCPEM list, click the following link:
> > https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCPEM&A=1
>
> --
>
>      ===============================================================
>      *                                                             *
>      * Gerard Bricogne                     gb10 at GlobalPhasing.com  *
>      *                                                             *
>      * Global Phasing Ltd.                                         *
>      * Sheraton House, Castle Park         Tel: +44-(0)1223-353033 *
>      * Cambridge CB3 0AX, UK               Fax: +44-(0)1223-366889 *
>      *                                                             *
>      ===============================================================
>
> ########################################################################
>
> To unsubscribe from the CCPEM list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCPEM&A=1
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ncmir.ucsd.edu/pipermail/3dem/attachments/20181225/322a133e/attachment-0001.html>