[3dem] [ccpem] on FSC curve (A can of worms...)
Gabor Herman
gabortherman at yahoo.com
Sun Aug 30 11:05:19 PDT 2015
Dear Disputants:
I have been looking with amazement at this exchange of e-mails, trying hard
not to get sucked into it. However, having seen in Ed Engelman's message:
"FSC measures self-consistency, and not resolution" I cannot resist saying
that OF COURSE this is so.
In a recent paper of mine on nondestructive testing by reconstructing 2D slices
(Schrapp, M.J., Herman, G.T.: Data fusion in X-ray computed tomography using
a superiorization approach, Review of Scientific Instruments 85:053701, 2014)
we say that: "We wish to make a comment on the use of FRC as applied here
for evaluating algorithms. If the FRC comparing reconstructions from two halves
of the data is very low at a certain frequency, then it is reasonable to conclude
that the reconstruction process is not reliable for recovering that frequency from
the data. However, the converse is not necessarily true, it is possible to acquire
by the described method FRC values that are near 1.0 at some frequency without
the algorithm being reliable for that frequency. An extreme of this is an “algorithm”
that totally ignores the data and always produces the same “reconstruction”
irrespective of the data. Such an algorithm is clearly useless in practice, but when
evaluated by the methodology we use here would result in an FRC of 1.0 at all
frequencies. Thus one has to be careful not to overstate the significance of the
FRC level near 1.0."
Cheers,
Gabor T. Herman, Ph.D.
Distinguished Professor of Computer Science
The Graduate Center of the City University of New York
www.dig.cs.gc.cuny.edu/~gabor/index.html
.
==========================================
.
ANNOUNCEMENTS
.
Sixth Annual Minisymposium on Computational Methods for Three-Dimensional Microscopy Reconstruction
August 5, 2015
City University of New York, Graduate Center
http://www.dig.cs.gc.cuny.edu/workshops/Minisymposium_2015_Gabor_6_4.pdf
.
SNARK09 (a programming system for the reconstruction of 2D images from 1D projections)
http://www.dig.cs.gc.cuny.edu/software/snark09/
.
How I became an object of slander by Pamela Geller, Ron Torossian and their Ilk
http://www.dig.cs.gc.cuny.edu/~gabor/How_with_revised_links.pdf
--------------------------------------------
On Sun, 8/30/15, Edward Egelman <egelman at virginia.edu> wrote:
Subject: Re: [3dem] [ccpem] on FSC curve (A can of worms...)
To: 3dem at ncmir.ucsd.edu
Date: Sunday, August 30, 2015, 1:40 PM
And just to reinforce another point that I have made
previously and
in a recent eLife paper, the FSC measures
self-consistency, and not
resolution. The examples I had used involved applying
the wrong
helical symmetry, where one can generate an FSC that has
no relation
to reality. But what about consistent, reproducible
errors in the
reconstruction engine? Imagine a bug that introduces a
large peak of
density at the center of every reconstruction. The two
half maps
will then show a better FSC than the ones generated by
the bug-free
reconstruction engine.
Ed
On Aug 30, 2015, at 12:14 PM, Ludtke, Steven J
<sludtke at bcm.edu>
wrote:
Ok, I've tried to avoid this discussion, as it seems
like somewhat pointless rehashing of old debates to little
real point. However, based on direct emails I've gotten
from some people new to the field, it may be causing a lot
of confusion and uncertainty among this group. They lack the
historical context to understand the point of the debate.
Let me add a couple of minor points to the discussion:
1) Compensating for statistical uncertainty through use of
an adjustment to the threshold is confusing to people raised
in experimental science. In essence, it is concealing the
fact that the FSC values have considerable uncertainty due
to counting statistics and other effects. That is, the final
resolution plots wind up being the intersection of two lines
with no presented uncertainty at all, and we find people
looking a specific intersection points between these two
lines with ridiculous levels of precision.
A much more sensible way to present this result would be to
produce FSC curves with error bars, which do a much better
job of expressing the fact that there is considerable
uncertainty in the resulting intersection! The difficulty
is how to best produce such error bars.
Once you have an FSC with error bars, you still have the
question of a threshold value/curve. I would argue that the
error bars subsume the uncertainty, and using Alexis
arguments about expectation values, you can then use a fixed
value threshold. I think Alexis arguments are spot-on in
this case (FSC relationship to SNR is an expectation value),
and Marin's orthogonality argument is fundamentally
incorrect. The cross-terms in the presence of noise do have
an expectation value of zero, of course! The cross-terms
contribute to the uncertainty in the estimator, not to its
asymptotic value.
2) Closely related to point #1 is the issue that our
resolution estimates simply are not that precise. They do
have considerable uncertainty (which an FSC with error bars
would help to express). They also ignore differences in the
FSC curve at resolutions lower than the cutoff resolution,
which are also significant from the perspective of map
interpretation. If I have an FSC curve up close to 1 which
smoothly and rapidly falls to zero near some target
resolution, the quality of the map is not equivalent to an
FSC which begins falling gradually at much lower resolution
and undergoes considerable gymnastics before finally falling
below the 'threshold' value.
----
Our field takes these resolution numbers MUCH too seriously,
and have unwisely turned them into the sole measure of map
quality. I do not believe it is possible to make the FSC
into a single catch-all measure.
Following the 'error-bar' approach (if we can agree
on one) would properly associate an uncertainty with each
measured resolution value, to point out the limits of this
estimator in a way that a reviewer from any field could
easily encompass. Like the X-ray community, we need to adopt
additional criteria rather than continue these pointless
debates trying to make the FSC more statistically accurate
than it is possible for it to be.
----------------------------------------------------------------------------
Steven Ludtke, Ph.D.
Professor, Dept of Biochemistry and Mol. Biol. (www.bcm.edu/biochem)
Co-Director National Center For Macromolecular Imaging
(ncmi.bcm.edu)
Co-Director CIBR Center (www.bcm.edu/research/cibr)
Baylor College of Medicine
sludtke at bcm.edu
--
#yiv4227369519 #yiv4227369519 --
_filtered #yiv4227369519 {}
_filtered #yiv4227369519 {}
_filtered #yiv4227369519 {font-family:Cambria;panose-1:2 4
5 3 5 4 6 3 2 4;}
_filtered #yiv4227369519 {panose-1:2 11 6 0 4 5 2 2 2 4;}
#yiv4227369519
#yiv4227369519 p.yiv4227369519MsoNormal, #yiv4227369519
li.yiv4227369519MsoNormal, #yiv4227369519
div.yiv4227369519MsoNormal
{margin:0in;margin-bottom:.0001pt;font-size:12.0pt;font-family:Cambria;}
#yiv4227369519 a:link, #yiv4227369519
span.yiv4227369519MsoHyperlink
{color:blue;text-decoration:underline;}
#yiv4227369519 a:visited, #yiv4227369519
span.yiv4227369519MsoHyperlinkFollowed
{color:purple;text-decoration:underline;}
#yiv4227369519 p.yiv4227369519MsoAcetate, #yiv4227369519
li.yiv4227369519MsoAcetate, #yiv4227369519
div.yiv4227369519MsoAcetate
{margin:0in;margin-bottom:.0001pt;font-size:9.0pt;}
#yiv4227369519 span.yiv4227369519BalloonTextChar
{}
#yiv4227369519 .yiv4227369519MsoChpDefault
{font-family:Cambria;}
_filtered #yiv4227369519 {margin:1.0in 1.0in 1.0in 1.0in;}
#yiv4227369519 div.yiv4227369519WordSection1
{}
#yiv4227369519
Edward H.
Egelman, Ph.D.
Professor
Dept. of
Biochemistry and Molecular
Genetics
University of
Virginia
President
Biophysical
Society
phone:
434-924-8210
fax:
434-924-5069
egelman at virginia.edu
http://www.people.virginia.edu/~ehe2n
-----Inline Attachment Follows-----
_______________________________________________
3dem mailing list
3dem at ncmir.ucsd.edu
https://mail.ncmir.ucsd.edu/mailman/listinfo/3dem
More information about the 3dem
mailing list