[3dem] [ccpem] lost gain reference image

Tue Dec 4 15:29:18 PST 2018

I guess you still have to define what R is😊

Regards,
Pawel

On Dec 4, 2018, at 4:56 PM, Carlos Oscar S. Sorzano <coss at cnb.csic.es<mailto:coss at cnb.csic.es>> wrote:

Dear Marin,

thank you for the reminder, since I had certainly forgotten. What you call a contrast reversal is most likely a nomenclature issue. If you have two images, I_observed and I_ideal, and the relationship between the two is simply a multiplicative term

I_observed = H I_ideal

exactly the same relationship is described by the relationship (assuming there are no zeros)

F I_observed = I_ideal

The relationship between F and H is F=1/H, which implies a contrast reversion. I have explictly avoided G to denote any of these two images (F or H) to avoid confusions. Which of the two is called "gain" is a matter of definition. I prefer describing H as the gain, but I understand other people may prefer the other option.

Regarding R^2, it is true I did not define it assuming it was general knowledge. It is a very standard statistical measure of quality called the coefficient of determination (https://en.wikipedia.org/wiki/Coefficient_of_determination<https://urldefense.proofpoint.com/v2/url?u=https-3A__en.wikipedia.org_wiki_Coefficient-5Fof-5Fdetermination&d=DwMDaQ&c=6vgNTiRn9_pqCD9hKx9JgXN1VapJQ8JVoF8oWH1AgfQ&r=vDDf9rsFxPMXm8JgJa6hc4B9V4qKr7wftnDkLIRdshI&m=BmtVdlQoD6zHW2o25IZa6vGz0lwSvq24Wch-T3W-Tcw&s=nYF1wmtGLXiMY93v2J7Kd4Eq6UmjU-dbkYDbd0-hMFU&e=>), and it expresses the fraction of the original variance explained by a model. If R^2=1, the model has totally explained the original variance, while if R^2=0, the model explains the same variance as the mean of the observations, which is the simplest, sensible model we could have.

Cheers, Carlos Oscar

El 04/12/2018 a las 21:48, Marin van Heel escribió:
Hi Carlos Oscar!
I just remembered I had posed a question about your camera normalisation paper ( https://www.ncbi.nlm.nih.gov/pubmed/29551714<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ncbi.nlm.nih.gov_pubmed_29551714&d=DwMDaQ&c=6vgNTiRn9_pqCD9hKx9JgXN1VapJQ8JVoF8oWH1AgfQ&r=vDDf9rsFxPMXm8JgJa6hc4B9V4qKr7wftnDkLIRdshI&m=BmtVdlQoD6zHW2o25IZa6vGz0lwSvq24Wch-T3W-Tcw&s=GbJK0WJKZBqEfGERNJc2mspMIWIiDrX5-tdTkJL6Fnw&e=>.) on this site some two months ago, in which you critisized our 2015 camera normalisation paper (https://www.nature.com/articles/srep10317<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.nature.com_articles_srep10317&d=DwMDaQ&c=6vgNTiRn9_pqCD9hKx9JgXN1VapJQ8JVoF8oWH1AgfQ&r=vDDf9rsFxPMXm8JgJa6hc4B9V4qKr7wftnDkLIRdshI&m=BmtVdlQoD6zHW2o25IZa6vGz0lwSvq24Wch-T3W-Tcw&s=V-yc_wFNb5n9w2eVCBL9uP-MzMmoxOdcVYCzjqm9fp4&e=>).  Did you already respond to my question and I missed your answer?

Cheers
Marin

My question was:

QUOTE:
What you call our "gain image" is - apart from an erroneous contrast reversal - actually more similar to the "official" gain image in your Fig 1 than does the one generated with your proposed algorithm.  I would be interested in knowing what the R2 turns out to be after you correct the contrast reversal since it visually is better than yours. It would be nice if you could respond to this mailing including that information!? By the way how exactly is this R2 metric defined (I could not find it anywhere in the paper)?
END QUOTE

On Tue, Oct 2, 2018 at 10:21 PM Marin van Heel <marin.vanheel at googlemail.com<mailto:marin.vanheel at googlemail.com>> wrote:
Dear Carlos Oscar and Dimitry,

Unfortunately, you seem to have missed the point of our Afanasyev 2015 paper. Our paper does not try to duplicate the "experimentally determined Gain image" but tries to normalize the signal from each pixel to the same average and the same standard deviation at the exposure and contrast level that the data set was recorded. Our approach typically improves significantly on standard "a priori" flat field/gain  corrections.

We are not directly interested in generic "gain images" as such and we certainly  do not generate "gain images" that have an inverted contrast when compared to the other ones you have in Figure #1 of your paper. Your comments on our methods are thus not appropriate:  "To the best of our knowledge, the only article that addresses a similar problem is that of Afanasyev et al. (2015). In their work, they assimilate the gain of the camera to the standard deviation of each pixel over a large number of movies, and they prove this is a successful way of identifying dead pixels. However, our results show that this approach does not provide a consistent gain estimation (Fig. 1)."

What you call our "gain image" is - apart from an erroneous contrast reversal - actually more similar to the "official" gain image in your Fig 1 than does the one generated with your proposed algorithm.  I would be interested in knowing what the R2 turns out to be after you correct the contrast reversal since it visually is better than yours. It would be nice if you could respond to this mailing including that information!? By the way how exactly is this R2 metric defined (I could not find it anywhere in the paper)?

I would want to suggest you and your colleagues to use the FRC metric to prove that your approach does indeed remove the influence of the various patterns of your detectors exhibits.

My two cents

Marin

=====================

On 02/10/2018 15:19, Carlos Oscar Sorzano wrote:
By the way, in our article we compared both methods (ours and Marin).

Kind regards, Carlos Oscar

On 01/10/2018 21:23, Marin van Heel wrote:
Dear Da,

In IMAGIC-4D  you can perform the necessary camera correction! (https://www.nature.com/articles/srep10317<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.nature.com_articles_srep10317&d=DwMDaQ&c=6vgNTiRn9_pqCD9hKx9JgXN1VapJQ8JVoF8oWH1AgfQ&r=vDDf9rsFxPMXm8JgJa6hc4B9V4qKr7wftnDkLIRdshI&m=BmtVdlQoD6zHW2o25IZa6vGz0lwSvq24Wch-T3W-Tcw&s=V-yc_wFNb5n9w2eVCBL9uP-MzMmoxOdcVYCzjqm9fp4&e=>).  It does it better than any manufactures correction and improves the data significantly even when performed after using the standard gain correction.

Cheers,

Marin

=====================================================

On 01/10/2018 15:36, Da Cui wrote:
Hi all,
    The gain reference image for one dataset was missing by accident. In order to achieve a more accurate motioncor result, does anyone have idea about how to generate a gain reference image from the dataset (around 3k movies)?
    Thank you so much for your help!!!
---Da

########################################################################

To unsubscribe from the CCPEM list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCPEM&A=1<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.jiscmail.ac.uk_cgi-2Dbin_webadmin-3FSUBED1-3DCCPEM-26A-3D1&d=DwMDaQ&c=6vgNTiRn9_pqCD9hKx9JgXN1VapJQ8JVoF8oWH1AgfQ&r=vDDf9rsFxPMXm8JgJa6hc4B9V4qKr7wftnDkLIRdshI&m=BmtVdlQoD6zHW2o25IZa6vGz0lwSvq24Wch-T3W-Tcw&s=yT3kjQpRGyp5cKZEwjnWpMpmpVgZ4oGFbYclNxESB2M&e=>

--
==============================================================

    Prof Dr Ir Marin van Heel

    Laboratório Nacional de Nanotecnologia - LNNano
    CNPEM/LNNano, Campinas, Brazil

    tel:    +55-19-3518-2316
    mobile  +55-19-983455450 (current)
    mobile  +55-19-981809332
                 (041-19-981809332 TIM)
    Skype:  Marin.van.Heel
    email:  marin.vanheel(A_T)gmail.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__gmail.com&d=DwMDaQ&c=6vgNTiRn9_pqCD9hKx9JgXN1VapJQ8JVoF8oWH1AgfQ&r=vDDf9rsFxPMXm8JgJa6hc4B9V4qKr7wftnDkLIRdshI&m=BmtVdlQoD6zHW2o25IZa6vGz0lwSvq24Wch-T3W-Tcw&s=ZQzkqvtzk56LQirMk_M_H7faPSmvycM_1nEcCV6dkxI&e=>
            marin.vanheel(A_T)lnnano.cnpem.br<https://urldefense.proofpoint.com/v2/url?u=http-3A__lnnano.cnpem.br&d=DwMDaQ&c=6vgNTiRn9_pqCD9hKx9JgXN1VapJQ8JVoF8oWH1AgfQ&r=vDDf9rsFxPMXm8JgJa6hc4B9V4qKr7wftnDkLIRdshI&m=BmtVdlQoD6zHW2o25IZa6vGz0lwSvq24Wch-T3W-Tcw&s=tvzbuokQ-c7H8ClCHLw_SKUYf6T2mQVycJEj9nQ3_vA&e=>
    and:    mvh.office(A_T)gmail.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__gmail.com&d=DwMDaQ&c=6vgNTiRn9_pqCD9hKx9JgXN1VapJQ8JVoF8oWH1AgfQ&r=vDDf9rsFxPMXm8JgJa6hc4B9V4qKr7wftnDkLIRdshI&m=BmtVdlQoD6zHW2o25IZa6vGz0lwSvq24Wch-T3W-Tcw&s=ZQzkqvtzk56LQirMk_M_H7faPSmvycM_1nEcCV6dkxI&e=>

--------------------------------------------------
    Emeritus Professor of Cryo-EM Data Processing
    Leiden University
    Mobile NL: +31(0)652736618 (ALWAYS ACTIVE SMS)
--------------------------------------------------
    Emeritus Professor of Structural Biology
    Imperial College London
    Faculty of Natural Sciences
    email: m.vanheel(A_T)imperial.ac.uk<https://urldefense.proofpoint.com/v2/url?u=http-3A__imperial.ac.uk&d=DwMDaQ&c=6vgNTiRn9_pqCD9hKx9JgXN1VapJQ8JVoF8oWH1AgfQ&r=vDDf9rsFxPMXm8JgJa6hc4B9V4qKr7wftnDkLIRdshI&m=BmtVdlQoD6zHW2o25IZa6vGz0lwSvq24Wch-T3W-Tcw&s=xjNoEFFbWleDIHeBdgmnx4pQ3hWrS4DA9L09cGZ7fX8&e=>
--------------------------------------------------

I receive many emails per day and, although I try,
there is no guarantee that I will actually read each incoming email.

--
------------------------------------------------------------------------
Carlos Oscar Sánchez Sorzano                  e-mail:   coss at cnb.csic.es<mailto:coss at cnb.csic.es>
Biocomputing unit                             http://i2pc.es/coss<https://urldefense.proofpoint.com/v2/url?u=http-3A__i2pc.es_coss&d=DwMDaQ&c=6vgNTiRn9_pqCD9hKx9JgXN1VapJQ8JVoF8oWH1AgfQ&r=vDDf9rsFxPMXm8JgJa6hc4B9V4qKr7wftnDkLIRdshI&m=BmtVdlQoD6zHW2o25IZa6vGz0lwSvq24Wch-T3W-Tcw&s=e4rYqG-wlLmE48eJd2mVvUW7P2JP8xA6uSYQ7Q41j-k&e=>
National Center of Biotechnology (CSIC)
c/Darwin, 3
Campus Universidad Autónoma (Cantoblanco)     Tlf: 34-91-585 4510
28049 MADRID (SPAIN)                          Fax: 34-91-585 4506
------------------------------------------------------------------------

_______________________________________________
3dem mailing list
3dem at ncmir.ucsd.edu<mailto:3dem at ncmir.ucsd.edu>
https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.ncmir.ucsd.edu_mailman_listinfo_3dem&d=DwICAg&c=6vgNTiRn9_pqCD9hKx9JgXN1VapJQ8JVoF8oWH1AgfQ&r=vDDf9rsFxPMXm8JgJa6hc4B9V4qKr7wftnDkLIRdshI&m=BmtVdlQoD6zHW2o25IZa6vGz0lwSvq24Wch-T3W-Tcw&s=Fnm7k8NmR1cuxq4VkEc5oznU31Aryw9_ULlCEVqzpC8&e=
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ncmir.ucsd.edu/pipermail/3dem/attachments/20181204/7daab416/attachment-0001.html>