[3dem] [ccpem] Converting MRC to tiff

Ludtke, Steven J sludtke at bcm.edu
Mon Jun 4 05:30:14 PDT 2018


The key, as a couple of others have mentioned, is (for counting-mode data) to store uncorrected raw movies, and independently store the gain reference. For most images in counting mode the number of counts doesn't extend beyond a few bits, and the (lossless) compression is both fast and extremely effective. For many people using SerialEM for their automated data collection this has become the default strategy.  (8 bits with lossless compression + gain reference)
This decision should be non-controversial, as no information loss is involved.

A bit more controversial:
It is absolutely true that with corrected images or integrating mode data (which is often scaled up to a large number of bits) that, since most of the 'information' in the image is actually noise it can only be losslessly compressed by a very small amount.

There is a strong argument to be made that, given that the number of electron impacts per pixel in per movie frame is unlikely to be more than, say, 16 or 32 e-, (4 or 5 bits), that amplifying the signal up to 2000 "counts" (11 bits, 1/2 of which are uncompressible noise) and storing all of these bits is pointless and extremely wasteful. Either an electron impacted or it did not impact. Over many frames of recording such discrete counts you can gain additional bits, but counting "fractional electrons" in integrating mode serves little actual purpose. That is to say that even in integrating mode scaling the data down to 8 bits (or even fewer) for individual movie frames is likely to be a lossless operation in terms of actual information about the pattern of electron impacts.

On the other hand, we have Fei Sun's argument that you should make corrections for Poisson statistics in integrating mode. This doesn't make the bit reduction argument go away, but it isn't immediately obvious to me how many extra bits one would like to retain to make this work well (if any). Regardless, such corrections could easily be made before bit reduction.

That said, I fear this argument doesn't move many CryoEM practitioners who fear losing information content from their "raw data". This choice comes with a very significant financial cost (perhaps ~5x higher storage costs). In the long run, however, I suspect these points will all be moot, as detectors become more universally capable of e- counting. If you can do counting, and you elect to do integration instead, you are already throwing away so much information that anything bit reduction could conceivably lose is negligible in comparison.

--------------------------------------------------------------------------------------
Steven Ludtke, Ph.D. <sludtke at bcm.edu<mailto:sludtke at bcm.edu>>                      Baylor College of Medicine
Charles C. Bell Jr., Professor of Structural Biology
Dept. of Biochemistry and Molecular Biology                      (www.bcm.edu/biochem<http://www.bcm.edu/biochem>)
Academic Director, CryoEM Core                                        (cryoem.bcm.edu<http://cryoem.bcm.edu>)
Co-Director CIBR Center                                    (www.bcm.edu/research/cibr<http://www.bcm.edu/research/cibr>)



On Jun 3, 2018, at 7:31 PM, Nicolas, William (William) <wjnicol at caltech.edu<mailto:wjnicol at caltech.edu>> wrote:

***CAUTION:*** This email is not from a BCM Source. Only click links or open attachments you know are safe.
________________________________
Hey Yehuda,

As Marin is saying this isn’t going to save any space to convert MRC in TIFF. However, if you happen to want to perform this task, this is how you can do it:

Both methods involve ImageJ. You can either use Bioformat plugin to open .mrc although I don’t like doing that. Instead there is another plugin called U759_inputoutput.jar (http://www.cmib.fr/en/download/softwares/input-output.html<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.cmib.fr_en_download_softwares_input-2Doutput.html&d=DwMGaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=Dk5VoQQ-wINYVssLMZihyC5Dj_sWYKxCyKz9E4Lp3gc&m=6Ls3H0A_8EAztVuLZM43JcJmYK-mQFz0dxDf064qk3s&s=LXLyCByhKGR4_7U5qlLUB_jdhgfeuuSOouh-GgdxJAA&e=>) that allows you to open .mrc with ImageJ. Then all you got to do is save as > TIFF.
Check if metadata are kept by doing so.

Cheers,

William Nicolas, HHMI Postdoc.

Jensen Laboratory - Meyerowitz Laboratory
Division of Biology and Biological Engineering
1200 East California Blvd
Postal code: 156-29
California Institute of Technology
Pasadena, CA 91125, USA





Le 3 juin 2018 à 03:18, Marin van Heel <marin.vanheel at googlemail.com<mailto:marin.vanheel at googlemail.com>> a écrit :


Dear Yehuda Halfon

The em2em converter (Image-Science.de<https://urldefense.proofpoint.com/v2/url?u=http-3A__image-2Dscience.de_&d=DwMGaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=Dk5VoQQ-wINYVssLMZihyC5Dj_sWYKxCyKz9E4Lp3gc&m=6Ls3H0A_8EAztVuLZM43JcJmYK-mQFz0dxDf064qk3s&s=oKtRm8XtxdcGvclFpNrn7XVOyea95slt5rqildDWY0c&e=>) should do the trick but storing (4-bit/8-bit?) MRC movies in tiff is not a good idea: very few programs can handle tiff stacks. Moreover if the TIFFs are not compressed you will not necessarily win any space. You are probably best off using a standard loss-less compression program ("zip"), and convert them back when you need it again. You must always keep your original raw data "forever" in a loss-less form.

Marin van Heel


On 03/06/2018 06:02, Yehuda Halfon wrote:
Hi there,

We have a bunch of MRC movie files the are eating at out storage, and since more are coming I was wondering if there is a good way to convert them into tiff to save space?

I know that the best way is to save them directly as tif from EPU/ serialEM and that is what we will to in the future. But we need to find a solution to the ones we have now.

Thanks,

Yehuda Halfon

________________________________

To unsubscribe from the CCPEM list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCPEM&A=1<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.jiscmail.ac.uk_cgi-2Dbin_webadmin-3FSUBED1-3DCCPEM-26A-3D1&d=DwMGaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=Dk5VoQQ-wINYVssLMZihyC5Dj_sWYKxCyKz9E4Lp3gc&m=6Ls3H0A_8EAztVuLZM43JcJmYK-mQFz0dxDf064qk3s&s=bwCiEZA0uoCodVMZSruCyh5KlRZzHUxV6Yiy4lcS6oA&e=>


--
==============================================================

    Prof Dr Ir Marin van Heel

    Laboratório Nacional de Nanotecnologia - LNNano
    CNPEM/LNNano, Campinas, Brazil

    tel:    +55-19-3518-2316
    mobile  +55-19-983455450 (current)
    mobile  +55-19-981809332
                 (041-19-981809332 TIM)
    Skype:  Marin.van.Heel
    email:  marin.vanheel(A_T)gmail.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__gmail.com_&d=DwMGaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=Dk5VoQQ-wINYVssLMZihyC5Dj_sWYKxCyKz9E4Lp3gc&m=6Ls3H0A_8EAztVuLZM43JcJmYK-mQFz0dxDf064qk3s&s=ZZItgauQuYwPavYgMV_B_b9RxqtBp22WoGwRcZbltRc&e=>
            marin.vanheel(A_T)lnnano.cnpem.br<https://urldefense.proofpoint.com/v2/url?u=http-3A__lnnano.cnpem.br_&d=DwMGaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=Dk5VoQQ-wINYVssLMZihyC5Dj_sWYKxCyKz9E4Lp3gc&m=6Ls3H0A_8EAztVuLZM43JcJmYK-mQFz0dxDf064qk3s&s=4JNgzSKk8X__jRgPhau73XCRH1wObg9rUApvvlBljOY&e=>
    and:    mvh.office(A_T)gmail.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__gmail.com_&d=DwMGaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=Dk5VoQQ-wINYVssLMZihyC5Dj_sWYKxCyKz9E4Lp3gc&m=6Ls3H0A_8EAztVuLZM43JcJmYK-mQFz0dxDf064qk3s&s=ZZItgauQuYwPavYgMV_B_b9RxqtBp22WoGwRcZbltRc&e=>

--------------------------------------------------
    Emeritus Professor of Cryo-EM Data Processing
    Leiden University
    Mobile NL: +31(0)652736618 (ALWAYS ACTIVE SMS)
--------------------------------------------------
    Emeritus Professor of Structural Biology
    Imperial College London
    Faculty of Natural Sciences
    email: m.vanheel(A_T)imperial.ac.uk<https://urldefense.proofpoint.com/v2/url?u=http-3A__imperial.ac.uk_&d=DwMGaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=Dk5VoQQ-wINYVssLMZihyC5Dj_sWYKxCyKz9E4Lp3gc&m=6Ls3H0A_8EAztVuLZM43JcJmYK-mQFz0dxDf064qk3s&s=xKUW5tG1lQiTkQOn44YiUzyAo1lt75MfBYDEyuD6BNM&e=>
--------------------------------------------------

I receive many emails per day and, although I try,
there is no guarantee that I will actually read each incoming email.

_______________________________________________
3dem mailing list
3dem at ncmir.ucsd.edu<mailto:3dem at ncmir.ucsd.edu>
https://mail.ncmir.ucsd.edu/mailman/listinfo/3dem<https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.ncmir.ucsd.edu_mailman_listinfo_3dem&d=DwMGaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=Dk5VoQQ-wINYVssLMZihyC5Dj_sWYKxCyKz9E4Lp3gc&m=6Ls3H0A_8EAztVuLZM43JcJmYK-mQFz0dxDf064qk3s&s=XM3eYNzjJV-BPYi3zHVY4YlSn6lsDkiRMz_7G9bNBIc&e=>

_______________________________________________
3dem mailing list
3dem at ncmir.ucsd.edu<mailto:3dem at ncmir.ucsd.edu>
https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.ncmir.ucsd.edu_mailman_listinfo_3dem&d=DwICAg&c=ZQs-KZ8oxEw0p81sqgiaRA&r=Dk5VoQQ-wINYVssLMZihyC5Dj_sWYKxCyKz9E4Lp3gc&m=6Ls3H0A_8EAztVuLZM43JcJmYK-mQFz0dxDf064qk3s&s=XM3eYNzjJV-BPYi3zHVY4YlSn6lsDkiRMz_7G9bNBIc&e=

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ncmir.ucsd.edu/pipermail/3dem/attachments/20180604/381f1752/attachment-0001.html>


More information about the 3dem mailing list