<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
Interested in all of your takes on 2 things:
<div class=""><br class="">
</div>
<div class="">(1) I think libtiff has supported building with Zstandard support for a bit (though it is not a common compilation option). My guess is TIFF+ZStandard will do quite a bit better in the general-case compression dept. than LZW. Does anyone expect
that software packages would automatically work with such images if I built libtiff with Zstandard and linked against it? I suppose I could just try it, but have not had the chance yet…</div>
<div class=""><br class="">
</div>
<div class="">(2) For cameras like the F3 that output movies with gains pre-applied- it would seem some amount of lossy compression is required to achieve reasonable compression savings- has anyone looked at using “error-bounded” lossy compression algorithms
in this use case? My very naive attempt at quantizing the pixel values (drop 5 LSB) was promising but needs a lot more characterization:</div>
<div class=""><br class="">
</div>
<div class=""> original Falcon 3 MRC to TIFF-LZW<span class="Apple-tab-span" style="white-space:pre">
</span>: 1.9GB -> 1.3GB</div>
<div class="">quantized Falcon 3 MRC to TIFF-LZW<span class="Apple-tab-span" style="white-space: pre;">
</span>: 1.9GB -> 295MB</div>
<div class=""><br class="">
</div>
<div class="">both movies got ~ the same CTF fit limit after motion correction and fitting (3.3 vs 3.1), the RMSD between the two final aligned averages was 0.041.</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
<div><br class="">
<blockquote type="cite" class="">
<div class="">On Aug 29, 2019, at 8:53 PM, Carlos Oscar S. Sorzano <<a href="mailto:coss@cnb.csic.es" class="">coss@cnb.csic.es</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div text="#000000" bgcolor="#FFFFFF" class="">
<p class="">Dear all,</p>
<p class="">as it seems to be the logical trend now, Scipion also keeps the movies in the original (compressed or not) format, and decompress them if necessary on the fly (the gain is also considered after this decompression). In this way, there is no need
for extra space.</p>
<p class="">Kind regards, Carlos Oscar<br class="">
</p>
<div class="moz-cite-prefix">El 29/08/2019 a las 21:35, Ali Punjani escribió:<br class="">
</div>
<blockquote type="cite" cite="mid:CAP_b-V_enQCdtXNkuQxhVDwBEs8+TR_zuTR93pKkuNG40S9XKA@mail.gmail.com" class="">
<div class="">
<div style="color:rgb(49,49,49);font-size:16px;word-spacing:1px" dir="auto" class="">
<div dir="auto" style="font-size:1rem" class="">Hi Matthew,</div>
</div>
<div dir="auto" style="color:rgb(49,49,49);font-size:16px;word-spacing:1px" class="">
<br class="">
</div>
<div dir="auto" style="color:rgb(49,49,49);word-spacing:1px;font-size:1rem" class="">
In cryoSPARC, LZW compressed tiff files are handled natively and decompressed in parallel on the fly as needed. They are never stored in decompressed form, so the original tiff files (created by IMOD, serialEM, etc) are retained throughout a project. This is
a very common workflow, though as noted, for effective tiff compression the movies should not be gain corrected - the gain reference must be stored separately and provided at import time. </div>
</div>
<div dir="auto" style="color:rgb(49,49,49);word-spacing:1px;font-size:1rem" class="">
<br class="">
</div>
<div dir="auto" style="color:rgb(49,49,49);word-spacing:1px;font-size:1rem" class="">
Ali</div>
<div class=""><br class="">
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Thu, Aug 29, 2019 at 3:19 PM Matthew H. Cahn <<a href="mailto:mcahn@princeton.edu" moz-do-not-send="true" class="">mcahn@princeton.edu</a>> wrote:<br class="">
</div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word" class="">Is it possible to convert mrc files to compressed tiff immediately after data collection, and use that compressed tiff for all the downstream processing? For example, if I compress the mrc files from our Krios to
tiff with LZW (lossless) compression, using IMOD’s mrc2tif, can motioncor2, RELION, CryoSPARC, etc. handle that format and keep everything compressed throughout the process?
<div class=""><br class="">
</div>
<div class="">— Matthew</div>
<div class=""><br class="">
<div class="">
<div style="word-wrap:break-word" class="">
<div style="font-variant-caps: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;" class="">
<font size="2" class=""><b class="">|| Matthew Cahn | Linux Administrator | Dept. of Molecular Biology / Research Computing | Princeton University | (609) 258-5404 | <a href="mailto:mcahn@princeton.edu" target="_blank" moz-do-not-send="true" class="">mcahn@princeton.edu</a>
||<br class="">
</b></font><br class="">
</div>
</div>
</div>
<br class="">
</div>
</div>
_______________________________________________<br class="">
3dem mailing list<br class="">
<a href="mailto:3dem@ncmir.ucsd.edu" target="_blank" moz-do-not-send="true" class="">3dem@ncmir.ucsd.edu</a><br class="">
<a href="https://mail.ncmir.ucsd.edu/mailman/listinfo/3dem" rel="noreferrer" target="_blank" moz-do-not-send="true" class="">https://mail.ncmir.ucsd.edu/mailman/listinfo/3dem</a><br class="">
</blockquote>
</div>
</div>
<br class="">
<fieldset class="mimeAttachmentHeader"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
3dem mailing list
<a class="moz-txt-link-abbreviated" href="mailto:3dem@ncmir.ucsd.edu">3dem@ncmir.ucsd.edu</a>
<a class="moz-txt-link-freetext" href="https://mail.ncmir.ucsd.edu/mailman/listinfo/3dem">https://mail.ncmir.ucsd.edu/mailman/listinfo/3dem</a>
</pre>
</blockquote>
<pre class="moz-signature" cols="72">--
------------------------------------------------------------------------
Carlos Oscar Sánchez Sorzano e-mail: <a class="moz-txt-link-abbreviated" href="mailto:coss@cnb.csic.es">coss@cnb.csic.es</a>
Biocomputing unit <a class="moz-txt-link-freetext" href="http://i2pc.es/coss">http://i2pc.es/coss</a>
National Center of Biotechnology (CSIC)
c/Darwin, 3
Campus Universidad Autónoma (Cantoblanco) Tlf: 34-91-585 4510
28049 MADRID (SPAIN) Fax: 34-91-585 4506
------------------------------------------------------------------------
</pre>
</div>
_______________________________________________<br class="">
3dem mailing list<br class="">
<a href="mailto:3dem@ncmir.ucsd.edu" class="">3dem@ncmir.ucsd.edu</a><br class="">
https://mail.ncmir.ucsd.edu/mailman/listinfo/3dem<br class="">
</div>
</blockquote>
</div>
<br class="">
</div>
</body>
</html>