<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<div class="moz-cite-prefix"><br>
There is absolutely no need to run such operations "in core" (that
is, in the central memory of the computer). You can calculate 3D
variances of thousands of volumes on a simple Notebook computer as
long as your disk (internal or external) can store all the data.
Don't waste your time re-inventing the wheel... ;)<br>
<br>
Marin<br>
<br>
On 23/07/2017 04:03, Dan Asarnow wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CALiNCbbqw+gkgO7_EYRgKJZtR52x+pKrOxhis=y3DucLfLbYsA@mail.gmail.com">
<div dir="ltr">If you don't have enough memory for all the
volumes, you can use the Welford running variance algorithm.
It's also more numerically stable than the standard formula.
<div><br>
</div>
<div>Here's an example: <a
href="https://github.com/asarnow/pyem/blob/master/varmap.py"
moz-do-not-send="true">https://github.com/asarnow/pyem/blob/master/varmap.py</a></div>
<div><br>
</div>
<div>Best,)</div>
<div>-da</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Fri, Jul 21, 2017 at 12:49 AM,
Dominik A. Herbst <span dir="ltr"><<a
href="mailto:dominik.herbst@unibas.ch" target="_blank"
moz-do-not-send="true">dominik.herbst@unibas.ch</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
<p><font face="Times New Roman, Times, serif">Hi Reza,</font></p>
<p><font face="Times New Roman, Times, serif">If it is
just about variance calculation, you can do that with
a few lines of python:</font></p>
<p><font face="Times New Roman, Times, serif"><br>
</font></p>
<p><font face="Times New Roman, Times, serif"> </font><tt>import
numpy as np</tt><tt><br>
</tt><tt>from mrcz import ioMRC</tt><tt><br>
</tt><tt>from glob</tt><tt> import glob<br>
</tt><tt><br>
</tt><tt>all_bootvols = glob("bootstrap_volumes/<wbr>bootstrap_vol_??????.mrc")</tt><tt>
# get list of all files<br>
</tt><tt><br>
</tt><tt>bootvols_data_arr_list = [
ioMRC.readMRC(bootvol)[0] for bootvol in all_bootvols
]</tt><tt> # read all volumes<br>
</tt><tt>bootvols_data_arr = np.concatenate([arr[np.<wbr>newaxis]
for arr in bootvols_data_arr_list])</tt><tt> # put
them into an array<br>
</tt><tt>del(bootvols_data_arr_list)</tt><tt><br>
</tt><tt>variance = np.var(bootvols_data_arr, axis=0)</tt><tt>
# calculate variance<br>
</tt><tt>ioMRC.writeMRC( variance, 'variance.mrc',
dtype='float32', pixelsize=[1.0]*3, pixelunits=u"\AA",
voltage=300.0, endian='le') # write variance map to
disk</tt><font face="Times New Roman, Times, serif"> <br>
</font></p>
<p><font face="Times New Roman, Times, serif"><br>
</font></p>
<p><font face="Times New Roman, Times, serif">... should
work.</font><br>
</p>
<p><font face="Times New Roman, Times, serif">^^ for that
you need the python module mrcz (pip install mrcz, <a
class="m_-844694785772443938moz-txt-link-freetext"
href="https://pypi.python.org/pypi/mrcz"
target="_blank" moz-do-not-send="true">https://pypi.python.org/pypi/<wbr>mrcz</a>).<br>
Please note, this will read all volumes into memory
and depending an the amount of bootstrap volumes you
might need a high-mem machine.</font></p>
<p><font face="Times New Roman, Times, serif">Best,</font></p>
<p><font face="Times New Roman, Times, serif">Dominik</font></p>
<span class="HOEnZb"><font color="#888888">
<p><br>
</p>
<pre class="m_-844694785772443938moz-signature" cols="72">Dominik A. Herbst | Dipl. Biochem, PhD Student | Biozentrum, University of Basel | Klingelbergstrasse 50/70 | CH-4056 Basel | Phone: <a href="tel:+41%2061%20207%2021%2003" value="+41612072103" target="_blank" moz-do-not-send="true">+41 61 207 21 03</a> | Fax: <a href="tel:+41%2061%20207%2021%2009" value="+41612072109" target="_blank" moz-do-not-send="true">+41 61 207 21 09</a> | Email: <a class="m_-844694785772443938moz-txt-link-abbreviated" href="mailto:dominik.herbst@unibas.ch" target="_blank" moz-do-not-send="true">dominik.herbst@unibas.ch</a> | <a class="m_-844694785772443938moz-txt-link-abbreviated" href="http://www.biozentrum.unibas.ch" target="_blank" moz-do-not-send="true">www.biozentrum.unibas.ch</a>
</pre>
</font></span>
<div>
<div class="h5">
<div class="m_-844694785772443938moz-cite-prefix">On
07/21/2017 12:39 AM, Reza Khayat wrote:<br>
</div>
<blockquote type="cite">
<div class="m_-844694785772443938WordSection1">
<p class="MsoNormal">Hi,</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">Can anyone recommend software
that can calculate variance maps among several
dozens of volumes? I’m aware of sxvar.py from
Sparx. Thanks.</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">Best wishes,<br>
Reza</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal"><span style="color:#1f497d">Reza
Khayat, PhD</span><span style="color:#212121"></span></p>
<p class="MsoNormal"><span style="color:#1f497d">Assistant
Professor</span><span style="color:#212121"></span></p>
<p class="MsoNormal"><span style="color:#1f497d">Department
of Chemistry</span><span style="color:#212121"></span></p>
<p class="MsoNormal"><span style="color:#1f497d">City
College of New York</span></p>
<p class="MsoNormal"><span style="color:#1f497d">85
Saint Nicholas Terrace, CDI 2.318</span><span
style="color:#212121"></span></p>
<p class="MsoNormal"><span style="color:#1f497d">New
York, NY 10031</span><span
style="color:#212121"></span></p>
<p class="MsoNormal"><span style="color:#1f497d"><a
href="http://www.khayatlab.org/"
target="_blank" moz-do-not-send="true"><span
style="color:#0563c1">http://www.khayatlab.org/</span></a></span><span
style="color:#212121"></span></p>
<p class="MsoNormal"><span style="color:#1f497d"><a
href="tel:%28212%29%20650-6070"
value="+12126506070" target="_blank"
moz-do-not-send="true">212-650-6070</a></span><span
style="color:#212121"></span></p>
<p class="MsoNormal"> </p>
</div>
</blockquote>
<br>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
<p><br>
</p>
<pre class="moz-signature" cols="72">--
==============================================================
Prof Dr Ir Marin van Heel
Laboratório Nacional de Nanotecnologia - LNNano
CNPEM/LNNano, Campinas, Brazil
tel: +55-19-3518-2316
mobile +55-19-981809332
(041-19-981809332 TIM)
Skype: Marin.van.Heel
email: marin.vanheel(A_T)gmail.com
and: mvh.office(A_T)gmail.com
--------------------------------------------------
Emeritus Professor of Cryo-EM Data Processing
Leiden University
Mobile NL: +31(0)652736618
Emeritus Professor of Structural Biology
Imperial College London
Faculty of Natural Sciences
--------------------------------------------------
I receive many emails per day and, although I try,
there is no guarantee that I will actually read each incoming email. </pre>
</body>
</html>