[3dem] [ccpem] Variance map of volumes

[Marin van Heel]

There is absolutely no need to run such operations "in core" (that is, 
in the central memory of the computer).  You can calculate 3D variances 
of thousands of volumes on a simple Notebook computer as long as your 
disk (internal or external) can store all the data. Don't waste your 
time re-inventing the wheel... ;)

Marin

On 23/07/2017 04:03, Dan Asarnow wrote:
> If you don't have enough memory for all the volumes, you can use the 
> Welford running variance algorithm. It's also more numerically stable 
> than the standard formula.
>
> Here's an example: https://github.com/asarnow/pyem/blob/master/varmap.py
>
> Best,)
> -da
>
> On Fri, Jul 21, 2017 at 12:49 AM, Dominik A. Herbst 
> <dominik.herbst at unibas.ch <mailto:dominik.herbst at unibas.ch>> wrote:
>
>     Hi Reza,
>
>     If it is just about variance calculation, you can do that with a
>     few lines of python:
>
>
>     import numpy as np
>     from mrcz import ioMRC
>     from globimport glob
>
>     all_bootvols = glob("bootstrap_volumes/bootstrap_vol_??????.mrc")#
>     get list of all files
>
>     bootvols_data_arr_list = [ ioMRC.readMRC(bootvol)[0] for bootvol
>     in all_bootvols ]# read all volumes
>     bootvols_data_arr = np.concatenate([arr[np.newaxis] for arr in
>     bootvols_data_arr_list])# put them into an array
>     del(bootvols_data_arr_list)
>     variance = np.var(bootvols_data_arr, axis=0)# calculate variance
>     ioMRC.writeMRC( variance, 'variance.mrc', dtype='float32',
>     pixelsize=[1.0]*3, pixelunits=u"\AA", voltage=300.0, endian='le')
>     # write variance map to disk
>
>
>     ... should work.
>
>     ^^ for that you need the python module mrcz (pip install mrcz,
>     https://pypi.python.org/pypi/mrcz
>     <https://pypi.python.org/pypi/mrcz>).
>     Please note, this will read all volumes into memory and depending
>     an the amount of bootstrap volumes you might need a high-mem machine.
>
>     Best,
>
>     Dominik
>
>
>     Dominik A. Herbst | Dipl. Biochem, PhD Student | Biozentrum, University of Basel | Klingelbergstrasse 50/70 | CH-4056 Basel | Phone:+41 61 207 21 03 <tel:+41%2061%20207%2021%2003>  | Fax:+41 61 207 21 09 <tel:+41%2061%20207%2021%2009>  | Email:dominik.herbst at unibas.ch <mailto:dominik.herbst at unibas.ch>  |www.biozentrum.unibas.ch <http://www.biozentrum.unibas.ch>
>
>     On 07/21/2017 12:39 AM, Reza Khayat wrote:
>>
>>     Hi,
>>
>>     Can anyone recommend software that can calculate variance maps
>>     among several dozens of volumes? I’m aware of sxvar.py from
>>     Sparx. Thanks.
>>
>>     Best wishes,
>>     Reza
>>
>>     Reza Khayat, PhD
>>
>>     Assistant Professor
>>
>>     Department of Chemistry
>>
>>     City College of New York
>>
>>     85 Saint Nicholas Terrace, CDI 2.318
>>
>>     New York, NY 10031
>>
>>     http://www.khayatlab.org/
>>
>>     212-650-6070 <tel:%28212%29%20650-6070>
>>
>
>

-- 
==============================================================

     Prof Dr Ir Marin van Heel

     Laboratório Nacional de Nanotecnologia - LNNano
     CNPEM/LNNano, Campinas, Brazil

     tel:    +55-19-3518-2316
     mobile  +55-19-981809332
                  (041-19-981809332 TIM)
     Skype:  Marin.van.Heel
     email:  marin.vanheel(A_T)gmail.com
     and:    mvh.office(A_T)gmail.com

--------------------------------------------------
     Emeritus Professor of Cryo-EM Data Processing
     Leiden University
     Mobile NL: +31(0)652736618

     Emeritus Professor of Structural Biology
     Imperial College London
     Faculty of Natural Sciences
--------------------------------------------------

I receive many emails per day and, although I try,
there is no guarantee that I will actually read each incoming email.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ncmir.ucsd.edu/pipermail/3dem/attachments/20170723/91969a6f/attachment.html>