[3dem] volume compression standard

[Ludtke, Steven J.]

Just for the sake of argument, I'll throw this out there:

https://urldefense.com/v3/__https://pmc.ncbi.nlm.nih.gov/articles/PMC9645247/__;!!Mih3wA!FJtTpAvtEfE4M_5GrMVUGjNh3aS0l2McQHQlxI1eVmDekUoQF6SnCEatLaRSoF5KYK34znp8OBlXgJUgP04$ 

gzipping only saves significant space if one discritizes the values stored in a float format or uses an int representation.

On Apr 28, 2026, at 7:38 AM, Daniel Larsson via 3dem <3dem at ncmir.ucsd.edu> wrote:

***CAUTION:*** This email is not from a BCM Source. Only click links or open attachments you know are safe.
________________________________
Would it not be very beneficiary of the community could agree on a loss-less compression standard for EM volumes (maps and masks)?

High-resolution maps take up significant disk space, often around 500-800 MB for large complexes. Considering that there are many volumes associated with each reconstruction (half-maps, full map, sharpened map, masks, etc) and multiply this by all the reconstructions for a typical project, and it adds up significantly. In addition, both RELION and CryoSPARC save volumes for intermediate iterations. These large files take a long time to download/upload, e.g. transferring between computers or when depositing structures. Opening maps for visualization in ChimeraX and Coot is also slowed down by having to read large files from disk.

So, we need a compression standard. Zip compression is fast and can reduce the file size considerably without changing the information in the file. Mask files are reduced to just a few percent of the original size (example 537 MB to 9 MB). What we need is:


  1.  Software generating volumes should by default write these in a lossless compressed format
  2.  Software visualizing volumes should accept and on-the-fly uncompress maps in memory
  3.  Refinement packages should accept and on-the-fly uncompress maps in memory
  4.  Repositories should accept compressed maps and un/recompress according to their own needs


Actors that immediately comes to my mind are CryoSPARC, RELION, Warp, CryoDRNG, IMOD, ChimeraX, PyMOL, Coot, Blender Molecular nodes, Phenix, Servalcat, wwPDB/EMDB. The exact format is up for debate, but why not something simple such as gzipped mrc files?

Best regards,
Daniel (written while impatiently waiting for files being uploaded to the wwPDB)









När du har kontakt med oss på Uppsala universitet med e-post så innebär det att vi behandlar dina personuppgifter. För att läsa mer om hur vi gör det kan du läsa här: https://urldefense.com/v3/__http://www.uu.se/om-uu/dataskydd-personuppgifter/__;!!Mih3wA!FJtTpAvtEfE4M_5GrMVUGjNh3aS0l2McQHQlxI1eVmDekUoQF6SnCEatLaRSoF5KYK34znp8OBlXyBjGZQc$ <https://urldefense.com/v3/__http://www.uu.se/om-uu/dataskydd-personuppgifter/__;!!Mih3wA!BUNKAv9U59Pq6jXGpg_HasLYL0bl-XFcbmYiAET8ZiKt_ru2lzuFBJJ-SqqvDQ3tw1khyJypThew9ThNQozonIt7z2lJrA$>

E-mailing Uppsala University means that we will process your personal data. For more information on how this is performed, please read here: https://urldefense.com/v3/__http://www.uu.se/en/about-uu/data-protection-policy__;!!Mih3wA!FJtTpAvtEfE4M_5GrMVUGjNh3aS0l2McQHQlxI1eVmDekUoQF6SnCEatLaRSoF5KYK34znp8OBlXX2neXhE$ <https://urldefense.com/v3/__http://www.uu.se/en/about-uu/data-protection-policy__;!!Mih3wA!BUNKAv9U59Pq6jXGpg_HasLYL0bl-XFcbmYiAET8ZiKt_ru2lzuFBJJ-SqqvDQ3tw1khyJypThew9ThNQozonIusG45pUg$> _______________________________________________
3dem mailing list
3dem at ncmir.ucsd.edu<mailto:3dem at ncmir.ucsd.edu>
https://urldefense.com/v3/__https://mail.ncmir.ucsd.edu/mailman/listinfo/3dem__;!!GKvG1w!c4KeXym_7XyZmm8YaD37WyGM_5btxOKmUoJ71fQnhHEHYyN7mE3s75UPV0cWhg8nk-lDiwJcXfWQbcBjv_kd$

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ncmir.ucsd.edu/pipermail/3dem/attachments/20260428/ee67ac8a/attachment.html>