[3dem] Seeking advice on Relion/CryoSPARC workstation

[Guillaume Gaullier]

Hi Artemis,

There are many things to consider, and you should consult with an IT expert for the details. But the big picture is as follows:

- You need 4 GPUs if you want to use cryoSPARC Live in a way that it can keep up with data collection (it assigns two GPUs for motion correction, CTF estimation and particle picking, then one GPU for 2D classification, then the 4th GPU for 3D reconstruction and refinement; if you have less then 4 GPUs, you need to stop one of these tasks to be able to start another, so it’s less automated).

- The model of GPU you choose will pretty much dictate which case you need (some cases have enough room for only two of the latest GPUs). A common choice a couple years ago was 4 Nvidia RTX 2080 Ti. I don’t know about the latest models, but whatever is the successor of this one should offer good performance for cryoEM data processing. In any case, you want the "blower" type cards, that exhaust the generated heat out of the case (as opposed to the regular type that exhausts heat around it inside the case, which puts more strain on the case’s cooling system).

- Don’t be cheap with RAM; 128 GB is an absolute minimum, and more will be helpful eventually (not all programs for cryoEM data analysis can use an SSD scratch disk).

- Some job types in RELION are not GPU-accelerated (motion correction if you don’t use MotionCor2, bayesian polishing, CTF refinement), so more CPU cores is beneficial for these job types. If you face a compromise between less but faster CPU cores and vice versa, it is almost certainly more beneficial to choose slightly slower CPU cores but more of them.

- You absolutely need a dedicated SSD scratch disk. The OS is typically on an SSD in these workstations, so some people use their account’s home directory as a scratch space; but I would recommend against doing this! if not managed well, this will eventually fill this disk, and you don’t want to run out of space on the disk housing the OS. The heavy read/write operations on the scratch space will also wear out the underlying disk faster, so again not a great idea for the disk housing the OS: in case of disk failure, it’s a lot easier to change a separate, dedicated scratch disk than having to also reinstall the OS and all programs.

- You absolutely need RAID6 for the workstation’s internal storage (or a 10 Gb/s link to the NAS, and the NAS setup in RAID6), this is the sweet spot between hardware fault tolerance and read/write performance. A slow storage space will be limiting for the initial step of motion correction, where whole movies need to be read fast (and there are often too many movies to put them all on the scratch disk). I am using two systems, one with a 2-disk RAID1 and another one with a RAID6 (forgot how many disks it has), and the difference in performance is very noticeable. On the RAID1 system, most of the time during a motion correction job is spent with the GPUs waiting for the disks; on the RAID6 system, the GPUs are all running at full capacity during the entire job. And don’t be cheap with total usable storage space: having to move datasets in and out of the workstation too often because of limiting storage space is quickly annoying. I would say no less than 40 TB, but of course this depends a lot on how many datasets you typically work on at any given time and how large each individual dataset can be.

These guidelines are meant to optimize for performance, and a workstation of this type will likely be very noisy; but it should not be a big problem if you already planned a dedicated space for it.

I hope this helps,

Guillaume


On 5 Jan 2022, at 18:44, akosta at imm.cnrs.fr<mailto:akosta at imm.cnrs.fr> wrote:

Dear colleagues,
I'm looking for advice on building a workstation for general compute tasks as well as cryo-EM data analysis (for example with Relion or cryoSPARC).
The workstation will be placed next to a NAS storage device in our Data Center and few people will have access (less that 10). The informatics team will deal with installation and maintenance.
Are there any minimal or/and optimal requirements, especially what would your GPU and RAM recommendation be?
I am completely ignorant on this matter, so I will appreciate any advice!
Best regards
Artemis

--
Artemis Kosta
Service de microscopie
FR3479 Institut de Microbiologie de la Méditerranée
Aix Marseille Université
31 chemin Joseph Aiguier, Bat N
CS 70071 - 13402 Marseille cedex 09
France
akosta at imm.cnrs.fr<mailto:akosta at imm.cnrs.fr>
0033 491 164055
https://urldefense.proofpoint.com/v2/url?u=http-3A__www.imm.cnrs.fr_microscopie_&d=DwIDaQ&c=-35OiAkTchMrZOngvJPOeA&r=L7-zyQ-04fFCMRqzLIOnx7H0exGZHwIQe_wMPuY600I&m=p2o4y9hYqgLOqKF0I9JDaDfJz-qh-YBAsE0yNa8KnAuQvopJQnP_drt_FwVVEslw&s=mMMscHYkoAtesPOcdF7Uewe7gPKjAmK9WQEdqKVp-Bw&e= _______________________________________________
3dem mailing list
3dem at ncmir.ucsd.edu
https://mail.ncmir.ucsd.edu/mailman/listinfo/3dem









När du har kontakt med oss på Uppsala universitet med e-post så innebär det att vi behandlar dina personuppgifter. För att läsa mer om hur vi gör det kan du läsa här: https://urldefense.proofpoint.com/v2/url?u=http-3A__www.uu.se_om-2Duu_dataskydd-2Dpersonuppgifter_&d=DwIGaQ&c=-35OiAkTchMrZOngvJPOeA&r=L7-zyQ-04fFCMRqzLIOnx7H0exGZHwIQe_wMPuY600I&m=QZUuJijXm5Q6fNo2M_C9yWo4bFwNDZmAcd8eYR4oyCnXcRpTHJ8Gsww7JB_XFO9k&s=AbtB1bnJqmeYE6NO5LiYMQTuKNEkr33dnSlWsGrX9TE&e= 

E-mailing Uppsala University means that we will process your personal data. For more information on how this is performed, please read here: https://urldefense.proofpoint.com/v2/url?u=http-3A__www.uu.se_en_about-2Duu_data-2Dprotection-2Dpolicy&d=DwIGaQ&c=-35OiAkTchMrZOngvJPOeA&r=L7-zyQ-04fFCMRqzLIOnx7H0exGZHwIQe_wMPuY600I&m=QZUuJijXm5Q6fNo2M_C9yWo4bFwNDZmAcd8eYR4oyCnXcRpTHJ8Gsww7JB_XFO9k&s=HibWtvA-BgmJj3H0IAi6xENHTEGmQV6x5iFG8V4zAxs&e= 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ncmir.ucsd.edu/pipermail/3dem/attachments/20220110/0acb7918/attachment.html>