[3dem] Operating system for local server environment

Tue May 10 14:34:10 PDT 2022

Dear Christian,

Princeton University and the Institute for Advanced Study maintains a Linux distribution called Springdale.  It’s based on Red Hat Enterprise Linux.  We use it on all of our high-performance computing clusters, and also on some workstations, and some virtual servers serving web apps and other things.  It’s available to the public, here: https://urldefense.proofpoint.com/v2/url?u=https-3A__springdale.math.ias.edu_&d=DwIGaQ&c=-35OiAkTchMrZOngvJPOeA&r=L7-zyQ-04fFCMRqzLIOnx7H0exGZHwIQe_wMPuY600I&m=KJGXzJpgEwv3KpnwM49PeoeDRRFhIFj_Vm4B5DjwYFm6uDE0n02VqJv93PDRLzKM&s=eVnyMT2VKkiL6eFglp2Enj6LYLg-XtKA5yj07kuWFys&e= .

I’m running CryoSPARC, RELION, and AlphaFold in this environment.  The only problem I’ve had is that the Nvidia driver installation on a workstation was pretty involved, though I imagine it’s similar on other Linux distrubtions.

Best,
Matthew Cahn
Linux Administrator/Programmer/Lecturer
Princeton University
Department of Molecular Biology

||  Matthew Cahn  |  Linux Administrator  |  Dept. of Molecular Biology / Research Computing  |  Princeton University  |  (609) 258-5404  |  mcahn at princeton.edu<mailto:mcahn at princeton.edu>  ||

On May 10, 2022, at 6:48 AM, 3dem-request at ncmir.ucsd.edu<mailto:3dem-request at ncmir.ucsd.edu> wrote:

Send 3dem mailing list submissions to
3dem at ncmir.ucsd.edu<mailto:3dem at ncmir.ucsd.edu>

To subscribe or unsubscribe via the World Wide Web, visit
https://urldefense.proofpoint.com/v2/url?u=https-3A__nam12.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Fmail.ncmir.ucsd.edu-252Fmailman-252Flistinfo-252F3dem-26amp-3Bdata-3D05-257C01-257Cmcahn-2540princeton.edu-257C2f550a2886914885887a08da3272ba4e-257C2ff601167431425db5af077d7791bda4-257C0-257C0-257C637877765618827138-257CUnknown-257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0-253D-257C3000-257C-257C-257C-26amp-3Bsdata-3DDDvXsfTEo7u-252FyNUXp9FH9gJfrJk8J7mngXx8HoFj13U-253D-26amp-3Breserved-3D0&d=DwIGaQ&c=-35OiAkTchMrZOngvJPOeA&r=L7-zyQ-04fFCMRqzLIOnx7H0exGZHwIQe_wMPuY600I&m=KJGXzJpgEwv3KpnwM49PeoeDRRFhIFj_Vm4B5DjwYFm6uDE0n02VqJv93PDRLzKM&s=FGPNvC1MgBm_R_NCWnRWwi66ohbDDDo0fM9FXTRkxHg&e= 
or, via email, send a message with subject or body 'help' to
3dem-request at ncmir.ucsd.edu

You can reach the person managing the list at
3dem-owner at ncmir.ucsd.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of 3dem digest..."

Today's Topics:

  1. Operating system for local server environment (Christian T?ting)
  2. Re: Operating system for local server environment (Grigory Sharov)
  3. Re: Operating system for local server environment
     (Guillaume Gaullier)
  4. Re: Operating system for local server environment
     (Kilian Schnelle)

----------------------------------------------------------------------

Message: 1
Date: Tue, 10 May 2022 10:28:43 +0200
From: Christian T?ting  <christian.tueting at biochemtech.uni-halle.de>
To: <3dem at ncmir.ucsd.edu>
Subject: [3dem] Operating system for local server environment
Message-ID: <627A223B020000B000063B45 at gwia0.itz.uni-halle.de>
Content-Type: text/plain; charset=UTF-8

Dear 3dem mailing list recipients,

I would like to get your opinion on which operating system is the most
suitable for our local server environment.

To frame the question:
We have multiple local workstations and storage servers with a very
heterogeneous architecture:
Different GPUs (2080Ti, 3090), different CPUs (Intel i9, Intel Xeon, AMD
EPYC), sometimes with a cache SSD, sometimes not. Different memory
sizes/types, different hard drive organization, etc.

Currently, Centos 7 is installed on all the machines, but as you might
know, this OS is pretty old and the EOL is just in 2 years. As some of
the workstations needs some maintenance, I would like to not install
Centos 7 again, just to redo this in a couple of months.

Our need for the machines is the following:
cryoSPARC, Scipion3, Relion
AlphaFold, NAMD/Gromacs

For the storage/main node servers, there is no special need. We don't
have any workload balancer like SLURM, and also the storage servers
"just" stores the data, without any special need. Interconnectivity is
established over NFS or sshfs, but these should be possible in any unix
OS.

I personally have experience with [X/K/U]buntu, Centos, and some Arch
Linux, so I am aware of the basic differences (e.g. apt vs yum).

Proposed by cryoSPARC is Ubuntu, at some other point (might be in the
Scipion mailing list), Rocky Linux was proposed. Also, afaik, Scientific
Linux is often used.

As I am used to Centos 7, handling outdated repos (e.g. GCC4 being the
default compiler) is a daily business, I would like to avoid in the
future. I think, this strongly points to Ubuntu, where a big and active
community is present.

Thank you for your feedback :)

Best regards
Christian
Dr. rer. nat. Christian T?ting

Kastritis Laboratory for Biomolecular Research
Cryo-Electron Microscopy & Computational Structural Biology
________________________________________________
Martin-Luther-Universit?t Halle-Wittenberg
Biozentrum, Room A.2.19
IWE ZIK HALOmem NWG III
"Kryo-Elektronenmikroskopie an Membranproteinkomplexen"
Weinbergweg 22, 06120 Halle
tel: +49 345 5524985
web (Lab): https://urldefense.proofpoint.com/v2/url?u=https-3A__nam12.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Furldefense.proofpoint.com-252Fv2-252Furl-253Fu-253Dhttps-2D3A-5F-5Fblogs.urz.uni-2D2Dhalle.de-5Fkastritislab-5F-2526d-253DDwIDaQ-2526c-253D-2D35OiAkTchMrZOngvJPOeA-2526r-253DL7-2DzyQ-2D04fFCMRqzLIOnx7H0exGZHwIQe-5FwMPuY600I-2526m-253DM5tV4i7vwtyCEWiBFwqltoUDbWfJrQ6Rq4OeG3w6Pf3szICL5sanWtIgUYWC3B5-5F-2526s-253DUJR09zZF0h0c-2DvOZd4S3RmffzgEiJ7y-5FXgkY01dJpm8-2526e-253D-26amp-3Bdata-3D05-257C01-257Cmcahn-2540princeton.edu-257C2f550a2886914885887a08da3272ba4e-257C2ff601167431425db5af077d7791bda4-257C0-257C0-257C637877765618827138-257CUnknown-257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0-253D-257C3000-257C-257C-257C-26amp-3Bsdata-3DNAH88OCfLYM1XJZX2-252FaOunPdawC8VEafwvAH8ivgpSY-253D-26amp-3Breserved-3D0&d=DwIGaQ&c=-35OiAkTchMrZOngvJPOeA&r=L7-zyQ-04fFCMRqzLIOnx7H0exGZHwIQe_wMPuY600I&m=KJGXzJpgEwv3KpnwM49PeoeDRRFhIFj_Vm4B5DjwYFm6uDE0n02VqJv93PDRLzKM&s=Qd6OOQyYGYWesnCtNu51irzRUCnvdsBN3SKqMHSeav8&e= 
web (HALOmem): https://urldefense.proofpoint.com/v2/url?u=https-3A__nam12.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Furldefense.proofpoint.com-252Fv2-252Furl-253Fu-253Dhttps-2D3A-5F-5Fwww.halomem.de-5Fen-5F-2526d-253DDwIDaQ-2526c-253D-2D35OiAkTchMrZOngvJPOeA-2526r-253DL7-2DzyQ-2D04fFCMRqzLIOnx7H0exGZHwIQe-5FwMPuY600I-2526m-253DM5tV4i7vwtyCEWiBFwqltoUDbWfJrQ6Rq4OeG3w6Pf3szICL5sanWtIgUYWC3B5-5F-2526s-253DYPBQ4XfDHsZK6U3AgwzMuzhoNUcTjc4S2ntJIQ9jrJo-2526e-253D-26amp-3Bdata-3D05-257C01-257Cmcahn-2540princeton.edu-257C2f550a2886914885887a08da3272ba4e-257C2ff601167431425db5af077d7791bda4-257C0-257C0-257C637877765618827138-257CUnknown-257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0-253D-257C3000-257C-257C-257C-26amp-3Bsdata-3D5-252BfInG2DDmsZyUz1Q-252FTzzows5ggBcAk-252BFHUVD3VAjpA-253D-26amp-3Breserved-3D0&d=DwIGaQ&c=-35OiAkTchMrZOngvJPOeA&r=L7-zyQ-04fFCMRqzLIOnx7H0exGZHwIQe_wMPuY600I&m=KJGXzJpgEwv3KpnwM49PeoeDRRFhIFj_Vm4B5DjwYFm6uDE0n02VqJv93PDRLzKM&s=i57lYdpC7Nnonq_Js-b4z48DcFsJrEeZrC22tI6bWDw&e= 

------------------------------

Message: 2
Date: Tue, 10 May 2022 09:47:35 +0100
From: Grigory Sharov <sharov.grigory at gmail.com>
To: Christian T?ting  <christian.tueting at biochemtech.uni-halle.de>
Cc: 3dem at ncmir.ucsd.edu
Subject: Re: [3dem] Operating system for local server environment
Message-ID:
<CAJZ7w0dQQaRBgXh9Jtokap47fqFWVYV8xyv+MWy2z0fyC-TvJg at mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Dear Christian,

I personally have experience with [X/K/U]buntu, Centos, and some Arch
Linux, so I am aware of the basic differences (e.g. apt vs yum).

If you are going to manage this server then this is your answer. Use the OS
that you have most experience with.

I wouldn't use Arch Linux for a server (it is mostly for "bleeding edge"
desktop enthusiasts) and if you go for Ubuntu, in my opinion a server does
not have GUI, so no X/Kubuntu. Personally, I'm not an Ubuntu fan and use
Debian.

Proposed by cryoSPARC is Ubuntu, at some other point (might be in the
Scipion mailing list), Rocky Linux was proposed. Also, afaik, Scientific
Linux is often used.

CentOS has been buried by Red Hat/IBM, so Rocky linux, Alma linux or
VZlinux are the alternatives currently being developed by the community.

Hope this helps,
Grigory

--------------------------------------------------------------------------------
Grigory Sharov, Ph.D.

MRC Laboratory of Molecular Biology,
Francis Crick Avenue,
Cambridge Biomedical Campus,
Cambridge CB2 0QH, UK.
tel. +44 (0) 1223 267228 <+44%201223%20267228>
e-mail: gsharov at mrc-lmb.cam.ac.uk

On Tue, May 10, 2022 at 9:29 AM Christian T?ting <
christian.tueting at biochemtech.uni-halle.de> wrote:

Dear 3dem mailing list recipients,

I would like to get your opinion on which operating system is the most
suitable for our local server environment.

To frame the question:
We have multiple local workstations and storage servers with a very
heterogeneous architecture:
Different GPUs (2080Ti, 3090), different CPUs (Intel i9, Intel Xeon, AMD
EPYC), sometimes with a cache SSD, sometimes not. Different memory
sizes/types, different hard drive organization, etc.

Currently, Centos 7 is installed on all the machines, but as you might
know, this OS is pretty old and the EOL is just in 2 years. As some of
the workstations needs some maintenance, I would like to not install
Centos 7 again, just to redo this in a couple of months.

Our need for the machines is the following:
cryoSPARC, Scipion3, Relion
AlphaFold, NAMD/Gromacs

For the storage/main node servers, there is no special need. We don't
have any workload balancer like SLURM, and also the storage servers
"just" stores the data, without any special need. Interconnectivity is
established over NFS or sshfs, but these should be possible in any unix
OS.

I personally have experience with [X/K/U]buntu, Centos, and some Arch
Linux, so I am aware of the basic differences (e.g. apt vs yum).

Proposed by cryoSPARC is Ubuntu, at some other point (might be in the
Scipion mailing list), Rocky Linux was proposed. Also, afaik, Scientific
Linux is often used.

As I am used to Centos 7, handling outdated repos (e.g. GCC4 being the
default compiler) is a daily business, I would like to avoid in the
future. I think, this strongly points to Ubuntu, where a big and active
community is present.

Thank you for your feedback :)

Best regards
Christian
Dr. rer. nat. Christian T?ting

Kastritis Laboratory for Biomolecular Research
Cryo-Electron Microscopy & Computational Structural Biology
________________________________________________
Martin-Luther-Universit?t Halle-Wittenberg
Biozentrum, Room A.2.19
IWE ZIK HALOmem NWG III
"Kryo-Elektronenmikroskopie an Membranproteinkomplexen"
Weinbergweg 22, 06120 Halle
tel: +49 345 5524985
web (Lab):
https://urldefense.proofpoint.com/v2/url?u=https-3A__nam12.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Furldefense.proofpoint.com-252Fv2-252Furl-253Fu-253Dhttps-2D3A-5F-5Fblogs.urz.uni-2D2Dhalle.de-5Fkastritislab-5F-2526d-253DDwIDaQ-2526c-253D-2D35OiAkTchMrZOngvJPOeA-2526r-253DL7-2DzyQ-2D04fFCMRqzLIOnx7H0exGZHwIQe-5FwMPuY600I-2526m-253DM5tV4i7vwtyCEWiBFwqltoUDbWfJrQ6Rq4OeG3w6Pf3szICL5sanWtIgUYWC3B5-5F-2526s-253DUJR09zZF0h0c-2DvOZd4S3RmffzgEiJ7y-5FXgkY01dJpm8-2526e-253D-26amp-3Bdata-3D05-257C01-257Cmcahn-2540princeton.edu-257C2f550a2886914885887a08da3272ba4e-257C2ff601167431425db5af077d7791bda4-257C0-257C0-257C637877765618827138-257CUnknown-257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0-253D-257C3000-257C-257C-257C-26amp-3Bsdata-3DNAH88OCfLYM1XJZX2-252FaOunPdawC8VEafwvAH8ivgpSY-253D-26amp-3Breserved-3D0&d=DwIGaQ&c=-35OiAkTchMrZOngvJPOeA&r=L7-zyQ-04fFCMRqzLIOnx7H0exGZHwIQe_wMPuY600I&m=KJGXzJpgEwv3KpnwM49PeoeDRRFhIFj_Vm4B5DjwYFm6uDE0n02VqJv93PDRLzKM&s=Qd6OOQyYGYWesnCtNu51irzRUCnvdsBN3SKqMHSeav8&e= 
web (HALOmem):
https://urldefense.proofpoint.com/v2/url?u=https-3A__nam12.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Furldefense.proofpoint.com-252Fv2-252Furl-253Fu-253Dhttps-2D3A-5F-5Fwww.halomem.de-5Fen-5F-2526d-253DDwIDaQ-2526c-253D-2D35OiAkTchMrZOngvJPOeA-2526r-253DL7-2DzyQ-2D04fFCMRqzLIOnx7H0exGZHwIQe-5FwMPuY600I-2526m-253DM5tV4i7vwtyCEWiBFwqltoUDbWfJrQ6Rq4OeG3w6Pf3szICL5sanWtIgUYWC3B5-5F-2526s-253DYPBQ4XfDHsZK6U3AgwzMuzhoNUcTjc4S2ntJIQ9jrJo-2526e-253D-26amp-3Bdata-3D05-257C01-257Cmcahn-2540princeton.edu-257C2f550a2886914885887a08da3272ba4e-257C2ff601167431425db5af077d7791bda4-257C0-257C0-257C637877765618827138-257CUnknown-257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0-253D-257C3000-257C-257C-257C-26amp-3Bsdata-3D5-252BfInG2DDmsZyUz1Q-252FTzzows5ggBcAk-252BFHUVD3VAjpA-253D-26amp-3Breserved-3D0&d=DwIGaQ&c=-35OiAkTchMrZOngvJPOeA&r=L7-zyQ-04fFCMRqzLIOnx7H0exGZHwIQe_wMPuY600I&m=KJGXzJpgEwv3KpnwM49PeoeDRRFhIFj_Vm4B5DjwYFm6uDE0n02VqJv93PDRLzKM&s=i57lYdpC7Nnonq_Js-b4z48DcFsJrEeZrC22tI6bWDw&e= 

_______________________________________________
3dem mailing list
3dem at ncmir.ucsd.edu
https://urldefense.proofpoint.com/v2/url?u=https-3A__nam12.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Fmail.ncmir.ucsd.edu-252Fmailman-252Flistinfo-252F3dem-26amp-3Bdata-3D05-257C01-257Cmcahn-2540princeton.edu-257C2f550a2886914885887a08da3272ba4e-257C2ff601167431425db5af077d7791bda4-257C0-257C0-257C637877765618827138-257CUnknown-257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0-253D-257C3000-257C-257C-257C-26amp-3Bsdata-3DDDvXsfTEo7u-252FyNUXp9FH9gJfrJk8J7mngXx8HoFj13U-253D-26amp-3Breserved-3D0&d=DwIGaQ&c=-35OiAkTchMrZOngvJPOeA&r=L7-zyQ-04fFCMRqzLIOnx7H0exGZHwIQe_wMPuY600I&m=KJGXzJpgEwv3KpnwM49PeoeDRRFhIFj_Vm4B5DjwYFm6uDE0n02VqJv93PDRLzKM&s=FGPNvC1MgBm_R_NCWnRWwi66ohbDDDo0fM9FXTRkxHg&e= 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://urldefense.proofpoint.com/v2/url?u=https-3A__nam12.safelinks.protection.outlook.com_-3Furl-3Dhttp-253A-252F-252Fmail.ncmir.ucsd.edu-252Fpipermail-252F3dem-252Fattachments-252F20220510-252F8499e87b-252Fattachment-2D0001.html-26amp-3Bdata-3D05-257C01-257Cmcahn-2540princeton.edu-257C2f550a2886914885887a08da3272ba4e-257C2ff601167431425db5af077d7791bda4-257C0-257C0-257C637877765618827138-257CUnknown-257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0-253D-257C3000-257C-257C-257C-26amp-3Bsdata-3DuKlImgtK-252B23nzznWmONotKSYqto7NcYvkMusd1uf2u8-253D-26amp-3Breserved-3D0&d=DwIGaQ&c=-35OiAkTchMrZOngvJPOeA&r=L7-zyQ-04fFCMRqzLIOnx7H0exGZHwIQe_wMPuY600I&m=KJGXzJpgEwv3KpnwM49PeoeDRRFhIFj_Vm4B5DjwYFm6uDE0n02VqJv93PDRLzKM&s=8VAx9z58-foYVItNugbTv0z9ZXN2xhgN3qC35vOxh9E&e= >

------------------------------

Message: 3
Date: Tue, 10 May 2022 09:36:59 +0000
From: Guillaume Gaullier <guillaume.gaullier at icm.uu.se>
To: Christian T?ting  <christian.tueting at biochemtech.uni-halle.de>
Cc: "3dem at ncmir.ucsd.edu" <3dem at ncmir.ucsd.edu>
Subject: Re: [3dem] Operating system for local server environment
Message-ID: <0857AEA3-1EC9-4F95-A48E-100BBF895E7E at icm.uu.se>
Content-Type: text/plain; charset="utf-8"

Hello Christian,

I only manage two workstations, but with very different hardware (all the types of differences you describe). They initially ran CentOS 8, but since its end of support was shifted earlier than originally scheduled, I did a fresh installation of Rocky Linux (8.4 at the time, now they are both on 8.5 through regular updates). I have had a very good experience with Rocky as the base system to run cryoSPARC and RELION (I now even have them set up to automatically queue jobs to SLURM, it works fine), and a bunch of other cryoEM programs (Topaz, cryoDRGN, etc.). It is pretty much "set up once and forget", with minimal maintenance efforts.

If you are already familiar with CentOS, Rocky will be a smooth transition. Alma Linux is probably also a sensible option (also a RHEL rebuild).

CentOS Stream is not a good option, because it updates the kernel more often than new versions of the Nvidia driver get released (because Nvidia follows RHEL updates, and CentOS Stream?s purpose is to be just ahead of RHEL; it doesn?t seem like Nvidia has any plan of supporting CentOS Stream).

Arch is updated even more often than CentOS Stream, so it will probably be the same kind of struggle to maintain than CentOS 7, but the opposite way: most things in the base OS will be too new and changing too often for the research software you want to run. Also note that Nvidia does not support Arch, so you would have to rely on the community repository. Since the Nvidia driver is so critical to a base system for GPU workstations, I would recommend limiting your choice of OS to the ones Nvidia supports, see: https://urldefense.proofpoint.com/v2/url?u=https-3A__nam12.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Furldefense.proofpoint.com-252Fv2-252Furl-253Fu-253Dhttps-2D3A-5F-5Fdeveloper.nvidia.com-5Fcuda-2D2Ddownloads-2D3Ftarget-2D5Fos-2D3DLinux-2D26target-2D5Farch-2D3Dx86-2D5F64-2526d-253DDwIGaQ-2526c-253D-2D35OiAkTchMrZOngvJPOeA-2526r-253DL7-2DzyQ-2D04fFCMRqzLIOnx7H0exGZHwIQe-5FwMPuY600I-2526m-253DCkvlK51L4BVJq-2DWOK3yXrz-5F7t-2DL3SNOr3WKUExu2ZcNWBojC7nAL5PdrAi5yHcbu-2526s-253D-5FcizLH5i2tmuI0nb8-2DuL8hV0aW0pDqboVN9toCsQZOc-2526e-253D-26amp-3Bdata-3D05-257C01-257Cmcahn-2540princeton.edu-257C2f550a2886914885887a08da3272ba4e-257C2ff601167431425db5af077d7791bda4-257C0-257C0-257C637877765618827138-257CUnknown-257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0-253D-257C3000-257C-257C-257C-26amp-3Bsdata-3D-252BvUfjaPbYi8FAJ2Ecz-252BsgQpMaFKuRvR8kfgb6bblPSI-253D-26amp-3Breserved-3D0&d=DwIGaQ&c=-35OiAkTchMrZOngvJPOeA&r=L7-zyQ-04fFCMRqzLIOnx7H0exGZHwIQe_wMPuY600I&m=KJGXzJpgEwv3KpnwM49PeoeDRRFhIFj_Vm4B5DjwYFm6uDE0n02VqJv93PDRLzKM&s=KChAWsYUjG6CIx3PJR0Fhc8FhnV_Hyo0V_scQmoI4ME&e=   (knowing that you can use Rocky or Alma instead of RHEL). Moreover, I would strongly recommend installing the Nvidia driver with their R!
PM or DEB
 package, not with the "runfile", as this will ensure you never end up with incompatible driver and kernel versions (which the runfile installation does not guard against; been there, done that).

I don?t have recent experience with the Debian family to say much about them, but it would probably be fine if you limit your choice to the versions supported by Nvidia (so Debian 11 or Ubuntu 20.04 LTS).

I hope this helps,

Guillaume

On 10 May 2022, at 10:28, Christian T?ting <christian.tueting at biochemtech.uni-halle.de<mailto:christian.tueting at biochemtech.uni-halle.de>> wrote:

Dear 3dem mailing list recipients,

I would like to get your opinion on which operating system is the most
suitable for our local server environment.

To frame the question:
We have multiple local workstations and storage servers with a very
heterogeneous architecture:
Different GPUs (2080Ti, 3090), different CPUs (Intel i9, Intel Xeon, AMD
EPYC), sometimes with a cache SSD, sometimes not. Different memory
sizes/types, different hard drive organization, etc.

Currently, Centos 7 is installed on all the machines, but as you might
know, this OS is pretty old and the EOL is just in 2 years. As some of
the workstations needs some maintenance, I would like to not install
Centos 7 again, just to redo this in a couple of months.

Our need for the machines is the following:
cryoSPARC, Scipion3, Relion
AlphaFold, NAMD/Gromacs

For the storage/main node servers, there is no special need. We don't
have any workload balancer like SLURM, and also the storage servers
"just" stores the data, without any special need. Interconnectivity is
established over NFS or sshfs, but these should be possible in any unix
OS.

I personally have experience with [X/K/U]buntu, Centos, and some Arch
Linux, so I am aware of the basic differences (e.g. apt vs yum).

Proposed by cryoSPARC is Ubuntu, at some other point (might be in the
Scipion mailing list), Rocky Linux was proposed. Also, afaik, Scientific
Linux is often used.

As I am used to Centos 7, handling outdated repos (e.g. GCC4 being the
default compiler) is a daily business, I would like to avoid in the
future. I think, this strongly points to Ubuntu, where a big and active
community is present.

Thank you for your feedback :)

Best regards
Christian
Dr. rer. nat. Christian T?ting

Kastritis Laboratory for Biomolecular Research
Cryo-Electron Microscopy & Computational Structural Biology
________________________________________________
Martin-Luther-Universit?t Halle-Wittenberg
Biozentrum, Room A.2.19
IWE ZIK HALOmem NWG III
"Kryo-Elektronenmikroskopie an Membranproteinkomplexen"
Weinbergweg 22, 06120 Halle
tel: +49 345 5524985
web (Lab): https://urldefense.proofpoint.com/v2/url?u=https-3A__nam12.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Furldefense.proofpoint.com-252Fv2-252Furl-253Fu-253Dhttps-2D3A-5F-5Fblogs.urz.uni-2D2Dhalle.de-5Fkastritislab-5F-2526d-253DDwIDaQ-2526c-253D-2D35OiAkTchMrZOngvJPOeA-2526r-253DL7-2DzyQ-2D04fFCMRqzLIOnx7H0exGZHwIQe-5FwMPuY600I-2526m-253DM5tV4i7vwtyCEWiBFwqltoUDbWfJrQ6Rq4OeG3w6Pf3szICL5sanWtIgUYWC3B5-5F-2526s-253DUJR09zZF0h0c-2DvOZd4S3RmffzgEiJ7y-5FXgkY01dJpm8-2526e-253D-26amp-3Bdata-3D05-257C01-257Cmcahn-2540princeton.edu-257C2f550a2886914885887a08da3272ba4e-257C2ff601167431425db5af077d7791bda4-257C0-257C0-257C637877765618827138-257CUnknown-257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0-253D-257C3000-257C-257C-257C-26amp-3Bsdata-3DNAH88OCfLYM1XJZX2-252FaOunPdawC8VEafwvAH8ivgpSY-253D-26amp-3Breserved-3D0&d=DwIGaQ&c=-35OiAkTchMrZOngvJPOeA&r=L7-zyQ-04fFCMRqzLIOnx7H0exGZHwIQe_wMPuY600I&m=KJGXzJpgEwv3KpnwM49PeoeDRRFhIFj_Vm4B5DjwYFm6uDE0n02VqJv93PDRLzKM&s=Qd6OOQyYGYWesnCtNu51irzRUCnvdsBN3SKqMHSeav8&e= 
web (HALOmem): https://urldefense.proofpoint.com/v2/url?u=https-3A__nam12.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Furldefense.proofpoint.com-252Fv2-252Furl-253Fu-253Dhttps-2D3A-5F-5Fwww.halomem.de-5Fen-5F-2526d-253DDwIDaQ-2526c-253D-2D35OiAkTchMrZOngvJPOeA-2526r-253DL7-2DzyQ-2D04fFCMRqzLIOnx7H0exGZHwIQe-5FwMPuY600I-2526m-253DM5tV4i7vwtyCEWiBFwqltoUDbWfJrQ6Rq4OeG3w6Pf3szICL5sanWtIgUYWC3B5-5F-2526s-253DYPBQ4XfDHsZK6U3AgwzMuzhoNUcTjc4S2ntJIQ9jrJo-2526e-253D-26amp-3Bdata-3D05-257C01-257Cmcahn-2540princeton.edu-257C2f550a2886914885887a08da3272ba4e-257C2ff601167431425db5af077d7791bda4-257C0-257C0-257C637877765618827138-257CUnknown-257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0-253D-257C3000-257C-257C-257C-26amp-3Bsdata-3D5-252BfInG2DDmsZyUz1Q-252FTzzows5ggBcAk-252BFHUVD3VAjpA-253D-26amp-3Breserved-3D0&d=DwIGaQ&c=-35OiAkTchMrZOngvJPOeA&r=L7-zyQ-04fFCMRqzLIOnx7H0exGZHwIQe_wMPuY600I&m=KJGXzJpgEwv3KpnwM49PeoeDRRFhIFj_Vm4B5DjwYFm6uDE0n02VqJv93PDRLzKM&s=i57lYdpC7Nnonq_Js-b4z48DcFsJrEeZrC22tI6bWDw&e= 

_______________________________________________
3dem mailing list
3dem at ncmir.ucsd.edu<mailto:3dem at ncmir.ucsd.edu>
https://urldefense.proofpoint.com/v2/url?u=https-3A__nam12.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Fmail.ncmir.ucsd.edu-252Fmailman-252Flistinfo-252F3dem-26amp-3Bdata-3D05-257C01-257Cmcahn-2540princeton.edu-257C2f550a2886914885887a08da3272ba4e-257C2ff601167431425db5af077d7791bda4-257C0-257C0-257C637877765618827138-257CUnknown-257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0-253D-257C3000-257C-257C-257C-26amp-3Bsdata-3DDDvXsfTEo7u-252FyNUXp9FH9gJfrJk8J7mngXx8HoFj13U-253D-26amp-3Breserved-3D0&d=DwIGaQ&c=-35OiAkTchMrZOngvJPOeA&r=L7-zyQ-04fFCMRqzLIOnx7H0exGZHwIQe_wMPuY600I&m=KJGXzJpgEwv3KpnwM49PeoeDRRFhIFj_Vm4B5DjwYFm6uDE0n02VqJv93PDRLzKM&s=FGPNvC1MgBm_R_NCWnRWwi66ohbDDDo0fM9FXTRkxHg&e= 

N?r du har kontakt med oss p? Uppsala universitet med e-post s? inneb?r det att vi behandlar dina personuppgifter. F?r att l?sa mer om hur vi g?r det kan du l?sa h?r: https://urldefense.proofpoint.com/v2/url?u=https-3A__nam12.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Furldefense.proofpoint.com-252Fv2-252Furl-253Fu-253Dhttp-2D3A-5F-5Fwww.uu.se-5Fom-2D2Duu-5Fdataskydd-2D2Dpersonuppgifter-5F-2526d-253DDwIGaQ-2526c-253D-2D35OiAkTchMrZOngvJPOeA-2526r-253DL7-2DzyQ-2D04fFCMRqzLIOnx7H0exGZHwIQe-5FwMPuY600I-2526m-253DCkvlK51L4BVJq-2DWOK3yXrz-5F7t-2DL3SNOr3WKUExu2ZcNWBojC7nAL5PdrAi5yHcbu-2526s-253D1H-2D3ATQw0LNN6FPBxOEq0wh03XKun3SLQSI-5FopFoY64-2526e-253D-26amp-3Bdata-3D05-257C01-257Cmcahn-2540princeton.edu-257C2f550a2886914885887a08da3272ba4e-257C2ff601167431425db5af077d7791bda4-257C0-257C0-257C637877765618827138-257CUnknown-257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0-253D-257C3000-257C-257C-257C-26amp-3Bsdata-3DYaLO1Dd7ItgIHxhkqW01ybzP85YEO5sCRZyLNhRxNao-253D-26amp-3Breserved-3D0&d=DwIGaQ&c=-35OiAkTchMrZOngvJPOeA&r=L7-zyQ-04fFCMRqzLIOnx7H0exGZHwIQe_wMPuY600I&m=KJGXzJpgEwv3KpnwM49PeoeDRRFhIFj_Vm4B5DjwYFm6uDE0n02VqJv93PDRLzKM&s=RnhG8KpuJ8hq_4rBx5kv1pk2hrYnfs4nhZ7KUOhrOcQ&e= 

E-mailing Uppsala University means that we will process your personal data. For more information on how this is performed, please read here: https://urldefense.proofpoint.com/v2/url?u=https-3A__nam12.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Furldefense.proofpoint.com-252Fv2-252Furl-253Fu-253Dhttp-2D3A-5F-5Fwww.uu.se-5Fen-5Fabout-2D2Duu-5Fdata-2D2Dprotection-2D2Dpolicy-2526d-253DDwIGaQ-2526c-253D-2D35OiAkTchMrZOngvJPOeA-2526r-253DL7-2DzyQ-2D04fFCMRqzLIOnx7H0exGZHwIQe-5FwMPuY600I-2526m-253DCkvlK51L4BVJq-2DWOK3yXrz-5F7t-2DL3SNOr3WKUExu2ZcNWBojC7nAL5PdrAi5yHcbu-2526s-253Doqq2d5QKG47Iu5cXgfEOcrLUtJtfziSF2Gww78VTp2k-2526e-253D-26amp-3Bdata-3D05-257C01-257Cmcahn-2540princeton.edu-257C2f550a2886914885887a08da3272ba4e-257C2ff601167431425db5af077d7791bda4-257C0-257C0-257C637877765618827138-257CUnknown-257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0-253D-257C3000-257C-257C-257C-26amp-3Bsdata-3Dn681fGjKYTqOgllht43l-252Fg8fNON51EhNAA4i3-252BVDsEE-253D-26amp-3Breserved-3D0&d=DwIGaQ&c=-35OiAkTchMrZOngvJPOeA&r=L7-zyQ-04fFCMRqzLIOnx7H0exGZHwIQe_wMPuY600I&m=KJGXzJpgEwv3KpnwM49PeoeDRRFhIFj_Vm4B5DjwYFm6uDE0n02VqJv93PDRLzKM&s=jjkvjsPmYvRVl_e8TvChN4JMLsAQ1FePr1GqGLsNrMw&e= 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://urldefense.proofpoint.com/v2/url?u=https-3A__nam12.safelinks.protection.outlook.com_-3Furl-3Dhttp-253A-252F-252Fmail.ncmir.ucsd.edu-252Fpipermail-252F3dem-252Fattachments-252F20220510-252Ff3649d44-252Fattachment-2D0001.html-26amp-3Bdata-3D05-257C01-257Cmcahn-2540princeton.edu-257C2f550a2886914885887a08da3272ba4e-257C2ff601167431425db5af077d7791bda4-257C0-257C0-257C637877765618827138-257CUnknown-257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0-253D-257C3000-257C-257C-257C-26amp-3Bsdata-3DBzgO1Eh5iP4JBw0nsjP4O0uooOgt6iGgLfTtq0pgk3U-253D-26amp-3Breserved-3D0&d=DwIGaQ&c=-35OiAkTchMrZOngvJPOeA&r=L7-zyQ-04fFCMRqzLIOnx7H0exGZHwIQe_wMPuY600I&m=KJGXzJpgEwv3KpnwM49PeoeDRRFhIFj_Vm4B5DjwYFm6uDE0n02VqJv93PDRLzKM&s=qwxqD9MU299CiIREQ_heM5l6Rdlknf3xSYf0NkRAwX4&e= >

------------------------------

Message: 4
Date: Tue, 10 May 2022 12:48:27 +0200
From: Kilian Schnelle <kilian.schnelle at uni-osnabrueck.de>
To: Christian T?ting  <christian.tueting at biochemtech.uni-halle.de>
Cc: "3dem at ncmir.ucsd.edu" <3dem at ncmir.ucsd.edu>
Subject: Re: [3dem] Operating system for local server environment
Message-ID: <BFE7853A-3C61-4841-A13F-DA9A05D3F821 at uni-osnabrueck.de>
Content-Type: text/plain; charset="utf-8"

We are running Ubuntu 20.04 LTS on our HPC without any problems so far.

I self had experience with CentOS 7 before but switching to Ubuntu was not that much of a difference for most, I mean mostly you may have to google the corresponding package name since they differ in yum/apt. I choose Ubuntu over Rocky/Alma since I didn?t knew what would ?win? and also the support times of Ubuntu LTS versions is great, same as the obvious advantage of having a huge user base.

Cheers
Kilian

Kilian Schnelle
Department of Structural Biology
University of Osnabrueck
Barbarastr. 13
D-49076 Osnabrueck, Germany
Phone: +49 541 969 7508
Email: kilian.schnelle at uni-osnabrueck.de

On 10. May 2022, at 10:28, Christian T?ting <christian.tueting at biochemtech.uni-halle.de> wrote:

Dear 3dem mailing list recipients,

I would like to get your opinion on which operating system is the most
suitable for our local server environment.

To frame the question:
We have multiple local workstations and storage servers with a very
heterogeneous architecture:
Different GPUs (2080Ti, 3090), different CPUs (Intel i9, Intel Xeon, AMD
EPYC), sometimes with a cache SSD, sometimes not. Different memory
sizes/types, different hard drive organization, etc.

Currently, Centos 7 is installed on all the machines, but as you might
know, this OS is pretty old and the EOL is just in 2 years. As some of
the workstations needs some maintenance, I would like to not install
Centos 7 again, just to redo this in a couple of months.

Our need for the machines is the following:
cryoSPARC, Scipion3, Relion
AlphaFold, NAMD/Gromacs

For the storage/main node servers, there is no special need. We don't
have any workload balancer like SLURM, and also the storage servers
"just" stores the data, without any special need. Interconnectivity is
established over NFS or sshfs, but these should be possible in any unix
OS.

I personally have experience with [X/K/U]buntu, Centos, and some Arch
Linux, so I am aware of the basic differences (e.g. apt vs yum).

Proposed by cryoSPARC is Ubuntu, at some other point (might be in the
Scipion mailing list), Rocky Linux was proposed. Also, afaik, Scientific
Linux is often used.

As I am used to Centos 7, handling outdated repos (e.g. GCC4 being the
default compiler) is a daily business, I would like to avoid in the
future. I think, this strongly points to Ubuntu, where a big and active
community is present.

Thank you for your feedback :)

Best regards
Christian
Dr. rer. nat. Christian T?ting

Kastritis Laboratory for Biomolecular Research
Cryo-Electron Microscopy & Computational Structural Biology
________________________________________________
Martin-Luther-Universit?t Halle-Wittenberg
Biozentrum, Room A.2.19
IWE ZIK HALOmem NWG III
"Kryo-Elektronenmikroskopie an Membranproteinkomplexen"
Weinbergweg 22, 06120 Halle
tel: +49 345 5524985
web (Lab): https://urldefense.proofpoint.com/v2/url?u=https-3A__nam12.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Furldefense.proofpoint.com-252Fv2-252Furl-253Fu-253Dhttps-2D3A-5F-5Fblogs.urz.uni-2D2Dhalle.de-5Fkastritislab-5F-2526d-253DDwIDaQ-2526c-253D-2D35OiAkTchMrZOngvJPOeA-2526r-253DL7-2DzyQ-2D04fFCMRqzLIOnx7H0exGZHwIQe-5FwMPuY600I-2526m-253DM5tV4i7vwtyCEWiBFwqltoUDbWfJrQ6Rq4OeG3w6Pf3szICL5sanWtIgUYWC3B5-5F-2526s-253DUJR09zZF0h0c-2DvOZd4S3RmffzgEiJ7y-5FXgkY01dJpm8-2526e-253D-26amp-3Bdata-3D05-257C01-257Cmcahn-2540princeton.edu-257C2f550a2886914885887a08da3272ba4e-257C2ff601167431425db5af077d7791bda4-257C0-257C0-257C637877765618827138-257CUnknown-257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0-253D-257C3000-257C-257C-257C-26amp-3Bsdata-3DNAH88OCfLYM1XJZX2-252FaOunPdawC8VEafwvAH8ivgpSY-253D-26amp-3Breserved-3D0&d=DwIGaQ&c=-35OiAkTchMrZOngvJPOeA&r=L7-zyQ-04fFCMRqzLIOnx7H0exGZHwIQe_wMPuY600I&m=KJGXzJpgEwv3KpnwM49PeoeDRRFhIFj_Vm4B5DjwYFm6uDE0n02VqJv93PDRLzKM&s=Qd6OOQyYGYWesnCtNu51irzRUCnvdsBN3SKqMHSeav8&e= 
web (HALOmem): https://urldefense.proofpoint.com/v2/url?u=https-3A__nam12.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Furldefense.proofpoint.com-252Fv2-252Furl-253Fu-253Dhttps-2D3A-5F-5Fwww.halomem.de-5Fen-5F-2526d-253DDwIDaQ-2526c-253D-2D35OiAkTchMrZOngvJPOeA-2526r-253DL7-2DzyQ-2D04fFCMRqzLIOnx7H0exGZHwIQe-5FwMPuY600I-2526m-253DM5tV4i7vwtyCEWiBFwqltoUDbWfJrQ6Rq4OeG3w6Pf3szICL5sanWtIgUYWC3B5-5F-2526s-253DYPBQ4XfDHsZK6U3AgwzMuzhoNUcTjc4S2ntJIQ9jrJo-2526e-253D-26amp-3Bdata-3D05-257C01-257Cmcahn-2540princeton.edu-257C2f550a2886914885887a08da3272ba4e-257C2ff601167431425db5af077d7791bda4-257C0-257C0-257C637877765618827138-257CUnknown-257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0-253D-257C3000-257C-257C-257C-26amp-3Bsdata-3D5-252BfInG2DDmsZyUz1Q-252FTzzows5ggBcAk-252BFHUVD3VAjpA-253D-26amp-3Breserved-3D0&d=DwIGaQ&c=-35OiAkTchMrZOngvJPOeA&r=L7-zyQ-04fFCMRqzLIOnx7H0exGZHwIQe_wMPuY600I&m=KJGXzJpgEwv3KpnwM49PeoeDRRFhIFj_Vm4B5DjwYFm6uDE0n02VqJv93PDRLzKM&s=i57lYdpC7Nnonq_Js-b4z48DcFsJrEeZrC22tI6bWDw&e= 

_______________________________________________
3dem mailing list
3dem at ncmir.ucsd.edu
https://urldefense.proofpoint.com/v2/url?u=https-3A__nam12.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Fmail.ncmir.ucsd.edu-252Fmailman-252Flistinfo-252F3dem-26amp-3Bdata-3D05-257C01-257Cmcahn-2540princeton.edu-257C2f550a2886914885887a08da3272ba4e-257C2ff601167431425db5af077d7791bda4-257C0-257C0-257C637877765618827138-257CUnknown-257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0-253D-257C3000-257C-257C-257C-26amp-3Bsdata-3DDDvXsfTEo7u-252FyNUXp9FH9gJfrJk8J7mngXx8HoFj13U-253D-26amp-3Breserved-3D0&d=DwIGaQ&c=-35OiAkTchMrZOngvJPOeA&r=L7-zyQ-04fFCMRqzLIOnx7H0exGZHwIQe_wMPuY600I&m=KJGXzJpgEwv3KpnwM49PeoeDRRFhIFj_Vm4B5DjwYFm6uDE0n02VqJv93PDRLzKM&s=FGPNvC1MgBm_R_NCWnRWwi66ohbDDDo0fM9FXTRkxHg&e= 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://urldefense.proofpoint.com/v2/url?u=https-3A__nam12.safelinks.protection.outlook.com_-3Furl-3Dhttp-253A-252F-252Fmail.ncmir.ucsd.edu-252Fpipermail-252F3dem-252Fattachments-252F20220510-252Fa27835e6-252Fattachment.html-26amp-3Bdata-3D05-257C01-257Cmcahn-2540princeton.edu-257C2f550a2886914885887a08da3272ba4e-257C2ff601167431425db5af077d7791bda4-257C0-257C0-257C637877765618827138-257CUnknown-257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0-253D-257C3000-257C-257C-257C-26amp-3Bsdata-3DKs-252F4HiLdBvUe4DhF1Pv2be2wQ50X7fLddO0f-252FWJ-252BS0o-253D-26amp-3Breserved-3D0&d=DwIGaQ&c=-35OiAkTchMrZOngvJPOeA&r=L7-zyQ-04fFCMRqzLIOnx7H0exGZHwIQe_wMPuY600I&m=KJGXzJpgEwv3KpnwM49PeoeDRRFhIFj_Vm4B5DjwYFm6uDE0n02VqJv93PDRLzKM&s=-bORKbz5k6kCkxTq_tzvWIcPJMjflQTq48D_ymPqMWE&e= >
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5926 bytes
Desc: not available
URL: <https://urldefense.proofpoint.com/v2/url?u=https-3A__nam12.safelinks.protection.outlook.com_-3Furl-3Dhttp-253A-252F-252Fmail.ncmir.ucsd.edu-252Fpipermail-252F3dem-252Fattachments-252F20220510-252Fa27835e6-252Fattachment.p7s-26amp-3Bdata-3D05-257C01-257Cmcahn-2540princeton.edu-257C2f550a2886914885887a08da3272ba4e-257C2ff601167431425db5af077d7791bda4-257C0-257C0-257C637877765618827138-257CUnknown-257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0-253D-257C3000-257C-257C-257C-26amp-3Bsdata-3D8Vddhgk4tE8anU2ZWwHoZEeMMecLG5NsVThekreIMqo-253D-26amp-3Breserved-3D0&d=DwIGaQ&c=-35OiAkTchMrZOngvJPOeA&r=L7-zyQ-04fFCMRqzLIOnx7H0exGZHwIQe_wMPuY600I&m=KJGXzJpgEwv3KpnwM49PeoeDRRFhIFj_Vm4B5DjwYFm6uDE0n02VqJv93PDRLzKM&s=9ZWSejGRbvRhBi88DphUn4rC1Apbyw3NdPSd8VXxuLU&e= >

------------------------------

Subject: Digest Footer

_______________________________________________
3dem mailing list
3dem at ncmir.ucsd.edu
https://urldefense.proofpoint.com/v2/url?u=https-3A__nam12.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Fmail.ncmir.ucsd.edu-252Fmailman-252Flistinfo-252F3dem-26amp-3Bdata-3D05-257C01-257Cmcahn-2540princeton.edu-257C2f550a2886914885887a08da3272ba4e-257C2ff601167431425db5af077d7791bda4-257C0-257C0-257C637877765618827138-257CUnknown-257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0-253D-257C3000-257C-257C-257C-26amp-3Bsdata-3DDDvXsfTEo7u-252FyNUXp9FH9gJfrJk8J7mngXx8HoFj13U-253D-26amp-3Breserved-3D0&d=DwIGaQ&c=-35OiAkTchMrZOngvJPOeA&r=L7-zyQ-04fFCMRqzLIOnx7H0exGZHwIQe_wMPuY600I&m=KJGXzJpgEwv3KpnwM49PeoeDRRFhIFj_Vm4B5DjwYFm6uDE0n02VqJv93PDRLzKM&s=FGPNvC1MgBm_R_NCWnRWwi66ohbDDDo0fM9FXTRkxHg&e= 

------------------------------

End of 3dem Digest, Vol 177, Issue 11
*************************************

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ncmir.ucsd.edu/pipermail/3dem/attachments/20220510/d3703d38/attachment-0001.html>