[3dem] Sync Data from Athena/Offload servers to storage

[Paolo Swuec]

Hi all,

I’m trying to understand how other EM facilities are handling data transfer from Athena/Offload servers to institutional storage. We currently have a Titan Krios G4 and a Glacios, both with Falcon 4i detectors, and we are about to install a new Krios G5 and a new Glacios 2. For the last six years, we have simply used rsync directly from the Offload servers to our central storage. The servers are connected over fibre and we can transfer at around 10 Gbps.

Recently, however, Thermo Fisher DSE told us that this setup is discouraged and not officially supported. Their suggested solution is to use an intermediate workstation or VDI to pull the data from the Offload server and then sync it to storage.

The problem is that, in our current setup, this would likely reduce transfer speed from 10 Gbps to around 1 Gbps.

So I’m curious: what is everyone else doing?

Are you syncing directly from the Offload server, using a dedicated transfer node, workstation, VDI, or some other setup? And what kind of transfer speeds are you achieving?

Any details on your architecture or experience would be very helpful.

Thanks a lot,

Paolo



 
Paolo Swuec
Head of National Facility for Structural Biology
Human Technopole
Palazzo Italia
Viale Rita Levi-Montalcini, 1
20157 Milan, Italy
T. +39 02-30247442
M. +39 3427758814
E. paolo.swuec at fht.org
humantechnopole.it
Prima di stampare questa e-mail, pensa all’ambiente.
Before printing this e-mail, think of the environment
Legal Disclaimer 
The information contained in or accompanying this email and any attachment thereto, is intended solely for the use of the stated recipient(s) and may contain information that is confidential and/or privileged. Any dissemination, distribution or copying of this email and any attachment by anyone who is not the stated recipient is strictly prohibited. If you receive this message in error, please notify the sender immediately and delete the message and any attachment from your system without retaining a copy.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ncmir.ucsd.edu/pipermail/3dem/attachments/20260915/d6852a52/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image707071.png
Type: image/png
Size: 701 bytes
Desc: image707071.png
URL: <http://mail.ncmir.ucsd.edu/pipermail/3dem/attachments/20260915/d6852a52/attachment-0004.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image503970.png
Type: image/png
Size: 680 bytes
Desc: image503970.png
URL: <http://mail.ncmir.ucsd.edu/pipermail/3dem/attachments/20260915/d6852a52/attachment-0005.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image903090.png
Type: image/png
Size: 464 bytes
Desc: image903090.png
URL: <http://mail.ncmir.ucsd.edu/pipermail/3dem/attachments/20260915/d6852a52/attachment-0006.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image049593.png
Type: image/png
Size: 721 bytes
Desc: image049593.png
URL: <http://mail.ncmir.ucsd.edu/pipermail/3dem/attachments/20260915/d6852a52/attachment-0007.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image321536.jpg
Type: image/jpeg
Size: 64981 bytes
Desc: image321536.jpg
URL: <http://mail.ncmir.ucsd.edu/pipermail/3dem/attachments/20260915/d6852a52/attachment-0001.jpg>