[3dem] Slow connection linux - falcon3 server

[benoit.zuber at ana.unibe.ch]

Interesting! I thought SMB / cifs would be worse than nfs. We’ll try more in the direction of SMB then.

De : Steven Ludtke <sludtke at bcm.edu>
Date : jeudi, 21 décembre 2017 à 14:56
À : Benoît Zuber <benoit.zuber at ana.unibe.ch>
Cc : "3dem at ncmir.ucsd.edu" <3dem at ncmir.ucsd.edu>
Objet : Re: [3dem] Slow connection linux - falcon3 server

Sorry, missed the fast windows -> windows transfer comment in the original message.

We went through similar sort of issue when configuring our K2 acquisition computer (windows) to a Linux GPU workstation with 8 drive RAID, also over 10Gb fiber. In our case we were trying to mount the Linux RAID as a volume on the K2 computer so we could just directly store frames there during acquisition. Similar to your experience we discovered that the performance of the NFS module in Windows is abysmal, even after tuning.

Strangely, the approach that gave the best performance was using SMB to share the Linux drive to the Windows machine. I don't remember the final performance numbers (we set this up some time ago), but I'm pretty sure it was coming in at around 500 MB/s.  No other approach we tried gave decent bandwidth.

--------------------------------------------------------------------------------------
Steven Ludtke, Ph.D.
Charles C. Bell Jr., Professor of Structural Biology
Dept. of Biochemistry and Molecular Biology                      (www.bcm.edu/biochem<http://www.bcm.edu/biochem>)
Co-Director CIBR Center                                    (www.bcm.edu/research/cibr<http://www.bcm.edu/research/cibr>)
Co-Director National Center For Macromolecular Imaging                  (ncmi.bcm.edu<http://ncmi.bcm.edu>)
Baylor College of Medicine
sludtke at bcm.edu<mailto:sludtke at bcm.edu>


On Dec 21, 2017, at 6:32 AM, benoit.zuber at ana.unibe.ch<mailto:benoit.zuber at ana.unibe.ch> wrote:

Hi Steve,

Thanks for your reply. It has 12 disks in RAID (not sure what type). I don’t think this is the problem since when we connect another windows machine to it, files can be transfered high speed in both directions.

Benoit

De : Steven Ludtke <sludtke at bcm.edu<mailto:sludtke at bcm.edu>>
Date : jeudi, 21 décembre 2017 à 14:06
À : Benoît Zuber <benoit.zuber at ana.unibe.ch<mailto:benoit.zuber at ana.unibe.ch>>
Cc : "3dem at ncmir.ucsd.edu<mailto:3dem at ncmir.ucsd.edu>" <3dem at ncmir.ucsd.edu<mailto:3dem at ncmir.ucsd.edu>>
Objet : Re: [3dem] Slow connection linux - falcon3 server

Hi Benoit,
what sort of storage is being used to store the files in question on the FEI server?  Your description sounds like the files being transferred are on a single 'spinning platter' hard drive, which will generally max out in data I/O at 150-200 MB/s. Most machines used for storing high bandwidth data will either use a RAID 5/6 array of traditional hard drive (which can achieve 1000-2000 MB/s depending on the number of drives) or high speed SSDs which will max out at ~600 MB/s on a standard SATA bus, but can achieve 1000-3000 MB/s if alternative interfaces are used.

--------------------------------------------------------------------------------------
Steven Ludtke, Ph.D.
Charles C. Bell Jr., Professor of Structural Biology
Dept. of Biochemistry and Molecular Biology                      (www.bcm.edu/biochem<http://www.bcm.edu/biochem>)
Co-Director CIBR Center                                    (www.bcm.edu/research/cibr<http://www.bcm.edu/research/cibr>)
Co-Director National Center For Macromolecular Imaging                  (ncmi.bcm.edu<http://ncmi.bcm.edu/>)
Baylor College of Medicine
sludtke at bcm.edu<mailto:sludtke at bcm.edu>



On Dec 21, 2017, at 4:19 AM, benoit.zuber at ana.unibe.ch<mailto:benoit.zuber at ana.unibe.ch> wrote:

***CAUTION:*** This email is not from a BCM Source. Only click links or open attachments you know are safe.
________________________________
Dear all

We have a linux workstation connected directly through fiber optics to the so-called FEI server, i.e. the computer where Falcon3 saves frames, which runs under Windows. Strangely, file transfer from linux to windows is fast (600 MB/s) whereas file transfer from windows to linux is slow (160 MB/s). We have tried mounting the windows drive on linux through cifs or nfs, as well as mounting the linux drive on windows through nfs and we always get the same result.

We also tested windows to windows (we connected another windows machine to the fei server) : fast both ways. And we tested linux to linux : fast both ways

We tested network speed using iperf, and it is high in either direction. It is only when we transfer files that we have problems.

Any suggestion welcome.

Thanks
Benoit

--
Prof. Benoît Zuber
University of Bern
Institute of Anatomy
Baltzerstrasse 2
Postfach 922
3000 Bern 9

+41 31 631 84 40
_______________________________________________
3dem mailing list
3dem at ncmir.ucsd.edu<mailto:3dem at ncmir.ucsd.edu>
https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.ncmir.ucsd.edu_mailman_listinfo_3dem&d=DwICAg&c=ZQs-KZ8oxEw0p81sqgiaRA&r=GWA2IF6nkq8sZMXHpp1Xpg&m=iM30kDS0pjLFsoGLC5GUR_9aRscO2JMiBlLeuLAqe-k&s=c228ksyoRZn9VrDIqrz1-8wQz_j3mEiCm-er4g6mrFE&e=


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ncmir.ucsd.edu/pipermail/3dem/attachments/20171221/1c45c77e/attachment-0001.html>