<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns:mv="http://macVmlSchemaUri" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Titre" content="">
<meta name="Mots clés" content="">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.apple-converted-space
{mso-style-name:apple-converted-space;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.msoIns
{mso-style-type:export-only;
mso-style-name:"";
text-decoration:underline;
color:teal;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:595.0pt 842.0pt;
margin:70.85pt 70.85pt 70.85pt 70.85pt;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body bgcolor="white" lang="FR" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span lang="DE" style="mso-fareast-language:EN-US">Interesting! I thought SMB / cifs would be worse than nfs. We’ll try more in the direction of SMB then.
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="DE" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style="font-size:12.0pt;color:black">De : </span></b><span style="font-size:12.0pt;color:black">Steven Ludtke <sludtke@bcm.edu><br>
<b>Date : </b>jeudi, 21 décembre 2017 à 14:56<br>
<b>À : </b>Benoît Zuber <benoit.zuber@ana.unibe.ch><br>
<b>Cc : </b>"3dem@ncmir.ucsd.edu" <3dem@ncmir.ucsd.edu><br>
<b>Objet : </b>Re: [3dem] Slow connection linux - falcon3 server<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal">Sorry, missed the fast windows -> windows transfer comment in the original message.
<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">We went through similar sort of issue when configuring our K2 acquisition computer (windows) to a Linux GPU workstation with 8 drive RAID, also over 10Gb fiber. In our case we were trying to mount the Linux RAID as a volume on the K2 computer
so we could just directly store frames there during acquisition. Similar to your experience we discovered that the performance of the NFS module in Windows is abysmal, even after tuning.
<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Strangely, the approach that gave the best performance was using SMB to share the Linux drive to the Windows machine. I don't remember the final performance numbers (we set this up some time ago), but I'm pretty sure it was coming in at
around 500 MB/s. No other approach we tried gave decent bandwidth.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Courier",serif;color:black">--------------------------------------------------------------------------------------<br>
Steven Ludtke, Ph.D.<br>
Charles C. Bell Jr., Professor of Structural Biology<br>
Dept. of Biochemistry and Molecular Biology (<a href="http://www.bcm.edu/biochem">www.bcm.edu/biochem</a>)<br>
Co-Director CIBR Center (<a href="http://www.bcm.edu/research/cibr">www.bcm.edu/research/cibr</a>)<br>
Co-Director National Center For Macromolecular Imaging (<a href="http://ncmi.bcm.edu">ncmi.bcm.edu</a>)<br>
Baylor College of Medicine <br>
<a href="mailto:sludtke@bcm.edu">sludtke@bcm.edu</a></span><span style="color:black"><o:p></o:p></span></p>
</div>
</div>
</div>
</div>
</div>
</div>
<div>
<p class="MsoNormal"><br>
<br>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal">On Dec 21, 2017, at 6:32 AM, <a href="mailto:benoit.zuber@ana.unibe.ch">
benoit.zuber@ana.unibe.ch</a> wrote:<o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal" style="background:white"><span lang="DE">Hi Steve,</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span lang="DE"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span lang="DE">Thanks for your reply. It has 12 disks in RAID (not sure what type). I don’t think this is the problem since when we connect another windows machine to it, files can be transfered high speed in both
directions.<span class="apple-converted-space"> </span></span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span lang="DE"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span lang="DE">Benoit</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span lang="DE"> </span><o:p></o:p></p>
</div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<div>
<p class="MsoNormal" style="background:white"><b><span style="font-size:12.0pt">De :<span class="apple-converted-space"> </span></span></b><span style="font-size:12.0pt">Steven Ludtke <<a href="mailto:sludtke@bcm.edu">sludtke@bcm.edu</a>><br>
<b>Date :<span class="apple-converted-space"> </span></b>jeudi, 21 décembre 2017 à 14:06<br>
<b>À :<span class="apple-converted-space"> </span></b>Benoît Zuber <<a href="mailto:benoit.zuber@ana.unibe.ch">benoit.zuber@ana.unibe.ch</a>><br>
<b>Cc :<span class="apple-converted-space"> </span></b>"<a href="mailto:3dem@ncmir.ucsd.edu">3dem@ncmir.ucsd.edu</a>" <<a href="mailto:3dem@ncmir.ucsd.edu">3dem@ncmir.ucsd.edu</a>><br>
<b>Objet :<span class="apple-converted-space"> </span></b>Re: [3dem] Slow connection linux - falcon3 server</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="background:white"> <o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal" style="background:white">Hi Benoit,<span class="apple-converted-space"> </span><o:p></o:p></p>
</div>
<div>
<div>
<p class="MsoNormal" style="background:white">what sort of storage is being used to store the files in question on the FEI server? Your description sounds like the files being transferred are on a single 'spinning platter' hard drive, which will generally
max out in data I/O at 150-200 MB/s. Most machines used for storing high bandwidth data will either use a RAID 5/6 array of traditional hard drive (which can achieve 1000-2000 MB/s depending on the number of drives) or high speed SSDs which will max out at
~600 MB/s on a standard SATA bus, but can achieve 1000-3000 MB/s if alternative interfaces are used.<o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="background:white"> <o:p></o:p></p>
</div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-size:10.5pt;font-family:"Courier",serif">--------------------------------------------------------------------------------------<br>
Steven Ludtke, Ph.D.<br>
Charles C. Bell Jr., Professor of Structural Biology<br>
Dept. of Biochemistry and Molecular Biology (<a href="http://www.bcm.edu/biochem"><span style="color:purple">www.bcm.edu/biochem</span></a>)<br>
Co-Director CIBR Center (<a href="http://www.bcm.edu/research/cibr"><span style="color:purple">www.bcm.edu/research/cibr</span></a>)<br>
Co-Director National Center For Macromolecular Imaging (<a href="http://ncmi.bcm.edu/"><span style="color:purple">ncmi.bcm.edu</span></a>)<br>
Baylor College of Medicine <br>
<a href="mailto:sludtke@bcm.edu"><span style="color:purple">sludtke@bcm.edu</span></a></span><o:p></o:p></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="background:white"><br>
<br>
<br>
<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal" style="background:white">On Dec 21, 2017, at 4:19 AM,<span class="apple-converted-space"> </span><a href="mailto:benoit.zuber@ana.unibe.ch"><span style="color:purple">benoit.zuber@ana.unibe.ch</span></a><span class="apple-converted-space"> </span>wrote:<o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal" style="background:white"> <o:p></o:p></p>
</div>
<div>
<div>
<p class="MsoNormal" style="background:white"><b><span style="font-size:10.0pt;font-family:"Helvetica",sans-serif;background:white">***CAUTION:*** This email is not from a BCM Source. Only click links or open attachments you know are safe.</span></b><o:p></o:p></p>
</div>
<div class="MsoNormal" align="center" style="text-align:center;background:white">
<hr size="2" width="100%" align="center">
</div>
<div>
<div>
<p class="MsoNormal" style="background:white"><span lang="DE">Dear all</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="background:white"><span lang="DE"> </span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="background:white"><span lang="DE">We have a linux workstation connected directly through fiber optics to the so-called FEI server, i.e. the computer where Falcon3 saves frames, which runs under Windows. Strangely, file transfer from
linux to windows is fast (600 MB/s) whereas file transfer from windows to linux is slow (160 MB/s). We have tried mounting the windows drive on linux through cifs or nfs, as well as mounting the linux drive on windows through nfs and we always get the same
result.</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="background:white"><span lang="DE"> </span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="background:white"><span lang="DE">We also tested windows to windows (we connected another windows machine to the fei server) : fast both ways. And we tested linux to linux : fast both ways</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="background:white"><span lang="DE"> </span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="background:white"><span lang="DE">We tested network speed using iperf, and it is high in either direction. It is only when we transfer files that we have problems.</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="background:white"><span lang="DE"> </span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="background:white"><span lang="DE">Any suggestion welcome.</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="background:white"><span lang="DE"> </span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="background:white"><span lang="DE">Thanks</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="background:white"><span lang="DE">Benoit</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="background:white"><span lang="DE"> </span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="background:white">-- <o:p></o:p></p>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal" style="background:white"><span lang="DE">Prof. Benoît Zuber</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="background:white"><span lang="DE">University of Bern</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="background:white"><span lang="DE">Institute of Anatomy</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="background:white"><span lang="DE">Baltzerstrasse 2</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="background:white">Postfach 922<o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="background:white"><span lang="DE">3000 Bern 9</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="background:white"><span lang="DE"> </span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="background:white"><span lang="DE">+41 31 631 84 40</span><o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-size:13.5pt;font-family:"Helvetica",sans-serif;background:white">_______________________________________________</span><span style="font-size:13.5pt;font-family:"Helvetica",sans-serif"><br>
<span style="background:white">3dem mailing list</span><br>
</span><a href="mailto:3dem@ncmir.ucsd.edu"><span style="font-size:13.5pt;font-family:"Helvetica",sans-serif;color:#954F72;background:white">3dem@ncmir.ucsd.edu</span></a><span style="font-size:13.5pt;font-family:"Helvetica",sans-serif"><br>
</span><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.ncmir.ucsd.edu_mailman_listinfo_3dem&d=DwICAg&c=ZQs-KZ8oxEw0p81sqgiaRA&r=GWA2IF6nkq8sZMXHpp1Xpg&m=iM30kDS0pjLFsoGLC5GUR_9aRscO2JMiBlLeuLAqe-k&s=c228ksyoRZn9VrDIqrz1-8wQz_j3mEiCm-er4g6mrFE&e="><span style="font-size:13.5pt;font-family:"Helvetica",sans-serif;color:#954F72;background:white">https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.ncmir.ucsd.edu_mailman_listinfo_3dem&d=DwICAg&c=ZQs-KZ8oxEw0p81sqgiaRA&r=GWA2IF6nkq8sZMXHpp1Xpg&m=iM30kDS0pjLFsoGLC5GUR_9aRscO2JMiBlLeuLAqe-k&s=c228ksyoRZn9VrDIqrz1-8wQz_j3mEiCm-er4g6mrFE&e=</span></a><o:p></o:p></p>
</div>
</div>
</blockquote>
</div>
<div>
<p class="MsoNormal" style="background:white"> <o:p></o:p></p>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</div>
</body>
</html>