<div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr">Dear All,<div><br></div><div><div>Thank you for your excellent suggestions regarding the issue. </div><div><br></div><div>Mike Strauss was right about EPU writing stacks from Falcon II to the Microscope PC's HDD before transferring it to the destination folder on Support PC. This is an obvious bottleneck, but I think FEI did this for a good reason (Win 7 upgrade was probably implemented at the very end of the product lifecycle, so there was no point in making the old product almost as good as the new one).</div><div>Thanks to Mike, I was able to locate the folder where intermediate frames are saved: <b>C:/Users/<username>/AppData/Local/Temp/FalconInterImages/ </b><br></div><div dir="auto">However, <i>IntermediateConfig.xml</i> points to the different location, not the one that is currently used (you'll find the attachment below). I've tried to find .XML files with the same name, but haven't found any, except for those that were in the backup copy made during the migration from Win XP to Win 7.</div><div dir="auto"><br></div><div>Now let me address your questions one at a time:</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">What type of disks are on Camera Support Rack #1? </blockquote></div><div>Camera Support Rack #1 is a "black box" - I don't have a direct access to it, so no info on the type of the memory used and/or additional computations performed on it. </div><div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">You are talking about the image transfer being performed by the FEI software, rather than copying a file via SMB? </blockquote><div>Yes, correct. As indicated on the network map, 1Gbps channel ("Support network") used for transferring via SMB is fully utilised.</div><div><br></div><div>Matthias, thank you for pointing out the possible workarounds, I've contacted Chen Xu already. I didn't know the camera can go to 32 fps or higher, that's fascinating!<br></div><div><br></div>Steven, of course you are right about the switch. It isn't (probably) a cause of the problem itself (since 1Gb/s Ethernet has been around for 20 years or so). My guess was that it's configured this way to "cap" incoming 1Gbps channels because there is only one outgoing 1Gbps channel.</div><div><br></div><div><br></div><div><br></div><div>Regards,</div><div><br></div><div><div dir="auto">Eugene </div></div><div><br></div><div><br><div><br></div><div><br></div><div dir="auto"><br></div><br class="gmail-Apple-interchange-newline"><div><br></div><div><br></div></div></div></div></div></div></div></div><br><div class="gmail_quote"><div dir="ltr">On Thu, Oct 18, 2018 at 10:40 PM Ludtke, Steven J <<a href="mailto:sludtke@bcm.edu">sludtke@bcm.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word;line-break:after-white-space">
You know, I think I misunderstood the question. You are talking about the image transfer being performed by the FEI software, rather than copying a file via SMB? If so, then almost certainly nothing to do with your hardware at all. Just poor software design
in handling the network transfer by FEI.
<div><br>
</div>
<div><br>
<div>
<div dir="auto" style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word;line-break:after-white-space">
<div dir="auto" style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word;line-break:after-white-space">
<div dir="auto" style="word-wrap:break-word;line-break:after-white-space">
<div style="color:rgb(0,0,0);font-variant-caps:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">
<font face="Courier"><span style="font-size:14px">--------------------------------------------------------------------------------------<br>
Steven Ludtke, Ph.D. <<a href="mailto:sludtke@bcm.edu" target="_blank">sludtke@bcm.edu</a>> Baylor College of Medicine <br>
Charles C. Bell Jr., Professor of Structural Biology<br>
Dept. of Biochemistry and Molecular Biology (<a href="http://www.bcm.edu/biochem" target="_blank">www.bcm.edu/biochem</a>)<br>
Academic Director, CryoEM Core (<a href="http://cryoem.bcm.edu" target="_blank">cryoem.bcm.edu</a>)<br>
Co-Director CIBR Center (<a href="http://www.bcm.edu/research/cibr" target="_blank">www.bcm.edu/research/cibr</a>)<br>
<br>
</span></font><br>
</div>
</div>
</div>
</div>
</div>
<div><br>
<blockquote type="cite">
<div>On Oct 18, 2018, at 1:29 PM, Matthias Wolf <<a href="mailto:matthias.wolf@oist.jp" target="_blank">matthias.wolf@oist.jp</a>> wrote:</div>
<br class="m_-5858926061959128986Apple-interchange-newline">
<div><font size="2" style="font-family:Helvetica;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none"><b>***CAUTION:***
This email is not from a BCM Source. Only click links or open attachments you know are safe.</b></font><span style="font-family:Helvetica;font-size:18px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;float:none;display:inline!important"></span>
<hr style="font-family:Helvetica;font-size:18px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none">
<div id="m_-5858926061959128986divtagdefaultwrapper" dir="ltr" style="font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;font-size:12pt;font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols">
<div id="m_-5858926061959128986divtagdefaultwrapper" dir="ltr">
<div style="margin-top:0px;margin-bottom:0px;font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols;font-size:12pt">
Hi Gene,</div>
<div style="margin-top:0px;margin-bottom:0px;font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols;font-size:12pt">
<br>
</div>
<div style="margin-top:0px;margin-bottom:0px;font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols;font-size:12pt">
Don't bother with the inadequate FEI microscope PC and its gigabit network.</div>
<div style="margin-top:0px;margin-bottom:0px;font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols;font-size:12pt">
<br>
</div>
<div style="margin-top:0px;margin-bottom:0px;font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols;font-size:12pt">
For proper performance, you have 2 options:<br>
<br>
</div>
<ol style="font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols;font-size:12pt;margin-bottom:0px;margin-top:0px">
<li>get FEI's storage server and frame saving hardware (CMTS) with dedicated 10Gbit/s upload. It'll cost you $$$. Needs Win7 on the scope PC.<br>
<br>
</li><li>set up Greg McMullans brilliant Falcon frame capture, which uses a beam splitter (fiber optical tap) to duplicate the raw camera signal and capture it on a dedicated linux PC with fast SSD. Total investment ~3k US$. This works on older Win XP systems,
but Greg has also written code for Win7 (FEI changes something in the Falcon data protocol upon upgrade to Win7). You may want to look up Chen Xu's "falcon hacking" page at Brandeis on Internet Waybackmachine. That way, the FEI system is completely bypassed
and your movie frames will be happily saved on the capture PC before you even get an image in TIA. I have done this with good success for several years until option 1 was installed in the course of a Win7+Falcon3 upgrade. You must obtain specific Solarflare
NICs for Greg's code to work and should follow his instructions to the letter (but best don't bother him and go with option 1)!</li></ol>
<br>
<div style="margin-top:0px;margin-bottom:0px;font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols;font-size:12pt">
Image size is 16 Mpixel * 14bit/pixel. At 18 fps, the data rate is ~4 Gb/s.</div>
<div style="margin-top:0px;margin-bottom:0px;font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols;font-size:12pt">
Decent hardware can deal with this. The camera can go to 32 fps or even higher, but then everything on the linux O/S must be tuned or you drop frames.</div>
<div style="margin-top:0px;margin-bottom:0px;font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols;font-size:12pt">
<br>
</div>
<div style="margin-top:0px;margin-bottom:0px;font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols;font-size:12pt">
Without frame capture, you won't get decent performance - both number of frames and frame rate will be severely limited and not up to current standards of cryo-EM movie data collection.</div>
<div style="margin-top:0px;margin-bottom:0px;font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols;font-size:12pt">
<br>
</div>
<div style="margin-top:0px;margin-bottom:0px;font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols;font-size:12pt">
As Steve wrote, your gigabit switch is unlikely to throttle bandwidth.<br>
</div>
<div style="margin-top:0px;margin-bottom:0px;font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols;font-size:12pt">
But the raw camera data is sent in jumbo packets. Maybe your switch is using standard MTU of 1500?</div>
<div style="margin-top:0px;margin-bottom:0px;font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols;font-size:12pt">
You could try changing this to use jumbo packets of 9014 bytes with telnet and the CLI.</div>
<div style="margin-top:0px;margin-bottom:0px;font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols;font-size:12pt">
<br>
</div>
<div style="margin-top:0px;margin-bottom:0px;font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols;font-size:12pt">
Matthias</div>
<div style="margin-top:0px;margin-bottom:0px;font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols;font-size:12pt">
<br>
</div>
</div>
<hr style="display:inline-block;width:2018.796875px">
<div id="m_-5858926061959128986divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt"><b>From:</b><span class="m_-5858926061959128986Apple-converted-space"> </span>3dem <<a href="mailto:3dem-bounces@ncmir.ucsd.edu" target="_blank">3dem-bounces@ncmir.ucsd.edu</a>>
on behalf of Eugene Pichkur <<a href="mailto:eugene.pichkur@gmail.com" target="_blank">eugene.pichkur@gmail.com</a>><br>
<b>Sent:</b><span class="m_-5858926061959128986Apple-converted-space"> </span>Tuesday, October 16, 2018 10:15:00 PM<br>
<b>To:</b><span class="m_-5858926061959128986Apple-converted-space"> </span><a href="mailto:3dem@ncmir.ucsd.edu" target="_blank">3dem@ncmir.ucsd.edu</a><br>
<b>Subject:</b><span class="m_-5858926061959128986Apple-converted-space"> </span>[3dem] Falcon II uses only half of the 1Gb/s bandwidth.</font>
<div> </div>
</div>
<div>
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="auto">Hello 3DEM community,
<div dir="auto"><br>
</div>
<div dir="auto">We've noticed that our Falcon II uses only half of the 1Gbps bandwidth to transfer data from the Camera Support Rack to the Microscope PC (I've attached a screenshot to illustrate the issue). </div>
<div dir="auto"><br>
<div dir="auto">Microscope PC runs Windows 7 and we are able to get the full frame rate (20fps) from the detector, however, it takes ~20s to transfer a typical movie (40frames, 1.2GB) to the PC which is a very frustrating bottleneck. </div>
<div dir="auto"><br>
</div>
<div dir="auto">Both Falcon II and Ceta are connected through the same 1Gb switch (HPE OfficeConnect 1420 24G 2SFP), so my guess is that the bandwidth is split 50/50. However, during data collection only Falcon II is used which makes this limitation
pointless, not to mention that Falcon II and Ceta can't be used simultaneously. </div>
<div dir="auto"><br>
</div>
<div dir="auto">Does anyone have a workaround or any info on the topic?</div>
<div dir="auto"><br>
</div>
<div dir="auto"><br>
</div>
<div dir="auto">Thanks in advance,</div>
<div dir="auto">Eugene Pichkur<br>
</div>
<div dir="auto"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px"><br>
</span></div>
<div dir="auto"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px">National Research Centre “Kurchatov Institute”<br>
</span><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:10pt">Akademika Kurchatova, 1, Moscow, </span><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px">Russia, </span><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:10pt">123098</span><br>
</div>
<div dir="auto"><br>
</div>
<div dir="auto"><br>
</div>
<div dir="auto"><br>
</div>
<div dir="auto"><br>
</div>
<div dir="auto"><br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<span style="font-family:Helvetica;font-size:18px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;float:none;display:inline!important">_______________________________________________</span><br style="font-family:Helvetica;font-size:18px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none">
<span style="font-family:Helvetica;font-size:18px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;float:none;display:inline!important">3dem
mailing list</span><br style="font-family:Helvetica;font-size:18px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none">
<a href="mailto:3dem@ncmir.ucsd.edu" style="font-family:Helvetica;font-size:18px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" target="_blank">3dem@ncmir.ucsd.edu</a><br style="font-family:Helvetica;font-size:18px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none">
<a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.ncmir.ucsd.edu_mailman_listinfo_3dem&d=DwICAg&c=ZQs-KZ8oxEw0p81sqgiaRA&r=GWA2IF6nkq8sZMXHpp1Xpg&m=OolwFF628X2ftLUpz8Ras6lKj2gYG24ZfiJOoYlUzP0&s=M4xY8AZbUX0K19ncPzZDEjD1hTrbXK6xsaTCOQGNQCk&e=" style="font-family:Helvetica;font-size:18px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" target="_blank">https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.ncmir.ucsd.edu_mailman_listinfo_3dem&d=DwICAg&c=ZQs-KZ8oxEw0p81sqgiaRA&r=GWA2IF6nkq8sZMXHpp1Xpg&m=OolwFF628X2ftLUpz8Ras6lKj2gYG24ZfiJOoYlUzP0&s=M4xY8AZbUX0K19ncPzZDEjD1hTrbXK6xsaTCOQGNQCk&e=</a></div>
</blockquote>
</div>
<br>
</div>
</div>
</blockquote></div>