<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
(certainly you can try bypassing the switch, but like I said, the specs say it is 32 Gbps, so it is extremely unlikely that it's causing the problem)<br class="">
<div class="">
<div dir="auto" style="color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div style="color: rgb(0, 0, 0); font-variant-caps: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;">
<font face="Courier" class=""><span style="font-size: 14px;" class="">--------------------------------------------------------------------------------------<br class="">
Steven Ludtke, Ph.D. <<a href="mailto:sludtke@bcm.edu" class="">sludtke@bcm.edu</a>> Baylor College of Medicine <br class="">
Charles C. Bell Jr., Professor of Structural Biology<br class="">
Dept. of Biochemistry and Molecular Biology (<a href="http://www.bcm.edu/biochem" class="">www.bcm.edu/biochem</a>)<br class="">
Academic Director, CryoEM Core (<a href="http://cryoem.bcm.edu" class="">cryoem.bcm.edu</a>)<br class="">
Co-Director CIBR Center (<a href="http://www.bcm.edu/research/cibr" class="">www.bcm.edu/research/cibr</a>)<br class="">
<br class="">
</span></font><br class="">
</div>
</div>
</div>
</div>
</div>
<div><br class="">
<blockquote type="cite" class="">
<div class="">On Oct 17, 2018, at 4:45 PM, Eugene Pichkur <<a href="mailto:eugene.pichkur@gmail.com" class="">eugene.pichkur@gmail.com</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div dir="ltr" class="">
<div dir="ltr" class="">
<div dir="ltr" class="">
<div dir="ltr" class="">
<div dir="ltr" class="">
<div dir="ltr" class="">Dear Bradford, Mike and Steven,<br class="">
<br class="">
</div>
<div dir="ltr" class=""><br class="">
</div>
<div dir="ltr" class="">Bypassing the switch is an excellent idea, I will try it. It might change the IP of the camera and make it inaccessible to the software, but one problem at a time, as they say. </div>
<div dir="ltr" class=""><br class="">
</div>
<div class="">Mike, thank you for providing some insight into the way FEI's software works. Our Microscope PC has 16GB of RAM, so it probably runs a 64bit Win 7. </div>
<div class="">I remember reading about Falcon hack at <a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__bio.brandeis.edu&d=DwMFaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=GWA2IF6nkq8sZMXHpp1Xpg&m=y_AfetL_zOeAJhvPoPdehQRmSpAeEUK_ww6h0Ig7wfM&s=VHucSm-tNXFGvJd_gfP0UebmFLSYeVKEbLZf8iCdWv4&e=" class="">
bio.brandeis.edu</a>, but I think, they've took down the page. As far as I remember, they used BlackBox optical tap to split signal and record frames on the separate computer. </div>
<div class=""><br class="">
</div>
<div class="">I will also dig into Samba related mess, thanks for sharing!</div>
</div>
</div>
</div>
</div>
</div>
<br class="">
<div class="gmail_quote">
<div dir="ltr" class="">On Thu, Oct 18, 2018 at 12:16 AM Ludtke, Steven J <<a href="mailto:sludtke@bcm.edu" class="">sludtke@bcm.edu</a>> wrote:<br class="">
</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word;line-break:after-white-space" class="">IOPS may be an issue, but I suspect it isn't really the cause of your slowdown. Certainly you could install a SSD in the Support PC and test, but I'm skeptical.
<div class=""><br class="">
</div>
<div class="">The Samba share is almost certainly the culprit in limited transfer rates here. I am not aware of any network cards anyone ships in a "full computer" nowadays which cannot trivially handle 1Gbps full duplex. 10Gbps can still have some hardware
bottlenecks, but not 1. Most of the tuning I've done has been for Linux <-> Windows, not Windows to Windows, but these may help:</div>
<div class=""><br class="">
</div>
<div class=""><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__www.experts-2Dexchange.com_articles_3545_Samba-2D3-2Dfor-2DWindows-2DVista-2Dand-2Dlater.html&d=DwMFaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=GWA2IF6nkq8sZMXHpp1Xpg&m=y_AfetL_zOeAJhvPoPdehQRmSpAeEUK_ww6h0Ig7wfM&s=UGnxBuoi9uXkLWvMu1cTW0aPA_xl3fXIwuMRp9Xs-ls&e=" target="_blank" class="">https://www.experts-exchange.com/articles/3545/Samba-3-for-Windows-Vista-and-later.html</a></div>
<div class=""><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.microsoft.com_en-2Dus_windows-2Dserver_administration_performance-2Dtuning_role_file-2Dserver_smb-2Dfile-2Dserver&d=DwMFaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=GWA2IF6nkq8sZMXHpp1Xpg&m=y_AfetL_zOeAJhvPoPdehQRmSpAeEUK_ww6h0Ig7wfM&s=07g4zDlaJKsZWWD20eubFFy-0sOSDWmEg_0a5zfNmpo&e=" target="_blank" class="">https://docs.microsoft.com/en-us/windows-server/administration/performance-tuning/role/file-server/smb-file-server</a></div>
<div class=""><br class="">
</div>
<div class="">
<div class="">
<div dir="auto" style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; word-wrap: break-word; line-break: after-white-space;" class="">
<div dir="auto" style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; word-wrap: break-word; line-break: after-white-space;" class="">
<div dir="auto" style="word-wrap:break-word;line-break:after-white-space" class="">
<div style="font-variant-caps: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;" class="">
<font face="Courier" class=""><span style="font-size:14px" class="">--------------------------------------------------------------------------------------<br class="">
Steven Ludtke, Ph.D. <<a href="mailto:sludtke@bcm.edu" target="_blank" class="">sludtke@bcm.edu</a>> Baylor College of Medicine <br class="">
Charles C. Bell Jr., Professor of Structural Biology<br class="">
Dept. of Biochemistry and Molecular Biology (<a href="http://www.bcm.edu/biochem" target="_blank" class="">www.bcm.edu/biochem</a>)<br class="">
Academic Director, CryoEM Core (<a href="http://cryoem.bcm.edu/" target="_blank" class="">cryoem.bcm.edu</a>)<br class="">
Co-Director CIBR Center (<a href="http://www.bcm.edu/research/cibr" target="_blank" class="">www.bcm.edu/research/cibr</a>)<br class="">
<br class="">
</span></font><br class="">
</div>
</div>
</div>
</div>
</div>
<div class=""><br class="">
<blockquote type="cite" class="">
<div class="">On Oct 17, 2018, at 3:40 PM, Eugene Pichkur <<a href="mailto:eugene.pichkur@gmail.com" target="_blank" class="">eugene.pichkur@gmail.com</a>> wrote:</div>
<br class="m_3465790147179592932Apple-interchange-newline">
<div class="">
<div dir="ltr" class="">
<div dir="ltr" class="">
<div dir="ltr" class="">
<div dir="ltr" class="">
<div dir="ltr" class="">
<div dir="ltr" class="">
<div dir="ltr" class="">
<div dir="ltr" class="">
<div dir="ltr" class="">
<div dir="ltr" class="">Dear Sargis and Steven,
<div class=""><br class="">
</div>
<div class="">Thank you for your help.<br class="">
</div>
<div class=""><br class="">
</div>
<div class="">Here is a map to provide a better understanding of the problem. <br class="">
</div>
<div class="">
<div class=""><span id="m_3465790147179592932cid:ii_jndjgnhg0" class=""><Network_map.jpeg></span><br class="">
</div>
</div>
<div class="">We transfer data to Support PC that has a RAID 0 (sequential write speed ~600MB/s) through a simple Samba share. Unless FEI's EPU writes stacks to Microscope PC's disk prior to transfer, which is unlikely, i don't see a problem with a disk access
speed. </div>
<div class=""><br class="">
</div>
<div class="">I do have doubts about the ethernet controller used in the Camera Support Main Unit, but I don't see a way to test it. </div>
<div class=""><br class="">
</div>
<div class="">IOPS is definitely a problem when there are multiple simultaneous writes/reads, which is probably a case for large CryoEM facilities like SEMC. </div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<br class="">
<div class="gmail_quote">
<div dir="ltr" class="">On Wed, Oct 17, 2018 at 4:48 AM Ludtke, Steven J <<a href="mailto:sludtke@bcm.edu" target="_blank" class="">sludtke@bcm.edu</a>> wrote:<br class="">
</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word;line-break:after-white-space" class="">Ok, can't comment specifically on the Falcon, but this seems more a computer issue than a camera issue.
<div class=""><br class="">
</div>
<div class="">Unless you are storing your data on an external drive (USB portable drive or similar) it is almost certainly not the disk speed. Even a cheap spinning platter internal drive can manage 100-120 MB/s sustained write, and most will do ~150 nowadays.
The theoretical peak for a 1 Gbps network is 120 MB/s and with overhead, 100 - 110 MB/s is a practical limit for the network. Non-SSD external USB drives (yes, even USB3/C) typically max out at ~50 MB/s. </div>
<div class=""><br class="">
</div>
<div class="">It is almost certainly not the network switch. There really aren't any half-duplex switches any more, and it looks like the one you have has a 32 Gbps backplane, so should not be a bottleneck.</div>
<div class=""><br class="">
</div>
<div class="">In most cases, the real issue is the configuration of the shared filesystem on the Windows PC. The K2 computers have a 10 Gbps port generally used for transfer to the microscope PC, and there, tuning the filesystem parameters (and having a sufficiently
fast set of drives on the microscope PC) are critical to getting good throughput. So:
<div class=""><br class="">
</div>
<div class="">1) no writing directly to shared external drives</div>
<div class="">2) check your shared filesystem configuration (if that's what you're using to move the images)</div>
<div class="">3) if you're moving the images some other way (sftp/scp or somesuch), we need to know that to comment<br class="">
<div class=""><br class="">
</div>
<div class=""><br class="">
<div class="">
<div dir="auto" style="letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word;line-break:after-white-space" class="">
<div dir="auto" style="letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word;line-break:after-white-space" class="">
<div dir="auto" style="word-wrap:break-word;line-break:after-white-space" class="">
<div style="font-variant-caps:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class="">
<font face="Courier" class=""><span style="font-size:14px" class="">--------------------------------------------------------------------------------------<br class="">
Steven Ludtke, Ph.D. <<a href="mailto:sludtke@bcm.edu" target="_blank" class="">sludtke@bcm.edu</a>> Baylor College of Medicine <br class="">
Charles C. Bell Jr., Professor of Structural Biology<br class="">
Dept. of Biochemistry and Molecular Biology (<a href="http://www.bcm.edu/biochem" target="_blank" class="">www.bcm.edu/biochem</a>)<br class="">
Academic Director, CryoEM Core (<a href="http://cryoem.bcm.edu/" target="_blank" class="">cryoem.bcm.edu</a>)<br class="">
Co-Director CIBR Center (<a href="http://www.bcm.edu/research/cibr" target="_blank" class="">www.bcm.edu/research/cibr</a>)<br class="">
<br class="">
</span></font><br class="">
</div>
</div>
</div>
</div>
</div>
<div class=""><br class="">
<blockquote type="cite" class="">
<div class="">On Oct 16, 2018, at 8:02 PM, Sargis Dallakyan <<a href="mailto:sargis@nysbc.org" target="_blank" class="">sargis@nysbc.org</a>> wrote:</div>
<br class="m_3465790147179592932m_6426025386482338208Apple-interchange-newline">
<div class=""><font size="2" style="font-family:Helvetica;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none" class=""><b class="">***CAUTION:***
This email is not from a BCM Source. Only click links or open attachments you know are safe.</b></font><span style="font-family:Helvetica;font-size:18px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;float:none;display:inline!important" class=""></span>
<hr style="font-family:Helvetica;font-size:18px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none" class="">
<div id="m_3465790147179592932m_6426025386482338208divtagdefaultwrapper" dir="ltr" style="font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;font-size:12pt;font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols" class="">
<div style="margin-top:0px;margin-bottom:0px" class="">Just to break an ice on this topic, what kind of disk is on a PC that you are transferring the data to? It's been mentioned in one of the<span class="m_3465790147179592932m_6426025386482338208Apple-converted-space"> </span><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__www.youtube.com_watch-3Fv-3DNabTsYl-5FmIQ-26list-3DPLohLhg8SlVmbG8W-5FTFIWQFFhg1LJ0-2Dxf1&d=DwMFAw&c=ZQs-KZ8oxEw0p81sqgiaRA&r=Dk5VoQQ-wINYVssLMZihyC5Dj_sWYKxCyKz9E4Lp3gc&m=4ZvqXtesRM4IBU-zUAPbIdFLZVEo1F1tixUiF6vUDKE&s=KUru33P_2mVMifpXMF33Nyz25BpRHmbk-Tg64ZHkg2I&e=" class="m_3465790147179592932m_6426025386482338208OWAAutoLink" id="m_3465790147179592932m_6426025386482338208LPlnk186991" disabled="" target="_blank">latest
DDN videos</a><span class="m_3465790147179592932m_6426025386482338208Apple-converted-space"> </span>that no matter how big is your bandwidth, you'll be disk IO limited if your storage can't support enough IOPS (<a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__en.wikipedia.org_wiki_IOPS&d=DwMFAw&c=ZQs-KZ8oxEw0p81sqgiaRA&r=Dk5VoQQ-wINYVssLMZihyC5Dj_sWYKxCyKz9E4Lp3gc&m=4ZvqXtesRM4IBU-zUAPbIdFLZVEo1F1tixUiF6vUDKE&s=0wiVG1LvTlAYZBXD_HJ_va_utU5R50hVDaW1XSqarA8&e=" class="m_3465790147179592932m_6426025386482338208OWAAutoLink" id="m_3465790147179592932m_6426025386482338208LPlnk502046" disabled="" target="_blank">https://en.wikipedia.org/wiki/IOPS</a>).
Maybe getting an SSD might solve this bottleneck, if I understand the problem correctly.<br class="">
</div>
<div style="margin-top:0px;margin-bottom:0px" class=""><br class="">
</div>
<div id="m_3465790147179592932m_6426025386482338208Signature" class="">
<div id="m_3465790147179592932m_6426025386482338208divtagdefaultwrapper" dir="ltr" style="font-size:12pt;background-color:rgb(255,255,255);font-family:Calibri,Arial,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols" class="">
<p style="margin-top:0px;margin-bottom:0px" class=""></p>
<div class=""><span id="m_3465790147179592932m_6426025386482338208ms-rterangepaste-start" class=""></span><span class="">Respectfully,</span><span id="m_3465790147179592932m_6426025386482338208ms-rterangepaste-end" class=""></span><br class="">
Sargis Dallakyan, PhD - Systems Administrator<br class="">
National Resource for Automated Molecular Microscopy<br class="">
Simons Electron Microscopy Center<br class="">
New York Structural Biology Center<br class="">
<a title="http://emg.nysbc.org/redmine/users/105" href="https://urldefense.proofpoint.com/v2/url?u=http-3A__emg.nysbc.org_redmine_users_105&d=DwMFAw&c=ZQs-KZ8oxEw0p81sqgiaRA&r=Dk5VoQQ-wINYVssLMZihyC5Dj_sWYKxCyKz9E4Lp3gc&m=4ZvqXtesRM4IBU-zUAPbIdFLZVEo1F1tixUiF6vUDKE&s=DK8kFnGbasHV5hu6_1CAytIhdp2d9bx8DhSXEt0od1U&e=" id="m_3465790147179592932m_6426025386482338208LPNoLP" disabled="" target="_blank" class="">http://emg.nysbc.org/redmine/users/105</a></div>
<br class="">
<p style="margin-top:0px;margin-bottom:0px" class=""></p>
</div>
</div>
<br class="">
<div class="">
<div class="m_3465790147179592932m_6426025386482338208BodyFragment"><font size="2" class=""><span style="font-size:11pt" class="">
<div class="m_3465790147179592932m_6426025386482338208PlainText">Date: Tue, 16 Oct 2018 16:15:00 +0300<br class="">
From: Eugene Pichkur <<a href="mailto:eugene.pichkur@gmail.com" target="_blank" class="">eugene.pichkur@gmail.com</a>><br class="">
To:<span class="m_3465790147179592932m_6426025386482338208Apple-converted-space"> </span><a href="mailto:3dem@ncmir.ucsd.edu" target="_blank" class="">3dem@ncmir.ucsd.edu</a><br class="">
Subject: [3dem] Falcon II uses only half of the 1Gb/s bandwidth.<br class="">
Message-ID:<br class="">
<<a href="mailto:CAJJJP0=9cSHBRxKLOQkaYmqvg3X6Ym4iq-DT_oMCvNYERErr4w@mail.gmail.com" target="_blank" class="">CAJJJP0=9cSHBRxKLOQkaYmqvg3X6Ym4iq-DT_oMCvNYERErr4w@mail.gmail.com</a>><br class="">
Content-Type: text/plain; charset="utf-8"<br class="">
<br class="">
Hello 3DEM community,<br class="">
<br class="">
We've noticed that our Falcon II uses only half of the 1Gbps bandwidth to<br class="">
transfer data from the Camera Support Rack to the Microscope PC (I've<br class="">
attached a screenshot to illustrate the issue).<br class="">
<br class="">
Microscope PC runs Windows 7 and we are able to get the full frame rate<br class="">
(20fps) from the detector, however, it takes ~20s to transfer a typical<br class="">
movie (40frames, 1.2GB) to the PC which is a very frustrating bottleneck.<br class="">
<br class="">
Both Falcon II and Ceta are connected through the same 1Gb switch (HPE<br class="">
OfficeConnect 1420 24G 2SFP), so my guess is that the bandwidth is split<br class="">
50/50. However, during data collection only Falcon II is used which makes<br class="">
this limitation pointless, not to mention that Falcon II and Ceta can't be<br class="">
used simultaneously.<br class="">
<br class="">
Does anyone have a workaround or any info on the topic?<br class="">
<br class="">
<br class="">
Thanks in advance,<br class="">
Eugene Pichkur<br class="">
<br class="">
National Research Centre ?Kurchatov Institute?<br class="">
Akademika Kurchatova, 1, Moscow, Russia, 123098<br class="">
-------------- next part --------------<br class="">
An HTML attachment was scrubbed...<br class="">
URL: <<a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__mail.ncmir.ucsd.edu_pipermail_3dem_attachments_20181016_a9ba7356_attachment.html&d=DwMFAw&c=ZQs-KZ8oxEw0p81sqgiaRA&r=Dk5VoQQ-wINYVssLMZihyC5Dj_sWYKxCyKz9E4Lp3gc&m=4ZvqXtesRM4IBU-zUAPbIdFLZVEo1F1tixUiF6vUDKE&s=ORrcT2ZtD3pxG1SOnfC4g8e6CT_-aLkzonc2aVUQsC0&e=" id="m_3465790147179592932m_6426025386482338208LPlnk596493" class="m_3465790147179592932m_6426025386482338208OWAAutoLink" disabled="" target="_blank">http://mail.ncmir.ucsd.edu/pipermail/3dem/attachments/20181016/a9ba7356/attachment.html</a>><br class="">
-------------- next part --------------<br class="">
A non-text attachment was scrubbed...<br class="">
Name: Data_transfer_Falcon2_crop.png<br class="">
Type: image/png<br class="">
Size: 36229 bytes<br class="">
Desc: not available<br class="">
URL: <<a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__mail.ncmir.ucsd.edu_pipermail_3dem_attachments_20181016_a9ba7356_attachment.png&d=DwMFAw&c=ZQs-KZ8oxEw0p81sqgiaRA&r=Dk5VoQQ-wINYVssLMZihyC5Dj_sWYKxCyKz9E4Lp3gc&m=4ZvqXtesRM4IBU-zUAPbIdFLZVEo1F1tixUiF6vUDKE&s=_tqIiPe4uB8Wu-2WRJmE3oz5GQqMxfyFfUP-nL5xIvU&e=" id="m_3465790147179592932m_6426025386482338208LPlnk400284" class="m_3465790147179592932m_6426025386482338208OWAAutoLink" disabled="" target="_blank">http://mail.ncmir.ucsd.edu/pipermail/3dem/attachments/20181016/a9ba7356/attachment.png</a>><br class="">
<br class="">
------------------------------<br class="">
<br class="">
Subject: Digest Footer<br class="">
<br class="">
_______________________________________________<br class="">
3dem mailing list<br class="">
<a href="mailto:3dem@ncmir.ucsd.edu" target="_blank" class="">3dem@ncmir.ucsd.edu</a><br class="">
<a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.ncmir.ucsd.edu_mailman_listinfo_3dem&d=DwMFAw&c=ZQs-KZ8oxEw0p81sqgiaRA&r=Dk5VoQQ-wINYVssLMZihyC5Dj_sWYKxCyKz9E4Lp3gc&m=4ZvqXtesRM4IBU-zUAPbIdFLZVEo1F1tixUiF6vUDKE&s=ivLCaBkOqzNDdkBqKbq_PHM4v4QIK6Qn1V0pq2BZ2OI&e=" id="m_3465790147179592932m_6426025386482338208LPlnk220647" class="m_3465790147179592932m_6426025386482338208OWAAutoLink" disabled="" target="_blank">https://mail.ncmir.ucsd.edu/mailman/listinfo/3dem</a><br class="">
<br class="">
<br class="">
------------------------------<br class="">
<br class="">
End of 3dem Digest, Vol 134, Issue 28<br class="">
*************************************<br class="">
</div>
</span></font></div>
</div>
</div>
<span style="font-family:Helvetica;font-size:18px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;float:none;display:inline!important" class="">_______________________________________________</span><br style="font-family:Helvetica;font-size:18px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none" class="">
<span style="font-family:Helvetica;font-size:18px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;float:none;display:inline!important" class="">3dem
mailing list</span><br style="font-family:Helvetica;font-size:18px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none" class="">
<a href="mailto:3dem@ncmir.ucsd.edu" style="font-family:Helvetica;font-size:18px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" target="_blank" class="">3dem@ncmir.ucsd.edu</a><br style="font-family:Helvetica;font-size:18px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none" class="">
<a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.ncmir.ucsd.edu_mailman_listinfo_3dem&d=DwICAg&c=ZQs-KZ8oxEw0p81sqgiaRA&r=Dk5VoQQ-wINYVssLMZihyC5Dj_sWYKxCyKz9E4Lp3gc&m=4ZvqXtesRM4IBU-zUAPbIdFLZVEo1F1tixUiF6vUDKE&s=ivLCaBkOqzNDdkBqKbq_PHM4v4QIK6Qn1V0pq2BZ2OI&e=" style="font-family:Helvetica;font-size:18px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" target="_blank" class="">https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.ncmir.ucsd.edu_mailman_listinfo_3dem&d=DwICAg&c=ZQs-KZ8oxEw0p81sqgiaRA&r=Dk5VoQQ-wINYVssLMZihyC5Dj_sWYKxCyKz9E4Lp3gc&m=4ZvqXtesRM4IBU-zUAPbIdFLZVEo1F1tixUiF6vUDKE&s=ivLCaBkOqzNDdkBqKbq_PHM4v4QIK6Qn1V0pq2BZ2OI&e=</a></div>
</blockquote>
</div>
<br class="">
</div>
</div>
</div>
</div>
_______________________________________________<br class="">
3dem mailing list<br class="">
<a href="mailto:3dem@ncmir.ucsd.edu" target="_blank" class="">3dem@ncmir.ucsd.edu</a><br class="">
<a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.ncmir.ucsd.edu_mailman_listinfo_3dem&d=DwMFaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=GWA2IF6nkq8sZMXHpp1Xpg&m=L8Esi10beB1R-ERlRCSQfmHZy_5muHF1yPo7souDjQI&s=xwSQw1wJyYBDsPfmQNSjL71FdnsXhuymUJJC7HgMgd4&e=" rel="noreferrer" target="_blank" class="">https://mail.ncmir.ucsd.edu/mailman/listinfo/3dem</a><br class="">
</blockquote>
</div>
<span id="m_3465790147179592932cid:f_jndm88tp1" class=""><Network_map.jpeg></span></div>
</blockquote>
</div>
<br class="">
</div>
</div>
</blockquote>
</div>
</div>
</blockquote>
</div>
<br class="">
</body>
</html>