[3dem] Workstation

Sat Nov 8 08:22:34 PST 2014

Hi Tim. I think your final comment is the key here (Windows vs Linux). Unless things have changed dramatically in the last year in the CUDA drivers, Linux boxes don’t distinguish between local and remote users, and headless operation is possible for any card. We have used GeForce, Tesla and Quadro cards remotely with no issues in the past. I confess we haven’t done much with the latest generation of GPUs as they aren’t really cost-effective any more for most situations (the exception being an inexpensive workstation with a bulked up consumer grade GPU).  That said, the general situation on linux is:

Consumer cards (Geforce, etc.) - Comparable to Tesla at the high-end, much less expensive, but not well suited to most rack-mount situations. ‘traditional’ 3-D stereo limited to full-screen.
Quadro - For professional interactive graphics. Specialized drivers on Windows for some programs, and able to do ‘traditional’ 3-D stereo in a window. Generally quite expensive. No performance advantage
Tesla - For use in clusters and other rack-mount situations. Hard to effectively use any of the others in this environment, but very expensive.

We spent a fair bit of manpower about 5 years ago and again about 2 years ago implementing GPU computing in EMAN2, but our experience is that for most 2-D tasks, the speedups were typically only 5-20x so it stopped making sense price/performance-wise in clusters. The one exception is single-particle tomography where it can give 100x speedups, and we still use this somewhat actively.  It can also still be viable when building a $1-2k desktop machine, but is a real pain to configure and use for most things.

----------------------------------------------------------------------------
Steven Ludtke, Ph.D.
Professor, Dept of Biochemistry and Mol. Biol.         (www.bcm.edu/biochem)
Co-Director National Center For Macromolecular Imaging        (ncmi.bcm.edu)
Co-Director CIBR Center                          (www.bcm.edu/research/cibr)
Baylor College of Medicine                             
sludtke at bcm.edu

> On Nov 8, 2014, at 2:17 AM, Tim Dahmen <Tim.Dahmen at web.de> wrote:
> 
> Hello Steven,
>  
> just my two bits about the Quadro vs. Consumer-GPU. There is one more thing that might or might not be relevant. The Quadro cards can be used as a Compute Device, for example via CUDA or OpenCL in a remote Desktop or server environment. For a personal workstation this is not an issue, because as long as you are logged in locally, consumer cards can do the same and a Titan will give lots more compute power for the same money. But for a 10K machine there might be the idea to use it as compute server for several people, build slave or similiar. Then Hernando would need to go for Quadro, because the relevant features (for example operate the  card without a monitor attached) are disabled for the consumer devices, and he will have trouble to access the GPU from a build slave environment or via remote desktop. This is all releated to Windows, I have no experience how it behaves on Linux, though.
>  
> best
>     Tim
>  
> Gesendet: Samstag, 08. November 2014 um 02:24 Uhr
> Von: "Steven Ludtke" <sludtke at bcm.edu>
> An: "Hernando J Sosa" <hernando.sosa at einstein.yu.edu>
> Cc: "3dem at ncmir.ucsd.edu" <3dem at ncmir.ucsd.edu>
> Betreff: Re: [3dem] Workstation
> 1 core on AMD is NOT equivalent to 1 core on Intel. For the sort of number crunching we do in image processing (floating point math), AMD is way behind per core. AMD was the clear leader about 10 years ago, but since Intel’s core 2 duo series, AMD has been playing catch-up. They can still be cost effective for web services, etc, but for number crunching, even if you get fewer cores you will get more bang for the buck with Intel. Generally the companies understand this, so it takes 1.5-2x more AMD cores to equal the compute power of a Xeon, and Xeon system costs will be close to 1.5 to 2x more expensive per core for an assembled computer.
>  
> I keep a page for EMAN users with current computer recommendations. My group manages about 2000 cores of cluster computing (5 different generations), so we pay fairly careful attention to these things. I expect in terms of computer purchasing, advice for EMAN will hold quite well for Relion, Frealign, etc.
>  
> http://blake.bcm.edu/emanwiki/EMAN2/FAQ/Computer <http://blake.bcm.edu/emanwiki/EMAN2/FAQ/Computer>
>  
>  
> There is very little reason to pay the significant premium for a Quadro card. You can get much better bang for the buck if you want to do GPU computation. The ONLY thing that the Quadro can do that consumer-grade cards cannot is Windowed frame-interleaved stereo 3D. If you buy a passive stereo monitor (using unpowered polarized glasses) many programs can do very effective stereo with any graphics card at all (like Chimera). 
>  
> The biggest thing to take into account in the movie-mode era is disk performance. Many compute operations can become disk speed limited quite easily. A single traditional hard drive can give ~150 MB/sec. A single SSD can give as high as 500-600 MB/sec for mainstream cards, but they are more expensive, and have somewhat poor durability for heavy write loads. A good inexpensive solution is to put 8 traditional 4TB drives in a case supporting hot-swapping with a PCIE RAID controller. This can give throughputs of ~1.5 GB/sec to the local computer, for a very modest cost.
>  
> See the page above for more...
> ----------------------------------------------------------------------------
> Steven Ludtke, Ph.D.
> Professor, Dept of Biochemistry and Mol. Biol.         (www.bcm.edu/biochem <http://www.bcm.edu/biochem>)
> Co-Director National Center For Macromolecular Imaging        (ncmi.bcm.edu <http://ncmi.bcm.edu/>)
> Co-Director CIBR Center                          (www.bcm.edu/research/cibr <http://www.bcm.edu/research/cibr>)
> Baylor College of Medicine                             
> sludtke at bcm.edu <x-msg://266/sludtke@bcm.edu>
>  
> 
>  
>  
> On Nov 7, 2014, at 6:07 PM, Hernando J Sosa <hernando.sosa at einstein.yu.edu <x-msg://266/hernando.sosa@einstein.yu.edu>> wrote:
>  
> Dear EM users,
>  
> We have some money to spend  (<10K) in a personal  workstation mainly to be used for number crunching  and single particle analysis. I came across  this system (see below) made by a company called Thinkmate.
> Does anybody have experience with this company?   
> Any comments on the system ?
>  
> As configured it has 64 cores (4 x 16) and 128 GB of RAM
>  
> I chose AMD Opteron  over Intel as it seems to produce the best price/performance but I may be wrong.
>   
> Any comments suggestions are welcomed.
>  
> Thanks
>  
> Hernando
> <image001.png>
> _______________________________________________
> 3dem mailing list
> 3dem at ncmir.ucsd.edu <x-msg://266/3dem@ncmir.ucsd.edu>
> https://mail.ncmir.ucsd.edu/mailman/listinfo/3dem <https://mail.ncmir.ucsd.edu/mailman/listinfo/3dem>_______________________________________________ 3dem mailing list 3dem at ncmir.ucsd.edu https://mail.ncmir.ucsd.edu/mailman/listinfo/3dem <https://mail.ncmir.ucsd.edu/mailman/listinfo/3dem>_______________________________________________
> 3dem mailing list
> 3dem at ncmir.ucsd.edu
> https://mail.ncmir.ucsd.edu/mailman/listinfo/3dem

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ncmir.ucsd.edu/pipermail/3dem/attachments/20141108/a171d189/attachment-0001.html>