[3dem] File systems

[Reza Khayat]

Thanks to Steve Ludtke and Bob Sinkovits for their fast reply. In response to Steve's questions:

1. The group's workstations will be accessing the file systems. I will not be tackling the issue of building and maintaining a cluster. I'll leave that to the more technically apt. 

2. There are currently no external storage capabilities (if that's what you're asking). I do have needs and funds to establish the system. I'd be grateful if suggestions can be made.  

3. Apologies, I'm reaching the limits of my networking knowledge. Ookla's Speedtest gives me 500Mbps for both upload and download; however, my particular workstation is a bit old. I don't use this system for data processing. Other workstations have Intel 82574L Gigabit Ethernet Controllers. All are located in different rooms of the same floor. Each is connected to its own Ethernet switch on the same floor.    

Thanks again.

Best wishes,
Reza

Reza Khayat, PhD
Assistant Professor
City College of New York
Department of Chemistry
New York, NY 10031

________________________________________
From: 3dem <3dem-bounces at ncmir.ucsd.edu> on behalf of Sinkovits, Robert <sinkovit at sdsc.edu>
Sent: Sunday, January 3, 2016 1:46 PM
To: 3dem at ncmir.ucsd.edu
Subject: Re: [3dem] File systems

Btw, I obviously meant NFS and not NSF in my previous email

On 1/3/16 10:43 AM, "Sinkovits, Robert" <sinkovit at sdsc.edu> wrote:

>Hi Reza,
>
>I'm no longer working in 3DEM, but felt that I could make some comments on
>file systems. We use NSF for users' home directories and Lustre where we
>need performance. You already know the limitations of NSF, but if you're
>not facing any scaling issues or performance bottlenecks, I wouldn't feel
>compelled to switch.
>
>Lustre is not for the faint of heart! While it's free and can deliver
>great performance, we often find that it requires a good deal of expert
>baby sitting. When we have problems with the nationally allocated systems
>at SDSC, nine times out of ten it's due to Lustre issues
>
>If you really need a parallel file system and have a budget to pay for it,
>you may want to think about GPFS. One of my colleagues who recently
>returned from NCAR said that they used GPFS on their big (multi-petaflop)
>systems and found that it was rock solid.
>
>I haven't had a chance to work with the other new filesystems that you
>mentioned, but do hear a lot of good things about Ceph. It's open source
>and there is a community of developers. The one downside (pulled verbatim
>from Wikipedia) is
>
>"Ceph currently lacks standard file system repair tools, and the Ceph user
>documentation currently does not recommend storing mission critical data
>on this architecture because it lacks disaster recovery capability and
>tools."
>
>Of course, I would expect mission critical data (e.g. Raw image data) to
>be backed up somewhere else just to be safe.
>
>-- Bob
>
>Robert Sinkovits, Ph.D.
>Director Scientific Applications Group
>San Diego Supercomputer Center
>University of California, San Diego
>
>
>
>
>On 1/3/16 9:53 AM, "Reza Khayat" <rkhayat at ccny.cuny.edu> wrote:
>
>>Hi,
>>
>>Can anyone describe some of their experience with deploying and using a
>>distributed filesystem for image analysis? Is it appropriate to say that
>>NFS is antiquated, slow and less secure than the younger systems like
>>Lustre, Gluster, Ceph, PVFS2, or Fraunhofer?
>>
>>Best wishes,
>>Reza
>>
>>Reza Khayat, PhD
>>Assistant Professor
>>City College of New York
>>85 St. Nicholas Terrace CDI 12308
>>New York, NY 10031
>>(212) 650-6070
>>www.khayatlab.org
>>
>>_______________________________________________
>>3dem mailing list
>>3dem at ncmir.ucsd.edu
>>https://mail.ncmir.ucsd.edu/mailman/listinfo/3dem
>
>_______________________________________________
>3dem mailing list
>3dem at ncmir.ucsd.edu
>https://mail.ncmir.ucsd.edu/mailman/listinfo/3dem

_______________________________________________
3dem mailing list
3dem at ncmir.ucsd.edu
https://mail.ncmir.ucsd.edu/mailman/listinfo/3dem