[3dem] File systems

Sinkovits, Robert sinkovit at sdsc.edu
Sun Jan 3 10:46:20 PST 2016


Btw, I obviously meant NFS and not NSF in my previous email

On 1/3/16 10:43 AM, "Sinkovits, Robert" <sinkovit at sdsc.edu> wrote:

>Hi Reza,
>
>I'm no longer working in 3DEM, but felt that I could make some comments on
>file systems. We use NSF for users' home directories and Lustre where we
>need performance. You already know the limitations of NSF, but if you're
>not facing any scaling issues or performance bottlenecks, I wouldn't feel
>compelled to switch.
>
>Lustre is not for the faint of heart! While it's free and can deliver
>great performance, we often find that it requires a good deal of expert
>baby sitting. When we have problems with the nationally allocated systems
>at SDSC, nine times out of ten it's due to Lustre issues
>
>If you really need a parallel file system and have a budget to pay for it,
>you may want to think about GPFS. One of my colleagues who recently
>returned from NCAR said that they used GPFS on their big (multi-petaflop)
>systems and found that it was rock solid.
>
>I haven't had a chance to work with the other new filesystems that you
>mentioned, but do hear a lot of good things about Ceph. It's open source
>and there is a community of developers. The one downside (pulled verbatim
>from Wikipedia) is
>
>"Ceph currently lacks standard file system repair tools, and the Ceph user
>documentation currently does not recommend storing mission critical data
>on this architecture because it lacks disaster recovery capability and
>tools."
>
>Of course, I would expect mission critical data (e.g. Raw image data) to
>be backed up somewhere else just to be safe.
>
>-- Bob
>
>Robert Sinkovits, Ph.D.
>Director Scientific Applications Group
>San Diego Supercomputer Center
>University of California, San Diego
>
>
>
>
>On 1/3/16 9:53 AM, "Reza Khayat" <rkhayat at ccny.cuny.edu> wrote:
>
>>Hi,
>>
>>Can anyone describe some of their experience with deploying and using a
>>distributed filesystem for image analysis? Is it appropriate to say that
>>NFS is antiquated, slow and less secure than the younger systems like
>>Lustre, Gluster, Ceph, PVFS2, or Fraunhofer?
>>
>>Best wishes,
>>Reza
>>
>>Reza Khayat, PhD
>>Assistant Professor
>>City College of New York
>>85 St. Nicholas Terrace CDI 12308
>>New York, NY 10031
>>(212) 650-6070
>>www.khayatlab.org
>>
>>_______________________________________________
>>3dem mailing list
>>3dem at ncmir.ucsd.edu
>>https://mail.ncmir.ucsd.edu/mailman/listinfo/3dem
>
>_______________________________________________
>3dem mailing list
>3dem at ncmir.ucsd.edu
>https://mail.ncmir.ucsd.edu/mailman/listinfo/3dem



More information about the 3dem mailing list