[3dem] File systems

[Sinkovits, Robert]

Hi Reza,

I'm no longer working in 3DEM, but felt that I could make some comments on
file systems. We use NSF for users' home directories and Lustre where we
need performance. You already know the limitations of NSF, but if you're
not facing any scaling issues or performance bottlenecks, I wouldn't feel
compelled to switch.

Lustre is not for the faint of heart! While it's free and can deliver
great performance, we often find that it requires a good deal of expert
baby sitting. When we have problems with the nationally allocated systems
at SDSC, nine times out of ten it's due to Lustre issues

If you really need a parallel file system and have a budget to pay for it,
you may want to think about GPFS. One of my colleagues who recently
returned from NCAR said that they used GPFS on their big (multi-petaflop)
systems and found that it was rock solid.

I haven't had a chance to work with the other new filesystems that you
mentioned, but do hear a lot of good things about Ceph. It's open source
and there is a community of developers. The one downside (pulled verbatim
from Wikipedia) is

"Ceph currently lacks standard file system repair tools, and the Ceph user
documentation currently does not recommend storing mission critical data
on this architecture because it lacks disaster recovery capability and
tools."

Of course, I would expect mission critical data (e.g. Raw image data) to
be backed up somewhere else just to be safe.

-- Bob

Robert Sinkovits, Ph.D.
Director Scientific Applications Group
San Diego Supercomputer Center
University of California, San Diego




On 1/3/16 9:53 AM, "Reza Khayat" <rkhayat at ccny.cuny.edu> wrote:

>Hi,
>
>Can anyone describe some of their experience with deploying and using a
>distributed filesystem for image analysis? Is it appropriate to say that
>NFS is antiquated, slow and less secure than the younger systems like
>Lustre, Gluster, Ceph, PVFS2, or Fraunhofer?
>
>Best wishes,
>Reza
>
>Reza Khayat, PhD
>Assistant Professor
>City College of New York
>85 St. Nicholas Terrace CDI 12308
>New York, NY 10031
>(212) 650-6070
>www.khayatlab.org
>
>_______________________________________________
>3dem mailing list
>3dem at ncmir.ucsd.edu
>https://mail.ncmir.ucsd.edu/mailman/listinfo/3dem