[3dem] File systems

[Reza Khayat]

Thanks to all, and particularly Steve for being so thorough. I think we’ll go with a hybrid solution for our workstations -Steve’s third to last paragraph. I’ll talk to the other PIs for our cluster needs. 

Best wishes and Happy New Year,
Reza

Reza Khayat, PhD
Assistant Professor
City College of New York
85 St. Nicholas Terrace CDI 12308
New York, NY 10031
(212) 650-6070
www.khayatlab.org

> On Jan 4, 2016, at 2:17 AM, Sjors Scheres <scheres at mrc-lmb.cam.ac.uk> wrote:
> 
> Hi Reza, Ben, Steve & the rest,
> 
> Happy New Year! That 2016 may bring you all many side chains. :-)
> 
> We've tested infiniband verus our standard ethernet and deemed the cost of
> it not worth it, as we saw hardly any improvement in speed of relion. We
> did however see an improvement of >30% speed when switching our cluster
> from a NFS to a Fraunhofer file system. The issue arises when one starts
> accessing the same disk from several hundreds of MPI processes, something
> that can easily happen when dozens of users use the same system.
> I think I agree with Steve that for not very large group-wide systems NFS
> will be better, as it is much  easier, but for larger clusters a more
> parallel file system will probably be worth the effort. At LMB, our
> (small) home areas are based on NFS, while the disks on our cluster were
> Fraunhofer and are now gpfs.
> HTH,
> Sjors
> 
> 
>> Hi Reza,
>> 
>> Our cluster was recently upgraded to InfiniBand and switched to gpfs.
>> Since then the rate of mysterious mpi crashes/hang during relion runs has
>> decreased quite a lot. And relion runs faster. Is it due to gpfs, to
>> InfiniBand or to the combination of both? I do not know.
>> 
>> Cheers
>> Ben
>> 
>> Le 3 janv. 2016 à 18:53, Reza Khayat <rkhayat at ccny.cuny.edu> a écrit :
>> 
>> Hi,
>> 
>> Can anyone describe some of their experience with deploying and using a
>> distributed filesystem for image analysis? Is it appropriate to say that
>> NFS is antiquated, slow and less secure than the younger systems like
>> Lustre, Gluster, Ceph, PVFS2, or Fraunhofer?
>> 
>> Best wishes,
>> Reza
>> 
>> Reza Khayat, PhD
>> Assistant Professor
>> City College of New York
>> 85 St. Nicholas Terrace CDI 12308
>> New York, NY 10031
>> (212) 650-6070
>> www.khayatlab.org
>> 
>> _______________________________________________
>> 3dem mailing list
>> 3dem at ncmir.ucsd.edu
>> https://mail.ncmir.ucsd.edu/mailman/listinfo/3dem
>> _______________________________________________
>> 3dem mailing list
>> 3dem at ncmir.ucsd.edu
>> https://mail.ncmir.ucsd.edu/mailman/listinfo/3dem
>> 
> 
> 
> -- 
> Sjors Scheres
> MRC Laboratory of Molecular Biology
> Francis Crick Avenue, Cambridge Biomedical Campus
> Cambridge CB2 0QH, U.K.
> tel: +44 (0)1223 267061
> http://www2.mrc-lmb.cam.ac.uk/groups/scheres
>