[3dem] DDN Solutions

Matthias Wolf matthias.wolf at oist.jp
Tue Aug 16 17:30:50 PDT 2016


Hi Bob,

In 2014, we bought a DDN SFA7700 storage controller with 240TB physical capacity. At the time it used 60x 4TB drives. In 2015 we added an expansion chassis with 84x 8TB drives. Note that the final usable capacity after striping (12-drive raid6 stripes) and formatting (XFS) is about 25% lower! They do this for purpose of redundancy and performance.

The system is in the datacenter about 100m away from the Krios, connected directly to our Leginon server (Dell R720) via quad fdr IB and has a real world write rate up to 2GB/sec. Leginon server has 10Gb optical link to K2 PC and Falcon hack capture PC. DDN are very good when it comes to on site maintenance - they do this routinely and most operations require no downtime (e.g. They hot-swapped an IB controller preventively before failure). Adding the expansion chassis required reformatting the volume and data backup/restore. We have had no drive failure so far and the system has been very reliable.

There doesn't seem to be an option for more efficient use of the physical capacity. Should buy annual maintenance contract. On a budget, I would do a FreeNAS, but our HPC team hates those and does not like them in the data center, which is why we chose DDN.
For DDN I think the 60x 8TB configuration on the 7700 is a good start and you can always add a 4U expansion chassis later. Drive capacity tends to increase, so might as well wait until you really need it. Infiniband is brilliant when it comes to latency and under heavy load. We don't worry about archival or backup at this time, but there is a tape robot somewhere.

Best regards,

   Matthias

On Aug 17, 2016, at 02:45, "bobashley at hi.umn.edu<mailto:bobashley at hi.umn.edu>" <bobashley at hi.umn.edu<mailto:bobashley at hi.umn.edu>> wrote:

Dear Colleagues,

Does anyone have an experiences they'd be willing to share about utilizing DDN solutions to manage Titan Krios data storage and processing? Particularly the size package you went with and if you found you bought too much or not enough initial hardware?

Sincerely,

Bob Ashley
CryoEM Facility Manager
The Hormel Institute
University of Minnesota/Mayo Clinic
801 16th Ave NE
Austin, MN 55912


_______________________________________________
3dem mailing list
3dem at ncmir.ucsd.edu<mailto:3dem at ncmir.ucsd.edu>
https://mail.ncmir.ucsd.edu/mailman/listinfo/3dem
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ncmir.ucsd.edu/pipermail/3dem/attachments/20160817/17671674/attachment.html>


More information about the 3dem mailing list