<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
</head>
<body dir="auto">
<div><span></span></div>
<div>
<div><span></span></div>
<div>
<div></div>
<div>Hi Bob,</div>
<div><br>
</div>
<div>In 2014, we bought a DDN SFA7700 storage controller with 240TB physical capacity. At the time it used 60x 4TB drives. In 2015 we added an expansion chassis with 84x 8TB drives. Note that the final usable capacity after striping (12-drive raid6 stripes)
and formatting (XFS) is about 25% lower! They do this for purpose of redundancy and performance.</div>
<div><br>
</div>
<div>The system is in the datacenter about 100m away from the Krios, connected directly to our Leginon server (Dell R720) via quad fdr IB and has a real world write rate up to 2GB/sec. Leginon server has 10Gb optical link to K2 PC and Falcon hack capture PC.
DDN are very good when it comes to on site maintenance - they do this routinely and most operations require no downtime (e.g. They hot-swapped an IB controller preventively before failure). Adding the expansion chassis required reformatting the volume and
data backup/restore. We have had no drive failure so far and the system has been very reliable.</div>
<div><br>
</div>
<div>There doesn't seem to be an option for more efficient use of the physical capacity. Should buy annual maintenance contract. On a budget, I would do a FreeNAS, but our HPC team hates those and does not like them in the data center, which is why we chose
DDN.</div>
<div>For DDN I think the 60x 8TB configuration on the 7700 is a good start and you can always add a 4U expansion chassis later. Drive capacity tends to increase, so might as well wait until you really need it. Infiniband is brilliant when it comes to latency
and under heavy load. We don't worry about archival or backup at this time, but there is a tape robot somewhere.</div>
<div><br>
</div>
<div>Best regards,</div>
<div><br>
</div>
<div> Matthias</div>
<div><br>
On Aug 17, 2016, at 02:45, "<a href="mailto:bobashley@hi.umn.edu">bobashley@hi.umn.edu</a>" <<a href="mailto:bobashley@hi.umn.edu">bobashley@hi.umn.edu</a>> wrote:<br>
<br>
</div>
<blockquote type="cite">
<div><font face="Tahoma" size="2">Dear Colleagues,<br>
<br>
Does anyone have an experiences they'd be willing to share about utilizing DDN solutions to manage Titan Krios data storage and processing? Particularly the size package you went with and if you found you bought too much or not enough initial hardware?<br>
<br>
Sincerely,<br>
<br>
Bob Ashley<br>
CryoEM Facility Manager<br>
The Hormel Institute<br class="">
University of Minnesota/Mayo Clinic<br class="">
801 16th Ave NE<br class="">
Austin, MN 55912 <br>
<br>
<br>
</font></div>
</blockquote>
<blockquote type="cite">
<div><span>_______________________________________________</span><br>
<span>3dem mailing list</span><br>
<span><a href="mailto:3dem@ncmir.ucsd.edu">3dem@ncmir.ucsd.edu</a></span><br>
<span><a href="https://mail.ncmir.ucsd.edu/mailman/listinfo/3dem">https://mail.ncmir.ucsd.edu/mailman/listinfo/3dem</a></span><br>
</div>
</blockquote>
</div>
</div>
</body>
</html>