<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">
If you don't expect to need to access it again, ie - purely an emergency backup, Amazon Glacier is a cost effective solution, as long as you have $ to continue paying for it. 100 TB of deep archive Glacier storage would run about $1200/year (+ additional cost
if you need to retrieve it).
<div><br>
</div>
<div>If you are storing it for possible additional processing, then you want the storage to be "close" in data transfer terms to the processing power. ie - if you are processing in the cloud, then storing the data in the cloud makes sense. Clearly you would
not want to process the data directly from cloud storage. Keep in mind the relative speeds of transfer for different devices/transfer methods:
<div><br>
</div>
<div>M.2 SSD -> 2-4 GB/s</div>
<div>8 drive RAID array with spinning platters directly on the machine -> ~1 GB/s</div>
<div>SATA SSD -> 0.6 GB/s</div>
<div>single spinning platter on machine -> 0.15 GB/s</div>
<div>gigabit network remote access -> 0.1 GB/s</div>
<div>less than gigabit remote access (cloud at typical institutions) -> <0.1 GB/s<br>
<div><br>
</div>
<div>For size comparison, a 4k x 4k x 1k tomogram at 8 bits is 16 GB, so opening that from an M.2 SSD might take 4-8 seconds, whereas opening the same file over a gigabit NAS would take almost 3 minutes.</div>
<div><br>
</div>
<div>Personally, I have a 12 bay Synology NAS box with a 10 Gb network card in it under my desk. With 16 TB drives and RAID6 this gives about 150 TB of usable storage space, which you can access at ~1 GB/s. Cost ~$5000, with an expected drive life of ~5 years,
ie - expect you will have to periodically replace bad drives occasionally after the first few years. </div>
<div><br>
</div>
<div>It's worth noting here that at $5000, with an expected life of ~5 years before you start having to pay for more drives, this is $1000/year and gives high speed access, compared to the $1200/year for deep Glacier storage above. However, the Glacier storage
has much better reliability than a single RAID6 array with no additional backup.</div>
<div><br>
</div>
<div>Anyway, some food for thought :^)</div>
<div><br id="lineBreakAtBeginningOfMessage">
<div>
<div dir="auto" style="text-align: start; text-indent: 0px; overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">
<div dir="auto" style="text-align: start; text-indent: 0px; overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">
<div dir="auto" style="text-align: start; text-indent: 0px; overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">
<div dir="auto" style="text-align: start; text-indent: 0px; overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">
<div dir="auto" style="text-align: start; text-indent: 0px; overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">
<div dir="auto" style="text-align: start; text-indent: 0px; overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">
<div><font face="Courier New"><span style="font-size: 14px;">---</span></font></div>
<div><font face="Courier New"><span style="font-size: 14px;">Steven Ludtke, Ph.D. <sludtke@bcm.edu> Baylor College of Medicine<br>
Charles C. Bell Jr., Professor of Structural Biology Dept. of Biochemistry <br>
Deputy Director, Advanced Technology Cores and Molecular Pharmacology<br>
Academic Director, CryoEM Core<br>
Co-Director CIBR Center<br>
</span></font><br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<div><br>
<blockquote type="cite">
<div>On Feb 14, 2024, at 6:14 PM, Jobichen <jobichenc@yahoo.com> wrote:</div>
<br class="Apple-interchange-newline">
<div>
<div>
<div class="yahoo-style-wrap" style="font-family:Helvetica Neue, Helvetica, Arial, sans-serif;font-size:13px;">
<div dir="ltr" data-setdir="false">Dear All,</div>
<div dir="ltr" data-setdir="false">We are looking for some suggestions on storing the raw datasets/movies. What will be best option for storing around 100TB of movies/processed data.</div>
<div dir="ltr" data-setdir="false">What will be pros/cons of having own storage server vs cloud storage options.</div>
<div dir="ltr" data-setdir="false">Thank you for your time.</div>
<div dir="ltr" data-setdir="false">Jobi</div>
<div dir="ltr" data-setdir="false"><br>
</div>
<div dir="ltr" data-setdir="false"><br>
</div>
<div dir="ltr" data-setdir="false"><br>
</div>
</div>
</div>
_______________________________________________<br>
3dem mailing list<br>
3dem@ncmir.ucsd.edu<br>
https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.ncmir.ucsd.edu_mailman_listinfo_3dem&d=DwICAg&c=ZQs-KZ8oxEw0p81sqgiaRA&r=GWA2IF6nkq8sZMXHpp1Xpg&m=-fMPusn_TT7DVAUweasDDQG4kEyzhEAyjRtShGQYPmx9cRVoBtVsmUUqEMrMPs9w&s=LBeNcMDu7IJx1_Y7BTp2_JFhuug6w0oVJobkLUozOFc&e=
<br>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</body>
</html>