[3dem] Gatan K3 reliability issues

[Craig Yoshioka]

Hi Terje,

Sounds like a dud?

My own opinion is I much prefer the overall reliability and quirks of the K3 to the K2.

Issues we’ve seen with K3’s:

1) Strange pixel defects on our first K3 installed Oct 2018. Even with these, the K3 was an upgrade in every area over the K2 (including reliability).  I have not seen this problem on any other K3.  These days, it now seems almost completely fixed: at some point I made a guess that the issue was in the communication between the FPGA boards, I pulled all the boards and moved the “unhappiest" board to the end of the chain (so it only connected to one FPGA neighbor).  I also cleaned and reseated all the SATA cables connecting the FPGAs, and cleaned and reseated all the FPGA DIMMs.

2) FPGAs reporting memory errors or losing sync: only robust fix seems to be a _full_ power cycle of the PC- I suspect the FPGAs are a fairly independent subsystem, and don’t re-initialize properly unless they lose power.

3) Other errors that show up in quick status, ie. Losing the i2C communication, etc. often seem to require rebooting the camera power supply.  These seem very rare compared to how often we’d have to power cycle the K2 digitizer and FPGAs.   In two years I can count on one hand the number of times we’ve had to power cycle the K3 power supply.

4) The USB-Ethernet adapter bundled with the system for scope comms is flaky.  I bought a bunch of $20 1Gbe NICs that I dropped into the K3 PCs when possible.  So far these are much more reliable.

To the above, I’ll add that Gatan moving to a commodity Supermicro server chassis is greatly appreciated from my perspective.  It has yielded some really nice benefits:

1) I feel way more empowered to troubleshoot issues related to the PC hardware
2) The troubleshooting cycle is so much faster without waiting for upload of gain references
3) The Supermicro chassis has a full RMU.  I can power cycle the PC remotely, if needed, to address occasional FPGA problems.






> On Jun 24, 2020, at 1:43 PM, Dokland, Terje <dokland at uab.edu> wrote:
> 
> I wanted to poll the community for others’ experience with the reliability of the Gatan K3 camera. We got ours installed in March last year, and In our experience the reliabilty has been appalling. 
> 
> So far we have had to send it back to the factory for repair THREE times, all within one year. Each time it takes 1-2 months before we get it back, during which we are not able to do any imaging. This is in addition to problems with the computer, the capture cards and the MIB (shutter control) that have all failed at least once in the past year. 
> 
> My question is whether this is typical for the K3, or did we receive a dud? 
> 
> I can imagine and accept a few initial teething problems, but this is getting ridiculous. We had the K2 for somewhat over a year before we got the K3, and had no problems with it. Before that, we had a Gatan CCD, which did not have any problems at all in its 10 year lifetime. Did Gatan run out an unfinished product, or have we just been unlucky? 
> 
> Any insights would be appreciated. 
> 
> Terje
> 
> 
> _______________________________________________
> 3dem mailing list
> 3dem at ncmir.ucsd.edu
> https://mail.ncmir.ucsd.edu/mailman/listinfo/3dem