[3dem] Introducing FOCUS: The interface between data collection and data processing in cryo-EM

[Michael Purdy]

Henning, this sounds great. Can FOCUS be used with EPU automated data 
collection?

Thanks,
Michael

****************************
Michael D. Purdy
University of Virginia
480 Ray C. Hunt Dr., Rm 320
Charlottesville, VA 22903
*****************************


On 05/10/2017 06:18 AM, Henning Stahlberg wrote:
> Dear Cryo-EM Users,
>
> We would like to draw your attention to a new software system called FOCUS, which we find quite handy to work together with automated cryo-EM data collection, as done for example with SerialEM.
>
> FOCUS is a C++ / Qt front end, running own and third-party softwares in the background. It features a GUI that allows you to define and edit parameters and C-shell or Python scripts, which are then executed in queues. Users can add or edit their own scripts, to adapt FOCUS to specific workflows or tasks. FOCUS comes with a set of default scripts, ready to run.
> Installation of FOCUS requires also the installation of any third-party programs that you want FOCUS to utilise, such as IMOD, EMAN, FREALIGN, or MotionCor2, UNBLUR, Zorro, CTFFIND4, gCTF, gAutomatch, or others.
>
> If you install FOCUS on a strong Linux machine on a computer adjacent to the automated SerialEM run, then FOCUS can for example be used to:
> - monitor the file system of the SerialEM run
> - fetch newly recorded movies from the SerialEM computer, together with the pixel-defect-list and gain reference files,
> - unpack the compressed movies (TIFF ZLW), gain correct,
> - Fourier crop 8k to 4k, or to any pixel size you want (optionally)
> - drift-correct (with ZORRO, MotionCor2, or Unblur)
> - computer 2D averages of the movie, and FFTs of all files
> - measure defocus (gCTF or CTFFIND4)
> - pick particles (gAutomatch)
> - present the last recorded image and its FFT (before and after drift correction), and the drift trajectory plot and CTF Thon ring plot on a website that you would have to setup (ours is at https://status.c-cina.unibas.ch. Ours is open and shows blurred images, but you can protect it with a password and then display the crisp images)
> - present the statistics of each recorded image on that web site (as plots over time)
> - organize all recorded movies in a large table (similar to an Excel sheet), which can be sorted by various parameters such as
> -- grey value (which images are too dark (contaminated, beam lost) or too white (empty hole?)
> -- defocus (which images are in focus or totally out of focus?)
> -- CTF resolution (for which images is CTF determination difficult?)
> -- iciness (which images have a too high ratio of crystalline ice, or show devitrified sample?)
> -- drift (which images have too long or too short or too jittery drift trajectories? which images have too high remaining interframe drift?)
> -- and various other parameters.
> This table can be sorted by any of such parameters, and images can be moved into a TRASH folder with one click.
> - remaining images can be exported in folder structures and STAR files ready for subsequent processing in, e.g., Relion.
>
> FOCUS can also be used for tomography sessions, to:
> - recognize parameters from the file name (e.g., specimen number and tilt angle) and organize files accordingly
> - drift-correct each recorded movie at a certain tilt angle, while taking the current electron dose and the prior electron dose from earlier tilt angle recordings on the same specimen into account for electron-dose-dependent B-factor resolution filtering
> - compute one 2D image per tilt angle movie
> - re-organize the tilt angle images by tilt angle (-60,-59,…,0,…,59,60) when recorded in the Hagen scheme (0,1,-1,-2,2,3,-3,-4,4,5,…60,-60)
> - as above: monitor data collection progress on a web site, sort data, assist in manually pruning of data,
> - export data in MRC stacks with one frame per tilt angle.
> - write out metadata for the stacks into STAR files
>
> FOCUS can also be used for 2D crystal sessions, where it can do the entire 2D crystal processing automatically (the full functionality of the 2dx software package is included in FOCUS). This allows direct 3D reconstructions during the 2D crystal data collection session.
>
> FOCUS maintains a batch queue, with which is runs all processes above in parallel.
>
> We have installed this on an Ubuntu PC with
> 2x12 cores (= 48 threads)
> 256 GB RAM (of which 96GB are used as a RAM disk. More would make life easier.)
> 70 TB HD (RAID5)
> 2 x GTX1080 GPU
> We also use a 40” 4k monitor, which is great to display the images full-screen in full quality.
> We have one such system per automated cryo-EM microscope, which is able to keep up with single particle data collection of 40-frame 8k movies at a rate of 90 movies per hour (Fourier cropping, drift correction wth MotionCor2 on 5x5 patches, CTFFIND4, gautomatch, etc.), and is thereby on par with data collection.
>
> FOCUS is described in Biyani et al., J. Struct. Biol., 2017 (https://www.ncbi.nlm.nih.gov/pubmed/28344036), also available on bioRxiv at https://doi.org/10.1101/105452.
> The software is open-source and freely available in precompiled versions for Linux and OS X. A manual in wiki form is available, all at http://focus-em.org.
>
> Nikhil Biyani, Ricardo D. Righetto, Robert McLeod, Daniel Caujolle-Bert, Daniel Castano-Diez, Kenneth N. Goldie, and Henning Stahlberg
>
>
>
> Henning Stahlberg, PhD
> Prof. for Structural Biology, C-CINA, Biozentrum, University Basel
> Mattenstrasse 26 | D-BSSE | WRO-1058 | CH-4058 Basel | Switzerland
> http://c-cina.org | Tel. +41-61-387 32 62
>
>
>
> _______________________________________________
> 3dem mailing list
> 3dem at ncmir.ucsd.edu
> https://mail.ncmir.ucsd.edu/mailman/listinfo/3dem