[3dem] Relion version of Topaz

[Bullitt, Esther]

Hi,

I am getting errors trying to run the RELION version of Topaz to autopick filaments.

I cannot use the Relion GUI to submit jobs, since our cluster is not in our control and is not set up to do so.
That is not a problem for other things, but for this I am getting ‘File Exists’ errors, when I know that these files are being created during the job and were not there prior to its submission.

Perhaps it is a problem caused by the different options that are available when I run the topaz gui vs when I get command help from

run_topaz_train.py -h. ?


Here is the command I get from the gui:

topaz train --train-images /projectnb/threepol/CS20Hst5set2e/Denoise/job002denoised_micrographs.star --train-targets /projectnb/threepol/CS20Hst5set2e/Extract/job045/particles.star --k-fold 5 --fold 0 --radius 3 --model resnet8 --image-ext .mrc --units 32 --dropout 0.0 --bn on --unit-scaling 2 --ngf 32 --method GE-binomial --autoencoder 0 --num-particles 1000 --l2 0 --learning-rate 0.0002 --minibatch-size 256 --minibatch-balance 0.0625 --epoch-size 5000 --num-epochs 10 --num-workers -1 --test-batch-size 1 --device 0 --save-prefix /projectnb/threepol/CS20Hst5set2e/Train/job002//model --output /projectnb/threepol/CS20Hst5set2e/Train/job002//results.txt

Here is the command I submit:

mpirun -np $NSLOTS `which run_topaz_train.py` --inputmic /projectnb/threepol/CS20Hst5set2e/Denoise/job002denoised_micrographs.star --inputpart /projectnb/threepol/CS20Hst5set2e/Extract/job045/particles.star --k-fold 5 --fold 0 --radius 3 --model resnet8 --image-ext .mrc --units 32 --dropout 0.0 --bn on --scalefactor 2 --ngf 32 --method GE-binomial --autoencoder 0 --num-particles 2000 --l2 0 --learning-rate 0.0002 --minibatch-size 256 --minibatch-balance 0.0625 --epoch-size 5000 --num-epochs 10 --num-workers -1 --test-batch-size 1 --device 0 --output /projectnb/threepol/CS20Hst5set2e/Train/job002/



Here is part of the error message:

Running command: relion_star_printtable /projectnb/threepol/CS20Hst5set2e/Extract/job045/particles.star data_particles _
rlnMicrographName _rlnCoordinateX _rlnCoordinateY | sed 's#.mrc##' | awk -F"/" '{print $NF}' | awk '{print $1"  "$2"
"$3}' >> /projectnb/threepol/CS20Hst5set2e/Train/job002/inputparts.txt
Converting particles.star to Topaz input...
Running command: relion_star_printtable /projectnb/threepol/CS20Hst5set2e/Extract/job045/particles.star data_particles _rlnMicrographName _rlnCoordinateX _rlnCoordinateY | sed 's#.mrc##' | awk -F"/" '{print $NF}' | awk '{print $1"  "$2"    "$3}' >> /projectnb/threepol/CS20Hst5set2e/Train/job002/inputparts.txt
Traceback (most recent call last):
  File "/share/pkg.7/topaz/0.2.5/install/topaz_venv/bin/run_topaz_train.py", line 71, in <module>
    os.makedirs(outargsPreProc)
  File "/share/pkg.7/python3/3.8.10/install/lib/python3.8/os.py", line 223, in makedirs
    mkdir(name, mode)
FileExistsError: [Errno 17] File exists: '/projectnb/threepol/CS20Hst5set2e/Train/job002/PreProcessed/'
Traceback (most recent call last):
  File "/share/pkg.7/topaz/0.2.5/install/topaz_venv/bin/run_topaz_train.py", line 71, in <module>
    os.makedirs(outargsPreProc)
  File "/share/pkg.7/python3/3.8.10/install/lib/python3.8/os.py", line 223, in makedirs
    mkdir(name, mode)

...
--------------------------------------------------------------------------
Primary job  terminated normally, but 1 process returned
a non-zero exit code. Per user-direction, the job has been aborted.

Lastly, I don’t see a relion version of the Topaz pre-processing command for the micrographs (I did denoise them).  Will that happen during training and auto picking, or am I missing a step?


Thanks very much.

Sincerely,

Esther

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ncmir.ucsd.edu/pipermail/3dem/attachments/20220316/078ad82a/attachment.html>