[3dem] Re: file conversion (aaron smith)

Wed Feb 6 23:26:09 PST 2013

Dear all,

Attached is the header description of the TIA ser format for TIA 3.x and later. It has remained unchanged for many years and can safely be used and its header interpreted.

The .emi file represent the internal state of the TIA application at the time of writing the files, and can be used to restore that situation within the TIA application. Not only does it contain the image - or images and/or spectra - but also the formatting of the layout, etc. It is in principle not meant for direct access by other packages, that's what the ser files are for, and can indeed change per TIA version. When the software changes (and thus perhaps its internal descriptors) this will be reflected in the .emi files. However, since some of the acquisition information is stored in XML inside the file, some packages still can try to extract that information, if present.

I hope this clarifies some of the 'myths' around the TIA formats ;-).

Kind regards
Remco Schoenmakers

> -----Original Message-----
> From: 3dem-bounces at ncmir.ucsd.edu [mailto:3dem-bounces at ncmir.ucsd.edu]
> On Behalf Of Jianlin Lei
> Sent: Thursday, February 07, 2013 4:52 AM
> To: Bob Grassucci; shid at janelia.hhmi.org; Jianlin Lei
> Cc: 3dem at ncmir.ucsd.edu
> Subject: Re: [3dem] Re: file conversion (aaron smith)
> 
> Hi Dan & Bob,
> 
> Ser files are routinely written alongside the emi files. Ser files can
> be easily converted to other files because ser files have a fixed file
> header of 124 bytes. For example, CP FROM RAW command in SPIDER works
> well for ser files (parameters: Bits/Pixel: 16; Header bytes to be
> skipped: 124; Users must know the columns and rows of the image).
> 
> I heard from an FEI software engineer that emi format keeps changed.
> You may notice that different emi files could have slightly different
> file size (bytes) although they are all 4k by 4k images.
> 
> CP FROM EMI command exists in SPIDER but did not work at least several
> years ago and it is very likely that it still does not work.
> 
> A few years ago, I raised the same issue regarding EMI format
> conversion to FEI. Then I wrote a program to convert either a single or
> a series of emi files to mrc files. It requires online or offline TIA
> and TecnaiTomo.dll from FEI. It works quite well and we have been
> routinely using it for several years. Therefore, almost all of our data
> are collected by using TIA for all Gatan, Tietz and Eagle cameras as
> long as the camera is supported by FEI. This function was also
> implemented in automated data collection package AutoEMation so that
> mrc files can be automatically saved.
> 
> Best regards,
> Jianlin
> 
> > -----Original Message-----
> > From: Bob Grassucci <rg2502 at columbia.edu>
> > Date: Wed, 30 Jan 2013 10:09:10 -0500
> > To: shid at janelia.hhmi.org, Jianlin Lei <jllei at tsinghua.edu.cn>
> > Cc: 3dem at ncmir.ucsd.edu
> > Subject: Re: [3dem] Re: file conversion (aaron smith) Hi Dan, While
> > working on AutoEmation in our lab which runs in the TIA environment
> > Jianlin Lei asked how we could output mrc files and the answer was
> > that we needed the file tecnaitomo.dll which is a com component that
> > needs to be registered.  From there AutoEMation output was mrc files.
> > I have copied Jianlin on this message. Perhaps he can comment.  I
> > believe there is also a cp from emi command in spider assuming the
> > file format has not changed since 2002.
> > Bob
> >
> > On 1/29/2013 3:54 PM, Shi PhD, Dan wrote:
> > > Dear colleagues,
> > > This message is to whom may concern FEI Falcon/Eagle cameras (made
> > > by FEI) which use Tia to control, if you're not interested in using
> > > Tia software, please ignore it.
> > >
> > > The Tia saves two proprietary files for each image i.e. image.emi
> > > and image_1.ser, they seems to contain the same data with different
> > > format; the program also can export an image to non-standard tif
> and
> > > bin(raw) formats without header information. Both ser and emi
> > > formats are currently incompatible with most of Cryo-EM data
> > > processing packages.  Only two or three software packages exist
> > > which claim capable to convert one of the two (not both) files into
> > > the .mrc or .tif "standard" format, we've tried all of them, so
> far,
> > > only EMAN2 seems to work. If FEI could modify Tia's output of each
> > > image to a single file with more popular or open format, like tiff
> > > or mrc, it would make the job much easier and save a lot storage
> space and time to all Tia user group especially to newer one.
> > >
> > > There have been sort of long email communications initialized by
> one
> > > FEI engineer with me, at the end, he seems not willing to do
> > > anything to improve TIA which was designed for material science
> over
> > > 20 years ago. As one Cryo-EM community,  we should all insist that
> > > FEI devotes a little extra effort to  improve or upgrade Tia to
> make
> > > our research more efficiently, which may also benefit FEI's image
> detector market share.
> > > Thanks for your time,
> > > Kind regards,
> > >
> > > Dan Shi, PhD
> > > Senior Scientist
> > > Dr Gonen lab
> > > HHMI Janelia Farm Research campus
> > > 19700 Helix Dr
> > >
> > > Ashburn, VA 20147
> > >
> > >
> > > On 1/21/13 8:19 PM, "Kaelber, Jason Toye" <kaelber at bcm.edu> wrote:
> > >
> > >> EMAN2 supports the .ser files that are written alongside the .emi
> files.
> > >> If you are processing in EMAN2 you can just start with the .ser
> > >> files as if they were .mrc or some other format without any
> > >> worries. If you are not processing in EMAN2 you can use the
> > >> e2proc2d.py command to convert them to some other format. For
> instance the command "
> > >> e2proc2d.py micrograph_0000.ser micrograph_0000.mrc " reads the
> > >> FEI-format file micrograph_0000.ser and writes it out in MRC
> > >> format.
> > >> Sincerely,
> > >> Jason Kaelber_______________________________________________
> > >> 3dem mailing list
> > >> 3dem at ncmir.ucsd.edu
> > >> https://mail.ncmir.ucsd.edu/mailman/listinfo/3dem
> > > _______________________________________________
> > > 3dem mailing list
> > > 3dem at ncmir.ucsd.edu
> > > https://mail.ncmir.ucsd.edu/mailman/listinfo/3dem
> > >
> >
> > ?
> 
> 
> 
> 
> _______________________________________________
> 3dem mailing list
> 3dem at ncmir.ucsd.edu
> https://mail.ncmir.ucsd.edu/mailman/listinfo/3dem
-------------- next part --------------
A non-text attachment was scrubbed...
Name: TIA series format.docx
Type: application/vnd.openxmlformats-officedocument.wordprocessingml.document
Size: 20869 bytes
Desc: TIA series format.docx
Url : https://mail.ncmir.ucsd.edu/mailman/private/3dem/attachments/20130207/87ab0e12/TIAseriesformat-0001.bin