Partner Organizations:
Scripps Research Institute: Collaborative Research; Personnel Exchanges

Other collaborators:

Ron Milligan, Research Institute Scripps Clinic,
Jenny Hinshaw, NIH
IBM Sponsored University Research Program
Informix Software, Inc.
David Agard, UCSF

Activities and findings:

Research Activities: 

The objectives of the project are to develop a portable and extensible
set of applications and tools for control and acquisition of data from
a remote electron microscope.  These tools will allow a researcher to
develop applications to design, oversee and manage the collection of
large numbers of low-dose electron micrographs.  Specific applications
which we propose developing will allow users without extensive
experience to collect images using cryo-TEM and low-dose imaging
techniques.

In the first year we developed an Instrument Control Server (ICS)
(which functions to provide device independence at the application
level) and an Application Programming Interface to the ICS (Specific
aims 1 and 2).  These software tools are based around a multi-layered
library that separates the low level control of the instrument from
the high level application programming.  These tools, collectively
known as the EMSCOPE library, provide the infrastructure to build the
higher level applications for large scale data collection.  The tools
also simplify porting and distributing the system to other sites. The
EMSCOPE library has been used to develop a number of standalone
applications (for example a system for automatically scanning an
entire grid at low magnification) as well as a web based remote
control interface for the TEM based on the JAVA programming language
[Kisseberth et al., 1999].  These web based tools are used in the
automated TEM acquisition system to monitor the progress of the
experiment. 

During the past year (year 2 of the proposal) we have concentrated on
Specific aims 3 and 4.  These specific aims centered on developing a
means of automatically acquiring low magnification images, assessing
the potential of these areas to yield suitable high magnification data
and finally acquiring high magnification images at suitable locations.
 We prototyped and tested this system by developing an application to
acquire images of negatively stained catalase crystals. We showed that
the automated system can acquire approximately 1000 images in 24 hours
of entirely unattended operation.  We also showed that, by adding
rudimentary 'intelligence' to the system, we could match the
performance of a human operator.  Some of this work was pursued in
collaboration with Brendan Frey, an expert in machine learning
techniques.  With his help we were able to implement an algorithm
capable of isolating individual crystalline patches from a complex of
overlapping crystals.  

For automated acquisition it is necessary to (i) accurately control
the goniometer (specimen stage) in order to locate the features of
interest in the center of the field of view of the low magnification
images; (ii) maintain the feature of interest in the center of the
field of view when the magnification is increased by several orders of
magnitude; and (iii) maintain the microscope in a stable configuration
while unattended for up to12 hours.  We have made a number of
improvements and modifications to the instrument to achieve this.
These include: 
Modeling the Goniometer to Improve Positioning Accuracy: We have
developed a method to improve the accuracy for absolute relocation of
a target specimen using the goniometer on a Philips transmission
electron microscope [Pulokas, et al. 1999]. Using the model, a target
may be located and repositioned to within about 100 nm when moving
over a distance of about 10,000nm.   This is an order of magnitude
more accurate than the best specification offered by the
manufacturer.
Precise Feature Centering during changes in Magnification: Maintaining
the feature of interest and the electron beam in the center of the
field of view as the magnification changes over several orders of
magnitude poses difficulties due to hysteresis effects in the
electromagnets controlling the magnification. Working closely with
Philips engineers we have implemented a solution to this problem which
involved low level programming of the lens currents on the microscope
column.  The beam positions are now stable over a 24 hour period
during which the magnification is changed thousands of times.
Long term operation:  We have extended the life of the LN2 dewar which
cools the anti-contamination device so that it lasts over 12 hours. 
We are also currently implementing a system employing a CryoMiser
(Torr Vacuum Products, Inc.) to allow us to sustain the temperature of
the cryo-stage so that an experiment can be paused for several hours
and then resumed using the same grid the next day.  This system uses a
liquid nitrogen-immersible thermocouple to activate a switch when the
temperature rises slightly.  The switch controls a valve that opens
and allows a controlled flow of LN2 into the vessel containing the
thermocouple.  As soon as the thermocouple is re-immersed in LN2, the
thermocouple detects the temperature change, the switch is closed, and
the LN2 delivery valve closes.

We are currently automatically acquiring low dose images of specimens
embedded in vitreous ice.  We are using specimens of either TMV or
microtubules prepared over a holey carbon foil.  The system is based
around a Philips CM200 TEM and a Gatan MSC CCD camera and is
controlled by the emScope software library.  The overall acquisition
protocol requires (i) obtaining a low magnification image [660x] of a
grid square from a Quantifoil grid; (ii) automatically identifying
holes containing ice of suitable thickness; (iii) acquiring an
intermediate magnification image [6600x] of the identified hole; (iv)
identifying features of interest within the hole; (v) focusing at high
magnification [38,000x] and finally (vi) acquiring a high
magnification image.

We have been using Quantifoil grids as our specimen substrate. These
grids provide holes of fixed size and geometry and greatly simplify
the algorithms required for correct identification of the holes.  The
algorithm which we have implemented uses a cross correlation template
matching and thresholding procedure combined with a filter to identify
the geometric parameters of the Quantifoil lattice.  The algorithm is
very accurate and is extremely robust even for grids where the carbon
foil has been damaged and the geometrical lattice distorted.

Once the holes have been identified the thickness of the ice for a
given hole is estimated  and a threshold on the ice thickness is set
to identify holes for further analysis in step (iii). We have shown
that we can automatically estimate the thickness of a vitreous ice
layer within the hole using the formula developed by Eusemann et al.,
1982).  We have used this formula to set parameters on the automated
hole finder so that we only find those areas of the grid which contain
ice of a specific thickness (e.g. 50 - 150 nm). This method requires
only that the unattenuated beam intensity be measured at the start of
the experiment.

Automated image acquisition required automated focus and astigmatism
correction under low dose conditions. We have implemented a system for
automated focus and astigmatism correction using beam tilt induced
image shifts [Koster and de Ruijter, 1992].  We have systematically
evaluated the performance of our automated focus and astigmatism
correction algorithms.  The results show that we can accurately set
focus to within +/-100 nm on a carbon grid and to within +/-200 nm on
a vitreous ice specimen even when the flatness of the grid required
shifts of many microns between target positions.  The focus could be
also be accurately set through a layer of vitreous ice if the
structure of the underlying substrate provided appropriate low
resolution targets for the focusing algorithm. 

The system as implemented can acquire approximately  500 high
magnification images of vitreous ice specimens in a 24 hour period. 
We are currently assessing the performance of the system and comparing
it against the performance of a human operator.  To achieve this we
have successfully ported the entire system to the laboratory of Ron
Milligan (Scripps Research Institute).  To achieve the technology
transfer of the system to the Scripps laboratory we have had very
close cooperation between that laboratory and members of our own team.
 Jim Pulokas, the research programmer on the project has visited
Scripps twice as has the PI.  A senior research programmer from the
Scripps labhas also spent a week at UIUC.  As a result of this
collaboration we are now sharing software between the two labs
(through a revision control system) and the software is completely
supported for both environments.  There are now several memebrs of the
Scripps laboratory using the system for acquisition of images related
to ongoing research projects within their laboratory.  This will
provide data as to the relative performance of the automated method
against the current manual methods used in that laboratory.  This data
will be used to evaluate and improve the system.

During the past year the software infrastructure that has been
developed as part of this project has also been exported to two other
laboratories. The emscope library has been incorporated into the
automated tomography application software developed in the laboratory
of Dr. David Agard.  This is an ongoing project with the long term
goal of providing a tomography package that will be portable acorss
many instruments and computer systems.
In addition, the software application for systematically scanning and
previewing an entire grid has been exported to the laboratory of Dr.
Jenny Hinshaw.

Research Training:

Mr. Stephen Hack, an undergraduate engineering student, has worked on
various software infrastructure projects related to the overall goals
of the proposal.  He investigated and implemented a system for
revision software source control.  He set this system up, imported all
of our existing code to the system, and trained all personnel
associated with the project in the use of the system.  He also gave a
public presentation of the system to interested outsiders
(http://www.itg.uiuc.edu/forums/1999-07-15/).  Mr. Hack has also been
deeply involved in the project to port all of the software
architecture developed for this project to a Linux system.  This
project is now very well advanced with all the software cross compiled
under Linux.  Complete testing is now underway and the project should
be completed within the next few months before Mr. Hack graduates.

Ms. Jennifer Slown, an undergraduate engineering student, has been
primarily responsible for porting the software architecture to a
Windows and NT platform.  This project is near completion and the
software will be extensively tested over the next few months before
Ms. Slown graduates.  She presented her experience in porting to these
platforms in a public forum (http://www.itg.uiuc.edu/forums/1999.htm)
.

Ms. Amy Reilein has been responsible for specimen preparation and cryo
electron microscopy on the project over the past year.  She has
attended a training workshop on cryo electron microscopy at Purdue
University and has in turn trained other members of our team in these
techniques.  She will graduate with her PhD in the summer of 2000.

Education and Outreach:

This work has been presented at a number of conferences and workshops
as listed in the publications section.  The PI (Bridget Carragher) and
the Co-PI (Clint Potter) were  also jointly responsible for organizing
a national workshop entitled 'Automated Control of Distributed
Instrumentation' held at the Beckman Institute for Advanced Science
and Technology on April 22-23, 1999
(http://www.itg.uiuc.edu/conferences/acdi99/). The workshop, funded by
NSF and the Beckman Institute, aimed at providing an interdisciplinary
forum for exchange of information on automated control of distributed
imaging instrumentation. The workshop focused on applications for
automated control of instrumentation including microscopy, robotics
and visual tracking as well as enabling technologies for these
applications such as middleware, networking and operating systems. The
workshop brought together researchers and developers working on
various aspects of distributed instrumentation and its automated
control.  The workshop addressed recent results and future directions
in automated distributed instrumentation, network protocols,
end-to-end timing, and different aspects of quality of service for
remote instrumentation.

Journal Publications:

C.S. Potter, H. Chu, B. Frey, C. Green, N. Kisseberth, T.J. Madden, K.L. Miller, K. Nahrstedt, J. Pulokas, A. Reilein, D. Tcheng, D. Weber, and B. Carragher, "Leginon: A system for fully automated acquisition of 1000 micrographs a day", Ultramicroscopy, vol. 77, (1999), p. 153. Published

N. Kisseberth, G. Brauer, B. Grosser, C. Potter, and B. Carragher, "JavaScope: A Web-Based TEM Control Interface", Journal of Structural Biology, vol. 125, (1999), p. 229. Published

B. Carragher and C. S. Potter, "The World Wide Laboratory: Remote and Automated Access to Imaging Instrumentation", Workshop Proceedings, vol. , (1999), p. 141. Published

Pulokas, J., C. Green, N. Kisseberth, C. S. Potter, and B. Carragher., "Improving the Positional Accuracy of the Goniometer on the Philips CM Series TEM.", Journal of Structural Biology, vol. , (1999), p. . Accepted

Book(s) of other one-time publications(s):

Other Specific Products:


Internet Dissemination:

http://www.itg.uiuc.edu/technology/autoem/ http://wwl.itg.uiuc.edu/

Contributions:

Contributions within Discipline:

 Molecular microscopy is, and will continue to be, one of the most
important structural approaches in cell biological investigations. 
Currently, the technique requires the acquisition of very large
numbers of high quality images from an electron microscope controlled
by an experienced microscopist.  This is a labor-intensive and slow
methodology and it is clear that this situation must change if
important biological problems are to be addressed in an expeditious
manner.  There is increasing interest in the field for fully
automating the entire process of acquiring high quality transmission
electron micrographs. 

Typically, a microscopist identifies potential features of interest by
visual inspection of a low magnification field of view.  High
magnification images of these identified features are then acquired
using techniques which minimize the exposure of the specimen to
electron beam damage.  As a result, the high magnification image is
never visually examined prior to acquisition.  The quality of the high
magnification image is assessed only after acquisition when the image
can be analyzed and a decision made as to whether it warrants further
processing.  An experienced microscopist assimilates this quality
assessment information and uses it to refine the choice of potentially
relevant low magnification features.  A simple brute force method in
which the entire low magnification field of view is systematically
examined is impractical because the field of view is very large and
the scale change between the low and high magnification images is
typically two orders of magnitude.

We have developed a system, called Leginon to automatically acquire
large numbers of high quality images under low dose conditions. There
are both short term and long term benefits to developing an automated
system for microscope control and image acquisition.  First, in the
short term, there is the advantage of increasing the efficiency of the
process of data collection.  The immediate goal in automating many of
the steps which are now performed manually is to make more efficient
use of the time spent by the researcher on the data acquisition task.
 The automated procedures will increase the throughput of data
collection and thus increase the numbers of structures which can be
analyzed.

The long term benefits of automation carry implications for how the
technique of biological structure analysis using TEM will develop in
the future.  In a recent publication, Henderson (1995) calculated that
to solve structures to atomic resolution using TEM will require the
collection of a very large number of high quality images (>10,000
images for a protein of molecular weight ~200KD).  The promise of
using these electron imaging techniques as a routine method for
analysis cannot be realized using the current manual data collection
procedures as the number of images involved is prohibitively high. 
While there may be isolated laboratories that are prepared to
undertake this labor intensive task in order to solve a particular
structure, it is unlikely to appear as an attractive prospect to young
researchers or graduate students.

We hope that the systems we are developing will contribute towards
turning cryo electron microscopy into a routine and accessible
technique.

An immediate benefit to the community has been provided by a number of
tools that have been developed as part of the infrastructure to
support hte project.   For example, the emscope library has been
incorporated into the automated tomography application software
developed in the laboratory of Dr. David Agard.  This is an ongoing
project with the long term goal of providing a tomography package that
will be portable acorss many instruments and computer systems.
In addition, the software application for systematically scanning and
previewing an entire grid has been exported to the laboratory of Dr.
Jenny Hinshaw.

 The software architecture for automated control of a remote instrument
has general applicability to other fields.  We have used the
expereince gained in developing the software to develop a similar
system to control an Environmental Scanning Electron Microscope.  The
system thus developed has been extensively used in a K-12 Education
and Outreach project called Bugscope
(http://bugscope.beckman.uiuc.edu). 

The improvements in the accuracy of the goniometer on the TEM have
been published and are of potential benefit to any users of the
Philips CM series of TEMs.

 The software architecture for automated control of a remote instrument
has general applicability to other fields.  We have used the
experience gained in developing the software to develop a similar
system to control an Environmental Scanning Electron Microscope.  The
system thus developed has been extensively used in a K-12 Education
and Outreach project called Bugscope
(http://bugscope.beckman.uiuc.edu). 

Two undergraduates and a graduate student have been extensively
involved in the project and have learned new techniques as a direct
result of their involvement.

 The software architecture for automated control of a remote instrument
has general applicability to other fields.  We have used the
experience gained in developing the software to develop a similar
system to control an Environmental Scanning Electron Microscope.  The
system thus developed has been extensively used in a K-12 Education
and Outreach project called Bugscope
(http://bugscope.beckman.uiuc.edu).