An Intelligent Microscope for Transmission Electron Microscopy

Sponsor: NSF DBI-9904547  -10/1/99- 9/30/2002

PIs: Clinton S. Potter, Bridget Carragher, David J. Kriegman and Ron Milligan

Summary:

We propose to develop, test, and implement an intelligent software system for the acquisition of images from a transmission electron microscope. This system will provide state-of-the-art quantitative image acquisition and analysis capabilities for the molecular microcopy community. Molecular microscopy can provide unique information about biological structure from the molecular to the cellular level. These structures will yield significant insight into biological function. One of the principal bottlenecks to this technique is that enormous number of images must be acquired for the structural analysis. Currently, the field is constrained by manual data acquisition methods that are slow, labor‑ intensive and result in a very low percentage of suitable images. An intelligent microscope that integrates instrument control, feature recognition algorithms and machine learning techniques would vastly improve both the quality and quantity of data collected and has the potential to revolutionize the field.

Macromolecular structure analysis using electron microscopy requires the acquisition of large numbers of good quality images at high magnification. Because of the effects of radiation damage of biological specimens, the quality of high magnification micrographs can not be judged prior to making the decision to acquire the high magnification image. The decision to acquire an image must be made based on the appearance of the specimen at low magnification. A good microscopist has the ability to link features in the low magnification image to the quality of the high magnification image. However, it is difficult for microscopists to maintain precision in their judgements between sessions at the microscope. Also, since this is empirical knowledge it is difficult to transfer this knowledge to new or inexperienced operators of the instrument. Finally, since there are substantial delays between acquiring the high magnification image and evaluating the quality, it is difficult to systematize the evaluation and improve the decision making process. Our goal is to develop a system which is capable of performing the same task as an expert microscopist in linking the scales, i.e. identifying relevant features in a low magnification field of view and learning to refine this choice based on assessment of the subsequently acquired high magnification image.

Specifically, in developing the intelligent microscope we will address the following questions. Can we quantify and model the quality of the image at high magnification? In the low magnification image, can we represent and model the features that result in an acceptable quality image at high magnification? Can we use this information to link the magnification scales to systematically improve both the quality and quantity of data collected?

We have pursued a number of projects related to the research proposed here. In particular we have developed a real-time distributed software infrastructure for automating the acquisition of transmission electron micrographs. The initial specific aims of that project have been achieved and have now laid the foundation for incorporating state-of-the-art image analysis and machine learning techniques into the data acquisition process. By incorporating machine intelligence into the system, we will vastly improve the quality and quantity of the data collected.

It is critical that the research proposed here be driven by problems of practical interest to the electron microscopy community. One of the co-PIs (Ron Milligan, Scripps Research Institute) is internationally recognized for his research on macromolecular structures imaged using a transmission electron microscope. The principal focus of Milligan’s group is in developing structural models for the motor proteins, in particular muscle filaments and microtubules. The solution of this problem involves collecting and analyzing many thousands of images of helical filaments using low dose cryo-TEM techniques. The proposed system will dramatically increase the efficiency of this task and have a fundamental impact on the progress of this research.

Once the system has been implemented successfully in the Milligan laboratory it will be made generally available to the biological community. Our group has had considerable experience in deploying and supporting large software packages in this manner.

Reports: