Tables for
Volume B
Reciprocal space
Edited by U. Shmueli

International Tables for Crystallography (2006). Vol. B. ch. 2.5, pp. 313-315   | 1 | 2 |

Section Image enhancement

B. K. Vainshteinc Image enhancement

| top | pdf |

The real electron-microscope image is subdivided into two components: [J(xy) = I(xy) + N(xy). \eqno(] The main of these, [I(xy)], is a two-dimensional image of the `ideal' object obtained in an electron microscope with instrumental functions inherent to it. However, in the process of object imaging and transfer of this information to the detector there are various sources of noise. In an electron microscope, these arise owing to emission-current and accelerating-voltage fluctuations, lens-supplying current (temporal fluctuations), or mechanical instabilities in a device, specimen or detector (spatial shifts). The two-dimensional detector (e.g. a photographic plate) has structural inhomogeneities affecting a response to the signal. In addition, the specimen is also unstable; during preparation or imaging it may change owing to chemical or some other transformations in its structure, thermal effects and so on. Biological specimens scatter electrons very weakly and their natural state is moist, while in the electron-microscope column they are under vacuum conditions. The methods of staining (negative or positive), e.g. of introducing into specimens substances containing heavy atoms, as well as the freeze-etching method, somewhat distort the structure of a specimen. Another source of structure perturbation is radiation damage, which can be eliminated at small radiation doses or by using the cryogenic technique. The structure of stained specimens is affected by stain graininess. We assume that all the deviations [\Delta I_{k}(xy)] of a specimen image from the `ideal' image [I_{k}(xy)] are included in the noise term [N_{k}(xy)]. The substrate may also be inhomogeneous. All kinds of perturbations cannot be separated and they appear on an electron microscope image as the full noise content [N(xy)].

The image enhancement involves maximum noise suppression [N(xy)] and hence the most accurate separation of a useful signal [I(xy)] from the real image [J(xy)] ([link]. At the signal/noise ratio [I/N \simeq 1] such a separation appears to be rather complicated. But in some cases the real image reflects the structure sufficiently well, e.g. during the atomic structure imaging of some crystals [(I/N \;\gt\; 10)]. In other cases, especially of biological specimen imaging, the noise N distorts substantially the image, [ (I/N) \sim 5\hbox{--} 10]. Here one should use the methods of enhancement. This problem is usually solved by the methods of statistical processing of sets of images [J_{k}] [(k = 1, \ldots, n)]. If one assumes that the informative signal [I_{k}(xy)] is always the same, then the noise error [N(xy)] may be reduced.

The image enhancement methods are subdivided into two classes:

  • (a) image averaging in real space xy;

  • (b) Fourier analysis and filtration in reciprocal space.

These methods can be used separately or in combination. The enhancement can be applied to both the original and the restored images; there are also methods of simultaneous restoration and enhancement.

The image can be enhanced by analogue (mainly optical and photographic) methods or by computational methods for processing digitized functions in real and reciprocal space.

The cases where the image has translational symmetry, rotational symmetry, and where the image is asymmetric will be considered.

Periodic images. An image of the crystal structure with atomic or molecular resolution may be brought to self-alignment by a shift by a and b periods in a structure projection. This can be performed photographically by printing the shifted image on the same photographic paper or, vice versa, by shifting the paper (McLachlan, 1958[link]).

The Fourier filtration method for a periodic image [I_{p}] with noise N is based on the fact that in Fourier space the components [ {\scr F}I_{p}] and [ {\scr F}N] are separated. Let us carry out the Fourier transformation of the periodic signal [I_{p}] with the periods a, b and noise N: [ \eqalignno{ {\scr F}J &= {\scr F}[I_{p}(xy) + N(xy)] &\cr &= {\textstyle\int} I_{p}(xy) \exp [2\pi i(hx + ky)]\;\hbox{d}x\;\hbox{d}y + {\scr F}N &\cr &= {\textstyle\sum} \Phi_{hk} \delta ({\bf u} - {\bf u}_{hk}) + {\scr F}N\hbox{;} &(\cr {\bf u}_{hk} &= h{\bf a}^{*} + k{\bf b}^{*}. &\cr}] The left part of ([link] represents the Fourier coefficients [\Phi_{hk}] distributed discretely with periods [a^{*}] and [b^{*}] in the plane [{\bf u}(uv)]. This is the two-dimensional reciprocal lattice. The right-hand side of ([link] is the Fourier transform [ {\scr F}N] distributed continuously in the plane. Thus these parts are separated. Let us `cut out' from distribution ([link] only [\Phi_{hk}] values using the `window' function [w(uv)]. The window should match each of the real peaks [\Phi_{hk}] which, owing to the finite dimensions of the initial periodic image, are not points, as this is written in an idealized form in ([link] with the aid of δ functions. In reality, the `windows' may be squares of about [a^{*}/10], [b^{*}/10] in size, or a circle. Performing the Fourier transformation of product ([link] without [ {\scr F}N], and set of windows [w({\bf u}) = w(uv) * {\textstyle\sum_{h, \, k}} \delta ({\bf u} - h{\bf a}^{*} - k{\bf b}^{*})], we obtain: [ \eqalignno{ J(xy) &= {\scr F}^{-1} \{w({\bf u}) {\textstyle\sum\limits_{h, \, k}} \Phi_{h, \, k} \delta ({\bf u} - {\bf u}_{h, \, k})\} &\cr &= W(xy) * I_{p}({\bf x}), &(\cr}] the periodic component without the background, [ W(xy) =] [{\scr F}^{-1} w({\bf u})]. The zero coefficient [\Phi_{00}] in ([link] should be decreased, since it is due, in part, to the noise. When the window w is sufficiently small, [I_{p}] in ([link] represents the periodic distribution [\langle I\rangle] (average over all the unit cells of the projection) included in [I_{p}] ([link]. Nevertheless, some error from noise in an image does exist, since with [\Phi_{hk}] we also introduced into the inverse Fourier transformation the background transform values [ {\scr F}^{-1} N_{hk}] which are within the `windows'.

This approach is realized by an analogue method [optical diffraction and filtering of electron micrographs in a laser beam (Klug & Berger, 1964[link])] and can also be carried out by computing.

As an example, Fig.[link] shows an electron micrograph of the periodic structure of a two-dimensional protein crystal, while Fig.[link] represents optical diffraction from this layer. In order to dissect the aperiodic component [ {\scr F}N] in a diffraction plane, according to the scheme in Fig.[link], one places a mask with windows covering reciprocal-lattice points. After such a filtration, only the [I_{p}] component makes a contribution during the image formation by means of a lens, while the component [ {\scr F}N] diffracted by the background is delayed. As a result, an optical pattern of the periodic structure is obtained (Fig.[link]).


Figure | top | pdf |

(a) Diagram of an optical diffractometer. D is the object (an electron micrograph), [M_{p}] is the diffraction plane and a mask that transmits only [\Phi_{hk}], [D_{p}] is the plane of the (filtered) image; (b) an electron micrograph of a crystalline layer of the protein phosphorylase b; (c) its optical diffraction pattern (the circles correspond to the windows in the mask that transmits only the [\Phi_{hk}] diffracted beams from the periodic component of the image); (d) the filtered image. Parts (b)–(d) are based on the article by Kiselev et al. (1971)[link].

Optical diffractometry also assists in determining the parameters of a two-dimensional lattice and its symmetry.

Using the same method, one can separate the superimposed images of two-dimensional structures with different periodicity and in different orientation, the images of the `near' and `far' sides of tubular periodic structures with monomolecular walls (Klug & DeRosier, 1966[link]; Kiselev et al., 1971[link]), and so on.

Computer filtering involves measuring the image optical density [J_{\rm obs}], digitization, and Fourier transformation (Crowther & Amos, 1971[link]). The sampling distance usually corresponds to one-third of the image resolution. When periodic weak phase objects are investigated, the transformation ([link] yields the Fourier coefficients. If necessary, we can immediately make corrections in them using the microscope transfer function according to ([link], (,[link]b[link]) and ([link]), and thereby obtain the true kinematic amplitudes [\Phi_{hk}]. The inverse transformation ([link] gives a projection of the structure (Unwin & Henderson, 1975[link]; Henderson & Unwin, 1975[link]).

Sometimes, an observed image [J({\bf x})] is `noised' by the [N({\bf x})] to a great extent. Then, one may combine data on real and reciprocal space to construct a sufficiently accurate image. In this case, the electron-diffraction pattern is measured and structure-factor moduli from diffraction reflection intensities [I_{hk, \, {\rm obs}}] are obtained: [|\Phi_{hk, \, {\rm obs}}| \sim \sqrt{I_{hk, \, {\rm obs}}}. \eqno(] At the same time, the structure factors [\Phi_{hk, \, {\rm calc}} = |\Phi_{hk, \, {\rm calc}}| \exp (i\alpha_{hk, \, {\rm calc}}) \eqno(] are calculated from the processed structure projection image by means of the Fourier transformation. However, owing to poor image quality we take from these data only the values of phases [\alpha_{hk}] since they are less sensitive to scattering density distortions than the moduli, and construct the Fourier synthesis [\eqalignno{ I(xy) &= {\textstyle\sum\limits_{hk}} | \Phi_{hk, \, {\rm obs}}| \exp (i\alpha_{hk, \, {\rm calc}}) &\cr &\quad\times \exp [2\pi i(hx + ky)]. &(\cr}]

Here the possibilities of combining various methods open up, e.g. for obtaining the structure-factor moduli from X-ray diffraction, and phases from electron microscopy, and so on (Gurskaya et al., 1971[link]).

Images with point symmetry. If a projection of an object (and consequently, the object itself) has a rotational N-fold axis of symmetry, the structure coincides with itself on rotation through the angle [2\pi /N]. If the image is rotated through arbitrary angles and is aligned photographically with the initial image, then the best density coincidence will take place at a rotation through [\alpha = (k2\pi / N)\ (k = 1,\ldots, N)] which defines N. The pattern averaging over all the rotations will give the enhanced structure image with an [(N)^{1/2}] times reduced background (Markham et al., 1963[link]).

Rotational filtering can be performed on the basis of the Fourier expansion of an image in polar coordinates over the angles (Crowther & Amos, 1971[link]). [I(r, \psi) = {\textstyle\sum\limits_{n = -\infty}^{+\infty}} g_{n}(r) \exp (in\varphi). \eqno(] The integral over the radius from azimuthal components [g_{n}] gives their power [p_{n} \sim \textstyle\int\limits_{0}^{a} |g_{n}|^{2}r\;\hbox{d}r, \eqno(] where a is the maximum radius of the particle. A set [p_{n}] forms a spectrum, the least common multiple N of strong peaks defining the N-fold symmetry. The two-dimensional reconstructed image of a particle with rotational symmetry is defined by the synthesis ([link] with [n = 0, N, 2N, 3N].

Asymmetric images. In this case, a set of images is processed by computational or analogue methods. The initial selection of images involves the fulfillment of the maximum similarity condition.

The averaging of n images in real space gives [I_{\rm enh} = (1/n) \textstyle\sum\limits_{k = 1}^{n} J_{k}(xy) = \langle I_{k}\rangle (xy) + (1/n) \textstyle\sum N_{k}(xy). \eqno(] The signal/noise ratio on an average image is [(n)^{1/2}] times enhanced.

The degree of similarity and accuracy of superposition of two images with an account both of translational and angular shifts is estimated by a cross-correlation function 3 of two selected images [J_{1}] and [J_{2}] (Frank, 1975[link], 1980[link]). [\eqalignno{ k({\bf x}') &= J_{1} * J_{2} = {\textstyle\int} J_{1} (x) J_{2} (x + x') \ \hbox{d}x &\cr &= k_{I_{1} I_{2}} + k_{I_{1} N_{2}} + k_{I_{2} N_{1}} + k_{N_{1} N_{2}}. &(\cr}] The value [k({\bf x}')] is the measure of image similarity, the x′ coordinate of the maximum indicates the shift of the images relative to each other. The first term of the resultant expression ([link] is the cross-correlation function of noise-corrected images being compared, the second and third terms are approximately equal to zero, since the noise does not correlate with the signal; the last term is the autocorrelation function of the noise (Cramér, 1954[link]; Frank, 1975[link], 1980[link]).

The calculation of a correlation function is performed by means of Fourier transformation on the basis of the convolution theorem, since the Fourier transformation of the product of the Fourier transform of function [J_{1}] and the conjugated Fourier transform function [J_{2}] gives the cross-correlation function of the initial functions: [k = {\scr F}^{-1} [{\scr F} J_{1} \cdot {\scr F}^{*} J_{2}]. \eqno(]

The probability density of samples for images has the form [\eqalignno{ p (J_{1} J_{2} \ldots J_{n}) &= {1 \over (\sigma \sqrt{2 \pi})^{n}} &\cr & \times \exp \left[{-1 \over 2 \sigma^{2}} \sum\limits^{n} \int [J_{k} ({\bf x} + {\bf x}_{k}) - J ({\bf x})]^{2}\ {\rm d}x\right].\cr &&(\cr}]

Here J is the tentative image (as such, a certain `best' image can first be selected, while at the repeated cycle an average image is obtained), [J_{k} ({\bf x})] is the image investigated, σ is the standard deviation of the normal distribution of noises and [x_{k}] the relative shift of the image. This function is called a likelihood function; it has maxima relative to the parameters [J (x)], [x_{k},\ \sigma]. The average image and dispersion are [\eqalignno{ &J ({\bf x}) = (1/n) {\textstyle\sum\limits^{n}} [J_{k} ({\bf x} - {\bf x}_{k})],\cr \sigma^{2} &= (1/n) {\textstyle\sum\limits^{n}} [J_{k} ({\bf x} - {\bf x}_{k}) - J ({\bf x})]^{2}. &(\cr}] This method is called the maximum-likelihood method (Cramér, 1954[link]; Kosykh et al., 1983[link]).

It is convenient to carry out the image alignment, in turn, with respect to translational and angular coordinates. If we start with an angular alignment we first use autocorrelation functions or power spectra, which have the maximum and the symmetry centre at the origin of the coordinates. The angular correlation maximum [f (\theta') = {\textstyle\int}\; f_{k} (\theta - \theta') f_{e} (\theta) \hbox{ d} \theta \eqno(] gives the mutual angle of rotation of two images.

Then we carry out the translational alignment of rotationally aligned images using the translational correlation function ([link] (Langer et al., 1970[link]).

In the iteration alignment method, the images are first translationally aligned and then an angular shift is determined in image space in polar coordinates with the centre at the point of the best translational alignment. After the angular alignment the whole procedure may be repeated (Steinkilberg & Schramm, 1980[link]).

The average image obtained may have false high-frequency components. They can be excluded by multiplying its Fourier components by some function and suppressing high-space frequencies, for instance by an `artificial temperature factor' [\exp \{- B | {\bf u} |^{2}\}].

For a set of similar images the Fourier filtration method can also be used (Ottensmeyer et al., 1977[link]). To do this, one should prepare from these images an artificial `two-dimensional crystal', i.e. place them in the same orientation at the points of the two-dimensional lattice with periods a, b. [J = {\textstyle\sum\limits_{k = 1}^{n}} J_{k} ({\bf x} - {\bf t}_{p})\hbox{;}\; {\bf t} = p_{1} {\bf a} + p_{2} {\bf b.} \eqno(] The processing is then performed according to ([link], ([link]; as a result one obtains [\langle I (xy)\rangle] with reduced background. Some translational and angular errors in the arrangement of the images at the artificial lattice points act as an artificial temperature factor. The method can be realized by computing or by optical diffraction.


First citation Cramér, H. (1954). Mathematical methods of statistics. University of Princeton.Google Scholar
First citation Crowther, R. A. & Amos, L. A. (1971). Harmonic analysis of electron microscope images with rotational symmetry. J. Mol. Biol. 60, 123–130.Google Scholar
First citation Frank, J. (1975). Averaging of low exposure electron micrographs of non-periodic objects. Ultramicroscopy, 1, 159–162.Google Scholar
First citation Frank, J. (1980). The role of correlation techniques in computer image processing. In Computer processing of electron microscopy images, edited by P. W. Hawkes, pp. 187–222. Berlin: Springer-Verlag.Google Scholar
First citation Gurskaya, G. V., Lobanova, G. M. & Vainshtein, B. K. (1971). X-ray diffraction and electron-microscope study of hexagonal catalase crystal. Sov. Phys. Crystallogr. 16, 662–669.Google Scholar
First citation Henderson, R. & Unwin, P. N. T. (1975). Three-dimensional model of purple membrane obtained by electron microscopy. Nature (London), 257, 28–32.Google Scholar
First citation Kiselev, N. A., Lerner, F. Ya. & Livanova, N. B. (1971). Electron microscopy of muscle phosphorylase B. J. Mol. Biol. 62, 537–549.Google Scholar
First citation Klug, A. & Berger, J. E. (1964). An optical method for the analysis of periodicities in electron micrographs and some observations on the mechanism of negative staining. J. Mol. Biol. 10, 565–569.Google Scholar
First citation Klug, A. & DeRosier, D. J. (1966). Optical filtering of electron micrographs: reconstruction of one-sided images. Nature (London), 212, 29–32.Google Scholar
First citation Kosykh, V. P., Pustovskikh, A. I., Kirichuk, V. S., Kühne, T., Orlova, E. V., Tsuprun, V. L. & Kiselev, N. A. (1983). Use of digital storage methods to recover images of monocrystalline layers of virus particles. Sov. Phys. Crystallogr. 28, 637–643.Google Scholar
First citation Langer, R., Frank, J., Feltynowski, A. & Hoppe, W. (1970). Anwendung des Bilddifferenzverfahrens auf die Untersuchung von Strukturänderungen dünner Kohlefolien bei Elektronenbestrahlung. Ber. Bunsenges Phys. Chem. 74(11), 1120–1126.Google Scholar
First citation Markham, R., Frey, S. & Hills, G. J. (1963). Methods for the enhancement of image detail and accentuation of structure in electron microscopy. Virology, 20, 88–102.Google Scholar
First citation McLachlan, D. (1958). Crystal structure and information theory. Proc. Natl Acad. Sci. USA, 44, 948–956.Google Scholar
First citation Ottensmeyer, F. P., Andrews, J. W., Basett-Jones, D. P., Chan, A. S. & Hewitt, J. (1977). Signal to noise enhancement in dark field electron micrographs of vasopressin: filtering of arrays of images in reciprocal space. J. Microsc. 109, 256–268.Google Scholar
First citation Steinkilberg, M. & Schramm, H. J. (1980). Eine verbesserte Drehkorrelations Methode für die Strukturbestimmung biologischer Macromoleküle durch Mittelung elektronenmikroskopischer Bilder. Hoppe–Seyler's Z. Physiol. Chem. 361, 1363–1369.Google Scholar
First citation Unwin, P. N. T. & Henderson, R. (1975). Molecular structure determination by electron microscopy of unstained crystalline specimens. J. Mol. Biol. 94, 425–440.Google Scholar

to end of page
to top of page