International
Tables for
Crystallography
Volume B
Reciprocal space
Edited by U. Shmueli

International Tables for Crystallography (2006). Vol. B. ch. 2.3, pp. 254-255   | 1 | 2 |

## Section 2.3.6.4. Sampling, background and interpretation

M. G. Rossmanna* and E. Arnoldb

aDepartment of Biological Sciences, Purdue University, West Lafayette, Indiana 47907, USA, and  bCABM & Rutgers University, 679 Hoes Lane, Piscataway, New Jersey 08854-5638, USA
Correspondence e-mail:  mgr@indiana.bio.purdue.edu

#### 2.3.6.4. Sampling, background and interpretation

| top | pdf |

If the origins are retained in the Pattersons, their product will form a high but constant plateau on which the rotation-function peaks are superimposed; this leads to a small apparent peak-to-noise ratio. The effect can be eliminated by removal of the origins through a modification of the Patterson coefficients. Irrespective of origin removal, a significant peak is one which is more than three r.m.s. deviations from the mean background.

As in all continuous functions sampled at discrete points, a convenient grid size must be chosen. Small intervals result in an excessive computing burden, while large intervals might miss peaks. Furthermore, equal increments of angles do not represent equal changes in rotation, which can result in distorted peaks (Lattman, 1972). In general, a crude idea of a useful sampling interval can be obtained by considering the angle necessary to move one reciprocal-lattice point onto its neighbour (separated by ) at the extremity of the resolution limit, R. This interval is given by

Simple sharpening of the rotation function can be useful. This can be achieved by restricting the computations to a shell in reciprocal space or by using normalized structure factors. Useful limits are frequently 10 to 6 Å for an average protein or 6 to 5 Å for a virus structure determination.

When exploring the rotation function in polar coordinates, there is no significance to the latitude ϕ (Fig. 2.3.6.4) when . For small values of ψ, the rotation function will be quite insensitive to ϕ, which therefore needs to be explored only at coarse intervals (say 45°). As ψ approaches the equator at 90°, optimal increments of ψ and ϕ will be about equal. A similar situation exists with Eulerian angles. When , the rotation function will be determined by , corresponding to and varying κ in polar coordinates. There will be no dependence on . Thus Eulerian searches can often be performed more economically in terms of the variables and , where which reduces to the simple rotation matrix when .

The computational effort to explore carefully a complete asymmetric unit of the rotation-function Eulerian group can be considerable. However, unless improper rotations are under investigation (as, for example, cross-rotation functions between different crystal forms of the same molecule), it is not generally necessary to perform such a global search. The number of molecules per crystallographic asymmetric unit, or the number of subunits per molecule, are often good indicators as to the possible types of noncrystallographic symmetry element. For instance, in the early investigation of insulin, the rotation function was used to explore only the plane in polar coordinates as there were only two molecules per crystallographic asymmetric unit (Dodson et al., 1966). Rotation functions of viruses, containing 532 icosahedral symmetry, are usually limited to exploration of the and 144° planes [e.g. Rayment et al. (1978) and Arnold et al. (1984)].

In general, the interpretation of the rotation function is straightforward. However, noise often builds up relative to the signal in high-symmetry space groups or if the data are limited or poor. One aid to a systematic interpretation is the locked rotation function (Rossmann et al., 1972) for use when a molecule has more than one noncrystallographic symmetry axis. It is then possible to determine the rotation-function values for each molecular axis for a chosen molecular orientation (Fig. 2.3.6.6).

 Figure 2.3.6.6 | top | pdf |The locked rotation function, L, applied to the determination of the orientation of the common cold virus (Arnold et al., 1984). There are four virus particles per cubic cell with each particle sitting on a threefold axis. The locked rotation function explores all positions of rotation about this axis and, hence, repeats itself after 120°. The locked rotation function is determined from the individual rotation-function values of the noncrystallographic symmetry directions of a 532 icosahedron. [Reprinted with permission from Arnold et al. (1984).]

Another problem in the interpretation of rotation functions is the appearance of apparent noncrystallographic symmetry that relates the self-Patterson of one molecule to the self-Patterson of a crystallographically related molecule. For example, take the case of α-chymotrypsin (Blow et al., 1964). The space group is with a molecular dimer in each of the two crystallographic asymmetric units. The noncrystallographic dimer axis was found to be perpendicular to the crystallographic axis. The product of the crystallographic twofold in the Patterson with the orthogonal twofold in the self-Patterson vectors around the origin creates a third twofold, orthogonal to both of the other twofolds. In real space this represents a twofold screw direction relating the two dimers in the cell. In other cases, the product of the crystallographic and noncrystallographic symmetry results in symmetry which only has meaning in terms of all the vectors in the vicinity of the Patterson origin, but not in real space. Rotation-function peaks arising from such products are called Klug peaks (Johnson et al., 1975). Such peaks normally refer to the total symmetry of all the vectors around the Patterson origin and may, therefore, be much larger than the peaks due to noncrystallographic symmetry within one molecule alone. Hence the Klug peaks, if not correctly recognized, can lead to erroneous conclusions (Åkervall et al., 1972). Litvin (1975) has shown how Klug peaks can be predicted. These usually occur only for special orientations of a particle with a given symmetry relative to the crystallographic symmetry axes. Prediction of Klug peaks requires the simultaneous consideration of the noncrystallographic point group, the crystallographic point group and their relative orientations.

### References

Arnold, E., Erickson, J. W., Fout, G. S., Frankenberger, E. A., Hecht, H. J., Luo, M., Rossmann, M. G. & Rueckert, R. R. (1984). Virion orientation in cubic crystals of the human common cold virus HRV14. J. Mol. Biol. 177, 417–430.Google Scholar
Blow, D. M., Rossmann, M. G. & Jeffery, B. A. (1964). The arrangement of α-chymotrypsin molecules in the monoclinic crystal form. J. Mol. Biol. 8, 65–78.Google Scholar
Dodson, E., Harding, M. M., Hodgkin, D. C. & Rossmann, M. G. (1966). The crystal structure of insulin. III. Evidence for a 2-fold axis in rhombohedral zinc insulin. J. Mol. Biol. 16, 227–241.Google Scholar
Johnson, J. E., Argos, P. & Rossmann, M. G. (1975). Rotation function studies of southern bean mosaic virus at 22 Å resolution. Acta Cryst. B31, 2577–2583.Google Scholar
Lattman, E. E. (1972). Optimal sampling of the rotation function. Acta Cryst. B28, 1065–1068.Google Scholar
Litvin, D. B. (1975). The molecular replacement method. I. The rotation function problem, application to bovine liver catalase and STNV. Acta Cryst. A31, 407–416.Google Scholar
Rayment, I., Johnson, J. E., Suck, D., Akimoto, T. & Rossmann, M. G. (1978). An 11 Å resolution electron density map of southern bean mosaic virus. Acta Cryst. B34, 567–578.Google Scholar
Rossmann, M. G., Ford, G. C., Watson, H. C. & Banaszak, L. J. (1972). Molecular symmetry of glyceraldehyde-3-phosphate dehydrogenase. J. Mol. Biol. 64, 237–249.Google Scholar
Åkervall, K., Strandberg, B., Rossmann, M. G., Bengtsson, U., Fridborg, K., Johannisen, H., Kannan, K. K., Lövgren, S., Petef, G., Öberg, B., Eaker, D., Hjertén, S., Rydén, L. & Moring, I. (1972). X-ray diffraction studies of the structure of satellite tobacco necrosis virus. Cold Spring Harbor Symp. Quant. Biol. 36, 469–488.Google Scholar