International
Tables for Crystallography Volume B Reciprocal space Edited by U. Shmueli © International Union of Crystallography 2006 
International Tables for Crystallography (2006). Vol. B. ch. 2.3, pp. 250252
Section 2.3.6.1. Introduction ^{a}Department of Biological Sciences, Purdue University, West Lafayette, Indiana 47907, USA, and ^{b}CABM & Rutgers University, 679 Hoes Lane, Piscataway, New Jersey 088545638, USA 
The rotation function is designed to detect noncrystallographic rotational symmetry (see Table 2.3.6.1). The normal rotation function definition is given as (Rossmann & Blow, 1962) where and are two Pattersons and U is an envelope centred at the superimposed origins. This convolution therefore measures the degree of similarity, or `overlap', between the two Pattersons when has been rotated relative to by an amount defined by The elements of [C] will depend on three rotation angles . Thus, R is a function of these three angles. Alternatively, the matrix [C] could be used to express mirror symmetry, permitting searches for noncrystallographic mirror or glide planes.

The basic concepts were first clearly stated by Rossmann & Blow (1962), although intuitive uses of the rotation function had been considered earlier. Hoppe (1957b) had also hinted at a convolution of the type given by (2.3.6.1) to find the orientation of known molecular fragments and these ideas were implemented by Huber (1965).
Consider a structure of two identical units which are in different orientations. The Patterson function of such a structure consists of three parts. There will be the selfPatterson vectors of one unit, being the set of interatomic vectors which can be formed within that unit, with appropriate weights. The set of selfPatterson vectors of the other unit will be identical, but they will be rotated away from the first due to the different orientation. Finally, there will be the crossPatterson vectors, or set of interatomic vectors which can be formed from one unit to another. The selfPatterson vectors of the two units will all lie in a volume centred at the origin and limited by the overall dimensions of the units. Some or all of the crossPatterson vectors will lie outside this volume. Suppose the Patterson function is now superposed on a rotated version of itself. There will be no particular agreement except when one set of selfPatterson vectors of one unit has the same orientation as the selfPatterson vectors from the other unit. In this position, we would expect a maximum of agreement or `overlap' between the two. Similarly, the superposition of the molecular selfPatterson derived from different crystal forms can provide the relative orientation of the two crystals when the molecules are aligned.
While it would be possible to evaluate R by interpolating in and forming the pointbypoint product with within the volume U for every combination of and , such a process is tedious and requires large computer storage for the Pattersons. Instead, the process is usually performed in reciprocal space where the number of independent structure amplitudes which form the Pattersons is about onethirtieth of the number of Patterson grid points. Thus, the computation of a rotation function is carried out directly on the structure amplitudes, while the overlap definition (2.3.6.1) simply serves as a physical basis for the technique.
The derivation of the reciprocalspace expression depends on the expansion of each Patterson either as a Fourier summation, the conventional approach of Rossmann & Blow (1962), or as a sum of spherical harmonics in Crowther's (1972) analysis. The conventional and mathematically easier treatment is discussed presently, but the reader is referred also to Section 2.3.6.5 for Crowther's elegant approach. The latter leads to a rapid technique for performing the computations, about one hundred times faster than conventional methods.
Let, omitting constant coefficients, and From (2.3.6.2) it follows that and, hence, by substitution in (2.3.6.1) where When the volume U is a sphere, has the analytical form where and . G is a spherical interference function whose form is shown in Fig. 2.3.6.1

Shape of the interference function G for a spherical envelope of radius R at a distance H from the reciprocalspace origin. [Reprinted from Rossmann & Blow (1962).] 
The expression (2.3.6.3) represents the rotation function in reciprocal space. If in the argument of , then h′ can be seen as the point in reciprocal space to which p is rotated by [C]. Only for those integral reciprocallattice points which are close to h′ will be of an appreciable size (Fig. 2.3.6.1). Thus, the number of significant terms is greatly reduced in the summation over p for every value of h, making the computation of the rotation function manageable.
The radius of integration R should be approximately equal to or a little smaller than the molecular diameter. If R were roughly equal to the length of a lattice translation, then the separation of reciprocallattice points would be about . Hence, when H is equal to one reciprocallattice separation, , and G is thus quite small. Indeed, all terms with might well be neglected. Thus, in general, the only terms that need be considered are those where is within one lattice point of h. However, in dealing with a small molecular fragment for which R is small compared to the unitcell dimensions, more reciprocallattice points must be included for the summation over p in the rotationfunction expression (2.3.6.3).
In practice, the equation that is or determines p, given a set of Miller indices h. This will give a nonintegral set of Miller indices. The terms included in the inner summation of (2.3.6.3) will be integral values of p around the nonintegral lattice point found by solving (2.3.6.5).
Details of the conventional program were given by Tollin & Rossmann (1966) and follow the principles outlined above. They discussed various strategies as to which crystal should be used to calculate the first (h) and second (p) Patterson. Rossmann & Blow (1962) noted that the factor in expression (2.3.6.3) represents an interpolation of the squared transform of the selfPatterson of the second (p) crystal. Thus, the rotation function is a sum of the products of the two molecular transforms taken over all the h reciprocallattice points. Lattman & Love (1970) therefore computed the molecular transform explicitly and stored it in the computer, sampling it as required by the rotation operation. A discussion on the suitable choice of variables in the computation of rotation functions has been given by Lifchitz (1983).
References
Crowther, R. A. (1972). The fast rotation function. In The molecular replacement method, edited by M. G. Rossmann, pp. 173–178. New York: Gordon & Breach.Google ScholarHoppe, W. (1957b). Die `Faltmolekülmethode' – eine neue Methode zur Bestimmung der Kristallstruktur bei ganz oder teilweise bekannter Molekülstruktur. Acta Cryst. 10, 750–751.Google Scholar
Huber, R. (1965). Die automatisierte Faltmolekülmethode. Acta Cryst. 19, 353–356.Google Scholar
Lattman, E. E. & Love, W. E. (1970). A rotational search procedure for detecting a known molecule in a crystal. Acta Cryst. B26, 1854–1857.Google Scholar
Lifchitz, A. (1983). On the choice of the model cell and the integration volume in the use of the rotation function. Acta Cryst. A39, 130–139.Google Scholar
Rossmann, M. G. & Blow, D. M. (1962). The detection of subunits within the crystallographic asymmetric unit. Acta Cryst. 15, 24–31.Google Scholar
Tollin, P. & Rossmann, M. G. (1966). A description of various rotation function programs. Acta Cryst. 21, 872–876.Google Scholar