Tables for
Volume F
Crystallography of biological macromolecules
Edited by M. G. Rossmann and E. Arnold

International Tables for Crystallography (2006). Vol. F, ch. 13.2, pp. 270-272   | 1 | 2 |

Section 13.2.3. The rotation function

J. Navazaa*

aLaboratoire de Génétique des Virus, CNRS-GIF, 1. Avenue de la Terrasse, 91198 Gif-sur-Yvette, France
Correspondence e-mail:

13.2.3. The rotation function

| top | pdf |

The absolute and relative orientations of the subunits that constitute a crystal may in principle be found by exploiting some properties of the Patterson function. We first express the crystal structure factors [F({\bf h})] in terms of the Fourier transform of the electron density of isolated molecules, [f_{m}({\bf s})], calculated with their centres of gravity placed at the origin. If [{\bf r}_{m}] denotes the position within the crystal of the centre of gravity of the mth molecule, we have [F({\bf h}) = {\textstyle\sum\limits_{m = 1}^{M}}\ f_{m}({\bf h}) \exp(2 \pi i{\bf hr}_{m}), \eqno(] where M is the number of molecules within the unit cell. This is a general expression for [F({\bf h})], although the space-group and noncrystallographic rotational symmetries are not explicitly exhibited. The presence of symmetry implies that some of the [f_{m}({\bf h})]'s are in fact samples of the same function at rotated arguments. The Fourier coefficients of the Patterson function can then be written as [\eqalignno{I({\bf h}) &= |F({\bf h})|^{2} = {\textstyle\sum\limits_{m=1}^{M}} |f_{m}({\bf h})|^{2}\cr &\quad + {\textstyle\sum\limits_{m \neq m'=1}^{M}} \overline{f_{m}({\bf h})}\ f_{m'}({\bf h})\exp[2\pi i{\bf h}({\bf r}_{m}-{\bf r}_{m'})], &(}] where [\overline{f_{m}}] is the complex conjugate of [f_{m}]. The first term in equation ([link] involves only intramolecular contributions or self-Patterson terms; they are centred at the origin of the Patterson function. The second term involves intermolecular contributions or cross-Patterson terms; they are centred at the intermolecular vectors [{\bf r}_{m} - {\bf r}_{m'}]. Therefore, if we restrict the Patterson function to a region Ω, of volume v, centred at the origin and having a dimension of the order of the dimensions of the isolated molecules, self-Patterson terms will dominate the crossed ones.

Thus it becomes possible to determine the rotation R that superimposes one subunit upon an independent, homologous one by calculating the overlap within the region Ω of the observed Patterson function (the target function [P_{t}]) and a rotated version of either itself or the Patterson function of an isolated known molecule (the search function [P_{s}]): [{\cal R}({\bf R}) = (1/v) \textstyle\int\limits_{\Omega} P_{t}({\bf r}) P_{s}({\bf R}^{-1}{\bf r})\ \hbox{d}^{3} {\bf r} \eqno(] (Rossmann & Blow, 1962[link]). [{\cal R}] should display a local maximum for the sought rotation. Note that when we rotate the search function [P_{s}] by R, its argument contains [{\bf R}^{-1}]. When the search and the target functions are the same, [{\cal R}] is called the self-rotation function; otherwise, it is called the cross-rotation function. When a model is available, the target data are calculated by placing the model within a P1 cell whose dimensions guarantee that Ω only contains self-Patterson contributions.

The reciprocal-space formulation of the above integral is obtained by substituting the Patterson functions by their Fourier summations: [P({\bf r}) = {\textstyle\sum\limits_{\bf h}}\ [I({\bf h})/V]\exp(-2 \pi i{\bf hr}). \eqno(] Taking into account that [I({\bf -h}) = I({\bf h})], we obtain [\eqalignno{{\cal R}({\bf R}) &= \sum\limits_{\bf h} \sum\limits_{\bf k} {I_{t}({\bf h}) \over V_{t}} {I_{s}({\bf k}) \over V_{s}} {1 \over v} \int\limits_{\Omega}\ \exp[2\pi i({\bf h - kR}^{-1}){\bf r}]\ \hbox{d}^{3}{\bf r}\cr &= \sum\limits_{\bf h} \sum\limits_{\bf k} {I_{t}({\bf h}) \over V_{t}} {I_{s}({\bf k}) \over V_{s}}\ \chi_{\Omega}({\bf h - kR}^{-1}). &(}] [\chi_{\Omega}], the interference function, is the Fourier transform of the characteristic function of Ω, i.e., a function that takes the value 1 within Ω and 0 outside. In principle, the domain of integration could have any shape. However, in order to take full advantage of the properties of the rotation group, Ω is usually chosen as a spherical domain of radius b. Calling [{\bf h - kR}^{-1} = {\bf s}] for short, we have [\eqalignno{\chi_{b}({\bf s}) &= (3/4\pi b^{3}) {\textstyle\int\limits_{0}^{b}} {\textstyle\int\limits_{0}^{\pi}} {\textstyle\int\limits_{0}^{2\pi}}\exp(2\pi i{\bf s r}) r^{2} \sin(\theta)\ \hbox{d}r\ \hbox{d}\theta\ \hbox{d}\varphi\cr &= 3 [\sin(2\pi sb) - 2\pi sb \cos(2 \pi sb)]\big/(2 \pi sb)^{3}. &(}] In the case of a spherical shell with inner and outer radii a and b, respectively, the interference function is obtained by subtraction: [[\chi_{b}({\bf s}) - (a/b)^{3}\chi_{a}({\bf s})]\big/[1-(a/b)^{3}]. \eqno(]

Although simple, the resulting expression for the rotation function has the disadvantage of containing entangled h, k and R contributions, which renders its computation time-consuming if the whole domain of rotations has to be explored. The difficulty may be overcome by expanding the exponentials entering equation ([link] in spherical harmonics, [Y_{\ell, \, m}]. Taking advantage of their transformation under rotations, and using recurrence relationships between spherical Bessel functions, [j_{\ell}], we obtain (see Appendix 13.2.1[link]) [\eqalignno{\chi_{b}({\bf h - kR}^{-1}) &= \sum\limits_{\ell = 0}^{\infty} \sum\limits_{m, \, m' = -\ell}^{\ell}\ \overline{Y_{\ell, \, m}({\hat {\bf h}})}\ Y_{\ell, \, m'}({\hat {\bf k}})\cr &\quad \times \left\{\sum\limits_{n = 1}^{\infty} 12\pi [2(\ell + 2n) - 1]\ {{j_{\ell + 2n - 1}(2\pi hb) \over 2\pi hb}\ {j_{\ell + 2n - 1}(2\pi kb) \over 2\pi kb}} \right\}\cr &\quad \times D^{\ell}_{m, \, m'}({\bf R}), &(}] where [D^{\ell}_{m, \, m'}] are the matrices of the irreducible representations of the rotation group and [{\hat {\bf h}}] stands for `angular part of vector h'. The awkwardness of equation ([link] is apparent rather than real. Indeed:

  • (1) The expression separates angular from crystal variables. It also separates target from search contributions (see Section[link]).

  • (2) The equation is accurate, even when truncating the summations on ℓ and n to reasonable values (Navaza, 1993[link]). The upper limit for ℓ is of the order of the highest argument of the spherical Bessel functions, [\ell_{\max} \simeq 2\pi b/d_{\min},] where [d_{\min}] is the resolution of the data. The upper limit for n also depends on the current value of ℓ, [n_{\max}(\ell) \simeq (\ell_{\rm max} - \ell + 2)/2.]

  • (3) When the rotations are parameterized in Euler angles (α, β, γ), the matrices [D^{\ell}_{m, \, m'}] take the form [D^{\ell}_{m, \, m'}(\alpha,\beta,\gamma) = d^{\ell}_{m, \, m'}(\beta)\exp[i(m\alpha + m'\gamma)], \eqno(] which enables the computation of [{\cal R}], for each given value of β, by means of two-dimensional fast Fourier transforms (FFTs).

This formulation is referred to as the fast rotation function (Crowther, 1972[link]).

It may be useful to compare rotation functions obtained under different conditions. For this, some kind of normalization is needed. One possibility is to cast [{\cal R}] into the form of a correlation coefficient by dividing equation ([link] by the norms of the truncated Patterson functions, [{\cal R}_{N}({\bf R}) = {\textstyle\int\limits_{\Omega}} P_{t}({\bf r}) P_{s}({\bf R}^{-1}{\bf r})\ \hbox{d}^{3} {\bf r} \bigg/\left[\ {\textstyle\int\limits_{\Omega}} P_{t}({\bf r})^{2}\ \hbox{d}^{3} {\bf r}\ {\textstyle\int\limits_{\Omega}} P_{s}({\bf r})^{2}\ \hbox{d}^{3} {\bf r}\right]^{1/2}. \eqno(] Computing the rotation function

| top | pdf |

A direct evaluation of equation ([link] is possible. However, since the values of the Patterson functions are only available at discrete sampling points, an interpolation is needed after each rotation. This method is known as the direct-space formulation of the rotation function (Nordman, 1966[link]; Steigemann, 1974[link]).

In the reciprocal-space formulation [equations ([link] and ([link]], some precautions are to be taken in order to save computing time:

  • (1) Use only strong reflections in the summations.

  • (2) For each [{\bf kR}^{-1}], limit the summation on h to vectors satisfying the condition [|{\bf h - kR}^{-1}| \leq 1/2\pi b].

In both formulations, it is worth using a distance between rotations to sample [{\cal R}] efficiently, as explained in Section[link].

For the fast rotation function [equations ([link] and ([link]], the calculations are organized as follows (Dodson, 1985[link]; Navaza, 1993[link]):

  • (1) Given the search and the target diffraction data, compute [\eqalignno{e_{\ell, \, m, \, n} =& \left\{12\pi [2(\ell + 2n) - 1]\right\}^{1/2}\cr &\times{\textstyle\sum\limits_{\bf h}} [I({\bf h})/V] Y_{\ell, \, m}({\hat {\bf h}}) [\;j_{\ell+2n-1}(2\pi hb)]/2\pi hb&(\cr}] and normalize [to compute equation ([link]]: [e_{\ell, \, m, \, n} \rightarrow e_{\ell, \, m, \, n} \bigg/\left({\textstyle\sum\limits_{\ell = 2}^{\ell_{\max}}}\ {\textstyle\sum\limits_{m = -\ell}^{\ell}}\ {\textstyle\sum\limits_{n = 1}^{n_{\max}(\ell)}} |e_{\ell, \, m, \, n}|^{2}\right)^{1/2}. \eqno(] Odd ℓ terms disappear, because the Friedel-related reflections contribute with opposite signs: [I({\bf -h}) Y_{\ell, \, m} (- \hat{{\bf h}}) = (-1)^{\ell} I({\bf h}) Y_{\ell, \, m}({\hat {\bf h}}).] Also, if the Patterson function has an n-fold rotation axis along z, only the terms with m equal to a multiple of n survive.

  • (2) Given the [e_{\ell, \, m, \, n}]'s, perform the sums [C^{\ell}_{m, \, m'} = {\textstyle\sum\limits_{n=1}^{n_{\max}(\ell)}} \overline{e_{\ell, \, m, \, n}^{(t)}} \ e_{\ell, \, m', \, n}^{(s)}. \eqno(]

  • (3) For each β value, calculate the reduced matrix elements and compute [S_{m, \, m'}(\beta) = {\textstyle\sum\limits_{\ell=2}^{\ell_{\max}}} C^{\ell}_{m, \, m'} d^{\ell}_{m, \, m'}(\beta). \eqno(] Then evaluate the β section of [{\cal R}] by FFT: [{\cal R}(\alpha,\beta,\gamma) = {\textstyle\sum\limits_{m, \, m'=-\ell_{\max}}^{\ell_{\max}}} S_{m, \, m'}(\beta) \exp [i(m\alpha+m'\gamma)]. \eqno(] The sampling in α and γ here is dictated by the standard FFT requirements. Plotting and sampling the rotation function

| top | pdf |

An odd feature of the rotation domain is that quite different values of the parameters may correspond to very close rotations. This causes the graphic representation of the rotation function to be distorted. For example, when the Euler angle β is small or close to 180, a peak plotted on a Cartesian grid in α and γ is strongly elongated in one direction. By redefining the reference orientation, the same peak near the [\beta = 90] section will have a quite different – natural – shape.

This problem may be partially overcome by using the distance between rotations introduced in Section[link]. Although the metric cannot be reduced to a Cartesian one in three dimensions, it is possible to do so for two-dimensional sections. Indeed, for fixed β, the transformation [\eqalignno{\omega_{+} &= \cos(\beta/2)(\alpha + \gamma)\cr \omega_{-} &= \sin(\beta/2)(\alpha - \gamma) &(}] reduces equation ([link] to [\hbox{d}s^{2} = \hbox{d}\omega_{+}^{2} + \hbox{d}\omega_{-}^{2}]. Therefore, distortion-free β sections may be plotted in terms of the variables [\omega_{\pm}] (Burdina, 1971[link]; Lattman, 1972[link]). Accordingly, a correct sampling of [{\cal R}] should consist of points regularly spaced in these variables. The number of points with respect to that of a regular spacing in α, β and γ is approximately [{\textstyle\int} \sin({\beta})\ \hbox{d}\alpha\ \hbox{d}\beta\ \hbox{d}\gamma\big/{\textstyle\int}\ \hbox{d}{\alpha}\ \hbox{d}{\beta}\ \hbox{d}{\gamma} = 2/\pi \simeq 0.64.] Notice that for β equal to 0 or 180, the sections reduce to lines.

In the case of the self-rotation function, when the order of the NCS may be anticipated, it is worth working with equation ([link] in the polar parameterization, [\hbox{d}s^{2} = \hbox{d}\chi^{2} + [2 \sin(\chi /2)]^{2} \hbox{d}{\bf u}^{2}. \eqno(] For constant χ, we have the topology of the spherical surface. It is then natural to plot the function as a stereographic projection.

In the case of the cross-rotation function, i.e., when the atomic distribution of the search model is known, we may use equation ([link] instead of equation ([link] to define the sampling. For this, we choose σ as the smallest root-mean-square shift in atomic coordinates that produces an appreciable change in the value of the cross-rotation function. An acceptable sampling set of [{\cal R}] should then satisfy the following conditions:

  • (1) For every point in the rotation domain, there is at least one sampling point at a distance less than σ.

  • (2) The number of sampling points is as small as possible. Strategies

| top | pdf |

Since the first attempts to detect subunits within the crystal asymmetric unit, a great deal of experience has been gained and the results collected in several treatises on molecular replacement (Rossmann, 1972[link]; Machin, 1985[link]; Dodson et al., 1992[link]; Carter & Sweet, 1997[link]).

The success of the rotation function relies in part on the choice of the domain of integration, i.e., inner and outer radii of the spherical domain, and limits on data resolution. The outer radius is chosen so as to maximize the ratio between the number of intramolecular and intermolecular vectors. A typical value of this radius is 50–75% of the subunit's diameter for spherical molecules. The choice of the inner radius is less crucial, provided that the origin peaks are subtracted from the Patterson functions; it is often set to zero.

The resolution range of the selected diffraction data depends on whether we are computing self- or cross-rotation functions. For self rotations, data up to the highest available resolution may be used. For cross rotations, the high-resolution limit is dictated by the degree of similarity between the probe and the actual molecules that constitute the crystal. The low-resolution limit is usually chosen so as to skip the solvent contribution.

Another parameter of interest concerns the angular resolution, which is rather specific to the fast rotation function. Although it was derived by expanding the interference function to follow the original reciprocal-space formulation as closely as possible, it may also be obtained by expanding the Patterson functions in spherical harmonics and then performing the integration (Crowther, 1972[link]). In this way, the concept of angular resolution is more evident. In fact, the spherical harmonics play the same role in the angular domain as the imaginary exponentials do in Cartesian coordinate space. The analogue of the Miller indexes here is the label ℓ, directly related to the angular resolution. The term with [\ell = 0] is invariant under rotations and is thus eliminated from the summation in equation ([link]. It represents the contribution of the best radial functions, in a least-squares sense, that approximate the Patterson functions within Ω. The origin peak is thus properly removed by omitting the [\ell = 0] term, as well as a substantial component of the original function. Also, by omitting low ℓ's, the angular resolution of peaks is enhanced. These eliminations may also be done in other formulations, but never as efficiently.

The parameters discussed above are not all independent. The relationship between them stems from the behaviour of the spherical Bessel functions for small values of their argument (Watson, 1958[link]), [j_{\ell}(2\pi hb) \simeq (2\pi hb)^{\ell}/(2\ell + 1)!!] [see equation ([link]]. So, when omitting low ℓ terms, we are also weighting down low-resolution data. Symmetry properties of the rotation function

| top | pdf |

The overlap integral that defines [{\cal R}] may be calculated by rotating [P_{t}] instead of [P_{s}], but with the inverse rotation, [\eqalignno{{\cal R} ({\bf R}) &= (1/v) {\textstyle\int\limits_{\Omega}} P_{t}({\bf r}) P_{s}({\bf R}^{-1}{\bf r})\ \hbox{d}^{3} {\bf r}\cr &= (1/v) {\textstyle\int\limits_{\Omega}} P_{t}({\bf Rr}) P_{s}({\bf r})\ \hbox{d}^{3} {\bf r}.&(}] This property enables the analysis of the consequence of the symmetries of the Patterson functions upon the rotation function (Tollin et al., 1966[link]; Moss, 1985[link]). For example, when the target and search functions are the same, a trivial symmetry of the self-rotation results: its values at R and [{\bf R}^{-1}] are the same or, in Euler angles, at (α, β, γ) and [(180 - \gamma, \beta, 180 - \alpha)]. More generally, if [P_{t}] is invariant under the rotation T, i.e., [P_{t}({\bf T}^{-1}{\bf r}) = P_{t}({\bf r})] [and similarly for the search function, [P_{s}({\bf S}^{-1}{\bf r}) = P_{s}({\bf r})]], then [\eqalignno{{\cal R}({\bf R}) &= (1/v) {\textstyle\int\limits_{\Omega}} P_{t}({\bf r}) P_{s}({\bf R}^{-1}{\bf r})\ \hbox{d}^{3} {\bf r}\cr &= (1/v) {\textstyle\int\limits_{\Omega}} P_{t}({\bf T}^{-1}{\bf r}) P_{s}({\bf S}^{-1}{\bf R}^{-1}{\bf r}) \hbox{ d}^{3} {\bf r}\cr &= (1/v) {\textstyle\int\limits_{\Omega}} P_{t}({\bf r}) P_{s}({\bf S}^{-1}{\bf R}^{-1}{\bf T} \ {\bf r}) \hbox{ d}^{3} {\bf r} = {\cal R}({\bf T}^{-1}{\bf RS}). \cr &&(}] However, the actual symmetry displayed by [{\cal R}] depends on the parameterization of the rotations and on the orientation of the orthonormal axes with respect to the crystal ones. Within the Euler parameterization, if the target function has an n-fold rotation axis parallel to the orthonormal z axis, then, according to equation ([link], [{\cal R}] will have a periodicity of [360/n] along α. Similarly, a rotation axis of order n along z of [P_{s}] gives rise to a periodicity of [360/n] along γ. Therefore, the amount of calculation is reduced by choosing z along the Patterson functions' highest rotational symmetry axes [see equation ([link]].


Burdina, V. I. (1971). Symmetry of rotation function. Sov. Phys. Crystallogr. 15, 545–550.Google Scholar
Carter, C. W. & Sweet, R. M. (1997). Molecular replacement. Methods Enzymol. 276, 558–619.Google Scholar
Crowther, R. A. (1972). In The molecular replacement method, edited by M. G. Rossmann, pp. 173–178. New York: Gordon and Breach.Google Scholar
Dodson, E. J. (1985). In Proceedings of the Daresbury study weekend. Molecular replacement, edited by P. A. Machin, pp. 33–45. Warrington: Daresbury Laboratory.Google Scholar
Dodson, E. J., Gover, S. & Wolf, W. (1992). Editors. Proceedings of the Daresbury study weekend. Molecular replacement. Warrington: Daresbury Laboratory.Google Scholar
Lattman, E. E. (1972). Optimal sampling of the rotation function. Acta Cryst. B28, 1065–1068.Google Scholar
Machin, P. A. (1985). Editor. Proceedings of the Daresbury study weekend. Molecular replacement. Warrington: Daresbury Laboratory.Google Scholar
Moss, D. S. (1985). The symmetry of the rotation function. Acta Cryst. A41, 470–475.Google Scholar
Navaza, J. (1993). On the computation of the fast rotation function. Acta Cryst. D49, 588–591.Google Scholar
Nordman, C. E. (1966). Vector space search and refinement procedures. Trans. Am. Crystallogr. Assoc. 2, 29–38.Google Scholar
Rossmann, M. G. (1972). Editor. The molecular replacement method. New York: Gordon and Breach.Google Scholar
Rossmann, M. G. & Blow, D. M. (1962). The detection of sub-units within the crystallographic asymmetric unit. Acta Cryst. 15, 24–31.Google Scholar
Steigemann, W. (1974). Die Entwicklung und Anwendung von Rechenverfahren und Regenprogrammen zur Strukturanalyse von Proteinen am beispiel des Trypsin–Trypsininhibitor Komplexes, des Freien und der L-Asparaginase. PhD thesis, Technische Universität, Munich, Germany.Google Scholar
Tollin, P., Main, P. & Rossmann, M. G. (1966). The symmetry of the rotation function. Acta Cryst. 20, 404–417.Google Scholar
Watson, G. N. (1958). A treatise on the theory of Bessel functions, 2nd ed. Cambridge University Press.Google Scholar

to end of page
to top of page