Section 13.1.4. Interpretation of generalized noncrystallographic symmetry where the molecular structure is partially known

The rotation function can be used in the generalized case to compare the Patterson functions of different crystals. When used in this way, it is called the cross-rotation function. In the most usual case, information providing some kind of structural model is available for one of the crystals.

The power of the cross-rotation function may be greatly improved by removing all intermolecular vectors from the `model' Patterson function. This may be done by constructing an imaginary crystal structure in which a single copy of the structural model is placed in a unit cell that is large enough for all intermolecular vectors to be longer than the longest intramolecular vector of the model. In this cell, the self-Patterson vectors may be completely isolated and used for comparison with the Patterson function of a crystal containing a molecule of unknown orientation.

In searching for the position of a molecule in the generalized case of noncrystallographic symmetry, a molecular model defines an origin of coordinates in the model structure, and the corresponding position can be sought in an unknown structure (Nordman & Nakatsu, 1963; Tollin & Cochran, 1964; Huber, 1965; Crowther & Blow, 1967). The procedure is to calculate a three-dimensional function whose peaks should lie at the inter-subunit vectors. In some procedures (Tollin & Cochran, 1964; Crowther & Blow, 1967), this function may be calculated as a Fourier series.

The translation functions will fail if the corresponding rotation is incorrect, or even if it is insufficiently accurate to give a good overlap between the structures. To avoid this danger, Brünger (1997) recommends computing translation functions using rotations corresponding to many (e.g. 200) high values of the rotation function. Though this is a huge increase in computing load, it still compares favourably with a full six-dimensional search.

These methods are considered further in Chapter 13.3 .

Table 13.1.4.1 distinguishes a number of different situations in which noncrystallographic symmetry can be used to aid structure determination. The most frequent application of molecular-replacement methods is to cases where a structure is partially known, but is not yet susceptible to refinement by standard techniques.

Two types of situation arise in the standard case, where noncrystallographically related subunits exist in the same crystal. Most frequently [type (2)], the noncrystallographic symmetry allows the electron density to be improved at the given resolution. Occasionally, high-order noncrystallographic symmetry may be used to extend the resolution to the point where conventional structural refinement becomes possible (Schevitz et al., 1981; McKenna et al., 1992). In the most favourable case, high-order noncrystallographic symmetry constraints may allow direct structure determination [type (1)], starting from the position of a symmetric particle in the asymmetric unit (Jack, 1973).

In the generalized case, most often, similarities with a known molecular structure can be employed to improve an unknown structure [types (5) and (6)]. Such techniques were first used by Tollin (1969) (before structural refinement was possible) and by Fehlhammer & Bode (1975).

It is also possible that a refinable structure could be generated from intensity data observed from several different crystal forms, using noncrystallographic symmetry constraints, but this is not known to have been done in practice [types (3) and (4)].