Least squares and maximum likelihood

Ten Eyck, L. F.; Watenpaugh, K. D.

doi:10.1107/97809553602060000693

International
Tables for
Crystallography
Volume F
Crystallography of biological macromolecules
Edited by M. G. Rossman and E. Arnold

pdf | chapter contents | chapter index | related articles

International Tables for Crystallography (2006). Vol. F. ch. 18.1, pp. 369-370 | 1 | 2 |

Section 18.1.4. Least squares and maximum likelihood

L. F. Ten Eyck^a ^* and K. D. Watenpaugh^b

^a San Diego Supercomputer Center 0505, University of California at San Diego, 9500 Gilman Drive, La Jolla, CA 92093-0505, USA, and ^bStructural, Analytical and Medicinal Chemistry, Pharmacia & Upjohn, Inc., Kalamazoo, MI 49001-0119, USA
Correspondence e-mail: [email protected]

18.1.4. Least squares and maximum likelihood

| top | pdf |

`Improving the agreement' between the observed and calculated data can only be done if one first decides the criteria to be used to measure the agreement. The most commonly used measure is the $[L_{2}]$ norm of the residuals, which is simply the sum of the squares of the differences between the observed and calculated data (IT C Chapter 8.1 ), $[ L_{2} ({\bf x}) = \left\| w_{i} [y_{i} - f_{i} ({\bf x})] \right\| = \textstyle\sum\limits_{i} w_{i} [y_{i} - f_{i} ({\bf x})]^{2}, \eqno(18.1.4.1)]$ where $[w_{i}]$ is the weight of observation $[y_{i}]$ and $[f_{i}({\bf x})]$ is the calculated value of observation i given the parameters x. In essence, least-squares refinement poses the problem as `Given these data, what are the parameters of the model that give the minimum variance of the observations?'. The $[L_{2}]$ norm is strongly affected by the largest deviations, which is not a desirable property in the early stages of refinement where the model may be seriously incomplete. In the early stages, it may be better to refine against the $[L_{1}]$ norm, $[ L_{1} = \textstyle\sum\limits_{i} w_{i} \left| [y_{i} - f_{i} ({\bf x})] \right|,]$ the sum of the absolute value of the residuals. At present, this technique is not used in macromolecular crystallography.

The observable quantity in crystallography is the diffracted intensity of radiation. Fourier inversion of the model gives us a complex structure factor. The phase information is normally lost in the formulation of $[f_i({\bf x})]$ . This is a root cause of some of the problems of least-squares refinement from poor starting models. Many of the problems of least-squares refinement can be addressed by changing the measure of agreement from least squares to maximum likelihood, which evaluates to the likelihood of the observations given the model. In this formulation, the problem is posed as `Given this model, what is the probability that the given set of data would be observed?'. The model is adjusted to maximize the probability of the given observations. This procedure is subtly different from least squares in that it is reasonably straightforward to account for incomplete models and errors in the model in computing the probability of the observations. Maximum-likelihood refinement is particularly useful for incomplete models because it produces residuals that are less biased by the current model than those produced by least squares. Maximum likelihood also provides a rigorous formulation for all forms of error in both the model and the observations, and allows incorporation of additional forms of prior knowledge (like additional phase information) into the probability distributions.

The likelihood of a model given a set of observations is the product of the probabilities of all of the observations given the model. If $[P_{a} ({\bf F}_{i}\hbox{;}\ {\bf F}_{i,\, c})]$ is the conditional probability distribution of the structure factor $[{\bf F}_{i}]$ given the model structure factor $[{\bf F}_{i,\, c}]$ , then the likelihood of the model is $[ L = \textstyle\prod\limits_{i} P_{a} ({\bf F}_{i}\semi\ {\bf F}_{i,\, c}).]$ This is usually transformed into a more tractable form by taking the logarithm, $[ \log L = \textstyle\sum\limits_{i} \log P_{a} ({\bf F}_{i}\semi\ {\bf F}_{i,\, c}).]$ Since the logarithm increases monotonically with its argument, the two versions of the equation have maxima at the same values of the parameters of the model. This formulation is described in more detail in Chapter 18.2 , in IT C Section 8.2.1 and by Bricogne (1997), Pannu & Read (1996), and Murshudov et al. (1997).

References

Bricogne, G. (1997). Bayesian statistical viewpoint on structure determination: basic concepts and examples. Methods Enzymol. 276, 361–423.Google Scholar

Murshudov, G. N., Vagin, A. A. & Dodson, E. J. (1997). Refinement of macromolecular structures by the maximum-likelihood method. Acta Cryst. D53, 240–253.Google Scholar

Pannu, N. S. & Read, R. J. (1996). Improved structure refinement through maximum likelihood. Acta Cryst. A52, 659–668.Google Scholar

International Tables for Crystallography (2006). Vol. F. ch. 18.1, pp. 369-370