International
Tables for
Crystallography
Volume F
Crystallography of biological macromolecules
Edited by M. G. Rossmann and E. Arnold

© International Union of Crystallography 2006
International Tables for Crystallography (2006). Vol. F. ch. 11.2, p. 216   | 1 | 2 |

Section 11.2.6.6. Improvement provided by profile fitting weak reflections

A. G. W. Lesliea*

aMRC Laboratory of Molecular Biology, Hills Road, Cambridge CB2 2QH, England
Correspondence e-mail: andrew@mrc-lmb.cam.ac.uk

11.2.6.6. Improvement provided by profile fitting weak reflections

| top | pdf |

For very weak reflections, where all the weights [w_{i}] are approximately the same, the variance in [I_{p}] using equation (11.2.6.21[link]) is given by [\sigma_{I_{p}}^{2} = {\textstyle\sum} \hbox{Var}\left(\rho_{i} - ap_{i} - bq_{i} - c\right) P_{i}^{2} \left({\textstyle\sum} P_{i} \big/ {\textstyle\sum} P_{i}^{2}\right)^{2}. \eqno(11.2.6.22)] Assuming a flat background and very weak intensity, then from Poisson statistics [\hbox{Var}\left(\rho_{i} - ap_{i} - bq_{i} - c\right) \simeq G\rho_{i}, \eqno(11.2.6.23)] and as [\rho_{i}] has approximately the same value [(\rho)] for all pixels, [\eqalignno{\sigma_{I_{p}}^{2} &= G\rho {\textstyle\sum} P_{i}^{2} \left({\textstyle\sum} P_{i} \big/ {\textstyle\sum} P_{i}^{2}\right)^{2} &(11.2.6.24)\cr &= G\rho \left({\textstyle\sum} P_{i}\right)^{2} \big/ {\textstyle\sum} P_{i}^{2}. &(11.2.6.25)\cr}] The variance in the summation integration intensity is simply [\sigma_{I_{s}}^{2} = Gm\rho. \eqno(11.2.6.26)] The ratio of the variances is thus [\sigma_{I_{s}}^{2} \big/ \sigma_{I_{p}}^{2} = m{\textstyle\sum} P_{i}^{2} \big/ \left({\textstyle\sum} P_{i}\right)^{2}. \eqno(11.2.6.27)] For a typical spot profile, the right-hand side (which depends only on the shape of the standard profile) has a value of 2, showing that profile fitting can reduce the standard deviation in the integrated intensity by a factor of [(2)^{1/2}].








































to end of page
to top of page