The effect of instrument or detector errors

Leslie, A. G. W.

doi:10.1107/97809553602060000675

International
Tables for
Crystallography
Volume F
Crystallography of biological macromolecules
Edited by M. G. Rossmann and E. Arnold

pdf | chapter contents | chapter index | related articles

International Tables for Crystallography (2006). Vol. F. ch. 11.2, p. 214 | 1 | 2 |

Section 11.2.5.3. The effect of instrument or detector errors

A. G. W. Leslie^a ^*

^aMRC Laboratory of Molecular Biology, Hills Road, Cambridge CB2 2QH, England
Correspondence e-mail: andrew@mrc-lmb.cam.ac.uk

11.2.5.3. The effect of instrument or detector errors

| top | pdf |

Standard-deviation estimates calculated using (11.2.5.11) are generally in quite good agreement with observed differences between the intensities of symmetry-related reflections for weak or medium intensities. This is particularly true if other sources of systematic error are minimized by measuring the same reflections five or more times, by doing multiple exposures of the same small oscillation range and then processing the data in space group P1. However, even in this latter case, the agreement between strong intensities is significantly worse than that predicted using equation (11.2.5.11). This is consistent with the observation that it is very unusual to obtain merging R factors lower than 0.01, even for very strong reflections where Poisson statistics would suggest merging R factors should be in the range 0.002–0.003.

An experiment in which a diffraction spot recorded on photographic film was scanned many times on an optical microdensitometer showed that the r.m.s. variation in individual pixel values between the scans was greatest for those pixels immediately surrounding the centre of the spot, where the gradient of the optical density was greatest. One explanation for this observation is that these optical densities will be most sensitive to small errors in positioning the reading head, due to vibration or mechanical defects. A simple model for the instrumental contribution to the standard deviation of the spot intensity is obtained by introducing an additional term for each pixel in the spot peak: $[\sigma_{\rm ins} = K {\delta \rho \over \delta x}, \eqno(11.2.5.12)]$ where $[\delta \rho /\delta x]$ is the average gradient and K is a proportionality constant. Taking a triangular reflection profile, the gradient and integrated intensity are related by $[I_{s} = {1 \over {12}}\left(x^{3} + 3x^{2} + 5x + 3 \right) {\delta \rho \over {\delta x}}, \eqno(11.2.5.13)]$ where x is the half-width of the reflection (in pixels).

Writing $[A = {1 \over {12}} \left(x^{3} + 3x^{2} + 5x + 3 \right) \eqno(11.2.5.14)]$ gives $[\sigma_{\rm ins} = (K/A)I_{s}, \eqno(11.2.5.15)]$ where the factor A allows for differences in spot size and K is, ideally, a constant for a given instrument.

The total variance in the integrated intensity is then $[\eqalignno{\sigma_{\rm tot}^{2} &= \sigma_{I_{s}}^{2} + m\sigma_{\rm ins}^{2} &(11.2.5.16)\cr &= G \left[I_{s} + I_{\rm bg} + (m/n) I_{{\rm bg}} \right] + m\left(K/A\right)^{2} I_{s}^{2}. &(11.2.5.17)}]$ A value for K can be determined by comparing the goodness-of-fit of the standard profiles to individual reflection profiles (of fully recorded reflections) with that calculated from combined Poisson statistics and the instrument error term. Standard deviations estimated using (11.2.5.17) give much more realistic estimates than those based on (11.2.5.11), even for data collected with charge-coupled-device (CCD) detectors where the physical model for the source of the error is clearly not appropriate.

References

International Tables for Crystallography (2006). Vol. F. ch. 11.2, p. 214