International
Tables for
Crystallography
Volume F
Crystallography of biological macromolecules
Edited by M. G. Rossmann and E. Arnold

International Tables for Crystallography (2006). Vol. F. ch. 11.2, pp. 215-216   | 1 | 2 |

Section 11.2.6.3. Modifications for very close spots

A. G. W. Lesliea*

aMRC Laboratory of Molecular Biology, Hills Road, Cambridge CB2 2QH, England
Correspondence e-mail: andrew@mrc-lmb.cam.ac.uk

11.2.6.3. Modifications for very close spots

| top | pdf |

In order to apply equation (11.2.6.5[link]), it is necessary to exclude all pixels in the measurement box that are overlapped by a neighbouring spot. This applies not only to the pixels of the reflection being integrated, but also to the pixels of all the reflections used to form the standard profile. Consequently, a pixel should be excluded even if it is only overlapped by a neighbouring spot for one of the reflections used in forming the standard profile. When processing data from large unit cells, this can lead to a very high percentage of the background pixels being rejected and therefore a poor determination of the background plane parameters. In these circumstances, the background plane is determined using only background pixels and excluding only those pixels that are overlapped by neighbours for the reflection actually being integrated. The profile-fitted intensity for both fully recorded and partially recorded reflections is then evaluated in the way described for partially recorded reflections in Section 11.2.6.2[link], with the summation in equation (11.2.6.15[link]) extending only over peak pixels. The standard deviation in the intensity for partially recorded reflections is derived from equation (11.2.5.17[link]) as before. For fully recorded reflections, the standard deviation has two components: the first is based on the fit of the scaled standard profile to the reflection profile and the second on the contribution from the background: [\eqalignno{\sigma_{I}^{2} &= \sigma_{\rm prof}^{2} + \sigma _{\rm bg}^{2} &(11.2.6.16)\cr &= \left[{\textstyle\sum\limits_{i = 1}^{m}} w_{i} \Delta_{i}^{2} \Big/ (m - 1)\right] \left[\left({\textstyle\sum\limits_{i = 1}^{m}} P_{i}\right)^{2} \bigg/ {\textstyle\sum\limits_{i = 1}^{m}} w_{i} P_{i}^{2}\right]\cr &\quad + (m/n) {\textstyle\sum\limits_{i = 1}^{n}} \left(\rho_{i} - ap_{i} - bq_{i} - c\right)^{2}, &(11.2.6.17)}] where m and n are the number of pixels in the peak and background, respectively.








































to end of page
to top of page