Chapter 4.7. Absolute measurement of X-ray absorption spectroscopy

The previous chapter (Chantler, 2024a) established the principles of data collection to obtain some measure of variance, standard deviation, standard error and precision to be used in analysis in a point-by-point manner without interpolation. For (X-ray) transmission measurements, it gave the key and the simplest consequences of using upstream and downstream measurement to derive variance from repeated measurements. A key issue raised was how to assess the data, whether in a directly Bayesian manner or by minimizing the variance and maximizing the precision and accuracy by removing or measuring particular systematic sources of variances of the datum. We repeat the comment that we take as given the standard error-propagation and statistical analysis discussed, for example, in undergraduate textbooks on statistical analysis (Bevington, 1969), Numerical Recipes (Press et al., 2007) and standard least-squares fitting code and theory (Ito, 1993; Newville, 2024), and discussion of Bayesian statistics or a priori probabilities (Gregory, 2005; James, 2006; Krappe et al., 2024), reverse Monte Carlo techniques (Timoshenko & Kuzmin, 2024) and wavelet-transform discussions.

Here, we expand upon that chapter, apply the logic to common specific experimental cases and illustrate how to address the problem of data analysis or pre-analysis and which data need to be collected for this purpose.

The previous chapter detailed processing for transmission measurement using the logarithm of the ratio of upstream and downstream (for example ion-chamber) detectors; that is, a monitor and a detector. No detector is perfectly linear, yet ion chambers and other detectors can be linear over six orders of magnitude and to four decimal places or more. However, almost all detectors have a time-dependent `zero signal' or dark current, which will affect and destroy the detector linearity that is required to measure attenuation with accuracy. Some experimental protocols set this offset to zero or less than zero, which invalidates the idea of counting statistics and increases the detector signal nonlinearities, meaning that taking a log ratio will generate a complex and ill-defined error and uncertainty. In general, data collected with negative dark-current signals should be remeasured and data collected without collecting dark-current signals should be remeasured. Hence, for ion-chamber detectors in particular, and in general for all detectors, a recommendation is that the dark current – the signal with the beam off – should always be set to be positive and then measured to provide a linear detection chain. Similarly, for solid-state detectors the dead time of the detector should be measured and corrected for in routine pre-analysis.

Secondly, all beam-path optics involve air or gas and windows or filters which produce absorption and scattering and fluorescence. To first order in transmission, these can be addressed and hence corrected for using (equation 2 in Best & Chantler, 2024), where D is the dark current (reading), I₀ and D₀ are readings in the upstream (monitor) detector I versus the downstream detector D, b indicates measurement of the blank or background signal, including attenuation from the air path, windows etc., and sb indicates measurement of the system with the sample of interest included. This improvement to the simple Beer–Lambert nominal equation demands that the datum become sevenfold, i.e. a data set of {E, I_0,sb, I_sb, I_0,b, I_b, D, D₀}, rather than the datum of the previous chapter. Hence, in the most simple experiment, an estimate of the uncertainty or variance of six quantities are required as f(E), and the propagation of uncertainty from raw measurements to an uncertainty in μt or [μ/ρ][ρt] requires the evaluation of a 6 × 6 correlation matrix for each energy. Under normal (common) circumstances uncertainties in D are small, with the additional common assumption that the sample measurement and the blank measurement were made at different times and under different experimental conditions, and thus are independent measurements, and the resulting variance arises from four independent variances as given by(equation 4 in Best & Chantler, 2024). Whilst not complicated, this procedure requires the estimation or determination of four (independent) energy-dependent variances, including a clear strategy for measurement of the blank. We have found that the time-dependence of D and D₀ can be significant in error propagation. This activity was called for in the original Q2XAFS conclusion summary (Ascone et al., 2012). Abe et al. (2018) discussed the need to correct (or compensate) for dead time and dark current to enable the detectors to be linear (over much wider ranges, for solid-state pixel detectors or for ion chambers etc.). In the absence of any such measurements, one can postulate a Bayesian approach of inference following the previous chapter, although estimating the unknowns may be hazardous. The consequence for a given fitting parameter is not always clear, but in general the variance will increase if any of these variates change through the measurement, and in this case the accuracy will decline. A simple analysis of this can be made from recent advanced data sets. See Chantler (2024b) for a series of sets of data and the corresponding publications analysing the uncertainties (Fig. 1).

Figure 1 The most significant systematics to address in transmission experiments for accurate XAS analysis.

For this Case 1, we present two illustrations. One is a data set collected from an oriented zinc selenide crystal at the second-generation Australian National Beamline, Tsukuba, Japan (Sier et al., 2020) and the other is a data set collected from zinc metal foil at the third-generation XAS beamline of the Australian Synchrotron (Sier et al., 2022; Ekanayake et al., 2021a,b).

It is often considered that dark-current, blank and related measurements are unnecessary, and perhaps for a range of qualitative comparisons this may be true. However, Sier et al. (2020) (the first data set) characterized the error of failing to measure and characterize dark current as up to 3.3% for a thin sample (25 µm) and 40.7% for a thicker sample (100 µm), noting that these errors are attenuation- and hence energy-dependent and thus are a complex functionality. Similarly, Ekanayake et al. (2021a) (the second data set) found errors due to failing to correct for a dark current of 57% for thicker foils but still of 1.3% for thin foils (10 µm). See, in particular, Figs. 6, 7, 10 and 11 in Ekanayake et al. (2021a) for the structural and energy-dependent effects. These percentages are systematic and energy-dependent errors in μ, [μ/ρ] and [μ/ρ][ρt]. This is often seen as a damping of the near-edge and oscillatory structure around the XANES and XAFS regions. Hence, it may correlate with errors in and ΔE and may also limit the effective working range of a Hanning window. Therefore, it affects the accuracy of nanostructural investigations.

Similarly, Sier et al. (2020) (the first data set) characterized the error of failing to measure and characterize the blank spectrum as 7.6–25% for a thick sample (100 µm) and up to 51% for a thin sample (25 µm), while Ekanayake et al. (2021a) (the second data set) found errors due to failing to correct for the blank signal of up to 97% for thicker foils (50 µm) but, perhaps surprisingly, of 22–536% for thin foils (10 µm). A traditional rubric states to use `thin' foils to eliminate transmission error and uncertainty, but the effects and errors of failing to make careful blank measurements can dramatically increase for thin foils, also in an energy-dependent and structural manner. This is a strong argument for recommending the use of multiple-thickness or multiple-concentration samples in order to extract accurate attenuation, absorption and XAS structure (Chantler et al., 1999; Sayers, 2000; Chantler, 2024c). These illustrative beamlines and applications are not poor examples, but are typical of many advanced beamlines.

In a related situation, where the sample is a liquid solution or a frozen solution in a solution cell, the meaning is similar: dark-current and blank measurements are just as important and for the same reasons, and with the same typical magnitudes of systematic energy-dependent error. Yet, there is a critical question as to whether the scientific need is to investigate the solvent or, more usually, the solution and solute environment. In the latter case, it is necessary to remove, i.e. measure, the large background and structure associated with solvent absorption and scattering, so the `blank' measurement should primarily be the measurement with no solute, but with the solvent, (equation 5 in Best & Chantler, 2024), where the sample (s) is to be interpreted as the solute (S) and the background (b) is to be interpreted to include contributions from the solvent (SV) and the background (BKG). This still represents two independent measurements. Under normal circumstances, the path-length and density in the cell or frozen sample with the sample or with the solvent are not identical. However, the background components from the air path, windows etc. might hopefully be identical, so it is useful and important to separate the BKG component from the SV component. This can be performed by either a separate measurement of an empty cell, a true blank BKG, or by measurement of the geometry of the beamline to the detector. Additionally, the effective path-length ratio t_ratio for the sample-cell thickness t to that of the solvent-cell measurement t_SV can be fitted to allow for known variations in density or cell path length. In this common case, the functional equation becomes (equation 6 in Best & Chantler, 2024), with three independent measurements or with two independent measurements and geometric characterization. Notice that these alternate equations or functionals represent valid Bayesian hypotheses which, if applicable, can and should be included in the analysis. In the simplest form, one observes the systematic, measures it and corrects for it directly. In this case, the uncertainty is formulated again in a sparse matrix as(equation 7 in Best & Chantler, 2024). Rather than making a Bayesian inference about what the contributions may be, this approach measures them directly, thereby ensuring that the inference is valid. For this Case 2, we present two illustrations. One is a data set collected from dilute nickel complexes at the second-generation Australian National Beamline, Tsukuba, Japan (Chantler et al., 2015; Islam et al., 2015; the first data set) and the second is from the medium-sized but subtle molecule ferrocene (Islam et al., 2016; Bourke et al., 2016; the second data set).

As seen in these references, the accurate measurement of XAS, XANES and XAFS for a solution of a moderately complex molecule with a single metal absorber at 15 mM, 1.5 mM or less is considered to be a very difficult problem (in transmission mode). Indeed, it is often considered to be impossible for a transmission measurement, and some beamline rubrics advise or require fluorescence measurement. In fact, it is addressable, noting that the beamline signal to noise is far more important than the signal to background [Figs. 3 and 4 of Chantler et al. (2015) and Figs. 2 and 3 of Islam et al. (2016)], so long as careful measurement of the blank and characterization of the solute is made. For an active centre or reaction in solution this is very important. This approach can then test different approaches to theoretical determinations of XAS. Of course, if there is a change of local coordination, local structure or phase with concentration, then this would be revealed as a functional change or anomaly.

Fluorescence measurements are not intrinsically linear and do not intrinsically measure attenuation or absorption (Lee et al., 1981; Chantler, 2024d; Calvin & Nariyama, 2024), yet they are used very widely to support and justify models for local structure by fingerprinting or by fitting the known or assumed structure from, for example, prior X-ray diffraction studies. However, much of the nonlinearity can be addressed by correcting for absorption and self-absorption (Trevorah et al., 2019, 2020; Bridges & Booth, 2024), which will still return a relative amplitude and thus still needs to be normalized or scaled appropriately. Some beamlines collect full-spectrum data, which can assist in normalization, whereas most collect and hence can deposit only region-of-interest (RoI) data.

The amplitude of XAFS oscillations decreases with energy and k. However, whenever there is structural variation in efficiency and dampening of the amplitude of oscillations as a function of energy, the use of fluorescence data for hypothesis testing becomes limited. Nonetheless, this can be partially addressed in several ways.

(i) The fluorescence spectrum can be `flattened', which brings it to a partially even footing for analysis (IFEFFIT, ATHENA and related packages). `Flattening' is defined in different ways, but typically involves scaling the edge jump to unity and scaling the above-edge baseline to unity. The below edge may be scaled accordingly or set to zero. Since the attenuation coefficient always decreases above the edge, roughly as (E − E₀)³, this `flattening' is itself a distortion, but can empirically align different pixels and can reduce the distortion from data which rise above the edge instead of decreasing.

(ii) Several implementations of correction for absorption and self-absorption exist and may be used or adapted (Goulon et al., 1982; Tröger et al., 1992; Eisebitt et al., 1993; Pfalzer et al., 1999; Booth & Bridges, 2005; Chantler, Rae et al., 2012; Trevorah et al., 2019; Bridges & Booth, 2024).

(iii) Samples can be made `infinitesimally thin' to avoid both absorption and self-absorption corrections (clearly, this idealization has challenges and is subject to other problems including zero flux and zero statistics and the dominance of other systematic sources of error).

(iv) The sample angle to the beam and the detector angle to the sample can be modified and optimized, especially to either minimize absorption or to minimize self-absorption. Note that the grazing-incidence methods are excellent and insightful but constitute a separate series of questions and systematics (Chantler, 2024e). Notice that since the fluorescence signal is never linear with the sample thickness, it cannot in general provide a linear attenuation or a linear absorption coefficient, which is the basis for all theoretical prediction of XANES, XAFS and XAS.

We recommend option (ii) (Fig. 2).

Figure 2 The most significant systematics to address in fluorescence experiments for accurate XAS analysis.

Despite these negative-sounding caveats, fluorescence is used, and used successfully, for many qualitative questions and dominant corrections can be made successfully with careful experiment to provide an absolute measurement with uncertainty. In particular, equation (2) of Trevorah et al. (2019), where I₀ is the incident flux on the sample, ρ is the sample density, f is the fluorescence yield (usually for the specific fluorescence spectrum given by the RoI, for example Kα fluorescence), the asterisk indicates that only the component absorbed in the active centre producing a fluorescent photon is relevant, `pe' signifies that only the photoelectric absorption coefficient is considered, Ω represents the solid angle subtended by each detector (pixel), θ_inc is the incident angle of the X-rays onto the sample relative to the normal, θ_out is the angle of emission of the fluorescent radiation from the sample relative to the normal, [μ_f/ρ] represents the mass attenuation coefficient of the material at the energy of the fluorescent photon, t represents the sample thickness and I_f is the total number of fluorescence photons produced into a small solid angle Ω centred on θ_out. This functional is not completely dissimilar from that of earlier authors (Tröger et al., 1992; Booth & Bridges, 2005). This is expanded in equations (3), (4), (5) and (6) of Trevorah et al. (2019) and indeed in the 40 equations therein. Notice that the slope and structure above and below the edge are not consistent with the theoretical prediction, yet the corrected spectra in units of can have high accuracy, noting that there will be an overall scaling coefficient and that the corrected spectra relate to the photoelectric coefficient for the active species, that is not to attenuation nor to the photoabsorption of the solute, non­resonant atoms or orbitals. As such, however, the spectra agree in detail and within uncertainty with simultaneous absorption measurements after background subtraction; see, for example, Fig. 11 in Trevorah et al. (2019). More excitingly, the typical range with the Hanning window is also well reproduced; see, for example, Fig. 12 in Trevorah et al. (2019). Perhaps surprisingly, this was successful for 15 mM solutions with 0.1%(w/w) nickel [that is, nickel(II) complexes, bis(N-n-propyl-salicylaldiminato)nickel(II) (ⁿPr) and bis(N-isopropylsalicylaldiminato)nickel(II) (ⁱPr) with 60% butyro­nitrile (BCN) + 40% acetonitrile (ACN) as the solvent] and could be used for subtle structural and hypothesis testing between conformers. The advantages and disadvantages of transmission or fluorescence for metals, solutions and dilute solutions are numerous; but it is important that in both cases it is possible to achieve high precision and high accuracy by careful analysis.

What are the limits for accurate XAS on dilute systems in fluorescence? What are the limits of dilution of the active species in solvent from which one might gain accurate XAS, meaning ab initio measurement of nanostructure beyond the first few shells? In the general case, this question has not yet been answered and will surely depend upon the beamline and flux and stability and error analysis. Nonetheless, on a bending-magnet beamline, on a second-generation source, for dilute systems of 15 and 1.5 mM solutions with 0.1%(w/w) nickel [nickel(II) complexes ⁿPr and ⁱPr with 60% BCN + 40% ACN as the solvent] this approach has been successful in fluorescence and in comparison with transmission to confirm that the oscillations and magnitude of XAFS structure are indeed analysable across a reasonable Hanning window (Trevorah & Chantler, 2022). More work and development is surely required to learn what limits are possible and what limits can become routine.

Some of the most convincing such studies have been simultaneous measurements of transmission and fluorescence, as the disparate systematic errors can be investigated and compared, and the potential agreement and insight can be quantitatively analysed (Chantler et al., 2015; Islam et al., 2015, 2016; Trevorah et al., 2019; Trevorah & Chantler, 2022; John et al., 2023). Here, the discussions of Best & Chantler (2024) are commended. A recent realization of this (John et al., 2023) has used similar approaches to the earlier cases, and in this case has looked at metal foils in transmission and fluorescence simultaneously. Nonetheless, the approaches and methodologies of the X-ray extended-range technique (XERT; Chantler, 2024e) could not be used directly, especially because the zinc metal sample was in a cryostat and a temperature series was investigated. In this non-ideal and common case, one key sample was measured in the cryostat; a reference measurement was made at room temperature downstream (Chantler, 2024f) and the results were normalized with respect to a room-temperature measurement using XERT (Chantler, 2024a,e). This is particularly insightful for graphing the effects of particular absorption corrections and systematics as a function of energy for dark current, blank normalization, fluorescence and scattering, and roughness. Note that for a `thin' 10 µm thick foil, the transmission corrections for dark current were 1–9% and the systematic corrections for blank normalization were −53% to +3.8%. Fluorescent scattering corrections were of the order of 3–11% and roughness corrections were up to 1.5%, with all of these being energy-dependent and oscillation-dependent.

For most fluorescence measurements, the dominant corrections in slope and structure are for absorption and self-absorption and the fan between detector pixels, with the required scaling or rescaling as discussed above.

Although not discussed in this chapter, the importance of correcting for other systematics (Chantler, 2009; Chantler, Barnea et al., 2012), including energy calibration (Rae et al., 2006; Tantau et al., 2014; Chantler, 2024f; Tran et al., 2024), absolute thickness (Newville, 2004; de Jonge et al., 2004a; Islam, Rae, Glover, Barnea & Chantler, 2010; Rae et al., 2010), monochromator drift (Glover et al., 2008; Sier et al., 2020), harmonic contamination (Tran et al., 2003; Glover & Chantler, 2009; Chantler, 2024g), bandwidth on sample (de Jonge et al., 2004b; Bunker, 2024), fluorescence radiation (Chantler, Tran, Paterson, Barnea et al., 2001; Tran et al., 2004; Chantler, 2024h) and nanoroughness (Glover et al., 2009) can affect structure and oscillation magnitude by large amounts and percentages. Further examples of the measurement and the implementation of absolute accuracy, and the discussion of alternate definitions of relative or XAFS accuracy, can be found in Chantler, Tran, Paterson, Cookson et al. (2001), Tran et al. (2005), de Jonge et al. (2005, 2007), Glover et al. (2008, 2010), Islam, Rae, Glover, Barnea et al. (2010), Islam et al. (2014) and Tantau et al. (2015), especially relating to Case 1, which has so far been the best studied. These studies are particularly concerned with accurate measurement. The earlier studies are concerned more with accuracy and measurement than with local nanostructure. These studies cover an extensive range of investigation of XAS and apply from atomic numbers Z = 6 to Z = 79 and from ∼5 to ∼60 keV.

Accurate measurement of X-ray absorption spectroscopy (XAS), and especially transmission X-ray absorption fine structure (XAFS), is possible and can be well defined. Accurate measurement of bond lengths, nanostructure, secondary shells and isotropic thermal parameters can be achieved with high accuracy. Such measurement requires the initial determination of variance, precision and uncertainty, propagation of uncertainty and, particularly, measurement during the experiment of dark current and a suitable blank, and of dead time for solid-state detectors, especially in typical fluorescence experiments. Hence, a recommendation for ion-chamber detectors in particular, and in general for all detectors, is that the dark current (the signal with the beam off) should always be set to be positive and then measured to provide a linear detection chain.

Whilst this does not address all possible systematic errors for transmission measurement, it addresses what have been found to be the largest, the most pervasive and the most significant. Further, these are relatively straightforward to implement on all beamlines, whether second generation or third generation, whether bending magnet, undulator or wiggler, and whether soft X-ray, medium X-ray or hard X-ray. Hence, a strong recommendation is to implement data collection, analysis, error propagation and pre-processing to make dead-time corrections, dark-current corrections and uncertainty (variance) estimation routine and reportable in scientific publications and data deposition.

Secondly, the range of systematic error and structure across the XAS spectrum leads to a strong recommendation to use multiple-thickness or multiple-concentration samples whenever possible in order to extract accurate attenuation, absorption and XAS structure (Chantler et al., 1999; Sayers, 2000; Chantler, 2024c).

Thirdly, there is no reason why these cannot be applied for X-ray absorption near-edge structure (XANES) and pre-edge measurement. Even if the user purpose is fingerprinting, it still permits a beamline-portable interpretation. If the purpose is principal component analysis, it allows the disproof of hypotheses, instead of finding a best fit of inequivalent reference samples of questionable provenance.

Turning to fluorescence measurement, the determination of variance and precision normally requires correction for absorption and self-absorption as a dominant systematic error, and the local calibration of a scale factor. Nonetheless, these measurements can also then and thereby determine an accurate measurement, especially for structural determination, hypothesis testing, including conformational changes, and general XAFS processing. Hence, a recommendation is to implement data collection, analysis, error propagation and pre-processing to make absorption and self-absorption and variance routine and reportable in scientific publications and data deposition for fluorescence.

Perhaps remarkably, accurate measurement has been achieved for 15 and 1.5 mM solutions, in transmission but also in fluorescence, with a 0.1%(w/w) concentration of the active absorber. These also allowed high-accuracy determination of local structure and explicit hypothesis testing. The concentration limits have not yet been further investigated. With the oncoming development of theory which can predict XANES, XAFS and pre-edge simultaneously, the testing of these theories, these packages and these structures becomes very important. Equally, this will drive and develop the potential to reliably investigate local and dynamic nanostructure. In the field of XAS, there are some lessons to be learned from the long-term development of statistical analysis in X-ray diffraction and crystallography.