Chapter 1.4. Perspectives for the future for X-ray absorption spectroscopy and related techniques

Can we explain extended X-ray absorption fine structure (EXAFS), X-ray absorption near-edge structure (XANES), resonant inelastic X-ray scattering (RIXS), high-energy-resolution fluorescence detection (HERFD) and the dependence of spectra on sample orientation, beam polarization and sample magnetization, or X-ray free-electron laser (XFEL) XAS data and spectra? The current answer is very clearly yes – see all of the chapters on theory in this volume – and no – the current scientific literature suggests that there is more understanding to come from detailed spectral structure. There are major areas where new theory, new understanding and new software will drive new insights and discoveries. These might require new and optimized beamlines to ask more specific or more detailed questions of the spectroscopy and local structure, and this is an exciting future for all of us. These current techniques have an area of overlap with diffraction anomalous fine structure (DAFS)-related techniques and multiwavelength anomalous diffraction (MAD); novel developments could allow inquiry with much higher sensitivity. Early understanding of coherence arose from infrared studies and also, much earlier, from Bragg diffraction. Much is talked about in the literature regarding coherence in visible and laser optics. In fact, the understanding of coherence is perhaps most advanced in the X-ray regime, yet major questions about the coherence of physical processes remain unanswered experimentally, and complex theoretically. There are emerging fields investigating dynamic structure, dynamic bond-length measurements, inelastic mean free paths and other phenomena which are nascent at present but are sure to grow, as discussed below.

The robustness and portability of data require the definition, measurement and use of uncertainty in all data science. This has been the case in more specialized experiments in X-ray absorption fine structure (XAFS) and X-ray absorption spectroscopy (XAS) for some 20 years now, but has not yet become widespread, routine or expected. Part of this is discussed in Hall et al. (2024) and in other chapters in this volume. Every step towards this will make a beamline much more useful and will also make the data more reliable.

In this volume, packages that use, propagate and fit data with defined uncertainty are described. Although these have been available for some 20 years, the worldwide community needs their use to be widespread and routine. Efforts by beamlines to achieve this will generate much higher quality data for reference materials, for solutions, for reactions, for novel materials and for all forms of condensed matter. This yields the ability to use the information content in the data and to justify and defend conclusions, and to have independent groups further investigate important conclusions.

The International Union of Crystallography's (IUCr's) Commission on XAFS roadmap includes a directive to promote the use of XAS by teaching advanced methods of measurement and analysis and to develop standards and criteria for the community to use in order to improve the overall quality of the research being performed in the field and to investigate the conclusions drawn (Chantler et al., 2019). This is a core topic of the regular Q2XAFS meetings (Ascone et al., 2012; Diaz-Moreno & Strange, 2018) and a joint direction of current efforts in both the Commission on XAFS and the International X-ray Absorption Society (IXAS).

Ab initio density-functional theory simulations currently provide an excellent starting point for the analysis and fitting of XAFS data. Beamlines can provide high-quality data with which to investigate the nanostructure of any material. Nano­structure is currently often fitted using a reference base from crystallography; that is, an analogous crystal structure. If the material has a specific space-group symmetry and hence is crystalline, then this is an excellent starting point. Often, however, the material is disordered, perhaps with significant local order but a lower symmetry. In this case, especially with phase changes or solutions or mixed phases, the XAS data analyst needs to be able to conduct hypothesis testing, which in turn requires a measure of significance. Some studies have worked towards this, but it is by no means yet routine.

The development of hypothesis testing of data sets includes investigations of challenging phenomena, including dynamical bond lengths from XAS versus crystallographic mean atom-position separations, bond-length increases with disorder or temperature, any temperature or temporal dependence and any extreme conditions. Such ab initio hypothesis testing also includes simple physically based investigations of XANES, principal component analysis and related techniques, and includes studies of beam damage and studies of reliability diagnostics for fragile (beam-sensitive) systems.

Exciting methodological developments relating to increased quality of data sets have led to greater information content than earlier work, and to new structural and dynamic insights into materials. In particular, an improved understanding of uncertainties and their propagation allows the evaluation of subtle features and discrimination between models with, for example, the same coordination number when high-quality data are provided. Novel techniques such as the X-ray extended-range technique (XERT) and hybrid techniques have pushed the sensitivity of transmission XAFS to 10 and 1.5 mM, and similarly have shown new structures to be obtainable in fluorescence XAFS.

Some scientific journals have criteria for publication of studies involving XAS- and XES-related techniques. Data sets are typically not included in, made available with or deposited with the publications, although work is ongoing to improve this, as explained in this volume. As an example, data recorded at facilities in Europe are required to be open access, after an embargo period, and this is one step along this route. The inclusion of information to permit portability, i.e. use by other researchers or at alternative beamlines, is in no way trivial, and much work and cooperation will be needed by all experts and beamline scientists to achieve this. Nonetheless, working towards such portability is a great opportunity for a bright future, for standards for data acquisition at new beamlines and for insight from these techniques. The Commission on XAFS roadmap is also working towards establishing adequate guidelines and standards for articles to be published in IUCr journals reporting structural investigations of materials by XAS. The IXAS has also been active in this area (Sayers, 2000a,b). In part, this also follows the work of the IUCr on this issue (Creagh & Hubbell, 1987, 1990; Creagh, 1987).

The literature demonstrates a developing understanding of and ability to present uncertainties representing both the relative and absolute accuracy of a datum. These are important for different experimental and theoretical purposes. So far, determining these typically involves specialized experiments, and a future endeavour would be to allow such experiments to become possible for routine users and routine evaluation.

There have been several recent and valuable efforts towards XAFS databases, which are essential for the portability and usability of data by other researchers. Some of these are currently in simple and easy-to-use formats and for a local beamline or regional group, or as a national depository (Lytle et al., 1995; Kelly et al., 2009; Abe et al., 2018; Asakura et al., 2018). However, the development of an authoritative and robust database is in its inception. The coordination of a database on XAS is also one of the key aims of the Commission, together with the promotion of inter-facility comparison of the results gained on different beamlines, in order to evaluate the influence of data quality on the extracted structural parameters (Chantler et al., 2018). A round-robin project across synchrotrons and XAFS beamlines is in progress (Chantler et al., 2019).

The development of quick XAFS over the last decade has been very exciting, with several new beamlines being built for this purpose. The reproducibility and speed are outstanding. One of the key advances is the development of new technology for fast data acquisition, which applies to this field, XFELs (see below) and many other fields. Although currently not unified or widespread, these approaches have great value and provide insights into many different areas of physics, especially time-dependent phenomena. At this stage, it appears that more widespread understanding and usage, including applications and limitations, are eagerly sought, including characterization of precision and accuracy.

In the recent decade, new advanced laboratory sources have been developed and marketed for routine XAS analysis at the local laboratory, i.e. avoiding the need to visit a synchrotron. Synchrotron beam time is typically oversubscribed and it can be difficult to obtain sufficient (or indeed any) beamtime for critical projects. One solution is of course new, adapted or advanced beamlines. Another emerging solution is regional laboratory sources. For local sources, the optics, systematic errors, precision and accuracy need to be clearly understood and characterized. There is doubtless great value and applicability of these new sources, which are sure to improve.

Free-electron lasers (FELs) can provide X-ray pulses that are characterized by extremely high peak brilliance, high coherence and femtosecond duration; the pulse-repetition rate depends on the particular source, ranging from 100 Hz to several megahertz. These features are opening new areas of research in many fields, including those that use X-ray spectroscopy. Since the pulse duration is shorter than the vibrational periods of atoms in condensed matter, the evolution of the atomic and electronic structure can be followed on a highly relevant timescale, leading to the realization of molecular movies. Exciting new possibilities are opening up as the scientific community learns to use these advanced sources, with a typical application being determination of the sequence of local structures formed as a consequence of absorption of visible or near-visible photons as the system (molecule or solid) passes though a number of intermediate states before relaxing back to the ground state. The standard XAFS-related methods (XANES, EXAFS, HERFD XAS and RIXS) can thus gain unprecedented time resolution. X-ray spectroscopy measurements require pulse-by-pulse normalization of the intrinsically unstable FEL source, and successful schemes to perform this have been devised. Femtosecond spectroscopy with FELs can further benefit from the advanced instruments and sample-conditioning apparatuses installed at these facilities, for example in the fields of photocatalysis and extreme conditions. The extremely high peak brilliance has opened a way to study and exploit non­linear interaction phenomena, which are well known in the visible and near-visible regions, in the X-ray range. This is both a challenge and an opportunity for spectroscopy, and it is to be expected that a large amont of work will be performed in the field. Novel experimental schemes are being devised, and applications to important classes of materials are expected as they are consolidated.

Many third-generation synchrotron-radiation (SR) sources are being, or have already been, upgraded using new magnetic lattices which offer great improvements in key parameters. For example, the European Synchrotron Radiation Facility has been upgraded to the Extremely Bright Source using the novel hybrid multi-bend achromat design, providing an approximate factor of 30 decrease in horizontal emittance and an increase in brilliance by a factor in the range 15–40, depending on the photon energy. XAFS-related methods have much to gain from these new ultra-low-emittance sources. The spatial resolution of imaging techniques based on X-ray absorption (microspectroscopy and nanospectroscopy) is dramatically improved and two-dimensional maps of chemical state and local electronic and atomic structure with ∼10 nm resolution are already achievable; further improvements are expected. The low-emittance source, coupled with the intrinsic stability, improves the sensitivity to low-concentration elements in fluorescence-detected XAFS; sophisticated experimental setups including full and high-resolution analysis of scattered photons improve the quantity and quality of information that can be gleaned from experiments. Last, but not least, the smaller circular source size of these new sources greatly improves all experimental approaches that use their high degree of polarization, both linear and circular.

RIXS and HERFD can now be combined with advanced theory to result in both new discoveries and physical processes, and to explain limitations in the techniques, analysis and theory of previously known processes. This is perhaps exemplified by extended-range HERFD (XR-HERFD). This is an emerging technique, and thus is well outside the scope of this volume at present. However, it is an example of a novel approach and the opportunity for new discoveries from XAS, XES and related techniques.

Another exciting development is that of complex atomic fine structure (CAFS) spectroscopy. This technique collects data on the real and imaginary components of the sample refractive index and incident wave modification simultaneously, and is analoguous to simultaneous XAFS and DAFS to measure the complex interaction coefficient for absorption, scattering and diffraction. As an idea, it is simple: measure the amplitude and phase evolution of the quantum-mechanical wave through an arbitrary nanomaterial to characterize bonding and many other processes. This is currently a nontrivial experiment, yet results are beginning to emerge that point to an exciting future for novel XAS and XES and related techniques.

There are many new and insightful approaches to analysis which are germinal or nascent in their application to XAFS, XES and related techniques. Bayesian methods are very widespread, but are not necessarily well understood. Apart from the obvious and χ² markers, criteria for significance are germinal. These are discussed in this volume, including where they have been applied so far, but much more is possible. For XANES and related data, the current trend remains fingerprinting. It is important that reference materials be comparable to provide insightful analysis. Extensions and advances in fingerprinting, as discussed in this volume, involve principal component analysis, linear combination analysis and several other modalities. Care still needs to be taken to avoid false Bayesian assumptions. Other new methods include multivariate curve analysis and multivariate curve regression with alternating least squares (MCR-ALS).

XANES, for example, is now developing into an exciting and nuanced field in which many insights from advanced theory and analysis are expected. On the basis of XANES calculations, the experimentalist can now quantitatively confirm or discard a structure, obtaining structural parameters with high reliability and accuracy. Sometimes this represents a unique approach to establishing the structure when dealing with extra-dilute metal centres in a protein or a catalyst. This improvement in XANES analysis is crucial when the dilution of the photoabsorber prevents the recording of high signal-to-noise ratio EXAFS data over the extended photoelectron wavevector k-range necessary for accurate EXAFS analysis, although advanced analysis is able to question past common understanding on the dilution limits of the techniques. The new finite-difference method for XAFS (FDMX) and other approaches are now working towards new theory that is able to model both the EXAFS and XANES regions to gain new information and more accurate physical parameters. Progress in ab initio calculations of XANES spectra permits new insights into the physical causes of spectral details and their magnitudes.

In a parallel set of developments, the onslaught of machine learning, neural networks, assisted supervision, training algorithms and related image-based data-analysis methods has been seen to show much promise in explaining unknown data sets, or in reproducing them without explanation, and in potentially providing an alternative reference source for an ab initio theoretical ansatz to a database or advanced or standard experimental data sets. We should proceed with both excitement and caution, remembering the basic principles of physics and science.

This has remained a critical and topical area of research because of how important and pervasive beam damage is across the scientific and biomedical communities. We discuss some key ideas in this volume. Advances and insight here address the enormous world of fragile and beam-sensitive biological and nonbiological materials and their analysis.