Chapter 7.3. The Open Source Database of the Japanese XAFS Society

Although it is possible that synchrotron-radiation facilities and facility members may deposit data that they are responsible for, or a company related to XAFS might collect data to create a database that is offered for sale, we decided on an open-access database in both download and upload directions. Thus, anyone involved in XAFS spectroscopy and interested in the database can deposit their data voluntarily and anyone who requires data can download the data from the database freely. This idea requires only a data server and a web site. In order to prevent illegal access to the database, anyone who wishes to deposit data has to be registered in advance. All of the necessary cost is now covered by ICAT, Hokkaido University.

We only accumulate the raw data in text format. The data should include metadata as shown later. The data format is simple. Each line should contain energy and absorbance. The deposited data are mainly those for standard compounds which have well-defined structures and properties. In order to obtain energy calibration, the foil XAFS data measured at the same beamline and the same time are deposited.

ICAT provides the server computer and manages the database. The login page can be found at https://www.cat.hokudai.ac.jp/catdb/index.php?action=xafs_login_form&opnid=2 .

A peer-review system is one way to confirm the validity of the data. However, it requires a great cost, human resources and time. Here, we apply the majority rule. We accept any data for standard compounds even if data for the same standard compounds have already been uploaded. This means that we have a large amount of data for the same standard compounds measured on different occasions, on different beamlines and by different experimentalists. If these data for the same compounds agree well with each other, the reliability of the data should be high. However, if the data for the same compound contradict each other, the data reliability becomes low and users should use the data with care. The database may function as a round robin for the beamline by comparing data for the same compounds.

When the View data buttom is pressed, the search window with keywords appears (Fig. 2). When one inputs search words such as the absorbing atom, edge and so on, a list of files appears at the bottom of the window (scrolling down may be necessary). The data can then be downloaded in ATHENA or REX2000 format, both of which contain energy and μt data with different headers.

Figure 2 Data-search window.

To deposit data, click on the `New entry' button shown in Fig. 3 when the deposition window is open. The data-deposition buttons are at the top of the window (Fig. 4). The 9806 format used at Photon Factory and SPring-8 can directly be accepted using the upload button. If the data are not in the 9806 format they can be entered manually. The header is composed of the following items: sample name, sample details, absorbing atom, edge, format (ATHENA or REX), sample data (energy and μt are separated by a space on one line; the data can be copied and pasted), reference data (energy and μt in the same format) and reference name, date of measurement, monochromator, facility, beamline, beam current, beam energy, method of rejection of higher harmonics, method of focusing, method of measurement, detection gases, name of the experimentalist and comments.

Figure 3 Deposition window.

Figure 4 Data-input window.

The problem is the small amount of data in the database. In November 2022, there were 1014 data sets in the database. The majority rule has been adopted to guarantee the data. However, it does not work well with such a small number of data. The only way to solve this is the automatic deposition of all data to one storage computer just after measurement, followed by categorizing the data as an open or a secret file. Consequently, we may obtain a larger database in a few years and the majority rule will work to confirm the data quality. The question then is who will draw useful information from such a large database?

The recent rapid developments in artificial intelligence (AI) will open new possibilities if such a large database is obtained. AI will search through the database and pick up XAFS spectra of the same standard samples. It will judge whether the data are valid or not by comparing them based on the majority rule, but minor spectra should be kept with a tag `minor or less reliable'. In the future, each data file may have its own reliability factor based on the majority rule and other pieces of information (facility, signal-to-noise ratio, operator and so on) which can be determined by AI after deep learning. AI may then create a new database with reliability factors. If such a large database is obtained and is open to all, AI will search through the database and pick up similar XAFS spectra of standard compounds to the unknown compound that one has measured. AI will provide some hints about the structure of the material. We can then further analyse the results. The XAFS data files are preferably connected to other data sets of physical and chemical properties, synthesis details and so on (Ishii et al., 2023), and a hierarchical data format (such as HDF5) will need to be developed for storage of these big data.