Chapter 2.5. Specification of the core CIF dictionary definition language (DDL1)

The structure or arrangement of data in a CIF is well understood and predictable because the CIF syntax may be specified succinctly (see Chapter 2.2 for CIF syntax expressed using extended Backus–Naur form). In contrast, the meaning of individual data values in a file is only known if the nature of these items is understood. For CIF data this critical link between the value and the meaning of an item is achieved using an electronic `data dictionary' in which the definitions of relevant data items are catalogued according to their data name and expressed as attribute values, one set of attributes per item.

The dictionary definitions describe the characteristics of each item, such as data type, enumeration states and relationships between data. The more precise the definitions, the higher the level of semantic knowledge of defined items and the better the efficiency achievable in their exchange. To a large degree, the precision achievable hinges on the attributes selected for use in dictionary definitions, these being the semantic vocabulary of a dictionary. Within this context, attribute descriptors constitute a dictionary definition language (DDL). Definitions of the attributes described in this chapter are provided in Chapter 4.9 .

The main purpose of this chapter is to describe the DDL attributes used to construct the core, powder, modulated structures and electron density CIF dictionaries detailed in Chapters 3.2–3.5 and 4.1–4.4. We shall see that each item definition in these dictionaries is constructed separately by appropriate choice from the attribute descriptors available, and that a sequence of definition blocks (one block per item) constitutes a CIF dictionary file. The organization of attributes and definition blocks in a dictionary file need not be related to the syntax of a data CIF, but in practice there are significant advantages if they are. Firstly, a common syntax for data and dictionary files enables the same software to be used to parse both. Secondly, and of equal importance, data descriptions and dictionaries need continual updating and additions, and the CIF syntax provides a high level of extensibility and flexibility, whereas most other formats do not. Finally, the use of a consistent syntax permits the dictionary attribute descriptors themselves to be described in their own DDL dictionary file.

While the basic functionality and flexibility of a dictionary is governed by the CIF syntax, the precision of the data definitions contained within it is determined entirely by the scope and number of the attribute descriptors representing the DDL. Indeed, the ability of a DDL to permit the simple and seamless evolution of data definitions and the scope of the DDL to precisely define data items both play absolutely pivotal roles via the supporting dictionaries in determining the power of the CIF data-exchange process.

Every numerical data item has distinct properties. Consider the number 20 as a measure of temperature in degrees. To understand this number it is essential to know its measurement units. If these are degrees Celsius, one knows the item is in the temperature class, degrees Celsius sub-class, and that a lower enumeration boundary of any value can be specified at −273.15. Such a constraint can be used in data validation. More to the point, without any knowledge of both the class and subclass, a numerical value has no meaning. The number 100 is unusable unless one knows what it is a measure of (e.g. temperature or intensity) and, equally, unless one knows what the units are (e.g. degrees Celsius, Kelvin or Fahrenheit; or electrons or volts).

Knowing the inter-dependency of one data item on another plays a major role in the understanding and validation of data. If a triplet of numbers 5, 3, 0 is identified as Miller indices h, k, l, one immediately appreciates the significance of the index triplet as a vector in reciprocal crystal space. This stipulates that the three numbers form a single non-scalar data item in which the indices are non-associative (e.g. 3, 5, 0 is not equivalent to 5, 3, 0) and irreducible (e.g. the index 3 alone has no meaning). As a reciprocal-space vector, the triplet has other properties if is part of a list of other reciprocal-space data items; namely, its value represents a key or list pointer (i.e. a unique key to a row of items in a list table) to other data items in the list associated with this vector. This means that data forming a `reflection' list are inaccessible if these indices are absent, or invalid if there is more than one occurrence of the same triplet in the list. Such interdependency and relational information is very important to the application of data, and needs to be specified in a dictionary to enable unambiguous access and validation. Other types of data dependencies will be described in Section 2.5.6.

A definition for formula units in a crystal unit cell (Z) is shown in Fig. 2.5.5.1. The definition includes only the attributes _name, _category, _type, _enumeration_range and _definition. These identify the item's unique properties and enable its validation. The _category attribute indicates the class of the defined data item. Since lists may contain only items of one category type, this attribute is critical to data items appearing in lists (see Section 2.5.5.5). The attribute _enumeration_range has the value ` 1:' which stipulates that the item must be `one or a higher number'.

Figure 2.5.5.1 | top | pdf | DDL1 definition with a few attributes.

A CIF dictionary file contains, in addition to the data definitions, information about the nature of the dictionary. Fig. 2.5.5.2 shows the typical organization of definitions in a dictionary. Each dictionary starts with the audit information giving its version and creation history. This is followed by definitions in separate data blocks. The name of a data block matches the defined data name or the initial portion thereof. Each definition data block contains a sequence of data declarations, one for each attribute. The attributes within a data block constitute the total definition information, and only those attributes appropriate to a given defined item need be specified.

Figure 2.5.5.2 | top | pdf | DDL1 definition showing basic dictionary organization.

Some data items are closely related to other data items. Fig. 2.5.5.3 shows the definition of the length components _cell_length_a, *_b and *_c of the crystal unit-cell metric tensor. Because of their different enumeration and units attributes, the definition blocks for cell lengths and angles are specified separately. This definition shows how data items closely related in function may be defined in the same data block. The loop_ command is used to list the multiple _name values. The attribute _type_conditions is used to indicate that these values are measurements and can have a standard uncertainty (`s.u.') value appended in parentheses e.g. 7.254(2). (The label `esd' reflects the historical but inaccurate terminology `estimated standard deviation'.) The _units and _units_detail attributes stipulate that the measurement unit of length is ångströms.

Figure 2.5.5.3 | top | pdf | DDL1 definition of irreducible data items.

The definition of the number of hydrogen atoms attached to an atom site is shown in Fig. 2.5.5.4. The attributes _list and _list_reference signal whether the item is used in a list and what key item is in that list. As described in Chapters 2.1 and 2.2 , a CIF list is a two-dimensional table where data items are in the columns (headed by the data names) and the values are the rows. The attribute _list is yes if a data item may be used in a list. The attribute _list_reference specifies the key item as _atom_site_label. Each value of a key item must be unique so that the row of associated items can be accessed unambiguously. The attribute _enumeration_default specifies a default value for the defined item if it is absent from the data file.

Figure 2.5.5.4 | top | pdf | DDL1 definition of a `list' data item.

Each CIF dictionary also contains example applications of defined data items, grouped according to category. Fig. 2.5.5.5 shows these for the category ATOM_SITE, as defined in the core dictionary (see Chapter 3.2 ). A _type attribute with a value of null specifies that a data block contains no definition information. The particular examples shown here illustrate that the reference key for the ATOM_SITE category is _atom_site_label. This data item is present in both examples shown, and in each packet the label is unique.

Figure 2.5.5.5 | top | pdf | DDL1 overview of a category of items.

The example definition in Fig. 2.5.5.6 shows the attributes for the item _atom_site_label. As illustrated in Section 2.5.5.5, this item is the reference key to a list of items belonging to the category ATOM_SITE. The attribute _list_mandatory, which is set to a value of yes, specifies that this item is a mandatory item to a list of category ATOM_SITE items. The attribute _list_link_child identifies items in other categories that are `linked' by derivation to this item and therefore share the same data values. This dependency is known as a child dependency. The definition shows that data items describing the labels of atom sites in a list of angular geometry, i.e. _geom_angle_atom_site_label_1 and _geom_angle_atom_site_label_2, are the same labels described by _atom_site_label in the atom-site list. This is because the geometry is derived directly from the atom-site data. Note that this dependency requires that the ATOM_SITE list be present in the same CIF as the GEOM_ANGLE list, otherwise the molecular geometry information cannot be linked to the three-dimensional structural information.

Figure 2.5.5.6 | top | pdf | DDL1 definition of a `mandatory' data item.

In the example definition shown in Fig. 2.5.5.7, the key item _atom_site_aniso_label is shown to have a special relationship with the key item _atom_site_label. This is because the data items in the list category ATOM_SITE_ANISO may be merged with (i.e. joined to) items in the list category ATOM_SITE. If this happens, the item _atom_site_label assumes the role as the key to the merged packets and this is signalled using the attribute _list_link_parent. When these two categories appear in separate lists, the category ATOM_SITE_ANISO data require the category ATOM_SITE data to be present in the data instantiation, but not vice versa. Note that the parent relationship, unlike that in the example of Section 2.5.5.6 arising from a derivation dependency, is because ATOM_SITE_ANISO is a subcategory of ATOM_SITE.

Figure 2.5.5.7 | top | pdf | DDL1 definition of a `parent' data item.

As with the preceding example definition, this example concerns anisotropic atomic displacement parameters. Fig. 2.5.5.8 shows the definition of the component items _atom_site_aniso_U_ that comprise an irreducible set of matrix elements. The key item in a list of ATOM_SITE_ANISO items is _atom_site_aniso_label. This definition illustrates how the attributes _related_item and _related_function are used to identify the similar data items _atom_site_aniso_B_ that are atomic displacement parameters related by a simple conversion factor. Other attributes used to specify data relationships are described in Section 2.5.6 below.

Figure 2.5.5.8 | top | pdf | DDL1 definition showing `related' data items.

The last example, in Fig. 2.5.5.9, shows the definition of an item whose value is restricted to a predictable set of values known as enumeration states. The attributes _enumeration and _enumeration_detail are used to specify which enumeration states are permitted for the defined data item. Only one of these states may appear as the value for the defined item in a CIF. The attribute _enumeration_default specifies the state value that is used if an item is not instantiated.

Figure 2.5.5.9 | top | pdf | DDL1 definition showing enumeration states.

Establishing the identity of a data item is a primary function of a dictionary. In a data instantiation each item is recognized by a unique code, known as its data name or tag. This tag provides the most fundamental level of data validation.

The `identification' attribute in DDL1 dictionaries is _name, and appears in a definition as

The tag of the defined item is the value of _name and, because it starts with an underscore, it must always be bounded by quotes to prevent it from being interpreted as the start of another tag–value pair. If there is more than one data item in a definition, as in the case of an irreducible set of items, the data names are entered as a list starting with the statement (see the example in Fig. 2.5.5.8).

The presence of the identification attribute _name in a definition is mandatory. Its value, the name of the defined items, provides for spelling validation. Note that if a data item in a CIF is not defined in a dictionary the CIF is still valid, although CIF parsers that employ dictionaries as part of the scanning process usually flag undefined items. The accepted practice to date is that undefined data are largely ignored by a dictionary validation process. In contrast, the need for certain defined items to be present in CIF data can be crucial because of list dependencies. The attributes that specify data relationships and links are considered in Section 2.5.6.4.

Three DDL1 attributes are used to provide text descriptions of a defined data item. These are present in a dictionary for human readability, browser software or for the production of text dictionaries such as those in Part 4. This group of attributes is not machine interpretable. The _definition attribute provides a text description and as such is the primary semantic content in a defined data item. It may also be used to provide supplementary information about other machine-parsable attributes in the definition (see the definition in Fig. 2.5.5.3). The attributes _example and _example_detail are used to show typical instantiations of the defined item, as also shown in Fig. 2.5.5.6.

This class of attributes is used to specify the fundamental characteristics of data items and how they may be instantiated. These attributes are machine-parsable and of particular importance in the validation of CIF data. They are Enumeration attributes are used to specify restricted values, or `states', of a data item (see the example in Fig. 2.5.5.9). They are not applicable in a definition of an item with unrestricted values. The attributes _enumeration and _enumeration_detail are used in definitions to specify permitted states and their descriptions. For instance, in a definition of atomic element symbols these attributes would be used to list the IUPAC `element symbols' and `element names'. When a data item is restricted to an ordinal set of values, the attribute _enumeration_range is used to define the minimum and maximum values of the logical sequence in the format <min>:<max> The attribute _enumeration_default may be used to specify the default value if an item is not present in a data instantiation.

List attributes specify how, and when, data items are used in a (looped) list. The attribute _list has a value of yes if a defined item must be in a list, and no if it must not. A value of both allows for both uses. The attribute _list_level specifies the nesting level of a list defined item. Only level 1 data are allowed in a CIF (see Chapter 2.2 ). Higher-nested list levels are permitted for MIF data (Allen et al., 1995; see Chapter 2.4 for a description of the MIF format). Fig. 2.5.6.1 shows MIF data specifying a 2D chemical molecular diagram. The first four items in the category DISPLAY are in a level 1 list and the next two items in the category DISPLAY_CONN are in a level 2 list. Note that, according to the STAR File syntax, the nest level is automatically incremented after the first four values, and only decremented when a stop_ signal is encountered.

Figure 2.5.6.1 | top | pdf | (a) Hypothetical example and (b) the typical definition of nested items.

Additional _list attributes for specifying relational dependencies between data items are described in the next attribute group.

Type attributes are used to specify the form of data. The attribute _type is restricted to the states numb, char and null that code for a number, a text string and a dictionary descriptor, respectively. The implications of this typing, in terms of how numbers or character strings are interpreted, is not specified by the CIF syntax. That is, a `number' is simply a string of characters having one of several possible representations (e.g. as an integer, a floating point or in scientific notation) and it is up to the parsing software to interpret these strings appropriately. The same applies in the treatment of text strings defined as `character'. Such strings may be bounded by white space, quotation characters or, in the case of multi-line text, by semicolon characters in column 1 of a line.

The attribute _type_conditions is used to identify application-specific conditions that apply to the typing of data items. This attribute is used to alert parsing software of string constructions that may disrupt the normal identification and validation of data types. The permitted attribute states are none, esd (and its preferred synonym su) and seq. In the definition example in Fig. 2.5.5.3 the code esd signals that cell-length values are measurements and may have a standard uncertainty value appended in parentheses. Fig 2.5.5.8 indicates the use of the code su for the same purpose. A full description of _type_conditions states is given in the DDL1 dictionary in Chapter 4.9 .

If a data item consists of a concatenation of values, the attribute _type_construct may be used to specify the encoding algorithm for the component values. The language used for this algorithm is POSIX regex (IEEE, 1991). The specification of _type_construct can include the data names of other defined items. Validation software interprets this attribute as format specifications and constraints, and expands the embedded data items according to their separate definition. For example, the chronological date is a composite of day, month and year. These may be represented in many different ways and the _type_construct attribute enables a specific date representation to be defined. The two definitions shown in Fig. 2.5.6.2 illustrate how a date value of the construction `1995/03/25' may be embedded into the dictionary definitions (the month and day definitions have been omitted here for brevity).

Figure 2.5.6.2 | top | pdf | Definition example: type constructions.

Units attributes specify the measurement units permitted for a numerical data item. The attribute _units specifies a code that uniquely identifies the measurement units. A description of the units identified by the code is given by the attribute _units_detail. Typical uses of this attribute are shown in Fig. 2.5.5.8.

This class of attributes is used to describe special relationships and dependencies between data items. These attributes are machine parsable and are formally defined in the DDL1 dictionary in Chapter 4.9 . They are

The attribute _category is used to specify to which group, or basis set, a data item belongs. The value of this attribute is a name string that identifies the group and it is usually the leading part of the tags for all items in this group. Data items in a list must have the same category value. Data items in the same category may, however, be divided into different lists, provided each list contains an appropriate key data item (see _list_reference below).

List-link attributes are used to specify dependencies between data items in different lists. The attribute _list_link_parent identifies an item in another list from which the defined item was derived. The parent data item, or items in the case of irreducible sets, must have a unique value within its own list. The _list_link_child attribute is declared in the definition of a parent item and identifies items in other lists that depend implicitly on the defined data item being present in the same data instantiation. The functionalities of these two attributes mirror each other. The parent item must be present in any data instantiation containing the child items, but not the converse. The definition examples in Figs 2.5.5.6 and 2.5.5.7 illustrate the use of these attributes.

The attributes _list_mandatory and _list_reference are also connected to each other. The former signals (with values of yes or no) whether the presence of a data item is essential to preserving the validity of a list of items. The latter attribute identifies the data item in the list that provides the unique key value to each packet or row of items in a list. A packet is made up of listed values, one for each item in the name list (i.e. the packet size matches the number of data names at the head of the list). The _list_reference attribute identifies items that are the keys to specific packets in the list. In Example 2 of Fig. 2.5.5.5 the key item is _atom_site_label and the listed labels S1, S2, N1 and C1 must be unique.

The attribute _list_uniqueness is used to identify items that must be unique for a list to be valid and accessible. This attribute is similar to _list_reference except that it appears only in a definition in which _list_mandatory is set to yes. This simplifies validation because it may be used as the placeholder for all items that jointly identify the uniqueness of a list packet. This is in contrast to the attribute _list_reference, which appears in the definition of every item dependent on this item.

Relational attributes are used to link equivalent data. The _related_item attribute identifies items related to the defined data item. The nature of this relationship is specified with the _related_function attribute according to the restricted value states of alternate, convention, conversion and replace. The definition of these states is detailed in Chapter 4.9 . Relational attributes are used to provide equivalent data items, to replace definitions when definitions are superseded or to change access pathways. These facilities are for archives, because they enable old data to be accessed and the associated definitions to remain in a dictionary even when superseded by new definitions. The old and new definitions are linked by these attributes so that all related data items can be validated and accessed.

This class of attributes is used to register the dictionary version and audit information. They are

Dictionary attributes are used to record the creation and update history of a dictionary. The attribute _dictionary_history specifies the entry and update information of the dictionary, and _dictionary_name specifies the generic name of the electronic file containing the dictionary (the actual name of the file can vary from site to site). The attributes _dictionary_version and _dictionary_update specify the version number and the date of the last change in the dictionary. Both items represent important external reference information. An example application of these attributes is shown in Fig. 2.5.5.2.