Chapter 5.4. CIFtbx : Fortran tools for manipulating CIFs

These variables are returned by CIFtbx and provide information about the general status of processing.

file_ : character string containing the file name of the current input file.

longf_ : integer variable containing the length of the file name in file.

precn_ : integer variable containing the line number (starting from 1) of the last line written to the output CIF.

recn_ : integer variable containing the line number (starting from 1) of the last line read from the input CIF.

tbxver_ : character*32 variable that is the CIFtbx version and date in the form 'CIFtbx version N.N.N DD MMM YYYY ' (some older versions of CIFtbx use a two-digit year and have a comma after the version number).

These variables control CIFtbx commands. The user may accept the default values or may store new values into these variables to change the behaviour of the commands.

alias_ : logical variable to control the use of data-name aliases for input items. If set to .true., aliases from the input dictionary may be used (see Section 5.4.7). The default is .true..

append_ : logical variable to control reuse of the direct-access file. If set to .true., it will cause each call to ocif_ to append the information found to the current CIF. The default is .false..

line_ : integer variable to set the input/output line limit for processing a CIF. The default value is 80 characters. This limit counts the visible printable characters of the line, not the system-dependent line terminators.

nblank_ : logical variable to control the treatment of input blank strings. If set to .true., char_ or test_, it will return the type as `null' rather than `char' when encountering a quoted blank.

recbeg_ : integer variable to give the record number of the first record to be used. May be changed by the user to restrict access to a CIF.

recend_ : integer variable to give the record number of the last record to be used. May be changed by the user to restrict access to a CIF.

tabx_ : logical variable is set to .true. for tab stops to be expanded to blanks during the reading of a CIF. The default is .true..

These variables are returned by CIFtbx tools and are used to decide on subsequent actions in the program. The lengths of the character strings that hold data names and block names are controlled by the parameter NUMCHAR in the common-block declarations.

bloc_ : character string containing the current data-block name.

decp_ : logical variable is .true. if a decimal point is present in the input numeric value.

diccat_ : character string containing the category name specified in the attached dictionaries.

dicname_ : character string containing the root alias data name (see Section 5.4.7) specified in the attached dictionaries or, after a call to dict_, the name of the dictionary.

dictype_ : character string containing the data-type code specified in the attached dictionaries. These types may be more specific (e.g. `float' or `int') than the types given by the variable type_ (e.g. `numb').

dicver_ : character string containing the version of a dictionary after a call to dict_.

glob_ : logical variable is .true. if the current data block is a global block. The application is responsible for managing the relationship of global data to other data blocks.

list_ : integer variable containing the sequence number of the current looped list. This value may be used by the application to identify variables that are in different lists or that are not in a list (a zero value).

long_ : integer variable containing the length of the data string in strg_.

loop_ : logical variable is .true. if another loop packet is present in the current looped list.

lzero_ : logical variable is .true. if the input numeric value is of the form [sign]0.nnnn rather than [sign].nnnn.

posdec_ : integer variable containing the column number (position along the line, counting from 1 at the left) of the decimal point for the last number read.

posend_ : integer variable containing the column number (position along the line, counting from 1 at the left) of the last character for the last string or number read.

posnam_ : integer variable containing the starting column (position along the line, counting from 1 at the left) of the last name or comment read.

posval_ : integer variable containing the starting column (position along the line, counting from 1 at the left) of the last data value read.

quote_ : character variable giving the quotation symbol found delimiting the last string read.

save_ : logical variable is .true. if the current data block is a save frame, otherwise .false..

strg_ : character variable containing the data name or string representing the data value last retrieved.

tagname_ : character variable containing the data name of the current data item as it was found in the CIF. May differ from dicname_ because of aliasing.

text_ : logical variable is .true. if another text line is present in the current input text block.

type_ : character variable containing the data-type code of the current input data item. This will be one of the four-character strings `null' (for missing data, the period or the question mark), `numb' (for numeric data), `char' (for most character data) or `text' (for semicolon-delimited multi-line character data). For most purposes the type `text' is a subtype of the type `char', not a distinct data type. CIFtbx permits multi-line text fields to be used whenever character strings are expected.

These variables are specified to control the processing by CIFtbx commands that write CIFs.

aliaso_ : logical variable to control the use of data-name aliases for output items. If set to .true., preferred synonyms from the input dictionary may be output (see Section 5.4.7). The default is .false..

align_ : logical variable to control the column alignment of data values in loop_ lists output to a CIF. The default is .true..

esdlim_ : integer variable to set the upper limit of appended standard uncertainty (e.s.d.) integers output by pnumb_. The default value is 19, which limits standard uncertainties to the range 2–19.

globo_ : logical variable which if set to .true. will cause the output data block from pdata_ to be written as a global block.

nblanko_ : logical variable controls the treatment of output blank strings. If set to .true., output quoted blank strings will be converted to an unquoted period (i.e. to a data item of type null). Recall that CIFtbx treats an unquoted period or question mark as being of type null.

pdecp_ : logical variable controls the treatment of output decimal numbers. If set to .true., a decimal point will be inserted into numbers output by pnumb_ or pnumbd_. If set to .false., a decimal point will be output only when needed. The default is .false..

plzero_ : logical variable controls the treatment of leading zeros in output decimal numbers. If set to .true., a zero will be inserted before a leading decimal point. The default is .false..

pposdec_ : integer variable to set the column number (position along the line, counting from 1 at the left) of the decimal point for the next number to be output.

pposend_ : integer variable to set the position of the ending column for the next number or character string to be output. Used to pad with zeros or blanks.

pposnam_ : integer variable to set the starting column of the next name or comment to be output.

pposval_ : integer variable to set the position of the starting column of the next data value to be output.

pquote_ : character variable containing the quotation symbol to be used for the next string written.

saveo_ : logical variable is set to .true. for pdata_ to output a save frame, otherwise a data block is output.

ptabx_ : logical variable is set to .true. for tab stops to be expanded to blanks during the creation of a CIF. The default is .true..

tabl_ : logical variable is set to .true. for tab stops to be used in the alignment of output data. The default is .true..

xmlout_ : logical variable is set to .true. to change the output style to XML conventions. Note that this is not a CML (Murray-Rust & Rzepa, 1999) output, but a literal translation from the input CIF. The default is .false..

xmlong_ : logical variable is set to .true. to change the style of XML output if xmlout_ is .true.. When .true. (the default), XML tag names are the full CIF tag names with the leading underscore, _, removed. When .false., an attempt is made to strip the leading category name as well.

The following fatal messages are issued if the CIFtbx array bounds are exceeded. Operation terminates immediately. Array bounds can be adjusted by changing the PARAMETER values in ciftbx.sys. If the value of MAXBUF needs to be changed, the file ciftbx.cmv must also be updated.

However, the message is not `fatal', in the sense that the function bkmrk_ returns .false. to permit appropriate action before termination. This is effectively a fatal error for which recompilation with a larger value of MAXBOOK is necessary. However, this is usually the result of a logic error in the application, and the error has been made non-fatal to allow the programmer to insert debugging code, if desired. The application should clean up and exit promptly.

Dict_ must precede ocif_

Dictionary files must be loaded before an input CIF is opened because some checking occurs during the CIF loading process.

Illegal tag/value construction

Data name (i.e. a `tag') and data values are not matched (outside a looped list). This usually means that a data name immediately follows another data name, or a data value was found without a preceding data name. The most likely cause of this error is the failure to provide `.' or `?' for missing or unknown data values or a failure to declare a loop_ when one was intended.

Item miscount in loop

Within a looped list the total number of data values must be an exact multiple of the number of data names in the loop_ header.

Prior save-frame not terminated

Save-frame terminator found out of context. Save frames must start with save_framecode and end with save_. These messages will be issued if this does not occur.

Syntax construction error

Within a data block or save frame the number of data values does not match the number of data names (ignoring loop structures). This message should occur only if there is an internal logic error in the library. Normally the program will terminate on Item miscount in loop first.

Unexpected end of data

When processing multi-line text the end of the CIF is encountered before the terminal semicolon.

The following messages are generated by calls with invalid arguments.

Category 〈cat-code〉 first implicitly defined in cif

The category code in the DDL2 data name is not matched by an explicit definition in the dictionary. This may be intentional but usually indicates a typographical error in the CIF or the dictionary.

Data name 〈name〉 not in dictionary!

The data item name 〈name〉 was used in the CIF but could not be found in the dictionary.

Data block header missing

No data_ or global_ was found when expected.

Duplicate data item 〈name〉

Two or more identical data names 〈name〉 have been detected in a data block or save frame.

Exponent overflow in numeric input

Exponent underflow in numeric input

The numeric value being processed has an exponent that cannot be processed on this machine. If the string involved is not intended as a number, then surrounding it with quotes may resolve the problem.

Heterogeneous categories in loop 〈new cat-code〉 vs 〈old cat-code〉

Looped lists should not contain data items belonging to different categories. This error occurs if the category of a new data item fails to match the category of a prior data item. A special category (none) is used to denote item names for which no category has been declared. Warnings are not issued on this level for a loop for which all data items have no declared category.

Input line length exceeds line_

Non-blank characters were found beyond the value given by the variable line_. The default value for line_ is 80 (optionally increased to 2048 in CIFtbx 2.7 and later for CIF 1.1 compliance). The extra characters in column positions line_ + 1 through MAXBUF will be processed but the input file may need to be reformatted for use with other CIF-handling programs.

Missing loop_ items set as DUMMY

A looped list of output items was truncated with an incomplete loop packet (i.e. the number of items did not match the number of loop_ data names). The missing values were set to the character string `DUMMY'.

Numb type violated 〈name〉

The data item 〈name〉 has been processed with an explicit dictionary type numb, but with a non-numeric value. Note that the values `?' or `.' will not generate this message.

Quoted string not closed

Character values may be enclosed by bounding quotes. The strict definition of a `quoted-string' value is that it must start with a 〈wq〉 digraph and end with a 〈qw〉 digraph, where w is a white-space character blank or tab and q is a single or double quote, and the same type of quote mark is used in the terminal digraph as was used in the initial digraph. This message is issued if these conditions are not met.

Converted pchar_ output to text for 〈string〉

An attempt was made to write a string with pchar_ instead of ptext_, but the string contains a combination of characters for which ptext_ must be used.

ESD less than precision of machine

Overflow of esd

Underflow of esd

A call to pnumb_ or numb_ was made with values of the number and standard uncertainty (e.s.d) which cannot be presented properly on this machine. A bounding value of 0 or 99999 is used for the e.s.d.

Invalid value of esdlim_ reset to 19

In processing numeric output, a value of esdlim_ less than 9 or greater than 99999 was found. esdlim_ is then set to 19.

Missing loop_ name set as _DUMMY

Missing loop_ items set as DUMMY

In processing a loop_, a dummy string has been inserted for a missing header or value.

Output CIF line longer than line_

In outputting a line, the data exceed the limit specified in line_. This occurs only if a single data name or a value exceeds this limit.

Out-of-sequence call to end text block

The termination of a text block has been invoked before a text block has been started. This can only occur with irregular use of the CIFtbx routines rather than the standard interface routines.

Output prefix may force line overflow

A prefix string placed in prefx_ exceeds line_ less the allowed length of tags.

Prefix string truncated

A prefix string specified to prefx_ is longer than the maximum line length allowed. The prefix string is truncated and processing continues.

Aliases and names in different loops; only using first alias

If a DDL2 dictionary contains a loop of alias declarations, the corresponding data-name declarations are expected to be in the same loop. Only the first alias name is used.

Attempt to redefine category for item

Attempt to redefine type for item

If a DDL2 dictionary contains a category or type for a data item that conflicts with an earlier declaration, the first is used.

Categories and names in different loops

Types and names in different loops

If a DDL2 dictionary contains a loop of category or type declarations, the corresponding data-name declarations are expected to be in the same loop. Only the first category name or type is used.

Category id does not match block name

In a DDL2 dictionary, the save-frame code is expected to start with the category name. If a category name within the frame is not within a loop, it is checked against that in the frame code and a warning is issued if these do not match.

Conflicting definition of alias

A DDL2 dictionary contains a new declaration of a data-name alias which is in conflict with a previous alias definition. The first alias declaration is used.

Duplicate definition of same alias

A DDL2 dictionary contains a new declaration of an alias for a data name which duplicates a previously defined alias pair.

Item name 〈name〉 does not match category name

If category checking is enabled and the category assigned to an item name does not match the intial characters of the item name, this message is issued. This may indicate a typographical error or a deprecated item in the dictionary.

Item type 〈type-code〉 not recognised

The DDL2 dictionary type codes are translated to the DDL type codes `numb', `char' and `text'. If an unrecognized type code is found no translation occurs.

Multiple DDL category definitions

Multiple DDL name definitions

Multiple DDL type definitions

Multiple DDL related item definitions

Multiple DDL related item functions

DDL1 and DDL2 declarations for categories, data names, data types and related items are used in the same data block or save frame.

Multiple categories for one name

Multiple types for one name

A dictionary contains a loop of category or type definitions and an unlooped declaration of a single data name. The first category or type definition is used.

No category defined in block 〈name〉 and name 〈name〉 does not match

A DDL2 dictionary contains no category for the defined data item and it was not possible to derive an implicit category from the block name. This usually indicates a typographical error in the dictionary.

No category specified for name 〈name〉

A dictionary contains categories and category checking is enabled but no category is defined for the named data item.

No name defined in block

No name in the block matches the block name

These messages are issued if a dictionary save frame or data block contains no name definition or if all the names defined fail to match the block name.

No type specified for name 〈name〉

A type code is missing from a dictionary and type checking was requested in the dict_ invocation.

One alias, looped names, linking to first

A DDL2 dictionary may contain a list of data names and a single alias outside this loop. In this case, the correct name to which to link the alias must be derived implicitly. If the save-frame code matches the first name in the loop no warning is issued, because the use of the block name was probably the intended result, but if no such match is found this warning is issued.

A traditional Fortran style of programming is used in CIFtbx. Common blocks are declared to report and control the state of the processing. This allows argument lists to be kept short and avoids the need to create complex data structure types, but introduces extensive `programming by side-effects'. In order to reduce the impact of this approach on users, two different views of the common blocks are provided. The declarations in ciftbx.cmn are needed by all users. The more extensive declarations in ciftbx.sys, which include the same common declarations as are found in ciftbx.cmn and additional declarations used internally within CIFtbx, are provided for use in maintaining the library. Caution is needed in making internal modifications to the library to maintain the desired relationships among the actions of various routines and the states of variables declared in the common blocks.

Statements are written in the first 72 columns of a line, reserving columns one through five for statement labels and using column six for continuation. Approaches that would require the use of C libraries or non-portable Fortran extensions are avoided. For this reason, all the internal service routines are written in Fortran, all memory needed is preallocated with DIMENSION statements and a direct-access file is used to hold the working copy of a CIF.

Since CIFtbx does static memory allocation with DIMENSION statements, it is sometimes necessary to adjust the array dimensions chosen to suit a particular application. It may also be necessary to increase the storage allocated for individual tags to allow for unusually long ones.

The sizes of most arrays and strings used in CIFtbx that might require adjustment are controlled by PARAMETER statements in the files ciftbx.sys and ciftbx.cmv (the variable-declaration portion of ciftbx.cmn). The parameters are shown in Table 5.4.12.1.

These values can result in CIFtbx requiring more than a megabyte of memory. On smaller machines working with a small dictionary and simple CIFs, considerable space can be saved by reducing the values of NUMDICT and NUMBLOCK.

On the other hand, an application working with several layered dictionaries and large and complex CIFs with many data items and many loops in a data block might require a version of CIFtbx with larger values of NUMDICT, NUMBLOCK and, perhaps, of NUMLOOP.

The variables NUMPAGE and NUMCPP control the amount of memory to be used to buffer the direct-access file and the size of the data transfers to and from that file. Smaller values will reduce the demand for memory at the expense of slower execution.

The INCLUDE statement allows the statements in the specified file to be treated as if they were being included in a program in place of the INCLUDE statement itself. This simplifies the maintenance of common-block declarations and is an important tool in keeping code well organized. In CIFtbx, the INCLUDE statement is used to bring the statements in the files ciftbx.cmn and ciftbx.sys into programs where they are needed, and to simplify ciftbx.cmn and ciftbx.sys by using INCLUDEs of the files ciftbx.cmv and ciftbx.cmf. The file ciftbx.cmv contains the definitions of the essential CIFtbx data structures as common blocks, for inclusion in both ciftbx.cmn for user applications and in ciftbx.sys for the CIFtbx library routines themselves. Most compilers handle the INCLUDE statement, but, if necessary, a user may replace any or all of the INCLUDE statements with the contents of the indicated file. For example, the only non-comments in ciftbx.cmn are

This means that the file ciftbx.cmn could be replaced by a concatenation of the two files ciftbx.cmv and ciftbx.cmf.

CIFtbx makes some use of the ENDDO statement (as well as nested IF, THEN, ELSE, ENDIF constructs) to improve readability of the source code. Most compilers accept the ENDDO statement, but if conversion is needed then constructs of the form should be changed to where nnn is a unique statement number, not used elsewhere in the same routine.

The following routines are used internally by later versions of CIFtbx. If these names are needed for other routines, then changes in the library will be needed to avoid conflicts.

(a) Variable initialization:

Critical CIFtbx variables are initialized with data statements in a block data routine.

(b) Control of floating-point exceptions:

If a system requires special handling of floating-point exceptions, the necessary calls should be added to this subroutine.

(c) Message processing:

Error and warning messages are processed through these three routines.

(d) Internal service routines:

These routines are used internally by the library. The subroutine dcheck validates names against dictionaries. The subroutines eotext and eoloop are used to ensure termination of loops and text strings. The subroutines getitm, getstr and getlin extract items, strings and lines from the input CIF. The subroutine putstr writes strings to the output CIF.

(e) Numeric routines:

The routine ctonum converts a string to a number and its standard uncertainty. The subroutine putnum converts a number and standard uncertainty to an output string.

(f) String manipulation:

The subroutine detab converts tabs to blanks. The function lastnb finds the column position of the last non-blank character in a string. The function locase converts a string to lower case.

(g) Hash-table processing:

These routines are used to manipulate the internal hash tables used by the library.

All the externally accessible CIFtbx commands and variables terminate with the underscore character. This works well on most systems, but can cause occasional problems, because traditional Fortran does not include the underscore in the character set and some operating systems reserve the underscore as a system flag, for example to distinguish C-language library routines from those written in Fortran. If conversion is needed, and the local compiler allows long variable and subroutine names, then the simplest approach would be to make a local variant of CIFtbx in which every occurrence of underscore in a function, subroutine or variable name is changed to a distinctive character pattern (e.g. `CIF' or `qq'), but caution is needed, since there are many character strings used in the library that include the underscore. For example, in changing the variable loop_ to loopCIF, it would be a mistake to change the statement to

CIFtbx uses some function, subroutine and variable names longer than six characters to improve readability, but, in most cases, consistent truncation of all uses of a name to six characters will not cause any problems.

CIFtbx allows the user to read from one CIF while writing to another. The input CIF is first copied to a direct-access file to allow random access to desired portions of the input CIF. Since CIF allows data items to be presented in any order, the alternatives to the use of a direct-access file would have been to create memory-resident data structures for the entire CIF or to track position and make multiple search passes through the file as data items are requested. When programming for personal and laboratory computers with limited memory and which may lack virtual memory capabilities, assuming the availability of enough memory for large CIFs would greatly restrict the applications within which CIFtbx could be used. However, the disk accesses involved in using a direct-access file slow execution. When working on larger computers, execution speed can be increased at the expense of memory by increasing the number of memory-resident pages (see the parameter NUMPAGE above). If the number of pages times the number of characters per page (NUMCPP) is large enough to hold the entire CIF, the application will run much faster.

Direct reading of the input CIF, making multiple passes when data items are requested in a different order to that in which they are presented in the CIF, is only practical when the number of out-of-order requests is small and the applications will not need to be used as a filter, perhaps reading the output of another program `on-the-fly'. Since we cannot predict the range of applications and CIFs for which CIFtbx will be used, and direct reading could become impossibly slow, CIFtbx uses a direct-access file.

The processing of an output CIF is simpler than reading a CIF. The application determines the order in which the writing is to be done. No sorting is normally needed. Therefore CIFtbx writes an output CIF directly.

A CIF may contain data names in upper, lower or a mixture of cases. Internally, CIFtbx does all its name comparisons in lower case, using the function locase (see above) to convert. Good style, however, dictates the use of certain case combinations in certain names. Therefore CIFtbx does this lower-case conversion as needed, preserving the original case for whatever use may be desired. An application needing maximum speed and which does not need to preserve the cases in the original CIF might consider doing the case conversion once and removing the use of locase.

CIF does not care about white space. One blank or tab is equivalent to many blanks or tabs or empty lines in separating data names from values and values from one another. The internal routine getstr extracts the next white-space-delimited string, using getlin to deliver input lines from the direct-access file as required. Since Fortran does not provide dynamic memory allocation, this approach presents a problem with multi-line text fields. Rather than allocate a large fixed space that might not hold still larger text fields, the library delivers those strings one line at a time. As with case sensitivity, CIFtbx does white-space scanning repeatedly, keeping the original presentation (including tabs) available should an application need access to it. The author of an application needing maximum speed, not needing the original presentation and wishing to conserve disk space might wish to modify the operation of CIFtbx to remove all comments and compress all separating white space to single blanks or line terminators in an initial sweep.