International
Tables for Crystallography Volume C Mathematical, physical and chemical tables Edited by E. Prince © International Union of Crystallography 2006 |
International Tables for Crystallography (2006). Vol. C. ch. 8.2, p. 691
|
Entropy maximization, like least squares, is of interest primarily as a framework within which to find or adjust parameters of a model. Rationalization of the name `entropy maximization' by analogy to thermodynamics is controversial, but there is formal proof (Shore & Johnson, 1980, Johnson & Shore, 1983
) supporting entropy maximization as the unique method of inference that satisfies basic consistency requirements (Livesey & Skilling, 1985
). The proof consists of discovering the consequences of four consistency axioms, which may be stated informally as follows:
|
The term `entropy' is used in this chapter as a name only, the name for variation functions that include the form , where
may represent probability or, more generally, a positive proportion. Any positive measure, either observed or derived, of the relative apportionment of a characteristic quantity among observations can serve as the proportion.
The method of entropy maximization may be formulated as follows: given a set of n observations, , that are measurements of quantities that can be described by model functions,
, where x is a vector of parameters, find the prior, positive proportions,
, and the values of the parameters for which the positive proportions
make the sum
where
and
, a maximum. S is called the Shannon–Jaynes entropy. For some applications (Collins, 1982
), it is desirable to include in the variation function additional terms or restraints that give S the form
where the λs are undetermined multipliers, but we shall discuss here only applications where λi = 0 for all i, and an unrestrained entropy is maximized. A necessary condition for S to be a maximum is for the gradient to vanish. Using
and
straightforward algebraic manipulation gives equations of the form
It should be noted that, although the entropy function should, in principle, have a unique stationary point corresponding to the global maximum, there are occasional circumstances, particularly with restrained problems where the undetermined multipliers are not all zero, where it may be necessary to verify that a stationary solution actually maximizes entropy.
References



