The relation that is an equality

Three phases that do not move when the origin does says the thing all of these relations rest on, and says it about a relation rather than about a structure:

This is a probability, not a rule, and the distinction is the subject’s whole difficulty. A relation right nine times in ten is useless applied once and decisive applied ten thousand times, and the art is in the bookkeeping that turns many weak statements into one strong one.

Every relation since has been of that kind. Cochran’s distribution for a triplet is a probability; the tangent formula is a weighted average of probabilities; a negative quartet is a probability pointing the other way. The bookkeeping is the subject.

One relation is not a probability. It is an equality, it is exact, and it is the thing all the others are approximations to.

Squaring a sum of atoms gives a sum of atoms #

The whole derivation is one observation about a density.

A crystal’s electron density is a sum of atoms. Square it. If the atoms are resolved — far enough apart that no two overlap appreciably — then the square is a sum of the individual atoms squared, in the same places, and a squared atom is a narrower atom of the same shape. So $\rho^2$ is a structure with the same positions and a different atomic form factor.

Two structures with the same positions have proportional structure factors, with the ratio a function of resolution alone. And the transform of a product is the convolution of the transforms. Put those together:

$$F(\mathbf h) \;=\; \theta(\mathbf h)\sum_{\mathbf k} F(\mathbf k)\,F(\mathbf h - \mathbf k),$$

with θ the ratio of the atom’s form factor to the squared atom’s. An equality, holding for every reflection at once, with no probability anywhere in it.

θ is not a constant, and what it is #

The factor is often written as though it were a scale, and it is not: a squared Gaussian is narrower in real space than the Gaussian, so its transform falls off more slowly, so θ falls with resolution.

For atoms with form factor exp(−B|h|²) the squared atom has exp(−B|h|²/2), and θ is therefore proportional to exp(−B|h|²/2). The measurement fits the logarithm of the ratio against the squared index and finds a slope of −0.0169 against a predicted −0.0175, with a correlation of 0.994 over sixty reflections.

And the ratio is real and positive. Its phase departs from nought by at most a quarter of a radian across the same sixty, which is the whole content of the identity for these purposes: an equality between complex numbers is an equality of phases as well as of magnitudes, so Sayre’s equation relates every phase to every other exactly.

Where all the probabilities come from #

Take the equality and throw away almost all of it.

Keep one term of the sum — the single product F(k)F(h−k) — and the equality becomes an approximation saying that F(h) is roughly proportional to that product. Take phases on both sides and it says

$$\varphi(\mathbf h) \;\approx\; \varphi(\mathbf k) + \varphi(\mathbf h - \mathbf k),$$

which is the triplet relation. Cochran’s distribution is the statement of how good that approximation is, given that the remaining terms of the sum are a large number of complex numbers of random phase whose total is a random walk.

Keep all the terms but replace the exact amplitudes with their expectations and the result is the tangent formula: the phase of F(h) estimated as the phase of the sum, weighted. The formula that has the answer already is that estimate iterated, and its awkward property — that the true phase set is very nearly a fixed point — is Sayre’s identity showing through. Hand the iteration the true phases and the sum is the true structure factor, exactly, because the identity says so.

So the subject’s shape is not three unrelated relations. It is one equality, read with different amounts of it thrown away, and what is thrown away is what turns an equality into a distribution.

The sign relation is the centrosymmetric case of the triplet and is the form Sayre’s paper itself works in: with all phases nought or π a phase becomes a sign, and the one-term truncation says the product of three signs is positive. That statement is right about nine times in ten and wrong about one, and the whole apparatus of the subject exists to make the nine outvote the one.

Set it against the identity it came from and the loss is visible. The identity determines the phase exactly; keeping one term of its sum leaves a statement with a probability attached; and everything the tangent formula and the multisolution search do is an attempt to get back some of what the truncation threw away.

Why an equality is useless #

An exact relation among the unknowns ought to be worth more than a probabilistic one, and it is not, for a reason worth stating plainly.

Sayre’s equation is N equations in N unknowns and every one of them involves every unknown. The phase of F(h) is determined by the phases of all the others; the phase of every other is determined by the rest. It is a fixed-point condition on the whole phase set with no starting point anywhere in it, and the set of solutions includes the structure, its mirror image, every origin shift of both, and — since the equation is satisfied by F ≡ 0 — the trivial one.

That is the same difficulty the tangent formula has, stated one level up: having a fixed point and finding it are different problems. Sayre’s identity is the cleanest possible statement of the fixed point and supplies nothing whatever about finding it.

A probability is worse as a statement and better as a tool, because a distribution can be sampled, ranked and combined, and an identity can only be satisfied or not. The whole of direct methods is the decision to work with the weaker statements.

The one-term truncation, counted #

It is worth being quantitative about how much is thrown away, because the number explains why the truncation is as good as it is.

The sum runs over every reflection in the window — for a window of eighteen orders that is thirteen hundred and sixty-nine terms. Keeping one of them and calling the rest noise is discarding more than ninety-nine point nine per cent of the statement.

What makes that survivable is what the discarded terms are. Each is a product of two structure factors, and their phases are effectively unrelated to one another, so their sum is a random walk of thirteen hundred steps and grows as the square root of the count rather than as the count. The one retained term grows as a product of two amplitudes and the noise grows as a square root, so a term built of two strong reflections can stand out above the rest.

That ratio is Cochran’s κ, and the $\sqrt{N}$ in it is the number of atoms rather than the number of terms — the two are related because both count how many independent contributions make up a structure factor. So the same square root appears in the truncation error and in the relations’ weakening with size, and it is the same square root.

A truncation that keeps one term in a thousand and is right nine times in ten is not a lucky accident. It is a random walk being smaller than a product, which is the one arithmetical fact the whole method runs on.

What “resolved” is doing #

The word in the derivation that carries the assumption is resolved, and the measurement shows exactly how much work it does.

Squaring a sum of atoms gives a sum of squared atoms plus cross terms, one for every pair, each a small bump halfway between two atoms. Those are what the derivation discards, and they are negligible only while the atoms are sharp relative to their separation.

At the sharpest setting the ratio’s phase is sixteen thousandths of a radian and the fitted slope is the predicted one to three places. Broaden the atoms fourfold and the phase reaches one and a half radians — a quarter turn and more — and the fitted slope is two thirds of what it should be. The identity does not degrade gracefully; the cross terms are a density in their own right and they carry phases of their own.

That is where the resolution limit of direct methods comes from, and it is a different statement from the one about the number of atoms. The $\sqrt{N}$ in Cochran’s κ says the relations weaken as a structure grows; this says they stop being true at all when the data will not resolve the atoms. Both limits are real and they bite at different places.

What it says about a map, rather than about phases #

There is a second reading of the identity that has nothing to do with solving anything, and it is the one refinement uses.

Written the other way round, the equation says: take a density, square it, and transform; the result is the original transform times a known function. A map that does not satisfy that is not a map of resolved equal atoms — it has ripples, or noise, or a wrong scale, or atoms where there are none.

So the identity is a test a map can fail, and the amount by which it fails is a number. A map computed from measured amplitudes and estimated phases can be squared, transformed, and compared with itself; the discrepancy measures how far the estimated phases are from a density made of atoms.

That is the same idea as reversing the sign of everything below a threshold, with a sharper statement of what is wrong and a correspondingly narrower range of structures it applies to. Flipping needs only positivity; Sayre’s test needs atomicity, which is much more, and gets much more in return.

Neither is a probability. Both are things a map can be caught failing, which is what a procedure needs and what the invariants cannot supply.

Where the exactness stops #

Computed here. The structure factors of an eight-atom structure with Gaussian atoms, the convolution of those structure factors over a window of eighteen orders, and the ratio of the two at sixty reflections; the fit of the ratio’s logarithm against the squared index; the departure of the ratio’s phase from nought; and the same at six atomic widths. Three checks that can fail, including that the fitted slope is minus half the width.

The window is finite and the sum is not. Sayre’s equation sums over the whole of reciprocal space, and a computation sums over a window. The window is large enough that the terms outside it are negligible for these atoms, which is why the fit is as good as it is; for sharper atoms the same window would not be enough, and the smallest width in the table is where the truncation begins to show.

Point atoms would not work. A delta function squared is not a function, and the identity for point atoms exists only as a limit; the computation uses Gaussians throughout and the factor θ is theirs.

Sixty reflections, not all of them. The ratio is computed for the reflections out to fifth order in each index, where the amplitudes are large enough that the ratio is well conditioned. At high resolution both sides of the identity are small and their ratio is a quotient of small numbers, which is a measurement problem rather than a failure of the identity.

One structure and one seed. The fit is of a particular arrangement of eight atoms. The identity is an algebraic consequence of resolvedness rather than a statistical statement, so it should hold for any such arrangement; what varies between arrangements is how resolved they happen to be, which is what the second table sweeps by another route.

And no structure is solved. The identity is verified, its truncations are named, and the procedure that uses them is the formula that has the answer already. Nothing here recovers a phase from an amplitude.

Every row of that table is a mean over many relations, because a mean is the only thing a probability supports. Nothing in this page’s identity needs a mean: it holds reflection by reflection, and the numbers quoted are departures rather than averages.

The last refusal is Sayre’s identity offered as a way of finding the phases. It is the mistake the equality most invites: it is exact, it involves nothing but measured amplitudes and unknown phases, and it looks like a system waiting to be solved. It is a system, and it has the answer among its solutions along with the answer’s mirror image and every shift of both and nothing at all.

Three statements, and what each can be asked to do #

Setting the three kinds of relation side by side is the shortest summary of what the subject is.

An equality — Sayre’s — determines every phase from all the others and supplies no way in. It can be checked, it can be used as a test on a finished map, and it cannot be solved.

A probability — Cochran’s triplet — determines nothing and is true often enough to be combined. It can be sampled, weighted and iterated, which is what makes a procedure possible at all, and it can never contradict itself.

A probability that can turn — a negative quartet — is the same kind of statement with a sign that depends on data outside the relation. It can rank two candidate answers, which neither of the others can.

A working method needs all three: something to iterate with, something to rank by, and something to check against. The subject took thirty years to assemble them and they were all present by 1975, in the order equality, probability, sign.

Who wrote it down #

David Sayre published the equation in 1952, in a two-page paper in Acta Crystallographica, and the year matters: Hauptman and Karle’s probabilistic theory arrived immediately after, and Cochran’s distribution in 1955. The exact relation came first and the approximations came second, which is the reverse of the order these pages present them in and the reverse of how the subject is usually taught.

Sayre’s own paper is careful about what it has. It notes that the equation holds for equal resolved atoms, it works an example, and it says the relation is unlikely to be useful directly — which turned out to be right for thirty years and then not. The equation came back in the 1990s as the basis of iterative phase retrieval for non-periodic objects, where the density is not a sum of identical atoms at all and the constraint used is a support rather than atomicity; and it is the ancestor of the alternation between two spaces that charge flipping is the crystallographic member of.

That is worth carrying, because it is the opposite of the usual account. The relation that looked useless for being exact is the one that generalised.

Still open: the identity with unequal atoms #

The derivation needs the atoms to be equal as well as resolved, and real structures are not.

With two kinds of atom the square is not proportional to the density, because a heavy atom squared is heavier relative to a light one squared than it was before. So θ is no longer a function of resolution alone: it depends on which atoms are where, which is the unknown. The standard treatment replaces the amplitudes by normalised ones — dividing out the scattering of the actual cell contents, which the average intensity in a resolution shell supplies — and that is what makes the relations usable on a real structure, and it is an approximation rather than a repair.

How much of one is a measurement this page could make and does not: build a structure with two atomic species, compute the ratio Sayre’s identity gives, and see whether its logarithm is still a straight line. The prediction is that it is not — that the scatter about the line measures the spread of atomic numbers — and if that is right then the scatter is itself an estimate of how heterogeneous a cell is, computable from amplitudes alone.

The other direction is the one the 1990s took. An identity that needs atomicity has an analogue for any constraint that squares well, and finding which constraints those are is the question the whole family of iterative phase-retrieval methods is built on.