The relation that can say no

Three phases that do not move when the origin does establishes the triplet and names what it leaves undone:

The triplet is the smallest invariant and not the only one. … Quartets are useful for a reason triplets are not: they can be negative. … Nothing here computes quartets.

That is a debt with a reason behind it, and the reason is worth stating before the computation. Cochran’s distribution for a triplet is proportional to exp(κ cos Φ) with κ positive, which is peaked at zero and nowhere else. A strong triplet says the sum is very near zero and a weak one says it is somewhat near zero. No triplet ever says anything but zero.

So a phase set can satisfy every triplet relation it has and still be wrong, and nothing in the relations detects it. The measurement makes the point flatly: over the two hundred and twenty-six triplets among the strong reflections of a small structure, the mean cosine is +0.773 and four per cent of them are negative — the four per cent being the tail of a distribution whose peak is at zero, not a relation pointing the other way.

Four in the sum, three outside it #

A quartet is four reflections whose indices add to nothing, so the sum of their phases is unmoved by a shift of origin — the same property that makes a triplet an invariant, for the same reason: every phase changes by 2π h·t, and the shifts cancel when the indices do.

What is different is what decides its distribution. A triplet’s is decided by the three amplitudes in it. A quartet’s is decided by the four in it and by three others: the sums of the four reflections taken in pairs, h+k, h+l and k+l, which are reflections the experiment also measured and which are not in the phase sum.

That is the whole mechanism. The quartet’s estimate is not made of the quartet.

The sign turns #

Binning every quartet by the mean strength of its three cross terms gives a curve that crosses zero.

Where the cross terms are strongest the mean cosine is +0.873 and not one quartet in the bin has a negative cosine — the quartet behaves exactly like a triplet, peaked at zero, and adds nothing a triplet could not have said.

Where they are weakest it is −0.698, and ninety-three per cent of the quartets in that bin have a cosine below nought. The distribution has moved from a peak at zero to a peak at π.

The four reflections in the sum are equally strong in every bin, because the sum runs only over reflections above the same cut. What changes across the figure is three reflections that are not in the sum. So a quartet’s estimate is a statement the four amplitudes cannot make, and a procedure that used only the reflections it was phasing would never find it.

A sum of three phases and a sum of four #

The two invariants are worth setting side by side as arithmetic, because the difference in what they can say comes from one term.

A triplet’s distribution is proportional to exp(κ cos Φ) with $\kappa = 2|E_1E_2E_3|/\sqrt{N}$, and κ is a product of amplitudes, which is never negative. So the exponent is largest at Φ = 0, always, and the only thing the amplitudes decide is how sharply.

A quartet’s has the same shape with an exponent that carries the cross terms in it, and the combination can be negative. When it is, the same exponential is largest at Φ = π rather than at Φ = 0 — the identical functional form, read at the other end. Nothing about the distribution changed except the sign of a coefficient.

That is why quartets are the natural next object rather than an unrelated trick. The statistical theory is one theory, and the quartet is the first input for which one of its coefficients can turn.

The triplet’s curve is the one the subject opens with, and it has no negative part anywhere. What the next figure adds is a second variable the triplet does not have.

Why weakness is information #

The direction of the effect is worth pausing on, because it is the opposite of what a reader expects from everything else in the subject.

Everywhere else the strong reflections carry the content. Cochran’s κ is proportional to the product of three amplitudes, so a triplet among weak reflections says almost nothing; the tangent formula works with the strongest few hundred and ignores the rest; a structure’s strong reflections are the ones its atoms agree about.

A negative quartet reverses that. The content is in three reflections being weak, and weak means near the noise. A reflection whose intensity is not distinguishable from background is, in a negative quartet, the thing that carries the information — because a cross term being weak says the structure has a particular kind of interference in it, and that interference is what pushes the phase sum to π.

So a quartet is a statement about a near-zero measurement, and near-zero measurements are the ones an experiment reports least reliably. That is the practical difficulty of the method and it is a difficulty of exactly the shape the physics predicts: the relations that say the most are the ones built on the data that are worst.

What a relation that can contradict is for #

A procedure that phases a structure has two problems and the relations solve only one of them.

Finding a solution is what the triplets do, badly: the tangent formula has the answer as a fixed point and cannot reach it from anywhere else, so a procedure restarts from many places and hopes.

Telling a good answer from a bad one is the other, and it is where the subject spent twenty years. Every phase set produced by a triplet-driven iteration satisfies its triplets — that is what the iteration did — so the triplets cannot rank the results. A figure of merit computed from them is, in the word that essay uses, internal: it is computed from the same relations the phases were built to satisfy.

A negative quartet is external to that iteration. It is built on reflections the iteration did not phase, it makes a statement in the opposite direction, and a phase set that satisfies its triplets can be caught violating it. That is why the figures of merit that work in practice are built on negative quartets and why the uniform phase set — the trivial solution every iteration drifts towards — violates them badly.

The same essay lists the three things that make a multisolution search work and says only the first is computed there. The second is this one, and it is computed here.

Where a negative quartet comes from, physically #

The arithmetic says the sign turns and it is worth having a picture of why, because the picture explains the practical difficulty as well.

The cross term h+k is the reflection whose amplitude measures how much the structure’s Fourier components at h and at k reinforce each other. If the structure had an arrangement making both strong and their sum strong too, the three would be in phase and the quartet’s four phases would be free to sit at zero. A cross term being weak says the reverse: the components at h and at k are strong individually and their product contributes nothing at h+k, which is an interference — they are arranged so as to cancel there.

A cancellation is a constraint, and the constraint is what forces the phase sum away from zero. There is no other place for it to go: the invariant takes one value, the distribution is symmetric about it, and the value that a cancellation is consistent with is π.

So a negative quartet is a measurement of destructive interference among reflections, read as a statement about phases. That is why its information cannot be in the four amplitudes — destructive interference is a fact about a reflection that is not one of the four — and why it is in the data anyway.

And it is why the practical difficulty is what it is. The stronger the cancellation the more the quartet says, and a perfect cancellation is a reflection with no intensity at all — which an experiment records as a number smaller than its own error bar, and a systematically absent reflection records the same way.

What the numbers are, and what a real experiment has #

The structure is twelve atoms in a plane cell with no symmetry, the reflections run to twelfth order, and the quartets are every one whose four reflections and three cross terms are all inside the window. There are nine hundred of them.

The bins are by the mean of the three cross terms rather than by the smallest, and that is a choice: a quartet with two strong cross terms and one weak one is estimated differently from one with three moderately weak, and the mean does not separate those. A more careful treatment uses all three amplitudes in the distribution, which is what Hauptman’s formula for the quartet does and what is not reproduced here.

And no noise is added anywhere. The cross terms are computed exactly from the atomic positions, so a cross term this page calls weak has an amplitude that a real experiment would report as some number plus an error of similar size. The whole difficulty of negative quartets in practice is that distinguishing a weak reflection from a missing one is hard, and nothing in this computation faces it.

The same shape, in two later relations #

A relation that can contradict is a shape rather than a fact about quartets, and it appears twice more in the subject with different apparatus.

The solver that knows no symmetry works by reversing the sign of the density wherever it is below a threshold. That step says this part of the map is wrong — a statement in the opposite direction from anything the amplitudes say, and the only one in the loop. The essay’s own refusal is exactly this point: run the loop with the flipping switched off and the agreement figure is perfect for ever while the map is noise.

Whether there is a centre is a statistic is the same shape asked of symmetry rather than of phases: the intensity statistics of a centrosymmetric structure differ from those of a non-centrosymmetric one, and the test can come out either way. A test that could only ever say “centrosymmetric, weakly or strongly” would be no test.

The pattern is that a procedure needs at least one statement its own output cannot manufacture. Triplets are manufactured by the iteration that uses them; a negative quartet is not, because the iteration never touched the reflections it is built on.

Where the exactness stops #

Computed here. The structure factors of a twelve-atom structure exactly, their normalised amplitudes, every triplet and every quartet among the reflections above a cut, the three cross terms of each quartet, and the mean cosine of the phase sum binned by cross-term strength. Three checks that can fail: the triplets’ mean cosine must be positive, the quartets’ must fall as the cross terms weaken, and the weakest bin’s must be negative.

The phases are known. Every phase in the computation comes from the atomic positions, so what is measured is the distribution the relations are supposed to have and not the recovery of anything. A procedure using these relations is the formula that has the answer already, and is not run again here.

One structure. The distributions are properties of a random arrangement of equal atoms, and the theory says so; measuring them on one arrangement is evidence about that arrangement. The direction of the effect is large — from +0.87 to −0.70 — and the direction is what the essay claims.

The quartets outnumber the triplets by four to one. Nine hundred against two hundred and twenty-six, from the same reflections, because choosing four from a list has more ways than choosing three. That is part of why the higher invariants were worth the trouble and part of why they are expensive: a procedure that evaluates every quartet at every cycle is doing a great deal more arithmetic than one that evaluates every triplet.

And the cut is a cut. Reflections below an amplitude of 1.2 are excluded from the sums, which is the usual practice and is a threshold. Lowering it admits weaker quartets whose distributions are flatter; the effect measured is between bins at one cut rather than across cuts.

The first refusal is a quartet estimated from its own four amplitudes alone, which is the natural thing to try and gives the same answer for quartets whose true estimates are at opposite ends of the range. It is the mistake this whole page exists to catch.

A count of what each invariant is worth #

One number is worth extracting from the bins, because it says how much of the gain is real.

Of the nine hundred quartets, the bins with a mean cross term below three quarters hold a hundred and fifty between them, and those are the ones saying anything a triplet could not. The rest — seven hundred and fifty — repeat what the triplets already say, more weakly, because a quartet is a product of four amplitudes over N rather than three and is correspondingly less sharp.

So a sixth of the quartets carry the whole of the new information, and finding them means computing all of them. That is the economics of the method in one sentence, and it is why the negative quartets were a computational development as much as a theoretical one: the theory was available in the 1950s and the programs that could afford it arrived in the 1970s.

It also says what a figure of merit built on them is doing. It is not summing over the invariants; it is summing over the ones with weak cross terms, which is a small and carefully chosen subset, and a phase set is being asked to violate none of them.

Who found the negative ones #

Herbert Hauptman and Jerome Karle built the probabilistic theory of the invariants through the 1950s, and their monograph of 1953 is where the higher invariants first appear. The negative quartet — the recognition that the estimate turns when the cross terms are weak, and that this is usable — is Hans Schenk’s, in a series of papers from 1973, and it changed what a multisolution search could do: a figure of merit built on negative quartets ranks solutions in a way the triplets cannot, and the programs of the 1970s and 1980s were built round it.

Hauptman and Karle shared the Nobel Prize for Chemistry in 1985. The citation is for the triplets and the tangent formula, which is fair; what made the method usable on structures of a hundred atoms was the quartets, and the difference between the two is the difference between a relation that says where to go and one that says where not to.

The method’s own successor uses neither. Charge flipping has no invariants in it at all, and the thing that replaced the quartet’s power to contradict is the map itself: a density that goes negative is wrong, which is a statement the amplitudes cannot make either.

Still open: the estimate as a formula #

What is measured here is a trend and what the theory supplies is a number.

Hauptman’s quartet distribution gives the estimate of the phase sum from all seven amplitudes at once — the four in the sum and the three cross terms — in closed form, and it is not reproduced above. Computing it and comparing it with the measured cosines bin by bin would be the same check Cochran’s distribution already gets, and would turn a trend into an agreement.

And the quintets are the next term. Five reflections adding to nothing give a five-phase invariant with more cross terms still, and the same logic applies: more of them are weak, so more of the estimates are negative. Whether the extra information is worth the cost is a question the subject answered in practice — quintets were used and were mostly not worth it — and the arithmetic that would say why is a comparison of how much each order of invariant constrains, which nothing here computes.