What refinement takes back from a wrong model

The molecule size that hides a disorder compared two disorder models sharing an occupancy as ideal averaged structures. Each model put its partial atoms on the loci its kept subgroup fixes, and the residual between the two came out between eight and twenty-four per cent for a ten-atom molecule, falling as one over the molecule’s size. It ended by naming what that comparison left out. A crystallographer never compares two ideal structures. They refine one model against data that came from the crystal, and a wrong model with adjustable positions and displacement parameters can absorb part of the difference. That essay could not say how much, and it could not say whether the practical diagnostic works: a partial atom on the wrong locus, refined, is supposed to show a suspiciously large and anisotropic displacement parameter.

This essay runs the refinements. Every pair of models sharing an occupancy at the nine usable site symmetries is taken in both directions: data are made from one model’s averaged structure, and the other is refined against them by least squares. The wrong model takes back more than half of the ideal difference, typically, and all of it in forty-seven cases. Which cases is decided by a single geometric fact. Adding noise does not rescue the old boundary at sixty atoms, because the test that decides is a sum over every reflection. And the diagnostic works through the size of the displacement parameter and not through its shape.

A refinement with only the parameters a refinement has #

The comparison is the narrowest one available, and it is worth being exact about what the wrong model is allowed to do. The data are the amplitudes of model A’s averaged structure on a cubic cell of side twelve ångströms, to a resolution of two ångströms, about four hundred and sixty reflections. Model B is then refined against them by Levenberg–Marquardt, minimising the sum of squared differences between observed and calculated amplitudes, with every parameter a crystallographer would refine and no others.

B gets an overall scale, and the position and displacement parameter of the general atom, which both models share. For each locus B keeps, it gets the partial atom’s position on that locus: one coordinate on a rotation axis, two in a mirror plane. So the atom’s site symmetry is respected exactly as a refinement program respects it, and no atom can leave its special position to find a better fit. Each partial atom also gets a displacement parameter with two components, one along its locus and one across it. Occupancies are held at the values the composition fixes, which is the constraint every disorder refinement imposes. A second variant frees them.

Model A is refined against the same data in the same way, so every comparison is between two fitted structures. Least squares from a single start can stall, and in a trial refinement it did: started at the ideal positions it stopped at a residual of seven per cent with one displacement parameter run up past four hundred, while starts with the partial atoms displaced or their displacement parameters raised reached half that residual every time. So the wrong model is refined from four starts and the best kept, which is what a crystallographer who suspected the model would do.

The single refinement shows the mechanism. The partial atoms in mirror planes slide a few hundredths of a cell within their planes, towards where the true model’s partial atoms sit. The displacement parameters do the rest. Some run up to ten or eighteen times their true value, smearing an atom over a region where the true structure has density spread across several orientations. Others run down to the floor, where the refinement wanted to make them negative, which a refinement program flags as physically impossible. The general atom does not move at all, since it is already right.

More than half, typically #

The picture at the head of this essay is the whole survey, 462 directed pairs. The median ideal residual is 16.9 per cent and the median residual after refinement is 7.0 per cent. Refinement absorbs 57 per cent of the difference at the median, 21 per cent at the tenth percentile and 97 per cent at the ninetieth. The earlier essay’s warning that the refined gap would be “smaller than the numbers here by a factor nothing on this page predicts” has its factor: about two and a half at the median, and anything from a quarter to all of it.

The spread matters more than the median. A crystallographer facing two candidate models cannot read off how different they will look after refinement from how different they look as ideal structures. Pairs twenty per cent apart as ideal structures end anywhere from nought to nearly twenty per cent apart. The horizontal band of points on the floor of the figure is the most striking feature. It runs out to ideal residuals of forty per cent, which is two models no one would confuse on paper that a refinement makes indistinguishable.

Containment decides who can fit exactly #

The points on the floor share a property that can be read off the site’s symmetry before any refinement. Call the fitted model’s loci the axes and planes on which its partial atoms may move. Every pair refined to an exact fit is one in which every locus of the true model lies inside some locus of the fitted one: an axis inside an axis or inside a mirror plane, a plane inside a plane, allowing the site’s own operations to move one onto the other.

The reason is almost a tautology once stated, and it is the kind of tautology worth having. A partial atom constrained to a mirror plane can slide anywhere in the plane, including onto a rotation axis lying in it. So a model keeping mirrors can put its partial atoms exactly where a model keeping the axes in those mirrors put its own. A model keeping only the axes cannot reach off them into the planes. Containment is the condition that the wrong model’s parameter space contains the right structure, and without it no refinement can fit exactly. With it an exact fit is possible, and in forty-seven of the 163 contained pairs the refinement finds it. None of the 299 uncontained pairs gets closer than six per cent of its ideal residual.

The 116 contained pairs that do not fit exactly are held back by the occupancies. The true model spreads its budget of scattering equally over its own loci, the fitted model over its own, and when the two models have different numbers of loci the fitted model can put atoms in the right places with the wrong weights.

Freeing the occupancies removes that obstacle, and in twenty-two of twenty-eight sampled contained pairs the wrong model then reproduces the data exactly. The remaining six are a warning about least squares rather than about symmetry. A model that can fit exactly is not thereby a refinement that will. With occupancies free the problem has more parameters and more local minima, and some of those refinements stall at residuals above the fixed-occupancy ones, from every start. So the containment condition says when a wrong model is dangerous. Whether a particular refinement falls into it depends on where it starts, which is the one thing a crystallographer controls and cannot see the consequences of.

Finer data help some pairs and not others #

A wrong model has few parameters and every added reflection is one more equation they must satisfy, so the natural expectation is that higher resolution resists absorption. It does, unevenly.

For the pair drawn at 4/mmm and the pair at m3̄m the absorbed share falls from 77 and 93 per cent at three ångströms to 25 and 32 per cent at 1.2 ångströms. The extra shells carry detail that the wrong model’s smeared partial atoms cannot reproduce. For the pair at mmm, two orientations of a model keeping 2/m, the share stays at 55 per cent whatever the resolution. There the wrong model’s partial atoms sit on loci of the same kind as the right model’s, turned, and a refinement puts them nearly where the right ones are at every scale. The earlier essay found the ideal difference nearly flat across resolution shells. After refinement it is not flat, and the direction it moves is decided pair by pair by the geometry of the loci, not by any rule about resolution.

The whole data set still decides #

The earlier essay set a boundary: above about sixty atoms in general positions the ideal difference falls under a three per cent measurement error, and “the data stop choosing”. That criterion counted reflections whose difference individually exceeded three standard deviations. A refinement is judged by a different test, the sum over every reflection, and the two can disagree.

With Gaussian noise of three per cent of the mean amplitude, both models are refined against the same noisy data and compared by the likelihood ratio: the difference between their sums of squares in units of the noise variance, with a decision declared when it exceeds 10.83, the 0.1 per cent point of $\chi^2$ with one degree of freedom. That threshold is a convention and stated as one. At sixty-four atoms the likelihood ratio still decides fifteen of the eighteen pairs, with a median of 92. The ideal residual there is 2.4 per cent, under the noise. The wrong model refines to 2.6 per cent and the right one to 2.3, and a difference of three tenths of a per cent in the residual, summed over four hundred and sixty reflections, is decisive. No single reflection shows it and the whole data set does.

That overturns the earlier boundary in one direction and leaves a more important one standing. The noise here is purely random, with a known variance, and against such noise the aggregate test is extremely powerful. Real diffraction data have systematic errors of a few per cent — absorption, extinction, a scale that drifts across the detector — which no amount of data averages away. A three per cent systematic error is exactly the size of difference the likelihood ratio relies on at sixty-four atoms. So the molecule size at which the data stop choosing is set by the systematic error of the experiment and not by its counting statistics. The earlier estimate of sixty atoms is roughly right for the wrong reason, and a better experiment moves it out much further than a counting argument would suggest.

One pair is never decided at any size: a model keeping 4̄ against one keeping mm2 at a cubic site, where the second model’s loci contain the first’s. Its likelihood ratio is about minus two, noise and nothing else. The wrong model fits the data from the right one as well as the right one does, at every size of molecule. Containment decides this one before any noise enters.

The tell is the size of B, not its shape #

The practical diagnostic for a wrong disorder model is a partial atom whose displacement parameter refines to something implausible. The refinements test whether that works, and which part of it.

Refined against the right data, the right model’s partial atoms keep displacement parameters no larger than seventeen, against a true value of three. The noise inflates them a little because a partial atom at a thirty-second of an atom is poorly determined. The wrong model’s largest displacement parameter exceeds thirty, and twice the right model’s, in fifteen of the eighteen pairs, often reaching hundreds or thousands: the refinement has effectively deleted an atom it has no right place for. In every pair where the wrong model’s loci do not contain the true ones, the size of B gives it away. The three pairs where it stays quiet, between five and nine, are all contained. There the wrong model can put its atoms in the right places and has no need to smear them.

The anisotropy is a much worse guide, and the reason is the same partial occupancy. With so little scattering in each partial atom, the right model’s own ratio of displacement along its locus to across it runs past a hundred and fifty in some pairs. It is noise, but noise shaped like the diagnostic. Several wrong models have ratios below that. A crystallographer who distrusted every partial atom with a strongly anisotropic displacement would reject right models, and one who trusted a modest anisotropy would accept wrong ones. What separates the two models in these refinements is how large the displacement is, not how elongated.

What can be known before refining #

The results sort into things a crystallographer can know in advance from the site symmetry alone and things only a refinement reveals, and the first group is larger than one might expect.

Whether a wrong model can fit exactly is decided before any data exist. List the loci each candidate keeps, which the census of orientations and the count of models at one occupancy already supply, and ask whether one candidate’s loci contain another’s. If they do, the containing model is the dangerous one: refined against data from the contained model, it has the right structure within reach, and the residual cannot be trusted to reject it. The asymmetry matters. The model keeping more loci of higher dimension, mirror planes rather than axes, is the one that can impersonate the other and not the other way round. So a refinement that settles on the model with more freedom is the one that should be checked against the model with less.

Whether a refinement will actually find the impersonation is not decided in advance. Some contained pairs fit exactly and some stall, and freeing the occupancies changes which. That part is least squares, and the only defence is the one used here: refine from several starts and compare the best of each model rather than the first.

And what to look at when a refinement has finished is now clear. The residual separates uncontained pairs even at molecule sizes where no single reflection does, as long as the error is random. The size of the partial atoms’ displacement parameters separates them too, and more robustly, because a wrong model with no right place for an atom smears it until it contributes almost nothing. Neither separates a contained pair, and there the choice between models has to come from outside the diffraction data: from the chemistry of the molecule, from its shape, or from diffuse scattering that the averaged structure throws away.

What the refinement has to refuse #

The first two tests are what make the refinement trustworthy as a refinement. It must recover a right answer exactly, and it must find its way back from a displaced start. The third is stated for the sum of squares and not for the residual, and the distinction is worth a sentence. Least squares minimises squared differences, and the residual quoted is a sum of absolute ones, so a refinement can lower the first while nudging the second up. On a few pairs it does, by half a per cent of the ideal value. A check written on the residual would have failed for a reason that is not a fault.

Still open: systematic error, and the full displacement tensor #

Two approximations in these refinements are the obvious next steps. The noise is random with a known variance. Replacing it with a smooth systematic error of the same size — a scale that varies across reciprocal space, as an uncorrected absorption does — would measure how much of the likelihood ratio’s power survives the error real data actually have, and it would put a number on the boundary this essay argues is systematic.

The other is the displacement model. Each partial atom here has two components, along its locus and across it, where a refinement program would give an atom on a two-fold axis or in a mirror plane a tensor with four free components. More freedom can only let the wrong model absorb more, so the absorption measured here is a lower bound and the decisions an upper bound, and by how much the full tensor moves them has not been measured.