A scale error between two wavelengths has a sign

A second wavelength completes the map formed dispersive differences, the change in a reflection’s magnitude between a wavelength at an absorption edge and one far from it, and built a Patterson synthesis of the anomalous atoms from their squares. It found that a scale error between the two datasets reaches the dispersive map and not the Bijvoet one, because the two halves of a Bijvoet pair are measured together and the two terms of a dispersive difference are not. The error it added was a factor of $1 + \varepsilon$ on the second dataset, and the Patterson of that error is the ordinary Patterson of the whole structure. On a model crystal of a hundred and twenty light atoms and two anomalous ones, that Patterson is a haze of fourteen thousand small vectors, and the substructure’s two vectors stayed the strongest peaks until the error passed fifty per cent.

It ended on the case that matters in practice: a crystal with a few heavier atoms that do not scatter anomalously at the chosen edge, a metal in a protein studied at the selenium edge, whose vectors are the strongest features of the ordinary Patterson. A scale error should write those vectors into the dispersive map at a height growing as the square of the error, and they should take over long before fifty per cent.

This essay adds the metals, and they do take over. But the more important result is that the fifty per cent was never a property of the map. It was a property of the sign of the error that was tried. Scale the second dataset the other way and the substructure’s vectors are erased at under four per cent, with or without metals. The reason is a term in the error that the square of the error hides, and the same term explains what scaling the two datasets onto each other costs.

The model, and the error in both directions #

The structure is the one the earlier maps used: a hundred and twenty light atoms of scattering factor one and two anomalous atoms of three, placed at random in a cell with no symmetry, with the anomalous atoms’ $f'$ and $f''$ taken from a selenium-like edge and scaled to that atom. The first dataset is at the edge’s inflection, where $f'$ is most negative, and the second at a high remote wavelength. Between them $f'$ changes by $\Delta f' = -0.503$, a sixth of the anomalous atom’s normal scattering. Every dispersive difference is taken between the two datasets’ averages over each Bijvoet pair, over the 1,681 reflections of indices up to twenty.

Metal atoms are then added, two of them, with scattering factors from 3 (as heavy as the anomalous atom) to 12 (four times as heavy). Their own absorption edges are taken to be far away, so their scattering is the same at both wavelengths and drops out of every dispersive difference except through the error.

The error multiplies every magnitude of the second dataset by $1 + \varepsilon$. The earlier essay tried $\varepsilon$ from one to thirty-two per cent. Here it runs from minus six per cent to plus fifty. The map is judged as before, by its contrast: the height of the weaker of the two substructure peaks divided by the height of the strongest peak that is not one of them. Below one, something else outranks the substructure.

The picture at the head of this essay is the finding. With no metal, a second dataset too strong by ten per cent leaves the contrast at 1.80, and by fifty per cent at 1.00. A second dataset too weak by one per cent leaves 4.46, by two per cent 3.16, by three per cent 1.82, and at four and a half per cent the substructure’s vectors are no longer among the map’s strongest peaks at all. The map is lost at 3.7 per cent in one direction and at fifty in the other.

Where the asymmetry comes from #

With the second dataset scaled by $1 + \varepsilon$, a dispersive difference is the true one, $\Delta$, less $\varepsilon|F|$, to first order in the error. The Patterson coefficient is its square:

$$(\Delta - \varepsilon|F|)^2 = \Delta^2 - 2\varepsilon\,\Delta\,|F| + \varepsilon^2|F|^2 .$$

The earlier essay followed the last term. Its Patterson is $\varepsilon^2$ times the whole structure’s Patterson, and it does not care about the sign of $\varepsilon$. The middle term does. It is linear in the error, and it is the product of the error with the signal.

The signal has a known form. The anomalous atoms’ normal scattering is $F_A$, and when $f'$ changes by $\Delta f'$ between the wavelengths, the whole structure’s magnitude changes by $\Delta \approx (\Delta f' / f_0)\,|F_A| \cos(\varphi_T - \varphi_A)$, where the angle is between the whole structure’s phase and the anomalous atoms’ phase and $f_0$ is the anomalous atom’s normal scattering factor. So $\Delta\,|F|$ is $(\Delta f'/f_0)\,|F_A|\,|F|\cos(\varphi_T - \varphi_A)$, which is the real part of $F_A$ times the complex conjugate of $F$. The Patterson of such a product is a cross-Patterson, the same kind of synthesis as the map that needs no phases but with two different structures in it: it has a peak at every vector from an anomalous atom to any atom of the structure, including the other anomalous atom.

At a vector between the two anomalous atoms, then, two terms compete. The signal’s own square contributes $(\Delta f'/f_0)^2 f_0^2 \cdot \tfrac12$, the half being the average of $\cos^2$. The linear term contributes $-2\varepsilon\,(\Delta f'/f_0)\, f_0^2$. With $\Delta f'$ negative, a positive error adds to the peak and a negative one subtracts from it, and the two cancel when

$$\varepsilon = \frac{\Delta f'}{4 f_0} .$$

The substructure’s peak reaches zero when the second dataset is too weak by a quarter of the fractional change in $f'$, here $-0.503 / 12 = -0.042$, a scale error of 4.2 per cent. The contrast crosses one a little before that, at 3.7, because the peak must stay above the background as well as above zero.

The physical reading is simple. At the inflection the anomalous atoms scatter less than at the remote wavelength, so the first dataset is systematically a little weaker, and the dispersive differences lean negative: their mean over all reflections is $-0.187$, where the mean magnitude is 10.3. That lean is part of the signal, and it is the same lean one experiment gives the cosine, the other gives the sine exploits: a dispersive difference measures the cosine of the angle between the whole structure’s phase and the anomalous atoms’. It is largest where the anomalous atoms contribute most, which is how the substructure’s vectors get into the map at all. Scale the remote dataset down and the inflection dataset looks relatively stronger, which undoes exactly what the edge did. Scale it up and the lean is exaggerated, which makes the substructure’s vectors stronger for a while, until the error’s square catches up with the haze.

The rate, measured #

The prediction can be checked without looking at contrast. The height of the substructure’s peak at errors of plus and minus half a per cent gives its slope, and a straight line through them reaches zero at $-0.0397$, five per cent short of the predicted $-0.0419$. The residual difference is the square term, which is small but not nothing at a few per cent. The curvature is visible in the picture, where the lines bend upward away from zero.

The slope is 1.695 with no metal and between 1.692 and 1.695 with every pair of metals, from as heavy as the anomalous atom to four times as heavy. The linear term does not see the metals at a substructure vector, because it is a cross-Patterson of the anomalous atoms with everything, and at the vector joining the two anomalous atoms only the anomalous atoms contribute. The metals lower the starting height a little, by weakening the dispersive differences relative to the whole, so the zero moves in from 0.040 to 0.035. The rate at which it is approached does not change.

What the metals do #

On the side where the second dataset is too weak, the metals change the threshold only a little: from 3.7 per cent with none to 3.6, 3.4, 3.3, 2.7 and 2.2 per cent as the metals’ scattering factor goes from 3 to 12. Whatever the crystal contains, the substructure cannot survive more than about four per cent in that direction, because what destroys it is its own cancellation.

On the other side the metals are decisive, as the earlier essay expected. Two metals as heavy as the anomalous atom move the threshold from fifty per cent to 48; one and a half times as heavy, to 40. Twice as heavy, to 19.5. Three times as heavy, to 6.1, and four times, to 3.4, at which point the two directions are nearly equal. The metals take over the map on the side that the anomalous signal had been protecting.

But the first vectors to take over are not the ones the earlier essay predicted. It expected the vectors between the metals, whose height in the error’s Patterson grows as $\varepsilon^2 f_M^2$, the way a heavy atom’s vectors dominate the ordinary Patterson. What outranks the substructure first, for every metal from twice the anomalous atom’s weight up, is a vector from a metal to an anomalous atom. Those vectors appear in the linear term’s cross-Patterson, at a height proportional to $\varepsilon\, f_M f_0$, so they grow with the first power of the error and arrive sooner. With the second dataset too strong the linear term makes them peaks; with it too weak, it makes them holes, which is why they never appear on that side.

The four maps show the order in which things happen. With no error the two circles hold the strongest features. With the second dataset three per cent too weak the substructure’s peaks are fading, and nothing in particular replaces them: the contrast is 1.28 and falling towards the point where the background outranks what is left. Twenty per cent too strong, the contrast is 0.995 and the peak that has just overtaken the substructure is a vector from a metal to an anomalous atom. At fifty per cent it is 0.70, and the strongest intruder is the vector between the two metals, the one the earlier essay expected, arriving last.

Scaling builds in the dangerous sign #

The remedy for a constant scale error is to scale the second dataset onto the first before taking differences, by the factor that best matches their magnitudes. It removes a constant error exactly, with or without metals: with one factor fitted, the contrast of the map with metals of $f_0 = 6$ is 3.3853 whether the error was nothing, five per cent, twenty or fifty. The earlier essay found that the scaled map is nevertheless worse than the unscaled map with no error, and put it down to scaling removing real signal along with the error. The quarter rule now says how much.

A least-squares scale factor between two datasets makes their mean magnitudes agree. The true dispersive differences do not average to zero; they lean negative, by a fixed physical amount. So scaling, applied to perfect data, multiplies the second dataset by a factor slightly less than one. It introduces a scale error of exactly the cancelling sign, of size close to the mean difference divided by the mean magnitude.

On perfect data, one scale factor builds in an error of $-1.68$ per cent and ten shell factors an average of $-1.75$. The quarter rule predicts that a peak falls in proportion to its distance from the cancelling point: to $1 - \varepsilon / 0.042$ of its height, which is 60 per cent for one factor and 58 for shells. The measured heights are 61 and 58 per cent. The loss the earlier essay attributed to scaling in general is this one mechanism: a least-squares scale treats the edge’s systematic weakening of the inflection dataset as an error, corrects it, and in doing so moves two-fifths of the way to the point where the substructure vanishes.

There is no scale factor that avoids this while being fitted from the data alone, since the lean is exactly what a fitted factor removes. The way out is to fit the scale against something that does not lean: the part of the scattering the edge does not touch. At high resolution the anomalous atoms’ share of the scattering is small, and a scale fitted to the light atoms’ average scattering, the kind of average the average that knows the atoms derives from composition alone, would not see the lean. How well that works depends on how well the composition is known, and it is not modelled here.

A temperature factor puts both signs in one map #

Real datasets differ by more than a constant. The earlier essay also applied a relative temperature factor, the second dataset falling off with resolution faster than the first, and found that one scale factor fails where scaling in ten resolution shells holds. The sign rule says why the single factor fails so badly.

A single factor fitted to a dataset that falls off with resolution matches it on average and gets every shell wrong: the low-resolution shells are left 10.8 per cent too strong and the edge 11.7 per cent too weak. Both signs of the error are in one map, the reinforcing one in the strong low-resolution reflections and the cancelling one at the edge, and the cancelling one needs only a quarter of $\Delta f'/f_0$. Ten shell factors leave between 0.7 and 2.3 per cent too weak in every shell, which is the lean again, fitted away shell by shell.

With the metals of $f_0 = 6$ in the crystal, the pattern is the one the earlier essay found without them. Unscaled, the map is lost at a fall-off of five per cent at the edge. One scale factor holds it to five and loses it by ten. Ten shells hold it to ten and lose it by eighteen. The metals do not make the temperature factor worse, because on the side where the single factor leaves the data too weak nothing the metals contribute can compete: what fails is the substructure’s own cancellation, and it would fail the same way in a crystal with no metal at all.

What this does not show #

The structure is a model, two-dimensional, with no symmetry, and the metals sit at two fixed places. In a space group the vectors between symmetry-related atoms fall on the sections symmetry stacks them on, which changes where a peak is looked for and not the arithmetic of the differences. The quarter rule itself uses nothing about the structure except the anomalous atom’s scattering factor and the change in $f'$, and it is expected to carry over. The thresholds for the metals depend on how many there are and where they sit, and are measurements on this model only.

The rule depends on a convention, which dataset is the first. Here it is the inflection, where $f'$ is most negative, and the error is applied to the remote dataset. Exchange them and $\Delta f'$ changes sign, and so does the direction of danger. The statement that survives the exchange is this: the dangerous error makes the dataset in which the anomalous atoms scatter more look weaker than it is.

The Bijvoet map is not affected. Both halves of a Bijvoet pair come from one dataset and are scaled together, so none of this reaches the map of the atoms that break the law, and a combined map inherits the sign rule through its dispersive half, which it was not tested with here.

Only one mechanism of error is modelled. A scale factor and a resolution-dependent scale. Radiation damage changes the structure itself between datasets, and absorption differs between the two halves of a Bijvoet pair measured at different orientations; neither is a scale factor, and neither is in these maps.

And a map is not a solution. A Patterson gives the vectors; solving from the vector set turns them into positions, and a map in which a metal–anomalous vector outranks the substructure would send that search to the wrong place first. How often the search recovers from that is not measured here.

Who worked the pieces out #

The multi-wavelength method, and the form of the intensities the argument above linearises, are due to Jerome Karle, whose analysis was published in 1980, and to Wayne Hendrickson and his colleagues, who made it routine around 1990. Scaling datasets onto each other, locally and in shells, is a standard step in data reduction, and the practice of measuring the wavelengths of a multi-wavelength experiment close together in time is the working answer to the error modelled here. The choice of which wavelengths to measure is the subject of the two most anomalous wavelengths are the wrong pair, and the value of a third, of the third wavelength is there to measure the edge. The linear cross term and the quarter rule that follows from it are worked out on this page.

What the calculation must refuse #

The refused claim is the one the earlier essay’s figure took for granted by testing one sign, and it is the natural assumption about any scale error: an error of one per cent is an error of one per cent. Here the two sides differ by a factor of fourteen. The test that would catch a mistake in the explanation is the slope. If the linear term were not the cause, there would be no reason for six structures with different metals to lose their substructure peak at the same rate, and no reason for that rate to land within five per cent of a number computed from $\Delta f'$ and $f_0$ alone. The cost of scaling is a second, independent check of the same rule. It uses no scale error at all, only the lean that scaling removes, and it predicts the loss to within a percentage point.

Still open: fitting the scale to what the edge does not touch #

The quarter rule says the scale between two datasets of a multi-wavelength experiment must be fitted to something other than their mean magnitudes, since the means differ for a physical reason and a fitted factor removes the difference. The candidate is the scattering the edge does not change: at high resolution the light atoms dominate, and a scale fitted there, or fitted to a Wilson average computed from the known composition, would leave the lean in place. The anomalous atoms’ share of the scattering never falls to zero, though, so any such fit carries some of the lean with it, and the question is how much. It is the same computation as the cost above, with the scale fitted to a model of the light atoms’ average rather than to the data, and it would say how much of the forty per cent a careful scaling recovers.