The average scatters sharply and the rest does not

Every diffraction argument in this collection so far has been about a structure: a set of atoms, periodic, with a group — and the absences and intensities that follow from one. A great many real crystals are not that. Two kinds of atom share one lattice, and which site holds which is decided differently in every cell — the lattice perfect, the occupation not.

Such a crystal still diffracts, and what it produces splits exactly in two. The average structure is periodic whatever the disorder does, so it gives sharp reflections at the reciprocal lattice and nowhere else. Everything the average leaves out is the variance, and it is spread across all of reciprocal space.

The split, written down #

Let each site carry an occupation s taking the values +1 and −1 for the two kinds of atom, and let F(k) be the sum of $s\,e^{2\pi i\,\mathbf{k}\cdot\mathbf{r}}$ over the sites — the structure factor of one arrangement. A measurement sees the average of |F|² over the arrangements the crystal contains, and that average splits by the identity that defines a variance:

$$\langle |F|^2 \rangle = |\langle F \rangle|^2 + \left( \langle |F|^2 \rangle - |\langle F \rangle|^2 \right)$$

The first term is the intensity of the average structure — every site carrying the mean of the two kinds — and the average structure is perfectly periodic, so this term is non-zero only at reciprocal lattice points. Those are the Bragg reflections.

The second term is the variance of the structure factor, and nothing makes it vanish away from the reciprocal lattice. It is the diffuse scattering, and it is present at every wavevector.

That is the whole of the phenomenon. The rest of this essay is what the two terms do, measured on ensembles built for the purpose.

Two routes to the diffuse term #

The variance can be computed directly — average the intensity, subtract the intensity of the average — and it can be computed from the correlations between sites, which is the route a crystallographer actually uses.

The correlation of two sites a vector r apart is α(r) = ⟨s(x) s(x + r)⟩, averaged over sites and arrangements; it is one at r = 0 and falls away with distance. The claim is that the diffuse intensity is the Fourier transform of that function, and this collection’s habit is to compute both and compare rather than to cite it.

They agree to the last bits of double precision — a relative difference of about 10⁻¹⁴ at every wavevector, on every ensemble tried. The two computations share no code: one squares structure factors over configurations, the other counts agreeing pairs of sites and transforms the result.

The sum rule, which does not move #

There is a conservation law here and it is the cheapest check in the subject.

Occupations of ±1 have mean square one, so the total scattering summed over every wavevector the block can be asked about is exactly the number of sites — one unit per site — whatever the correlations are. Order moves intensity about; it does not create it.

Measured on the ensembles here the sum is 1.000000000 per site in every regime: strongly alternating, uncorrelated, strongly clustering. That the check is exact rather than approximate is a property of the finite block: the wavevectors an N × N block can be asked about are the multiples of 1/N, and summing over exactly those recovers the sites by an identity rather than by an integral.

The conservation is why a diffuse pattern is informative. Intensity that has left the diffuse background has gone somewhere — into the Bragg reflections, if the crystal has ordered — and the intensity in a superlattice reflection at a transition is intensity the diffuse scattering had before it.

The wavevectors a block can answer about #

That last point deserves separating, because it is a restriction on the measurement rather than on the crystal.

A configuration on an N × N block is periodic under translations by N, so the only wavevectors it is defined at are multiples of 1/N. Asking about anything else is asking a question the sample cannot answer: the intensity between those points depends on the shape of the block rather than on the crystal.

This collection has made the same point in another context, where a finite block holds exactly one wavevector per cell and everything else is absent rather than approximated. Here the consequence is practical: how finely a diffuse pattern can be measured is set by how large a coherent region of the crystal is, and a computation on a small block reports a coarse pattern for the same reason.

The bias every ensemble has #

An ensemble of n arrangements does not give the true split. It gives a biased one, and the size of the bias is exactly one part in n.

The reason is that the split subtracts the intensity of the sample mean rather than of the true mean, and a mean of n draws carries a 1/n share of the variance with it. So the “Bragg” term is inflated by that share, the diffuse term is short of it, and a measurement of the diffuse intensity per site from an ensemble of sixteen arrangements comes back at about 1 − 1/16 rather than at one.

Measured here at five ensemble sizes it follows the curve 1 − 1/n closely: 0.9285 at sixteen against 0.9375, and 0.9836 at sixty-four against 0.9844. A finite ensemble reports part of its own diffuse scattering as sharp, and the amount is predictable rather than mysterious.

That is worth knowing outside this computation, because the same bias appears whenever an average structure is refined from data and the difference taken as diffuse: the refined average is a sample mean, and it absorbs a share of the variance that scales inversely with how much has been averaged.

The average structure is not a structure #

A statement that is easy to make and worth being careful with: the average structure is an average, and it need not be anything a crystal could be.

Its sites carry the mean of the two kinds of atom — half of each, in the symmetric case — which is not an atom. A refinement of the Bragg intensities alone returns that fiction, faithfully, and reports partial occupancies at every site.

That is not a defect of the refinement. It is what the Bragg intensities contain: the periodic part of the structure, which is the average, and nothing else. Everything about the arrangement is in the diffuse scattering, which a Bragg-only refinement discards.

This collection has an essay about the same phenomenon in the context of a molecule taking several orientations, where the averaged structure comes back with symmetry no molecule has. The occupational case is the same statement with two kinds of atom instead of two orientations, and the same warning: the average is more symmetric than anything in the crystal, and a refinement that stops at the average reports the higher symmetry.

What a completely disordered crystal scatters #

The uncorrelated case is the one worth having a picture of, because the intuition is usually wrong.

A crystal with no correlations at all — every site an independent coin toss — scatters flatly: one unit per site at every wavevector, and no feature anywhere. Not zero, which is the common expectation, and not concentrated anywhere.

That flatness is the baseline every diffuse pattern is read against. Structure in a diffuse pattern is departure from flat, and a pattern with a maximum somewhere is a crystal with a preference; the next essay is about reading that preference off.

Measuring the flatness is where the finite-ensemble noise shows. The scatter of the measured values about their mean falls as the ensemble grows — 0.185 at sixteen arrangements, 0.135 at sixty-four — which is what distinguishes noise from structure, and is the reason both figures are reported rather than one.

Where the intensity goes when the crystal orders #

The sum rule turns the ordering transition into a bookkeeping statement, and it is the clearest way to see what a superlattice reflection is.

Above the transition the crystal has correlations and no long-range order: its diffuse intensity peaks at the wavevector it is trying to order at, and the peak is broad. Below the transition the crystal has ordered: the peak has become a sharp reflection, and the diffuse intensity around it has fallen by exactly what the reflection gained.

Nothing has been created. The total was one unit per site above the transition and it is one unit per site below it, and what changed is the distribution. That is why the intensity of a superlattice reflection is a measurement of the order parameter — it counts the intensity that has left the background — and why the reflections a superlattice adds grow from nothing rather than appearing at full strength.

Reading the transition this way also says what a partially ordered crystal looks like: a sharp reflection sitting on a broad diffuse maximum at the same place, with the two sharing the available intensity. Both are measured routinely, and their ratio is the standard measure of how complete the ordering is.

Why the split is exact and the reading is not #

The identity that splits the intensity is exact, and every difficulty in using it is about deciding which part of a measured pattern is which.

A Bragg reflection is a delta function in an ideal crystal and is not one in a real measurement: it is broadened by the instrument, by the finite size of the coherent regions, and by strain. So the sharp part sits on the diffuse part with no clean line between them, and separating the two is a modelling decision rather than an arithmetic one.

That is worth stating plainly next to a computation where the separation is exact. Here the block is finite, periodic, and its Bragg positions are known exactly, so the split is a matter of reading two numbers from the same array. In a measurement it is the whole difficulty of the subject.

The consequence is that the sum rule, which is exact here, is not directly usable there: it holds over all of reciprocal space, and a measurement covers a finite region. What survives is the qualitative form — intensity is conserved and redistributed — which is the reasoning above about the transition.

The two questions a diffuse pattern answers #

Being clear about what this kind of scattering does and does not report is the useful summary.

It reports the correlations, completely: the transform relationship above means that measuring the diffuse intensity everywhere and transforming back gives the whole correlation function, with no information lost either way.

It does not report the arrangement. Many different arrangements share a correlation function, and the diffuse intensity cannot tell them apart — which is the same underdetermination this collection meets as two structures with one Patterson, in the disordered setting. The pattern fixes the pair statistics and nothing beyond them.

That second point is why modelling disorder is hard: a measurement constrains the two-site correlations and leaves the three-site and higher ones open, and a model that reproduces the diffuse pattern is not thereby the right model. What can be said with confidence is exactly what the pair correlations say, and the next essay is about reading them.

What is model and what is measurement #

Every number in this essay is measured on configurations generated by a rule, and the rule is stated rather than hidden.

Each configuration is a block of ±1 values, produced by single-site updates at a fixed coupling from a fixed seed. A positive coupling favours like neighbours, a negative one unlike, and zero accepts every flip. That is an Ising arrangement, and nothing here claims it is a model of any material: what it is for is producing ensembles whose correlations are known because they are measured afterwards.

The seed matters and is fixed deliberately. A figure whose numbers moved between two builds of the same commit would be a figure nobody could check, so the generator is seeded and two runs of the same seed give the same configuration — which is itself one of the checks.

What is imported: the split identity, which is the definition of a variance; the statement that a periodic structure scatters only at its reciprocal lattice, which this collection derived in the reciprocal lattice essay. What is measured: the sum rule, the agreement of the two routes, the position of the diffuse maximum, and the bias.

The same split, in three other places #

The identity behind this essay is not about crystals, and naming where else it appears makes clear which part of the argument is physics.

A powder pattern. The average over orientations rather than over occupations, giving rings whose positions are exact and whose intensities merge the reflections a single crystal separates — which this collection treats as what a powder pattern loses. Same structure of argument: an average that is periodic, plus a spread that is not.

Thermal motion. The average over time rather than over sites. The average structure is the atoms at their mean positions, giving Bragg reflections attenuated by the Debye–Waller factor, and the variance is thermal diffuse scattering. The split is the same identity with displacement in place of occupation.

Stacking faults. The average over layer sequences, where the diffuse part concentrates into streaks rather than spreading evenly, because the disorder is one-dimensional.

In every case the sharp part is the square of a mean and the spread part is a variance. What differs is what is being averaged over, and the shape the variance takes in reciprocal space — which is decided by the geometry of the correlations rather than by the identity.

What the pictures cannot show #

A single crystal’s diffuse pattern is not an ensemble average. A real measurement is made on one crystal containing an enormous number of cells, and the average it produces is a spatial one rather than an average over arrangements. The two agree when the crystal is large and its regions independent — which is an assumption, standard and not checked here.

Nothing here is three-dimensional, and diffuse scattering in three dimensions is qualitatively richer: it can concentrate on planes, on lines, or on surfaces in reciprocal space, and each of those shapes says something about the dimensionality of the correlations producing it. The plane’s version has only points.

And no atom scatters differently from another here. Every site carries ±1, which stands for the difference between two scattering powers. A real alloy’s two atoms differ by a real number, and the diffuse intensity scales with the square of that difference — which is why an ordered alloy of two neighbouring elements can be invisible to X-rays and obvious to neutrons.

Reading the correlations back out #

The essay computes the diffuse intensity from the correlations. The measurement runs the other way, and it is worth setting out because it is the whole reason diffuse scattering is collected at all.

The diffuse intensity and the correlation function are a Fourier pair: the intensity at each wavevector is the transform of the correlations, so transforming the measured diffuse pattern back gives the correlations directly, one number per interatomic vector.

Those numbers are the Warren–Cowley parameters. The one at the nearest-neighbour vector says whether unlike atoms prefer each other — negative for alternation, positive for clustering, zero for a coin toss — and the ones further out say how far that preference propagates.

So a diffuse pattern is a measurement of local order, shell by shell, made without any model of the structure and without the crystal being ordered at all. That is an unusual thing for a diffraction experiment to deliver: everything else it reports is a property of the average, and this is the part the average threw away.

The practical difficulty is that the transform needs the whole pattern. Bragg reflections are a handful of strong spots and the diffuse intensity is spread over everything between them, weakly — so the measurement is one of collecting a large three-dimensional volume of weak data, subtracting several backgrounds, and transforming what is left.

And the sum rule is what keeps it honest. The transform’s value at zero vector is the total scattering per site, which is one — so a measured pattern whose transform does not come to one has lost intensity to a background subtraction, and the check costs nothing.