Where the experiment runs out

The previous rung showed a space group being read off a pattern of missing reflections. This one is about how often that fails, which is more often than the technique’s reputation suggests, and about what exactly is missing when it does.

There are two separate failures and they are worth separating from the start, because only one of them is about the equipment. Some groups produce no absences at all, so there is nothing to read. Others produce absences that are identical to another group’s, so what is read does not distinguish them. The first is a shortage of evidence and the second is a shortage of resolution in the evidence itself, and no improvement in the measurement touches either.

The silent groups #

Start with the extreme case. A group with no screw axis, no glide plane and no centring translation extinguishes nothing at all, and there are a great many of them: every one of the seventy-three symmorphic groups on a primitive lattice.

For those, the absences carry zero information and the assignment rests entirely on the other half of the experiment — the intensities, which give the Laue class. And the Laue class is a coarse instrument.

What Friedel’s law removes #

The intensities have a symmetry the crystal does not, and it is the single largest limitation in this whole business.

The structure factor of the reflection −h−k−l is the complex conjugate of that of hkl, so the two have the same magnitude. An experiment measures magnitudes, so it cannot distinguish them: every diffraction pattern appears centrosymmetric whether the crystal is or not. That is Friedel’s law, and this site has an essay on the two-dimensional case whose finding transfers exactly.

The consequence is that intensities give the point group with an inversion added — the Laue class — and there are eleven of those rather than thirty-two. So an experiment that determines the Laue class as 2/m has narrowed the point group to 2, m or 2/m, three possibilities, and cannot go further.

Combine that with silent absences and the ambiguity is real: P2, Pm and P2/m have the same Laue class and no absences at all, so a diffraction pattern is consistent with all three.

Note carefully what the families have in common. P2, Pm and P2/m are silent together. P2₁, Pc and P2₁/c all extinguish something, and they extinguish different things — so within that trio the absences do separate them, which is the case where the technique works.

The pattern is that the inversion is invisible. Adding a centre of symmetry to a group changes no absence, because the inversion has no intrinsic translation and contributes a phase of zero to every coefficient. So a centrosymmetric group and its non-centrosymmetric subgroup are always confused with each other, always, by every measurement of intensity magnitude.

The diffraction symbol #

Crystallographers have a name for what the experiment actually determines: the diffraction symbol, which is the Laue class plus the lattice letter plus the extinction conditions. It is what is known before a single atom has been located.

There are 122 possible diffraction symbols and 230 space groups. About fifty of the 230 are uniquely determined by their symbol; the rest share one with between one and five others.

The distribution is not even. The non-symmorphic groups with several screws and glides are the ones best determined, because every additional intrinsic translation adds a condition and conditions accumulate. P2₁2₁2₁ is uniquely determined: three axial conditions, no other group produces that combination. The symmorphic groups on primitive lattices are the worst determined, because they say nothing.

Counting the ambiguity #

It is worth putting numbers on how bad this is, because the technique is usually described as though it worked.

Of the 230 space groups, the ones this site can speak to directly are the ones whose conditions it computes, and among those the pattern is clear. P1 and P1̅ share a diffraction symbol and are silent. P2, Pm and P2/m share one. P2₁ and P2₁/m share one, since both extinguish 0k0 with k odd and the inversion adds nothing. Pc and P2/c share one. P2₁/c shares its symbol with P2₁/c alone, because the combination of an axial and a zonal condition in the monoclinic system is produced by nothing else — which is a large part of why the commonest space group in the literature is also one of the most confidently assigned.

The general shape is that each centrosymmetric group is confused with its own non-centrosymmetric subgroups, and the number of those depends on the group. That is Friedel’s law doing exactly one thing, systematically, across the whole classification.

A worked case, and why it was a scandal #

The ambiguity is not academic and the standard cautionary tale is worth telling, because it shows exactly what the technique cannot do.

A large number of structures have been published in a centrosymmetric group when the true group was its non-centrosymmetric subgroup, and a comparable number the other way. The two errors have different symptoms. Assuming a centre that is not there averages two slightly different molecules into one, producing bond lengths that are the mean of two real values and belong to neither. Assuming no centre when there is one leaves the refinement with twice as many parameters as the data supports, and they drift.

The reason it happens is entirely the one this essay is about: the absences do not distinguish the two, the intensities do not distinguish them, and the statistical test that does is a test on a distribution and can be misled by a heavy atom or a small data set.

What settles it in the end is chemistry. If the centrosymmetric refinement gives two chemically identical bonds different lengths, the centre is wrong; if the non-centrosymmetric one gives two independent molecules that are the same to within their uncertainties, the centre is right and was omitted. That is a judgement about a structure, made after the fact, and it is not something the diffraction pattern contains.

What resolves it #

Three things, in increasing order of effort.

Statistics of the intensities. A centrosymmetric structure has a different distribution of intensity values from a non-centrosymmetric one — centrosymmetric structure factors are real, so they cluster near zero more heavily. Comparing the observed distribution against the two theoretical ones gives an answer that is usually right and is a statistical inference rather than a determination. This site’s near-symmetry essay is about exactly this kind of judgement and about how it fails.

Solving the structure. Try one group, solve, see whether the answer is chemically sensible. If it is not, try the other. This is the honest brute-force method and it is what most people do.

Anomalous scattering. Friedel’s law is not exact. Near an absorption edge, the scattering factor acquires an imaginary component and the magnitudes of hkl and −h−k−l stop being equal. The difference is small — a percent or two — and it is measurable, and it is the standard way absolute configuration is determined. That is how a chiral crystal’s handedness is assigned, and it works because the failure of a symmetry is measurable when the symmetry is nearly exact.

The centring is visible and the inversion is not #

Two operations, both of which add nothing to the shape of a structure that a chemist would notice, and one of which the experiment sees clearly while the other is invisible. It is worth asking why.

A centring translation halves or quarters the number of reflections. It is the loudest thing a group can do to a diffraction pattern, and it is detectable at a glance from a photograph.

An inversion centre does nothing at all. Not a subtle effect, not a weak signal — a phase of exactly zero contributed to every coefficient, so every intensity is identical with and without it.

The difference is in the intrinsic translation. A centring vector is an intrinsic translation, of a pure-translation operation, and the extinction derivation is entirely about intrinsic translations. An inversion has none: it is (−I, 0) at the right origin, and there is always a right origin because a centrosymmetric group’s inversion centres are exactly where the origin gets put.

So the two things a group can carry that produce absences are the centring and the intrinsic translations of its operations, and everything else — every rotation, every mirror, the inversion — is silent by construction. That is not a limitation of the method; it is the method’s content, and knowing it settles exactly which half of a group an experiment can see.

The two ambiguities are different in kind #

It is worth separating them, because they get run together and only one of them is about the technique.

The Friedel ambiguity is physical. An experiment measures magnitudes and the magnitudes genuinely are centrosymmetric. Resolving it requires measuring something else, and anomalous scattering is measuring something else. This is a limit of the measurement, and it is removable at a cost.

The extinction ambiguity is informational. Two groups extinguishing the same reflections are not distinguishable by absences even in principle, no matter how perfect the measurement, because there is nothing to measure — the absences are identical and both are exact. Resolving it requires information from somewhere other than the absences, and no improvement in the experiment helps.

And the derivation turns up a third kind, which the argument above does not cover. P4₁ and P4₃ share a diffraction symbol, and so do P3₁ and P3₂. Their point groups are not merely of the same Laue class, they are identical: 4 in the first pair and 3 in the second. What differs is the hand of the screw — a fourfold that advances a quarter of a cell per turn against one that advances three quarters, which is the same screw seen in a mirror. A screw of either hand extinguishes 00l with l not divisible by four, exactly, so the absences say a screw is there and cannot say which way it turns.

That ambiguity is not removable by measuring more intensities and it is not about a centre. It is removable the same way absolute configuration is, by anomalous scattering, and for the same reason: the two candidates are mirror images of one another, and a mirror image is what Friedel’s law hides.

What a powder loses on top of all this #

Everything above assumes a single crystal, and the powder case is worse in a way worth naming, because most materials never grow a single crystal.

A powder pattern collapses the three-dimensional reciprocal lattice onto a single axis: only the magnitude of each reciprocal vector survives, so reflections with different indices and the same spacing overlap and cannot be separated. That destroys the Laue class outright — the pattern has no directions in it — and it makes many absences unobservable, because an absent reflection overlapping a present one is not absent in the data.

So a powder pattern gives the lattice parameters and, with care, some of the absences, and the space group has to be inferred with much weaker evidence. Structure determination from powder data is a real technique and it is hard for exactly this reason.

The absence that is not systematic #

One more distinction, because it causes trouble in practice and the words for it are bad.

A systematic absence is a reflection that vanishes for every arrangement of atoms in the group. It is a property of the group and it is what this essay and the last are about.

A structural absence is a reflection that happens to vanish for a particular structure — because the atoms sit where they do, and the phases happen to cancel. It looks identical in the data.

The classic case is a structure whose heavy atoms lie on a sublattice of higher symmetry than the whole. If the scattering is dominated by those atoms, the reflections violating the sublattice’s conditions are weak rather than absent, and a data set that does not reach them looks like a data set for the smaller group. Structures have been solved in the wrong space group this way and published.

The machinery here computes systematic absences and cannot see structural ones, which is correct — a structural absence is not a fact about the group, and the machinery here knows only about groups. But it means an experimentalist comparing a computed table against an observed pattern is comparing two things that can differ for a reason the table is not wrong about.

What the absences do determine, reliably #

It would be a distortion to end on the failures without saying what the technique is good for, because it is the standard method and it is standard for a reason.

The lattice type is determined outright. A general condition on hkl is unmistakable, it is a large effect, and it identifies the centring letter with no ambiguity at all. That is the first character of the symbol, given free.

The presence of screws and glides is determined. An axial condition means a screw on that axis and nothing else means it; a zonal condition means a glide on that plane. So the non-symmorphic half of a group’s content is fully visible.

The Laue class is determined, from the intensities, which fixes the crystal system and narrows the point group to at most three candidates.

What is left undetermined is precisely the part that produces no absence and no intensity difference: whether an inversion is present. Everything else the experiment sees.

Pna2₁ is the shape of that in one group. It has two glides and a screw, and the derived conditions are two: 0kl absent when k + l is odd, from the n glide perpendicular to a, and h0l absent when h is odd, from the a glide perpendicular to b. The screw contributes nothing of its own — its condition, 00l absent for odd l, is the k = 0 row of the n glide’s and is already there — which is a small reminder that the conditions a group produces are not in one-to-one correspondence with the operations that produce them. Those two together are held by no other group in this collection’s table, so the pattern names the group outright, without a single atomic position having been determined.

Put like that the score is better than the essay’s title suggests: the experiment determines the lattice, the system, the non-symmorphic content and the point group up to an inversion, before any structure solving happens at all. It is a remarkable amount to get from a list of directions and brightnesses, and the residue it cannot reach is one bit.

Why a better fit is not evidence #

There is a fourth thing people do to resolve the ambiguity, it is the most tempting, and it is the one that does not work on its own.

Two candidate groups related by an inversion are nested models: the centrosymmetric one is the non-centrosymmetric one with constraints imposed. Refining in the larger, less constrained group therefore always produces a fit at least as good, and in practice always slightly better, because more parameters always fit noise better. A lower residual in the non-centrosymmetric group is not evidence that the crystal lacks a centre. It is arithmetic, and it happens whether or not the centre is there.

What the improvement can be is evidence, once it is compared against how much improvement the extra parameters buy by themselves. Hamilton’s ratio test of 1965 is the standard form: it says how large a drop in the residual has to be, given the number of added parameters and the number of observations, before it is more than the fitting of noise. That is a statistical statement with a stated confidence, and it can be — and often is — reported as a decision without one.

The failure mode is specific and it produces a recognisable structure. Refined in the wrong, larger group, an essentially centrosymmetric structure has parameters that are nearly related by the centre it was not allowed to have, so the refinement is close to singular: correlations between paired parameters approach one, displacement parameters go strange, and bond lengths that should be equal come out differing by more than their own uncertainties. Those symptoms are the honest signal, and they are visible in the correlation matrix rather than in the residual.

So the order that works is the one this essay’s list gives: the intensity statistics first, because they test the property directly and cost nothing; then chemistry; then anomalous differences if the crystal has an atom heavy enough to supply them. The residual is not on that list, and neither is the absence pattern, which is where this essay started.

Why this is the right note to end the field on #

The rest of this collection computes. Groups are generated, forgotten and rediscovered; counts are enumerated; conditions are derived from sums. Every claim is decidable and the machinery decides it.

This essay is about the one place where that stops being the relevant question. The space group of a real crystal is not decided, it is inferred, and the inference is underdetermined in a way that has nothing to do with the quality of the arithmetic.

That gap is worth keeping in view when reading the rest of this field. The diagrams here are exact, and they are diagrams of groups rather than of crystals. Assigning one of them to a substance on a bench is a different activity with different standards of evidence, and the two hundred and thirty are a catalogue of possibilities rather than an index of answers.

Where the exactness stops #

Two limits on this essay’s own claims.

The counts of uniquely-determined groups are from the literature. Roughly fifty of the 230, and 122 diffraction symbols, are figures from the crystallographic literature rather than computations run here. What is computed is the extinction condition of every group this site defines, and that those conditions coincide within the families shown.

“Diffraction cannot tell them apart” means intensities and absences. A modern experiment measures more than that — anomalous differences, and in favourable cases the phases themselves through multiple-beam effects — and the sentence is about the classical reading of a data set rather than about the physical limit. The physical limit is lower and harder to state.