Two conditions are not one condition twice

Going up costs the cell a parameter priced every step up the diagram of plane-group supergroups in conditions on the cell. From p2 to p2mm needs a right angle; to c2mm, two primitive vectors of equal length; to p4, both at once; to p6, equal vectors at 120°. Going down is free and going up is conditional, and the conditions are the codimensions of the five plane lattices: nought, one, one, two, two.

It ended where every such account has to end, on a measured cell. An oblique cell whose angle refines to 90.02(4)° is rectangular or it is not, and the diagram says nothing about which. What it could say, the essay suggested, is how far the cell sits from each condition in the units of its own standard uncertainties — so that a list of conditional supergroups becomes an ordered list.

That distance is easy to compute and it does not order the list on its own. The strata impose different numbers of conditions, and a distance means something different on each. Read at the right number of conditions, the distances become probabilities; and then a square cell can score better than the equal-edged cells that contain it, which no ranking should allow. What rescues the ranking is walking up the diagram one condition at a time — and that has a price in its error rate, which the computation below measures rather than argues about.

A tenth of a degree can be further than a degree #

Start with one condition. A rectangular supergroup needs the cell angle to be a right angle, and a measured angle $\hat\gamma$ with standard uncertainty $\sigma_\gamma$ is $(\hat\gamma - 90°)/\sigma_\gamma$ standard uncertainties from it.

A cell measured at 90.12(3)° is four standard uncertainties off, and the chance of landing that far from a true right angle is six in a hundred thousand. A cell measured at 91.2(8)° is a degree and a fifth off and one and a half standard uncertainties, with a chance of thirteen per cent. The cell ten times closer in degrees is the one the measurement excludes. Nothing about that is subtle, and it is the first thing the tolerance on a symmetry search gets wrong when the tolerance is set in degrees: a fixed angular window is loose for a precise measurement and tight for a poor one, and the same structure can pass or fail according to how well its cell was measured rather than according to what it is. A tolerance set on coordinates has the same defect for the same reason.

So the distance has to be in standard uncertainties. That is the easy half.

Distances on strata of different codimension #

The hard half arrives with the square. A square cell needs two things: a right angle and equal edges. For a measured cell $(\hat a, \hat b, \hat\gamma)$ with independent uncertainties, the least $\chi^2$ over all square cells is the sum of the two separate ones,

$$\chi^2_{\text{square}} = \frac{(\hat\gamma - 90°)^2}{\sigma_\gamma^2} + \frac{(\hat a - \hat b)^2}{\sigma_a^2 + \sigma_b^2},$$

and the same construction gives every stratum its distance: one term for rectangular, one for rhombic, two for square, two for hexagonal. The trouble is what to compare those numbers against. A true rectangular cell, measured with Gaussian errors, produces a $\chi^2$ distributed with one degree of freedom; a true square cell produces one with two. A $\chi^2$ of 5 is unusual for the first and ordinary for the second — its probability is 2.5 per cent with one degree of freedom and 8.2 with two.

The third cell in the table is the one that shows the problem. It is measured at $a = 5.002(2)$, $b = 5.010(3)$, $\gamma = 90.03(4)°$ — an illustration, not any particular compound’s. Its right angle is 0.75 standard uncertainties off, so rectangular passes easily, at a probability of 0.45. Its edges differ by 0.008 with an uncertainty in the difference of 0.0036, which is 2.22 standard uncertainties, so the equal-edged rhombic stratum is at a probability of 0.027 and fails at five per cent. And the square stratum, which needs both, has $\chi^2 = 5.49$ on two conditions: probability 0.064, which passes.

So the cell is consistent with a square lattice and inconsistent with a rhombic one. Every square lattice is rhombic — equal edges at a right angle are equal edges — so that verdict places a set above a set that contains it, and no ordering of candidate supergroups can be read off a list that does that.

Three rules are three shapes #

The paradox is easiest to see as geometry. Put the measured cell in a plane whose two axes are the two conditions, each in its own standard uncertainties: distance from equal edges across, distance from a right angle up. The square cells are the origin.

Each rule for accepting “square” at five per cent is a region around the origin. The two-condition $\chi^2$ accepts inside a circle of radius 2.45, because that is where $\chi^2$ with two degrees of freedom has five per cent beyond it. Testing each condition on its own accepts inside the square of half-side 1.96, because each condition alone has five per cent beyond 1.96 standard uncertainties. And reading the distance as if it were one condition — the natural thing to do with a number in standard uncertainties — accepts inside a circle of radius 1.96.

The measured cell is inside the first region and outside the other two. It sits 2.22 across and 0.75 up: out of the square because of the first coordinate alone, out of the small circle because its total distance is 2.34, and inside the large circle because 2.34 is less than 2.45. The three rules give three verdicts for one cell, and none of them is an arithmetic error.

What each rule costs a cell that really is square #

A rule for deciding is characterised by how often it is wrong, and that can be measured. Draw twenty thousand cells at random around a true square cell, with the same uncertainties as the measurement — the draws seeded, so that the numbers are reproducible — and ask each rule whether each cell is square.

The two-condition $\chi^2$ rejects 4.85 per cent of them, which is five per cent within the sampling error of twenty thousand draws: that rule does what it says, and its p-values come out spread evenly between nought and one, as a calibrated test’s must. Testing each condition on its own rejects 9.80 per cent, because a true square cell has two chances to fail at five per cent each and $1 - 0.95^2 = 0.0975$. And reading the two-condition distance as one condition rejects 14.53 per cent, against the $e^{-1.92} = 0.1465$ that the geometry predicts: a rule nominally at five per cent that throws out one true square cell in seven, with its p-values piled up near nought.

The last rule is the one a tolerance quoted in standard uncertainties invites, and the calibration says what it is: wrong by a factor of three on every stratum with two conditions, and right only by accident on strata with one.

Walking up the diagram, one condition at a time #

The rule that keeps the ranking coherent is the one that respects the diagram’s own structure. A supergroup edge adds conditions, and the strata are nested: square is reached from rectangular by adding equal edges, or from rhombic by adding a right angle. Test each added condition on its own, one degree of freedom each, and accept a stratum only if every step on the way up to it passes.

For the nearly square cell both paths fail at the same step. Via the rectangle, the right angle passes at 0.45 and equal edges then fail at 0.027. Via the rhombus, equal edges fail first. So the walk rejects the square that the direct test accepts, and it cannot do the incoherent thing: a stratum is accepted only if every stratum below it on some path is, so no set ever outranks a set containing it.

The price is the error rate the calibration has just measured, 9.80 per cent for a nominal five. The standard remedy divides the level between the steps — two independent steps each at $1 - \sqrt{0.95} = 2.53$ per cent keep the whole walk at five per cent — and the calibration confirms it at 5.04 per cent. And at that divided level the nearly square cell passes, because its equal-edges probability of 0.0265 is above 0.0253. The verdict on this cell has now changed three times, with the rule, and it is a borderline cell by construction; what does not change is the pair of numbers underneath.

That is the useful conclusion. A structure report that says “no higher symmetry within tolerance” has made a decision with a rule it has not named, and the decision can flip under a rule equally defensible. The report that survives any rule is the list of one-condition $\chi^2$ values — 0.56 for the right angle and 4.92 for equal edges, in this case — from which a reader can reconstruct the verdict under whatever rule the question in hand needs.

Which cell the conditions are stated on #

Every condition above is a statement about three numbers, and the three numbers depend on which cell was chosen to describe the lattice. A rectangular lattice described by two vectors that are not its edges has no right angle anywhere in its description; a square lattice described by one edge and a diagonal has neither equal lengths nor a right angle. So “the angle is 90°” is a test of the lattice only if the cell it is stated on is one in which the lattice’s special angles show up as angles of the cell.

That cell is the reduced one: the shortest two vectors, with the angle between them taken between 90° and 120° — the shortest basis of the lattice, which is unique up to a finite set of choices. On it, every stratum of the plane is a condition of the form listed above, and a measured cell must be reduced before it is tested. The cell is a choice, and the tests here are tests of the lattice only because the choice has been made in the one way that shows its strata.

The reduction has its own edge case, and it is the same kind of edge. A cell near a boundary of the reduction conditions — an angle near 90° or near 120°, or two vectors of nearly equal length — has two reduced forms within its uncertainty, and the standard uncertainties of one are not simply those of the other. A cell near a stratum is exactly such a cell. The tests are unaffected where the two forms give the same conditions, and a cell close enough to a corner of the reduction domain to change which conditions apply is a cell whose description itself is uncertain, which no test on its parameters can repair.

What a cell near a stratum does to the crystal #

A cell that sits within its uncertainties of a higher stratum is not only hard to classify; it is a cell on which a crystal can grow in two orientations that nearly coincide. A rectangular cell whose edges are nearly equal can be rotated by a quarter-turn and still nearly fit itself, and a crystal that does this is twinned — two domains related by an operation of the higher stratum’s holohedry that is not an operation of the structure. The index and the angle a twin misses by measures how far from fitting such an operation can be and still be a twin law, and a twin hides in the statistics detects one from the intensities.

So a cell near a stratum raises two questions that are easy to confuse. Does the structure have the higher symmetry? And is the crystal twinned by it? The first is what the tests above address, and a clear “no” there makes the second more likely rather than less: a structure that is genuinely lower in symmetry, on a cell that is nearly higher, is exactly the structure that twins. The test for a supergroup and the search for a twin law ask the same arithmetic of the same cell and want opposite answers from it.

The covariance nobody reports #

One of those numbers depends on something a structure report rarely carries. The equal-edges test is about the difference $\hat a - \hat b$, and the uncertainty of a difference depends on how the two errors move together.

With no correlation the edges are 2.22 standard uncertainties apart. Refined cell edges are usually positively correlated — an error in the wavelength or in the distance to the detector stretches both the same way — and a positive correlation shrinks the uncertainty of their difference: at 0.6 the same edges are 3.32 apart, and at 0.9, 5.39. A negative correlation widens it, and at −0.6 the edges are only 1.78 apart and pass. The verdict on equal edges is decided by a covariance that most reported cells do not include, and a search that assumes independence is wrong in a direction set by that missing number.

What the computation depends on #

The cells are illustrations. The three measured cells are chosen to show the three effects and are not the cells of any compound; the calibration draws cells around one of them with its own uncertainties. Nothing here says how often real structures sit near a stratum, only how a decision made about one behaves.

The errors are taken as Gaussian and, except where stated, independent. Least-squares refinement gives standard uncertainties under assumptions of that kind, and they are known to be optimistic; underestimated uncertainties inflate every $\chi^2$ above, and a factor of two in σ is a factor of four in $\chi^2$. What the geometry of the three regions says does not depend on that: whatever the scale, one condition and two conditions are compared against different distributions.

The rhombic stratum has a second branch that the table leaves out. A reduced cell can be centred-rectangular with unequal edges when twice the projection of one edge on the other equals its length, and a cell near that condition would need the same test on that expression. The cells here are far from it, and the branch changes nothing about the argument.

And no figure here shows a symmetry. The regions are regions of a plane of measurements, and the diagram is a diagram of conditions; whether a structure on an accepted cell actually has the supergroup’s operations is a question about its atoms, which the pricing of each edge already said is a necessary condition and never a sufficient one.

The diagram as a sequence of questions #

The subgroup diagram of the plane groups has been read two ways. Downwards, every step is either fewer operations or fewer translations, and a maximal step does one or the other and never both. Upwards, the steps that add operations cost the cell conditions. What a measured cell adds is that the upward reading is also a sequence of statistical questions, one per added condition, and the diagram is exactly the structure that keeps their answers consistent: a path up is a chain of nested hypotheses, and testing along a path is what makes the answers monotone.

That is a better use for the diagram than the one it usually has. A symmetry search that tests every candidate supergroup against a fixed tolerance treats the candidates as a flat list and can return a set that no path through the diagram would reach. A search that walks the diagram cannot, and the only thing it has to be told is how to share its error rate among the steps.

Still open: three dimensions, where the strata are not a chain #

The plane has five lattice types and the conditions between them are two kinds, right angles and equal lengths. In three dimensions there are fourteen Bravais types over seven crystal families, the strata are nested in a lattice of inclusions rather than a chain, and some steps add three conditions at once — a cubic cell from a triclinic one needs three equal edges and three right angles. The number of paths to a stratum grows with the conditions it needs, and a walk that divides its level among the steps of every path is dividing it among more than two.

Whether the divided walk stays usable there — whether a level shared among five or six steps leaves each step enough power to reject anything — is the question the plane cannot ask. The calibration that answers it is the same one run here, on cells drawn around a true cubic cell with the six parameters of a real refinement and their covariances; it would say, for a cubic structure measured as well as a good diffractometer measures one, how often a correct search would call it tetragonal.