The parameter table#
This page gives the table of parameters a refinement builds from its models: the fields of a row, why a row cannot be freed, ties and restraints, the bounds a fit runs in, what the editing calls return and refuse, the rows a result reports, and the profile interval for a row on its bound.
A fit does not refine the Structure and Instrument objects directly.
It refines a flat vector, and the parameter table stands between the two.
The table walks both models, gives every Parameter it finds a name, and records which may be varied and which follow another.
(9.1) is the mapping, pphys = C·pfree + d.
The table is rebuilt from the models at every stage boundary, so a row is never stale against the objects it came from.
Parameter map lists every name with its unit, default and typical range. About parameters groups them by what they belong to.
Names#
A name is a dot-path that starts at phases, instrument or vars, the last for a variable you declared with Refinement.add_variable.
A number in the middle is a list index, counted from 0 in the order the model stores the list.
The same names, with * and ? as wildcards, are what Refinement.set_vary, Refinement.hold, Refinement.untie and a stage’s turn_on list take.
A name never contains square brackets.
The matching reads [0] as a set of characters, so phases[0].cell.a matches nothing and raises no error.
Some names exist only when the model declares the block they belong to.
phases.0.microstrain.dof.* appears once the phase carries a StephensStrain, phases.0.atoms.2.adp.* once that atom carries an AnisoU, and instrument.geometry.capillary_offset_along_beam only on a capillary geometry.
A wildcard over an absent block matches nothing, so the shipped plans’ broad names are safe on any model.
Rows#
Refinement.parameters returns the whole table as a list of ParameterRow, in the order the free vector uses.
It needs no fit.
import rietx as rx
lab6 = rx.Structure(phases=[rx.Phase(
name="LaB6",
space_group="P m -3 m",
cell=rx.Cell.cubic(4.15689, vary=True),
atoms=[
rx.Atom(label="La", species="La", x=rx.Parameter(value=0.0),
y=rx.Parameter(value=0.0), z=rx.Parameter(value=0.0)),
rx.Atom(label="B", species="B", x=rx.Parameter(value=0.19964),
y=rx.Parameter(value=0.5), z=rx.Parameter(value=0.5)),
],
)])
ref = rx.Refinement(lab6, rx.Instrument.debye_scherrer(wavelength=0.4139),
history=False)
rows = ref.parameters()
assert len(rows) == 42 # every scalar, held ones included
assert sum(row.refinable for row in rows) == 25 # what set_vary could free
held = {row.path: row.held_because for row in rows if not row.refinable}
assert held["phases.0.cell.b"] == "tied: = 1·phases.0.cell.a"
assert held["phases.0.cell.alpha"] == "structurally fixed by symmetry or by the model"
Two atoms and a default instrument make 42 rows, of which 25 could be freed.
Field |
Type |
Holds |
|---|---|---|
|
|
the name |
|
|
the current value, in the parameter’s own unit |
|
|
whether it is free in the next fit |
|
|
the stored bounds, inclusive, ±inf where unbounded; a stage can add one the row does not show (Bounds a stage adds) |
|
|
what this value follows (Ties) |
|
|
fixed by the structure; no call can free it |
|
|
held by |
|
|
the esd from the most recent fit, where the row was free or tied in it |
|
|
fixed by the Le Bail or Pawley mode |
|
|
a wavelength that cannot be freed while this pattern’s cell is free |
|
|
the direction along which this row holds a polar group’s floating origin |
|
|
the parameter map entry that describes this row’s family |
The first eight fields mirror the solver’s own row type field for field, and a test holds the two equal.
The rest are deliberate additions.
esd is a property of a completed fit, and merging it in lets one listing say both what a parameter is worth and how well it is known.
A tied row’s esd is its sources’ propagated through C, as σ² = diag(C·Cov·Cᵀ), so an identity tie reports its source’s esd.
A row gets an esd only when at least one of its sources was free, and None means the esd is unavailable rather than zero.
ParameterRow.refinable and ParameterRow.held_because are derived.
refinable is the one test a program should grey a row by, and held_because is the sentence to show beside it.
Held rows#
A row cannot be freed for one of five reasons, and the remedy differs.
Reason |
Means |
Lifted by |
|---|---|---|
|
fixed by the structure: a cell angle the space group fixes, a coordinate on a fully fixed special position, the first emission line’s weight, a non-primary line’s wavelength, |
nothing |
|
an affine function of other rows, so the freedom lives in its sources |
|
|
refinable in principle, but the Le Bail or Pawley mode fixes it: every atom parameter, each phase’s scale and a texture coefficient |
switching back to |
|
you declared it does not move, whatever a plan frees; a coordinate held to fix a polar group’s origin reads |
|
|
a wavelength, while this pattern’s cell is free, since d = λ/(2 sin θ) fixes only their product |
holding the cell; the one reason another parameter decides |
ParameterRow.refinable is false if any of the five holds.
ParameterRow.held_because reports the first that applies, in an order set by what you can do about it.
A row a rigid body places names the body first and says to refine the body’s origin and rotation.
The structural reasons come next, then the mode, then a hold, then the wavelength.
A Le Bail phase’s held atom therefore names the mode, which is the reason that would still hold if the hold went.
The counts of the five do not add up to the number of held rows.
On the LaB6 table above, the Le Bail listing marks 12 rows mode_fixed, while the refinable count falls only from 26 to 20, because 6 of the 12 were already locked or tied.
Read refinable for the decision, and the five flags only to explain it.
import rietx as rx
lab6 = rx.Structure(phases=[rx.Phase(
name="LaB6", space_group="P m -3 m", cell=rx.Cell.cubic(4.15689),
atoms=[rx.Atom(label="La", species="La", x=rx.Parameter(value=0.0),
y=rx.Parameter(value=0.0), z=rx.Parameter(value=0.0)),
rx.Atom(label="B", species="B", x=rx.Parameter(value=0.19964),
y=rx.Parameter(value=0.5), z=rx.Parameter(value=0.5))],
)])
ref = rx.Refinement(lab6, rx.Instrument.debye_scherrer(wavelength=0.4139),
history=False)
lebail = ref.parameters(mode="lebail")
assert sum(row.mode_fixed for row in lebail) == 12
assert sum(row.refinable for row in lebail) == 20
Refinement.parameters takes mode because the mode the object carries is the one its last stage ran in, which before the first run is the rietveld default.
A program that knows the mode of the next run passes it.
Without it, a Le Bail project’s atom rows come back looking editable.
A Le Bail phase must carry a dummy atom to exist at all, and that atom’s biso is not a parameter to offer anyone.
Ties#
TieSpec is the stored form of value = Σ c·source + k.
Field |
Holds |
|---|---|
|
the (path, coefficient) pairs |
|
the additive constant k |
|
|
|
the paths this value follows, which are the ones to edit instead |
|
the right-hand side as text, such as |
A symmetry tie and a tie you declared hold a row the same way, and held_because reads the same for both.
TieSpec.user tells them apart.
A symmetry tie is derived again from the space group every time the table is built, and nothing removes it.
A tie you declared lives in the history, and Refinement.untie removes it.
Symmetry outranks a tie of your own, so a coordinate refuses one.
How to tie parameters together gives the calls, what they refuse and how a dependent’s bounds carry onto its source.
A cell tie is an identity, 1·phases.0.cell.a.
A coordinate tie carries the starting position in its constant: on the LaB6 table above, phases.0.atoms.1.x describes itself as 0.19964 + 1·phases.0.atoms.1.dof.0.
A coordinate’s degree of freedom is a displacement from the stored coordinate (9.2).
Anisotropic displacement and Stephens degrees of freedom are absolute instead, which holds their site symmetry exactly.
Tying a coordinate’s degree of freedom to another is how two atoms are made to move together.
The anchor is the coordinate as it stood when the tie was declared.
phases.0.atoms.1.dof.0 then reads the displacement its tie implies, and phases.0.atoms.1.x reads the sum of the two.
That holds at every rebuild for as long as the tie is declared, so a second Refinement.fit reports the amplitude the first one did.
A dependent’s bounds carry onto its source exactly where the tie has one source, which is every tie rietx derives and most that anyone writes. With several sources the allowed region is a slanted boundary, and the solver can only be given a range. It is given the smallest range containing every allowed point, which rules out no answer you asked for and can leave a corner where two sources together push the dependent outside its bounds. If a fit lands in that corner, writing the values back to the models refuses, naming the parameter, its bounds and the tie that drove it.
Restraints#
A restraint adds a row to the residual and leaves the table as it is.
It is an observation with an uncertainty, a distance, an angle or a value known from chemistry, and the parameters it touches stay free.
Restraints are declared on Phase.restraints, and How to restrain a bond length or an angle gives the procedure.
Field |
|
|
|
|---|---|---|---|
the quantity |
|
|
|
the target |
|
|
|
the uncertainty you claim |
|
|
|
a multiplier on the row |
|
|
|
the symmetry image |
|
|
none |
An atom field is a position in Phase.atoms, counted from 0, as in the names.
Each restraint adds one row, √weight·(computed − target)/σ, after the data rows (9.17).
sigma sets how hard the restraint pulls, and weight multiplies the row on top of it.
Stage.restraint_weight_scale multiplies every row for one stage (9.18).
The rows enter the covariance, so they tighten the esds of the parameters they touch.
They are left out of Rwp, the Durbin-Watson statistic and the Bérar-Lelann inflation, because they are not measurements of the pattern.
RefinementResult.restraints reports each one’s computed value and deviation (Restraint report).
A distance obeys periodic boundary conditions, so the second atom is taken at a symmetry image. Leave the image unnamed and the nearest one is found once, at the coordinates a stage starts from, and kept for that stage. Name it whenever an atom may move far enough to change which image is nearest.
Two one-sided kinds, a tether and an anti-bump, hold a distance within a range. They belong to the rigid-body technique, which is provisional by declaration (Provisional by declaration), and are documented with it.
Bounds#
ParameterRow.value, ParameterRow.lo and ParameterRow.hi are what the solver sees.
Every parameter refines in its own unit, and the bounds go to the solver as a box.
The default solver solves each step inside that box, so a parameter the data push against a bound lands exactly on it (Solvers).
A width or a scale can therefore end on its off state at 0.
A field whose physics divides by the value carries a floor above 0 instead (Parameter).
A Parameter you pass to a field inherits the min, max and unit you left unset from the field’s own default, and a bound you state yourself wins (Parameter).
A value outside the inherited range is refused when the model is built, naming the field and the range.
State the range you mean on the Parameter, or, for a coarse instrument’s widths, build them with ProfileTCHZ.coarse.
A file written before schema 0.46 stored a transform word instead of bounds, and it is read as the box it enforced.
Stored |
Bounds it becomes |
|---|---|
|
unchanged |
|
|
|
|
So Parameter(value=0.5, min=-1.0, transform="softplus") builds with min=0.0, and a project saved by an earlier release opens with the bounds its fits ran in.
Bounds a stage adds#
A stage adds three default bounds of its own, each only on a side you left at ±inf.
A bound you set wins, so setting any finite bound on that side turns the default off.
None of them is written to the stored parameter, so ParameterRow.hi can read inf on a row that stopped on one.
A phase whose summed contribution sits below 3σ of the counting noise when the stage begins contributes nothing the fit can see.
Every structural parameter of a phase reaches the pattern only through scale × |F|² × profile, so its cell could then move without changing Rwp at all, out of the physical range.
The stage bounds that phase’s cell: each length to ±5 % of the value the stage starts from, plus 0.05 Å, with a floor of 1.5 Å, and each angle to ±2°.
The bounds follow TOPAS’s default cell limits, re-centred at each stage rather than at each iteration.
Only that phase’s cell is bounded.
Under the default solver, a bound the step never reaches leaves each step unchanged.
Under solver="trf" a bound changes how a parameter moves even when it is never reached, because TRF scales its step by the distance to the bounds, so the window is kept off every phase the data can see.
Such a cell can report BOUND_HIT while its ParameterRow.hi still reads inf.
In a staged plan the window is rarely reached, because the stage holds that phase’s structural parameters instead (StageResult.held, Held parameters).
PHASE_UNCONSTRAINED names a phase the data cannot tell from absent and says what the run did with its parameters.
A free cell that moves more than ±15 % in length or ±6° in angle within one stage has run away.
The stage puts that phase’s free cell parameters back at the values the stage started from, holds them, and solves the stage again.
Before 1.10 the check clamped the value to the 15 % edge, which left a number that was no measurement.
StageResult.cell_reset lists the parameters put back, and CELL_RUNAWAY reports it.
That check reads the answer and is never a bound the solver sees.
A runaway in the final stage leaves the cell at its start values and withholds its esd, and the esd of every parameter tied to it.
The sample strain and size widths get the other two default bounds, armed only on a term that has already reached them.
A strain width is held to the width the fitted 2θ range can express, and a size width to the same width read through 1/cos θ (the strain bound, the size bound).
No default bound limits the crystallite size itself.
SIZE_UNUSUALLY_SMALL speaks below its threshold and bounds nothing.
Editing the table#
Refinement.set_vary and Refinement.set_values edit the table, and each records a history node, because freeing a parameter and setting one are refinement moves.
The tree is created on the first Refinement.fit or Refinement.run_stage, since it is pinned to its pattern by a fingerprint.
Both calls change the working state before then, but a call before the first fit is not in the log (The refinement history).
Refinement.set_vary takes a name or a list of names, with wildcards, and returns the paths it changed.
Read the return rather than the list you offered it.
A locked or tied row never matches, however broad the name, so the hits are the account of what the call did.
import rietx as rx
lab6 = rx.Structure(phases=[rx.Phase(
name="LaB6", space_group="P m -3 m", cell=rx.Cell.cubic(4.15689),
atoms=[rx.Atom(label="La", species="La", x=rx.Parameter(value=0.0),
y=rx.Parameter(value=0.0), z=rx.Parameter(value=0.0)),
rx.Atom(label="B", species="B", x=rx.Parameter(value=0.19964),
y=rx.Parameter(value=0.5), z=rx.Parameter(value=0.5))],
)])
ref = rx.Refinement(lab6, rx.Instrument.debye_scherrer(wavelength=0.4139),
history=False)
assert ref.set_vary("phases.*.cell.*") == ["phases.0.cell.a"] # five of six are held
assert ref.set_vary("phases.*.cell.alpha") == [] # locked by symmetry
assert ref.set_vary("phases.*.atoms.*.x") == [] # tied to a site-symmetry direction
assert ref.set_vary("instrument.profile.u", vary=False) == ["instrument.profile.u"]
A cubic cell returns one path from a name that matches six, and that one hit is the whole of the cell’s freedom.
A held row is skipped by a wildcard name, as a locked one is.
A literal path to a row you held raises ValueError, saying to Refinement.unhold it first.
A row the current mode fixes is the exception to the matching rule: set_vary frees it, and the next stage drops it again and reports it as mode_fixed.
The vary flags on the models you passed in are the starting declaration only.
Once set_vary, a hold, a tie or a stage has declared a free set, the table reads that set, even when it is empty.
A plan replaces the flags at every stage, so vary=False alone does not keep a parameter fixed, and Refinement.hold does (How to keep a parameter fixed whatever the plan frees).
Refinement.set_values takes a dict of paths to values.
It is plural because a table is edited several cells at a time, and one node per value would bury the log.
It raises ValueError on four refusals, each with its own remedy.
Refusal |
Message |
Remedy |
|---|---|---|
unknown path |
|
correct the name |
locked |
|
none; the structure owns the value |
tied |
|
set the source |
out of bounds |
|
a value inside them, since the bounded solver cannot start outside |
A tied parameter follows its sources.
Setting phases.0.cell.a on a cubic phase moves b and c with it, and the change reaches the objects.
Refinement.structure and Refinement.instrument are the refinement’s own deep copies of what you passed in, and they are what the table writes back to.
Refinement.fitted_structure and Refinement.fitted_instrument return the same two objects, under names that say when you are reading them.
Setting a value also clears the fitted curve and its statistics, which described the previous values.
Refined rows#
A result carries its own view of the table.
RefinementResult.parameters is a list of RefinedParameter, and a row appears if its parameter varied or was tied.
A fixed parameter is absent rather than present with vary=False.
RefinementResult.parameter returns the row for one path.
Field |
Holds |
|---|---|
|
the name |
|
the value the fit ended at |
|
the esd, or |
|
the same esd without the Bérar-Lelann factor, as GSAS-II and TOPAS print it (Estimation); |
|
|
|
|
|
the parametric declarations that set the row (Parametric refinement), empty where none did, and |
|
the profile interval (Profile interval) of a moment angle the fit left out of its covariance at a direction the symmetry fixes (Magnetic refinement), and |
A parameter the data said nothing about reports no esd rather than a small one.
A free parameter can end in a direction the residual does not move at all, such as the centre of a declared peak whose area refined to zero, and there is no variance to report for it.
Its stderr is None, and so is the esd of every tied row that draws on it.
Read a None esd on a row you meant to refine as a sign to take that parameter out of the plan.
A scale or a peak area that refined to zero is not such a direction: its column is still the pattern it would add, so it keeps its esd, and at_bound says whether the floor held it.
On a bound#
A value resting on a bound the data push against is not a measurement, so do not quote it with its esd.
|
Means |
|---|---|
|
the fit stopped against a bound with the data still pushing outward; the same rows |
|
tested, and the bound carried no load |
|
not tested, so no answer either way |
None covers two cases.
A tied row is never tested, because it is not in the free vector the fit solves, and it can sit on its own declared bound while every source is interior.
A result built without a fit behind it has nothing to report, so every row of a replay is None.
A free scale or width on its floor of 0 is tested like any other bound, and an absent phase’s scale reads True when the data push it onto that floor.
It keeps its value and its esd, and so does every other row: no esd is computed as if the floored row were fixed.
A floor reports as FLOOR_HIT (a lower bound of 0, or W’s floor) and any other bound as BOUND_HIT, because only the second can be widened.
What counts as on a bound depends on the solver.
Under the default solver="lm" a row is on its bound when it is in the solver’s exact active set: it sits on the limit and the gradient there points out of the allowed range.
Under solver="trf" a value only comes close, so a row is on its bound when it sits within a hundredth of its own esd of the limit and the residual still pushes it outward.
The default solver takes that second test too when a stage frees more than 128 columns.
Neither test reads the distance alone, so a wide min=1e-14, max=1e14 written to mean “leave this alone” does not make a value read as pinned.
The flag and the two diagnostics come from one bound test, so they cannot disagree. Read the diagnostic for the list, and the flag when you are already iterating rows.
The table against the result#
The two views differ in size.
A Le Bail fit of the 11-BM NAC pattern over 2-24° gives 73 rows from Refinement.parameters and 31 from RefinementResult.parameters: 13 free, 18 tied and 42 fixed rows that the result leaves out.
All 13 free rows come back False, and all 18 tied rows None: cell.b, cell.c and 16 symmetry-tied coordinates.
Capping cell.a at 10.2500 Å against a free optimum of 10.2512 Å turns that one row True and takes Rwp from 0.1435 to 0.2163.
The other parameters cover for a cell the fit is not allowed to reach.
Refinement.parameters merges the most recent fit’s esds onto ParameterRow.esd.
In that fit phases.0.cell.b and phases.0.cell.c both carry 5.42×10⁻⁵ Å, a’s esd.
The tied coordinate rows carry none, because the fit freed no coordinate degree of freedom.
Use the result to report a fit, and the table to decide what to do next.
Profile interval#
An esd read as value ± esd is symmetric.
On a bound that interval reaches into values the bound forbids, and the estimate there has no symmetric sampling distribution to give it meaning.
Refinement.profile_interval gives such a row a range to quote, as a ParameterInterval (How to quote a parameter that ended on its bound).
It pins the parameter at a value, refits everything the last fit freed, reads the data’s χ² there, and finds where that χ² has risen by a threshold on each open side.
interval = ref.profile_interval(data, "instrument.profile.y")
print(interval.lower, interval.upper, interval.one_sided)
A row whose RefinedParameter.at_bound is True gets a one-sided interval.
The bound is one end, and the other is where χ² has risen by 2.71 × χ²red × f², with f the fit’s Statistics.esd_inflation.
Any other row gets a two-sided interval, at 3.84 × χ²red × f² on each side.
Both are 95 % points, and level moves them.
The two factors put the threshold on the scale of the esds, so on a quadratic profile a two-sided interval is the value ± 1.96 esd.
Each side steps out by 1, 2, 4, … esd until χ² passes the threshold, then solves for the end to a thousandth of an esd.
A side that reaches the parameter’s own limit first reports the limit as its end.
Each pinned value is refitted until its χ² stops falling, up to four times, because a stage keeps the peak windows it sized when it started.
max_refits caps each side at 60 refits by default.
Every refit runs on a branch, so the working state, Refinement.result_ and the history’s head are left as they were.
A fit must have run, and the path must have been free in it; a tied, locked or fixed path is refused with the reason.
Field |
Holds |
|---|---|
|
the parameter profiled |
|
the fit’s value, copied from its result row |
|
the fit’s esd for it |
|
the row’s |
|
the confidence level |
|
whether the bound is one end |
|
the χ²₁ point the threshold is built from, 3.84 or 2.71 at 95 % |
|
the threshold in χ², the quantile × χ²red × f² |
|
the two ends, or |
|
why each side stopped |
|
the fit’s own data χ², not reduced |
|
the lowest data χ² found, the fit’s included |
|
whether every refit stayed at or above the fit’s χ² |
|
one |
|
the refits those points took |
Field |
Holds |
|---|---|
|
the pinned value |
|
how many refits it took to settle |
|
the lowest data χ² those refits reached |
|
its distance above |
|
how the refit’s stage ended |
|
why a refit raised |
|
the history node the refit recorded, and |
Stop |
Means |
|---|---|
|
χ² rose by the threshold, and the end is that point |
|
the side reached the parameter’s own limit first, and the end is the limit; a one-sided interval’s bounded end reads this |
|
the side ran out of refits, and has no end |
|
a refit raised, and the side has no end |
ParameterInterval.fit_is_minimum is False when a refit found χ² below the fit’s by more than 1 % of the threshold.
The fit then did not reach the minimum along this parameter, and the ends, measured from the fit, are too wide.
Refit before quoting them.