Sequential refinement#
This page gives the calls that fit a series of patterns one after another, the arguments of a series, what crosses from one pattern to the next, the SeriesResult they return with its trajectories, the checks against a trend the chain made, and the events and cancellation of a series.
A series is N separate refinements in order, each starting from the answer of the one before. A joint fit is one refinement whose residual stacks several patterns, and the patterns share the parameters that describe the specimen (Joint refinement).
Series |
Joint fit |
|
|---|---|---|
Residual |
N of them, solved one at a time |
one, with every pattern’s points in it |
What crosses a pattern |
the starting values |
the shared parameters themselves |
Answer |
a trajectory, such as a(T) or w(t) |
one set of numbers, informed by every pattern |
Used for |
an in-situ ramp, a parametric sweep, a tray of specimens |
one specimen measured twice, at two wavelengths or on two instruments |
Entry point |
|
|
Modes |
any |
Rietveld only |
The question that separates them is whether the specimen changed. Eight mixtures with different compositions are eight specimens, so their cells are eight measurements and a series is the right shape. One powder measured at two wavelengths is one specimen, so its cell is one number and a joint fit is the right shape.
Fitting a(T) to a functional form of T across every pattern at once is parametric refinement [Stinton and Evans, 2007]. It puts every pattern in one residual, so it is a joint fit (Parametric refinement). A chain cannot do it, and a line drawn through a sequential trajectory afterwards is a different measurement. How to fit a temperature series runs a series from start to finish.
Entry points#
SequentialRefinement takes the starting models once and the patterns at SequentialRefinement.fit.
refine_sequential is the same run as one call.
import rietx as rx
series = rx.SequentialRefinement(structure, instrument)
result = series.fit(patterns, x=temperatures, x_label="T (K)")
result = rx.refine_sequential(patterns, structure, instrument,
x=temperatures, x_label="T (K)")
Both return a SeriesResult.
The runner keeps more after a fit:
Attribute |
Holds |
|---|---|
|
each pattern’s full |
|
each pattern’s history tree |
|
the |
|
under |
|
the backward chain’s |
|
the patterns on which every attempt raised, as |
|
each entry’s refined models, in series order; a pattern that failed has none, so these can be shorter than the patterns |
|
the package’s own deep copies of the models you passed, so a fit does not move your originals |
SequentialRefinement.forward_ lets an events= callback judge the forward trajectory when the first event stamped series_pass="backward" arrives.
It carries every diagnostic of the forward chain and none from the comparison of the two chains, and it is not rewritten when the run finishes.
Five settings are constructor arguments, because they describe the whole chain: backend, solver, history, SequentialRefinement.carry and SequentialRefinement.carry_hkl_intensities.
Entry points gives where each setting of a single fit goes.
history behaves as it does for a single refinement (The history setting), with one addition.
Given a directory, each pattern’s first attempt is written to <dir>/<label>.jsonl.
A later attempt on the same pattern writes <label>.warm_staged.jsonl or <label>.cold.jsonl, and the backward chain of direction="both" adds .backward before that suffix.
So the tree a refitted pattern’s SeriesEntry.node_id points into is in the suffixed file.
There is one tree per fit and none for the series.
A tree is pinned to its pattern by a data fingerprint, and that pin stops a node being replayed against the wrong data.
The chain is recorded instead as annotation notes on each tree’s root node (Tags and annotations).
The default is False, because a long series makes many trees.
Series coordinate#
x is the series coordinate: the quantity the experiment varied, or on an in-situ run the point of the experiment.
A vendor file records it where its format has a field for it, and the reader puts it in the pattern’s own metadata.
import rietx as rx
patterns = [rx.read_pattern("ramp.raw", scan=i) for i in range(68)]
temperatures = [float(p.metadata["temperature_k"]) for p in patterns]
PatternData.metadata holds strings, so the conversion is yours.
Read the key with dict.get and refuse to go on when it is missing.
An absent key is a file that recorded nothing, and not a specimen at ambient temperature.
As of rietx 1.10, the Bruker .raw version 3 range header is the only format here with such a field.
A reader does not guess a temperature from an axis named for something else.
rietx.io.readers.list_scans answers the same question without reading the patterns.
It returns one rietx.io.formats.base.ScanInfo per scan, each carrying its index, n_points, the stepped range and the temperature where the file gave one.
The temperature also appears in its label, since the scans of a reel are otherwise indistinguishable.
Arguments of fit#
SequentialRefinement.fit and refine_sequential take these.
Argument |
Default |
Does |
|---|---|---|
|
required |
the series, in order; each pattern keeps its own σ, and patterns are never pooled |
|
|
the series coordinate and its name; without |
|
|
a name per pattern, used in messages, tables and history filenames |
|
|
as for a single fit: |
|
|
the plan run on the first pattern and on any pattern refitted from the starting models |
|
|
|
|
|
one fit range, applied to every pattern |
|
|
|
|
|
the fence that rejects a bad warm start, and how far above the median Rwp it fires (Restart ladder) |
|
3.0 |
how much the first attempt may spend, as a multiple of the dearest first attempt this chain has converged; |
|
|
refit each flagged step’s two patterns cold and independently (Discontinuity check) |
|
|
|
|
|
|
|
|
|
|
|
what a pattern on which every attempt raised does to the chain: |
|
|
as on |
refit sets the first attempt on each warm-started pattern, and the ladder may use the full plan later (Restart ladder).
The staged order keeps early stages well conditioned from a poor starting model, and a converged neighbour is not a poor starting model, so the default collapses the plan.
When the neighbour is not a good start either, the fence catches the pattern whose Rwp jumps past the median.
It does not catch a collapse every pattern shares, because the median rises with it.
With a plan that frees U, V and W, "single" settled 1.04-1.29 times above "stages" in Rwp on every round-robin mixture after the first, and no pattern was refitted.
What named it was SEQUENTIAL_PERSISTENT_FINDING on RESOLUTION_UNCONSTRAINED for U, V and W, in six or seven of the eight patterns and in none under "stages".
Carried parameters#
SequentialRefinement.carry is a list of dot-path globs, matched as Refinement.set_vary matches them, naming the parameters carried from one pattern to the next.
The default is ["*"], every parameter.
A parameter it leaves out restarts from the starting model on every pattern.
A refined atomic coordinate crosses as its site’s displacement.
The degree of freedom a plan frees (phases.*.atoms.*.dof.*) is rebuilt at zero for every fit, so the chain carries where the fit moved the atom.
A glob naming either the coordinate or its degree of freedom moves every coordinate the site ties to it, and the atom stays on its special position.
The degree of freedom’s value is measured from where its fit began, which in a chain is the previous pattern’s answer, so its trajectory is each pattern’s step.
Chart the coordinate rows instead.
They carry the same esd, and the discontinuity and path-dependence checks judge them.
A Le Bail or Pawley pattern’s per-hkl intensities are not parameters, so no glob reaches them.
SequentialRefinement.carry_hkl_intensities decides whether they cross.
The default, True, seeds each warm pattern from its predecessor’s intensities, matched by hkl.
With False, every pattern’s extraction starts afresh, as the first pattern’s does.
On a synthetic ten-pattern fluorapatite ramp, the carried and fresh Pawley chains agree to five digits of Rwp on every pattern, and the carried Le Bail chain ends lower, at 3.95 % against 4.29 %.
Carrying everything is cheap. The eight IUCr round-robin sample-1 mixtures share three phases and one goniometer, while their composition swings from about 1 to 94 wt % across the set. On rietx 1.9.0 the chain took 365 least-squares iterations against 1214 for the same patterns fitted independently, 3.0 times fewer, at the same weight fractions.
prepare is for a parameter estimated afresh from each pattern, which no carry glob can express.
Excluding the phase scales from carry on that round-robin series would fall back to the first mixture’s guess.
Constraints on every pattern#
A tie, a named variable and a hold live on the Refinement and in no model.
No carry glob reaches them, and prepare runs before that Refinement exists.
constrain is where they go (How to tie parameters together, How to keep a parameter fixed whatever the plan frees).
def constrain(index, ref):
ref.add_variable("B_site", 0.5, min=0.0, max=5.0)
ref.tie_equal(["phases.0.atoms.0.biso", "phases.0.atoms.1.biso"],
source="vars.B_site")
ref.hold("phases.0.cell.*")
result = series.fit(patterns, constrain=constrain)
Each pattern has its own parameter table, and tie_equal resolves its globs against the live one.
Declaring the constraint again on each pattern makes it mean the same thing on all of them.
So the function runs once per fit, and a pattern can be fitted several times: every attempt of the restart ladder calls it, and so does the cold refit of the discontinuity check.
A raise inside it ends the series, as a raise in prepare does.
A held path drops out of that pattern’s SeriesEntry.parameters and out of the trajectory, because a held value is your input and not a measurement.
On a two-pattern synthetic chain, the trajectory of a held cell edge comes back empty, against two points with esds without the hold.
vars.B_site is an ordinary dot-path, so carry governs it like any other parameter, and the variable warm-starts from the last accepted pattern.
On a seven-pattern synthetic whose tied biso walks from 0.4 to 3.4 Ų, that saves about 5 % of the chain’s iterations, 176 against 186, and moves no fitted value beyond the fourth decimal.
carry=["phases.*.cell.*", "vars.*"] carries exactly what that glob says.
Only the chain knows which pattern was accepted, so a quarantined pattern seeds no successor here either.
A coordinate’s degree of freedom tied to a variable is where the two carries meet.
The carried coordinate already holds the variable’s fitted displacement.
The chain therefore takes that displacement back out before warming the variable, and the pattern starts where its predecessor left the atom rather than one displacement further on.
On a synthetic LaB6 ramp that start was 0.1996, against the 0.2092 that warming it naively gave.
Leave vars.* out of carry and the coordinate restarts with its variable.
Series result#
SeriesResult is the serialisable answer, a few kB of refined values, esds, agreement indices and diagnostics.
It stores no curves, which for nine patterns come to about 2 MB of JSON already on disk as the input files.
The curves stay on SequentialRefinement.results_.
Member |
Holds |
|---|---|
|
one |
|
the mode the series was fitted in |
|
|
|
under |
|
the axis name |
|
the series diagnostics (Series diagnostics) |
|
one |
|
the paths whose value is a step from where each fit began, and the angle paths (Paths to plot) |
|
the patterns on which every attempt raised, and how many (Restart ladder) |
|
package version, timestamp and settings, as for a single fit |
|
the axis: the coordinate given, or the pattern index |
|
one label per entry |
|
one Rwp per entry |
|
residual evaluations summed over the entries: the reported chain, not the whole run |
|
the printed view, which |
It iterates, indexes and has a length, so for entry in result walks the patterns in order.
SeriesResult.diagnostics is the place to start reading a series.
A series is where one unread warning multiplies: in the in-situ run this diagnostic comes from, 425 BOUND_HIT findings went unread across 68 patterns, and the parameters they named were quoted as a measured trajectory.
Series entry#
SeriesEntry is one pattern’s place in the series and how its fit went.
Field |
Holds |
|---|---|
|
position in the series |
|
its name |
|
its coordinate, or |
|
|
|
that pattern’s own |
|
the |
|
the phase quantities, when the fit produced them |
|
per-phase |
|
one |
|
that pattern’s own diagnostics |
|
residual evaluations over every attempt on this pattern that returned a fit |
|
the warm start was rejected and the pattern was refitted cold |
|
the Rwp the first, warm attempt reached, set whenever the ladder escalated |
|
which attempt produced these values: |
|
every attempt made, in ladder order |
|
the attempts among those whose fit raised, each with |
|
the Rwp limit the fence set at this pattern: |
|
the kept attempt is still above |
|
where this pattern’s history lives |
SeriesEntry.value and SeriesEntry.stderr look a path up in the entry’s parameters and return None when it is absent.
SeriesEntry.rung and SeriesEntry.reseeded answer different questions.
rung says where the numbers came from, and the first pattern of a chain is always "cold" because it has no predecessor.
reseeded says whether the chain was broken here, and only that has a fence.
"warm_staged" does not set it, because it is still a warm start.
SeriesEntry.phase_agreement does not test whether a minor phase is real.
Both indices are biased towards the model being tested, and a trace phase’s index is not comparable with the major phase’s (Structure agreement indices).
In a series the field serves the trajectory: one pattern’s RBragg is a value, and sixty of them are a shape.
Whether the data can see a phase at all is measured by PHASE_UNCONSTRAINED, from each phase scale’s value over its counting-noise esd.
It reaches an entry through SeriesEntry.diagnostics, and aggregates over the chain as SEQUENTIAL_PERSISTENT_FINDING, which says what no per-pattern diagnostic can: 42 of 68.
Read RBragg beside that, and beside the weight fraction with its esd.
A phase that appears part-way#
In an in-situ series a product phase is often in the model from the first pattern and in the specimen only from some point on. Below that point the fit holds the phase’s structural parameters, because the data cannot see the phase. The entries say so three ways:
SeriesEntry.parametersdoes not list them, because a held parameter was not free, has no esd and did not move.SeriesEntry.diagnosticscarriesPHASE_UNCONSTRAINED, naming the phase and the stages that held it.The fitted model keeps the value you supplied. Read it as your own input.
Above that point nothing is held, including in the pattern where the phase first appears. If the phase appears while a stage is still solving, that stage lifts the hold and solves again rather than deferring to the next pattern.
SeriesResult.trajectory reads the entries’ parameter lists, so a held pattern contributes no point.
Trajectory.x begins at the onset, and its length is the number of patterns that measured the parameter.
Trajectories#
SeriesResult.trajectory returns one parameter’s path across the series as a Trajectory.
Member |
Holds |
|---|---|
|
the dot-path |
|
the axis and its name |
|
the value at each point |
|
its esd, |
|
the pattern label at each point |
|
which entry of |
|
|
A pattern where the path is absent has no point.
The gap is a real thing, such as a phase not yet in the model or a stage that did not run.
len(trajectory) is therefore the number of points with a value, and not the number of patterns.
SeriesResult.qpa_trajectory gives a phase’s weight fraction as a percentage.
SeriesResult.agreement_trajectory gives a phase’s structure agreement index, with metric="r_bragg" (the default) or "r_f".
SeriesResult.agreement_phases lists the phases that carry one.
A phase can carry an agreement index without a weight fraction, because a weight fraction needs Z and a molar mass and an R index does not.
Every esd of an agreement index is None, because RBragg is a residual and has no covariance entry to propagate from.
arrays() turns them into NaN, for which a plot draws no error bar.
Outside Rietveld mode the agreement trajectory is empty.
In Le Bail mode the partition is the fit, and in Pawley mode the intensities are refined, so a zero would read as a perfect fit.
SeriesResult.resolve_trajectory is the one call behind these and the moment’s trajectory (Magnetic moment across a transition).
It dispatches on a display path’s qpa., r_bragg., r_f. or magnetic. prefix, and passes any other dot-path to trajectory.
The plots and the GUI call it.
SeriesResult.is_derived_path says whether a display path names one of those four derived curves.
SeriesResult.plot plots one or more trajectories against the series axis, and style="dark" draws it for a dark page.
It rings a point refitted from the starting models, crosses a SEQUENTIAL_UNRECOVERED point, boxes a SEQUENTIAL_RWP_OUTLIER point, shades a flagged discontinuity’s pair, and tags the title of a path-dependent trajectory.
Paths to plot#
SeriesResult.paths lists every parameter path present anywhere in the series, in first-seen order.
varied_only=True drops the tied paths.
The default keeps them, because a hexagonal cell.b is not free but is as measured as cell.a.
On the round-robin series that is 49 paths, 41 of them varied.
SeriesResult.moved_paths returns at most n display paths (default 6), largest movement first.
Movement is a path’s peak-to-peak spread divided by the median of its own esds, so a cell edge does not outrank a weight fraction for being larger in its units.
Only paths that moved at least min_sigma esds (default 3) are returned.
The list goes straight to SeriesResult.plot:
result.plot(result.moved_paths(), path="moved.png")
The spread is taken over measured points: a point with an esd, on a pattern that did not diverge.
Three kinds of path are never ranked.
A path with no esd, which includes every agreement index, has no scale to rank on.
A path whose value is a step from where each fit began, a coordinate degree of freedom or a rigid body’s rotation, gives way to the coordinate rows it drives.
A tied path that repeats an earlier candidate’s values exactly, such as a cubic cell.b, is the same curve.
An angle path is spread over the shortest arc that covers its values.
Only the run knows which paths are steps and which are angles, so it records them in SeriesResult.relative_paths and SeriesResult.angle_paths.
Both are None in a document written before they existed, and moved_paths then treats no path as either.
Tables and CSV files#
SeriesResult.to_table returns (header, rows) in wide form, one row per pattern with a value and an esd per parameter.
SeriesResult.write_csv writes it, with a tab delimiter for a .tsv or .tab suffix and a comma otherwise.
The columns are index, label, the axis, status, rung, rwp and gof, then each path followed by its _esd column.
rung travels beside status, because a point refitted from the starting models is a good fit whose start did not come from its neighbour.
paths= takes the derived kinds too.
to_table(paths=["qpa.LaB6"]) exports the weight-fraction curve, and r_bragg. and r_f. export the agreement indices, all resolved by SeriesResult.resolve_trajectory.
A path no pattern carries raises, naming it and listing what does exist.
A kind with no esd by construction, an agreement index, gets no _esd column.
A kind that has esds keeps its column even where this series estimated none, because there a blank says that the pattern did not estimate one.
Where a trajectory skips a pattern, the table still has a row for it and leaves the cell empty.
Trajectory.positions keeps the two aligned, since neither x nor label identifies an entry on its own.
The axis column takes SeriesResult.x_label, unless that name is already one of the fixed columns, in which case it is x.
A series with no coordinate has the x_label "index", which is such a name.
Magnetic moment across a transition#
A neutron ramp through TN grows a moment from nothing.
The magnetic model is stated once (Atom.moment and Phase.magnetic_symmetry, Moments), and the chain refines it on every pattern.
The magnetic intensity |Fm|² is proportional to m², so a moment the data cannot see is a flat direction of the least-squares problem and the fit leaves it near zero.
The report of that is “the data does not support a moment here”, and never a small measured moment such as 0.02(1) μB.
Each entry carries the evidence beside the value.
SeriesEntry.magnetic is one MomentEvidence row per magnetic site, the same rows a single pattern’s FitReport.magnetic carries (Moment evidence).
SeriesEntry.moment fetches one of them by its MomentEvidence.path, the modulus dot-path phases.*.atoms.*.moment.dof0, or by its atom label.
Naming nothing takes the only site, and refuses when there are two, because “the moment” is not one quantity on a two-sublattice structure.
SeriesResult.magnetic_trajectory is |m| against the axis for one site, and SeriesResult.magnetic_sites lists the sites that carry one.
It returns a MagneticTrajectory, which is a Trajectory with more columns, so it exports and plots through the same code.
Member |
Holds |
|---|---|
|
the axis and its name |
|
|m| in μB at each point, from that pattern’s own fit |
|
its esd, |
|
per point, whether |m| cleared three of its own esds |
|
the complement of |
|
per point, whether the pattern gave a verdict at all; |
|
that pattern’s fit status, because |
|
the pattern label at each point, and its index in the series |
|
the site and its form-factor key |
|
the modulus dot-path |
|
where the moment stops being supported, as a |
A held point keeps its value and loses its esd.
The value is the modulus the fit reached, and the numerator of the ratio that called the point unsupported.
Deleting it would delete the evidence, and zero would claim a measurement no fit made.
The esd is withheld because an error bar on an unsupported number reads as a small measured moment.
Trajectory.arrays hands a plotter NaN there, so no error bar is drawn.
result.plot("magnetic.Mn") draws held points hollow and a pattern with no verdict as a cross, and shades the onset bracket.
The onset is read off the verdicts, and never off the values. A warm-started chain carries a smooth |m| column straight through the transition, so a threshold on the numbers finds wherever the chain relaxed. The first pattern whose moment is held finds where the data stops carrying one.
Member |
Holds |
|---|---|
|
the midpoint of the bracket and half its width |
|
the last released pattern and the first held one |
|
|
|
whether there is one boundary, and each boundary as the pair of x values either side of it |
|
the counts over the patterns that gave a verdict |
|
the patterns that gave none, left out of the bracket |
|
whether neither bracket pattern owes a supported verdict to a fit that stopped early; |
|
which site it is about |
|
the reading in words, including why it is not quotable when it is not |
A pattern that gives no verdict is left out of the bracket rather than read as held.
A closed-shutter frame is the plain case: the phase is invisible, its structural paths are held for the stage, and the modulus comes back with no esd.
Counted as held, such a frame one step below TN would end the bracket at itself, a confident onset one pattern early.
Left out, the bracket spans it and is wider, and MagneticOnset.note names it.
MagneticOnset.x_esd is the spacing of the measurements, not a fitted uncertainty.
A ramp in 50 K steps cannot place TN better than ±25 K however good each fit is.
Nothing here fits m ∝ (1 − T/TN)β.
That form is nonlinear in its coefficients, and the exponent is yours to fit to the trajectory.
SEQUENTIAL_MOMENT_HOLD counts the patterns whose moment came back unsupported, and SEQUENTIAL_MOMENT_ONSET gives the bracket.
import rietx as rx
result = rx.refine_sequential(patterns, magnetic_structure, instrument,
plan=plan, x=temperatures, x_label="T (K)",
direction="both")
traj = result.magnetic_trajectory() # the only site
print(traj) # |m| per pattern, held marked
print(traj.onset) # the bracket and its half-width
result.plot("magnetic." + traj.atom, path="moment_vs_T.png")
A modulus a pattern did not support is reseeded to its floor before it becomes the next pattern’s start. The pattern’s own entry keeps what the fit reached, and the site’s direction is preserved. The modulus stays free, because it is the one direction that is not flat, and a moment climbs back out through it on a cooling ramp. A modulus with no verdict is not reseeded, so the next pattern starts from the last value a fit judged.
Warning
The reseed cannot protect the first pattern above the transition, which is warm-started from a supported neighbour.
Whether that pattern comes back unsupported is a measurement.
Two checks cover it, and you have to ask for both.
Under direction="both", SEQUENTIAL_MOMENT_ONSET carries the other chain’s bracket and says whether the two agree.
They agree when the brackets overlap, both chains read the same sense, and neither bracket owes a verdict to a fit that stopped early.
A bracket only one chain found is a warning, and so is one chain supporting every pattern the other holds.
MagneticOnset.bracket_verdicts_final catches a supported verdict from a fit that stopped at its iteration cap.
On a synthetic ramp starved to max_iter=1, the moment above the transition came back supported and the onset moved to a confident, wrong bracket, with Rwp fine and no other fence firing.
Series diagnostics#
A sequential fit is path-dependent by construction. Every pattern’s answer depends on its neighbour’s, so one bad pattern’s error passes to every successor and still draws a smooth curve. These diagnostics fence that, and none of them alters a fitted value. Series and joint fits gives each code’s full entry.
Code |
Says |
|---|---|
|
the warm start was rejected and the pattern was refitted cold |
|
the pattern diverged on every attempt; it seeded no successor and joined no median |
|
every attempt left the pattern above the Rwp fence (with |
|
a step much larger than the local trend, from the science or from a chain failure |
|
under |
|
under |
|
a phase width reached 3× the first value the series measured above three esds, by more than three combined esds, while GoF reached 2× its own at the same pattern; a width that grows at a flat GoF is real broadening and fires nothing |
|
how many patterns’ moment came back unsupported, and which successors’ modulus was reseeded to its floor |
|
where along the axis the moment stops being supported, as a bracket |
|
a per-pattern code fired in more than half the patterns, so it describes the model |
|
the series stopped before its last pattern (Events, history and cancellation) |
|
every attempt on a pattern raised (Restart ladder) |
A per-pattern diagnostic can only say “this pattern”, so in a run of 68 it cannot say “42 of 68”.
One BOUND_HIT is a pattern that hit a bound, while a BOUND_HIT in most patterns is a bound that is wrong.
SEQUENTIAL_PERSISTENT_FINDING counts each pair of code and parameter over the entries and states the fraction once, in value, while the entries still carry every occurrence.
Its threshold is half the series: above it the finding describes the series, and below it the per-pattern diagnostics say everything there is.
It needs a series of rietx.sequential.MIN_POINTS_FOR_PERSISTENCE patterns, five, and takes the worst level among the occurrences it counts.
A code about how a pattern was measured, listed in rietx.sequential.NOT_A_SERIES_FINDING, is never counted.
Discontinuity check#
SEQUENTIAL_DISCONTINUITY fires at info on a step more than five times the path’s median absolute step and more than three combined esds, in a trajectory of five points or more.
It names both readings, the science or a chain failure, and asks you to open that pattern’s own fit.
verify_discontinuities=True runs that check.
Each flagged step’s two patterns are refitted cold and independently, with no warm start and no neighbour.
The diagnostic’s value is then the cold step over the chain’s (How to check a series for path dependence).
result = rx.refine_sequential(patterns, structure, instrument, x=temperatures,
verify_discontinuities=True)
Near 1 the step is in the data.
Near 0 the chain made it, because the two patterns agree when nothing carried an error between them.
The ratio is signed, so a cold pair that stepped the other way reads about −1.
The refits are separate Refinement runs writing their own <label>.verify histories, and no fitted value, rung or median moves because of one.
It is off by default. A cold fit runs the full staged plan from the starting models, and a series flagging s steps runs up to 2s of them, once per pattern, since two paths flagged at the same step share a refit. On a 68-pattern thermal ramp flagging four steps over four patterns, the chain took 11.6-12.0 s and the chain with the check 12.1-12.2 s, 5 % more. The time scales with the patterns flagged, not with the series length.
SeriesResult.discontinuities holds one SeriesStep per flagged step, in the diagnostics’ order.
SeriesStep.path is the diagnostic’s where[0].
SeriesStep.labels and SeriesStep.indices name the pair, earlier first, and SeriesStep.step is the signed change, later minus earlier.
The check refits this pair, and the trajectory plot shades it.
Each path carries at most one flag, the largest step that passes both tests.
A step into or out of a pattern the fence rejected, SEQUENTIAL_UNRECOVERED or SEQUENTIAL_RWP_OUTLIER, is left out of the scan, because the pattern already has its own code.
Otherwise its step takes the flag.
In a 24-pattern ramp with one blank frame, the blank’s cell was fitted to noise and stepped 33 times the median.
The check confirmed that step at 1.00, because two cold fits reproduce the same noise, and the real step five patterns later went unreported.
The steps are dropped, never bridged, because a bridge over several rejected patterns spans that many steps, and on a clean ramp it read as an 8× jump.
Restart ladder#
A rejected warm fit escalates one attempt at a time: the collapsed warm refit, then the full staged plan from the warm state, then the full staged plan from the starting models.
Under refit="stages" the first attempt already is the staged warm fit, so the ladder has two attempts, "warm_staged" and "cold", and "warm" never appears.
Each attempt runs only when the fence still fires on the best attempt so far, and the entry keeps the best.
SeriesEntry.rungs_tried names the attempts, and SeriesEntry.n_iterations sums those that returned a fit.
The middle attempt matters most.
Throwing a warm start away roughly triples the work, and before the middle attempt existed that work went on a start not shown to be the problem.
What triggers the ladder is divergence, or an Rwp above reseed_factor times the median of the accepted patterns.
Each trigger is a property of this pattern’s own fit, readable the moment it finishes, and something a different start could plausibly fix.
A guard finding such as HIGH_CORRELATION is neither, since it fires on converged patterns and no attempt changes the model or data behind it.
A discontinuity is a property of the finished trajectory, so making it a trigger would mean walking a finished chain again.
The first attempt is bounded, because it is a bet rather than the answer.
Its budget is first_rung_factor times the dearest first attempt this chain has already converged, and it applies once three have.
Under refit="stages" the bound does nothing.
A first attempt that spends its budget escalates, and the next attempt starts from the same warm state, so a bounded chain reports the values it would have reported without the bound.
On the benchmark’s ten-pattern series, the solver made 1603 evaluations without the bound and 1395 with it, both converging to Rwp 0.01943 with every accepted value identical.
On the eight-mixture round-robin series the two runs are identical to the evaluation.
first_rung_factor=None reproduces a pre-1.1 run.
An attempt whose fit raises has lost in the same sense as a diverged one, and the ladder escalates past it the same way.
SeriesEntry.rungs_raised records what each such attempt raised, and SeriesEntry.rwp_warm is None when the warm attempt was one of them.
The case that needs this is a neighbour that converged with a cell far outside the physical range.
Every warm attempt inherits that cell, and the reflection enumerator refuses it before the fit starts, while the cold attempt starts from the starting models.
A pattern on which every attempt raised has no entry.
It is recorded in SeriesResult.failures with a SERIES_PATTERN_FAILED warning, and the on_error policy decides what the chain does next.
Each SeriesFailure carries the pattern’s SeriesFailure.index and SeriesFailure.label, and SeriesFailure.exception is the last attempt’s exception as repr.
SeriesResult.n_failed counts them, and SequentialRefinement.failures_ is the same list on the runner, where to read it after catching the exception on_error="raise" lets through.
Under the default, "carry", the successor warm-starts from the last accepted pattern, because what failed was this pattern and not the state its neighbour handed it.
A chain that fitted no pattern at all still raises, whatever the policy.
Your own prepare and constrain are never guarded: a raise there ends the series.
Quarantine#
A fit still "diverged" after the last attempt is quarantined, and SEQUENTIAL_UNRECOVERED names it.
Its successor warm-starts from the last accepted pattern, and the reseed median never sees the failure.
Otherwise one failure would seed its neighbour with a bad start and drag the median that decides every later trigger.
A pattern that converged but stayed above the Rwp fence on every attempt is reported and not quarantined.
SEQUENTIAL_RWP_OUTLIER names it, and it seeds its successor and joins the median like any accepted pattern.
Rwp cannot say why it rose.
Near a converged fit Rwp is GoF times Rexp, and three measured causes divide between the two factors:
A blank frame has a GoF like its neighbours’ (1.00 against 1.06) and an Rexp four times theirs, because it is background and noise, fitted perfectly.
A correct model over counts that fell 4× also keeps its GoF and doubles its Rexp. It is a sound measurement.
An unmodelled phase raises the GoF: 16.7 against 1.04.
Quarantined, a lasting change could never become the chain’s new normal. On a 24-pattern ramp whose counts fell from pattern 6, quarantine made every later pattern climb the ladder, with 55 % more iterations, and flagged all 18 rather than 7. With an unmodelled phase from pattern 6 the extra was 107 %. On the blank frame, quarantine saved 2-8 % of the chain, and the values were identical either way. The message’s GoF and Rexp against the last pattern inside the fence tell the causes apart. The fence follows the median, so a run of outliers that stops is a lasting change the median caught up with.
Forward and backward chains#
direction="both" runs the series forward and backward and compares the two trajectories.
SeriesResult.entries are the forward ones, and the comparison arrives as SEQUENTIAL_PATH_DEPENDENT diagnostics, one per parameter that disagrees (How to check a series for path dependence).
It is the only check that separates a measured trajectory from an ordering artefact, and on real data it is selective.
On the round-robin series it flagged nine parameters, every one a peak-shape term: phases.*.lor_size, gauss_size, gauss_strain, lor_strain, instrument.profile.x and instrument.geometry.axial_sl.
It flagged no cell parameter and no scale.
The σ multiples in those messages are ratios to a fitted esd.
A parameter near zero with an esd near zero reports a spread of thousands of σ with no physical size, so read the two values the message quotes.
The trajectory the messages compare against is SeriesResult.backward, the reverse chain’s own SeriesResult, and SequentialRefinement.backward_ is the same object.
Its own backward, provenance and discontinuities are None, and its diagnostics hold only the moment diagnostics.
The comparison and every other series diagnostic are on the forward result.
No SEQUENTIAL_PATH_DEPENDENT finding is also what a comparison that never happened would give, so a comparison that did not run says so.
When the forward chain was cancelled, or the backward chain was cancelled or raised, SEQUENTIAL_PATH_CHECK_INCOMPLETE fires at warning, naming the chain and the pattern it stopped on.
When the comparison ran on less than the series, the same code fires at info, and where names the patterns one chain has no entry for, or the paths no pattern has an esd for in both chains.
So no finding with neither of these present means the comparison was made and found nothing.
A backward chain’s own failures are on result.backward.failures.
Its raise under on_error="raise" keeps the forward chain, which is on SequentialRefinement.result_ and on the exception as series_result.
SeriesResult.n_iterations counts the chain the result reports, the forward one under direction="both".
The backward chain is a second complete set of fits, and result.backward.n_iterations counts it.
Events, history and cancellation#
events= and cancel= act per pattern.
Every event a pattern’s fit emits is forwarded with its place in the series stamped into the event’s data: series_index, series_label, series_n and series_pass, plus series_rung and series_cold on a restart.
series_pass is "forward", "backward" or "verify", the last for the discontinuity check’s refits.
These are added fields on existing kinds, so no event kind is new and the event schema version does not move (Events).
Read data with .get, and “pattern k of N” off fit_start.
progress= stamps each line with the pattern’s place (Progress lines), and label= names the whole series’ run directory (Recording and run names).
Cancelling a series returns what completed rather than raising (Cancellation).
The pattern in flight is abandoned by Refinement.fit itself, with no node, no commit and its models restored.
The patterns already walked are finished fits with committed nodes, and raising would throw them away.
SEQUENTIAL_CANCELLED says how many of how many patterns were reached.
A truncated series and a finished one are the same shape, so check for it before reading the last entry as the end of a ramp, or a slope over the entries as a slope over the experiment.