Project and run directories#
A rietx project is a directory ending in .rex, and this page lists every file rietx writes into one, the fields of project.json, what Project.open checks, and the run directories a fit writes outside a project.
The tree comes from a project built from the fap example, so the entries are the ones a project holds.
Project.create makes the directory, Project.open reads it, and rietx gui my_sample.rex opens it in the GUI.
About the project gives why a project is laid out this way.
fap.rex/
├── project.json
├── FAP.XRA
├── history.jsonl
├── peaks.json
├── live/
│ ├── .gitignore
│ └── 20261016-084102-4242/
│ ├── meta.json
│ ├── run.lock
│ ├── status.json
│ ├── events.jsonl
│ ├── snapshot.json
│ ├── summary.txt
│ └── cancel
└── exports/
├── refinement.cif
├── reflections.csv
├── qpa.csv
├── fit.html
├── result.json
└── instrument_profile.json
A new project holds project.json, FAP.XRA, history.jsonl, live/ and exports/. live/ and exports/ start empty. A fit, the GUI or an export writes everything else.
Entry |
Named by |
What it holds |
|---|---|---|
|
|
The pattern file, copied byte for byte. |
|
|
Settings: the plan, the mode, the limits, the excluded regions and the GUI’s own keys. |
|
|
Every model state, one history node a line, appended. The last node is the working state. |
|
|
The peak list the GUI’s peak picker edits, tied to the pattern’s fingerprint. |
|
|
One directory per recorded fit. |
|
|
What the GUI’s export verbs write. |
|
|
Holds |
|
|
The run’s description of itself. |
|
|
Held locked while the fitting process lives. |
|
|
Whether the run is running, done, failed or cancelled. |
|
|
The event stream, one JSON object a line. |
|
|
The latest stage’s curves and statistics, for a viewer. |
|
|
The result’s text summary, once one exists. |
|
|
A stop request. A viewer writes it and the fit reads it. |
|
|
One file per export kind, under these default names. |
Creating a project#
project = rx.Project.create("my_sample.rex", pattern="my_sample.xye",
structure=structure, instrument=instrument)
project.fit()
project.save()
project = rx.Project.open("my_sample.rex")
Project.create copies the pattern file into the directory byte for byte, reads it, writes project.json and starts history.jsonl with a root node.
pattern= is a path, not a PatternData, because the file’s bytes and its esd column are what the project keeps.
Its other keywords set the first settings: mode=, plan=, two_theta_limits=, excluded_regions=, reader_options= (block or scan, recorded as the reader call) and ui=.
Project.create refuses a directory that already holds a project.json, a pattern file that does not exist, and a time-of-flight pattern.
Opening a project#
Project.open resumes at the history head.
It re-checks every binding rather than assuming it, and each failure raises with its own message, because each has a different cause and a different fix:
Refused when |
Raises |
|---|---|
the directory holds no |
|
|
|
|
|
the pattern file is missing from the directory |
|
the pattern file’s bytes no longer hash to the recorded sha256 |
|
the pattern reads as time of flight |
|
the bytes match but this build parses them to different numbers |
|
|
|
the history was recorded against a different pattern |
|
the history log holds no node to resume from |
|
A project.json that fails validation raises pydantic’s error before any of these.
An older release’s project may spell component paths the old way in its saved plan.
Project.open rewrites those paths in the text before reading it, so the plan’s globs still free what they freed.
Opening appends a line to the history log before any verb runs.
Every verb that changes a project writes into its directory as it runs.
To look at a project without changing it, open a copy: rietx gui PROJECT.rex --scratch copies the directory byte for byte to a temporary one and opens that (The command line).
A project with no structure#
structure= may be left out, which makes a pattern-only project: zero phases, for a pattern whose phase you do not know yet.
project = rx.Project.create("unknown.rex", pattern="unknown.xye",
instrument=instrument)
Peak picking and indexing work over it, and the parameter table holds the instrument and the background.
Both routes out of it end in a phase: index the peaks and adopt a candidate cell, or build a Le Bail scaffold from a space-group symbol and the cell parameters that setting leaves free (rietx.schemas.structure.lebail_scaffold).
In the GUI it is the third answer to the wizard’s structure step.
It cannot be refined.
A phase reaches the pattern only through scale × |F|² × profile, so with no phase only the background is left to fit, and a plan run over it would converge on the background and report success.
Refinement.fit, Refinement.run_stage, refine_multi and refine_sequential therefore raise NoPhasesError before they start.
NoPhasesError.code is the string NO_PHASES, which the GUI also gives as its reason for a disabled Run button:
import rietx as rx
print(rx.NoPhasesError.code)
NO_PHASES
Refinement.predict still works, because evaluating the background as it stands is not a refinement.
The open project#
An open project holds the session in seven attributes.
Attribute |
Holds |
|---|---|
|
the project directory |
|
the |
|
the |
|
the |
|
that refinement’s |
|
what the pattern reader repaired or assumed on the last read |
|
what reading the history log repaired, on the last open (The refinement history) |
The two diagnostics lists are held in memory and are not project.json fields.
The pattern’s repairs are a function of the bytes, the reader and its options, which the data reference already records.
The history’s repairs are a function of the log and of the release reading it.
Project.parameters, Project.fit and Project.run_stage are the session verbs, with the meaning they have on Refinement.
Each takes the mode from the project’s settings unless you pass one.
Project.fit takes the 2θ limits and the plan from them too, Project.run_stage the limits and the phase_support_sigma of the plan, and both record the run into Project.live_dir.
Project.exports_dir and Project.live_dir are the paths of exports/ and live/.
Project.data_ref returns the pattern’s DataRef.
project.json#
ProjectDoc is project.json, the half of a session that nothing else records.
The model state is in history.jsonl, whose head is the working state, so no parameter value is written in both places.
Every verb that changes the model commits a history node as it runs, so the work is on disk whether or not anyone calls Project.save.
Project.save rewrites project.json alone.
Field |
Holds |
|---|---|
|
the data references, one per pattern |
|
the plan the next run will use, as a |
|
the intensity mode the next run will use |
|
the range the next run will fit |
|
2θ regions masked out of the residual |
|
the next indexing run’s controls |
|
the log’s filename inside the directory |
|
the version of the |
|
the version of rietx that wrote it |
|
when it was created, and when it was last saved |
|
keys a front end persists, with no schema |
The plan, the mode and the limits are what Project.fit and Project.run_stage will be called with.
A history node records a mode and limits too, as the ones a past run used.
The document says what the next run will use.
Before the first run there is no node to ask.
ProjectDoc.patterns is a list because stacking several patterns into one joint residual is a later milestone’s work.
A project holds one pattern today, and Project.open refuses a document carrying more rather than opening the first.
ProjectDoc.ui has no schema on purpose.
A front end owns those keys and the document only stores them, so a layout change is not a schema change.
The pattern reference#
The pattern file is copied byte for byte, because the reader takes σ from the file’s own esd column.
Project.data_ref returns the DataRef that makes those bytes trustworthy on re-open.
Field |
Holds |
|---|---|
|
the pattern file’s name inside the directory |
|
the digest of the file as copied, which catches an edit in place |
|
the digest of the parsed 2θ and intensity arrays, which catches a change of reader |
|
which pattern format claimed the file, and the reader options it was read with |
|
the number of points and the 2θ range |
|
whether σ was measured, from an esd column or the format’s own counting convention, rather than the Poisson fallback |
The reader call is part of the reference, because a pdCIF holding a measured and a calculated block is a different pattern depending on the block.
Agreeing bytes with a disagreeing fingerprint mean the reader changed, and the file is intact.
DataRef.has_sigma is a property of every fit in the project that is invisible once the data are read, so it is written down.
Fit range and excluded regions#
Project.set_excluded_regions records regions to leave out of the fit, in the document and on the data together.
They live in project.json because they are protocol that neither the pattern file nor a history node records.
An inverted or empty interval is refused rather than reordered.
Project.fitted_mask answers which channels the next run fits: the excluded regions intersected with ProjectDoc.two_theta_limits.
rietx.project.fitted_mask asks the same of any pattern:
import rietx as rx
from rietx.examples import examples_dir
from rietx.project import fitted_mask
data = rx.read_pattern(examples_dir() / "11BM_NAC.fxye")
print(len(data.two_theta), int(fitted_mask(data, (2.0, 24.0)).sum()))
data.excluded_regions = [(7.4, 7.6)]
print(int(fitted_mask(data, (2.0, 24.0)).sum()))
59498 22003
21803
On the 11-BM NAC pattern, limits of 2–24° keep 22 003 of 59 498 channels, and excluding 7.4–7.6° as well removes 200 more.
Run directories outside a project#
A fit records itself into a run directory unless recording is switched off (How to watch or stop a running fit).
A fit from a project records into the project’s live/.
Any other fit records under .rietx/runs/ in its working directory, one directory per fit, named for the date, the time and the process id, such as 20261020-030217-66184.
A fit given telemetry= records into the directory it names instead.
.rietx/runs/
.gitignore
20261020-030217-66184/
events.jsonl
snapshot.json
summary.txt
meta.json
status.json
run.lock
The generated table above gives what each file holds.
The .gitignore holds *, so runs stay out of git status.
rietx writes it only into a runs root it chose itself, and leaves a directory named with telemetry= alone.
Two directories are spelled .rietx, and they have nothing to do with each other.
$HOME/.rietx holds the GUI’s per-user state: the recent list and the theme.
--state-dir or RIETX_STATE_DIR moves it.
The runs root is relative to the working directory, and no environment variable moves it.
A fit run inside somebody else’s package leaves .rietx/ in their working directory, and RIETX_TELEMETRY=0 is how a process declines.
Retention#
The first time a process records under a runs root that rietx chose, it prunes that root by age and size, never by count.
When the root holds more than 1 GiB, rietx deletes finished runs older than a week, oldest first, until the root is under 1 GiB.
If the root is over the limit with nothing old enough to delete, rietx warns and keeps everything.
A directory is deleted only if its name is a run id, it sits directly under the root, and its meta.json carries rietx’s record tag, so a directory a user made there is never pruned.
What a run records#
events.jsonl holds every free parameter’s value at every recorded evaluation, with the stage boundaries and the statistics at each.
snapshot.json holds the observed, calculated and difference curves of the last finished stage, decimated for drawing, and is rewritten each stage.
No pattern file is copied into a run.
The decimation keeps the minimum and the maximum of every curve in each of 2000 bins, the 4000-point budget the GUI and the comparison page use too. Where the curves’ extremes fall at different points a bin keeps more than two, so the drawn count can pass 4000. A long pattern reaches a run as a picture of itself, and a short one arrives nearly whole: the fluorapatite pattern’s 5753 points reached the snapshot as 5221.
meta.json holds the run’s label, its start time, the rietx version, the working directory and the command line.
status.json holds the state (running, done, failed or cancelled), the process id and host, a heartbeat time, the current stage and its index of how many, and Rwp, the goodness of fit, χ² and the number of free parameters at that stage.
A series member’s run adds its place in the series, and a run that stopped recording, or declined a stop request, says why.
run.lock is held locked while the fitting process lives, and cancel appears when a viewer asks the fit to stop.
The run directory carries no version string (Provisional by declaration).
A run holds every free parameter’s trajectory and most of the observed pattern, so set RIETX_TELEMETRY=0 where the refinement is confidential and the working directory is shared.