Output files

A run writes HDF5 files through the package extension Runes2PHDF5Ext, which loads with HDF5 (Extensions). Without using HDF5 the writers and readers below have no methods. A run that does not fit one allocation is a chain of segments. The page Time stepping and solvers says when a segment stops and how segments are chained.

Files of a run

write_output writes one TimeSolution to a file of any name, in place unless atomic = true. A run whose CheckpointSettings have the name name writes three kinds of file to its directory:

The segment files and the checkpoints are written by write_atomically: to a temporary file that is synced and renamed. A path therefore holds either its old content or the new one, and a failed write changes neither the path nor the previous checkpoint. The last checkpoint is renamed to the previous one just before the new file is renamed onto its path. Between the two renames the previous checkpoint is the only one, which is why read_checkpoint falls back on it. A last checkpoint that is not HDF5 does not replace the previous one.

At each checkpoint the segment file is written first, then the checkpoint. A checkpoint then never names states that its segment file lacks. An attempt killed between the two writes leaves a segment file that the checkpoint does not name. A run with resume = true replaces that file. Any other run refuses to replace an existing segment file of its number unless overwrite = true. A run refuses to start while segment files of its name with higher numbers than its segment exist, with overwrite = true as well, and it deletes no file: the reason is at the end of Joining segments.

Root of a file

Output and checkpoint files share one root:

The version counts up with every change of the format. read_output and read_state refuse a file whose format_version is older than 0.0.1 or newer than OUTPUT_FORMAT_VERSION. A segment without a saved state has no time, no unknowns and no datasets under update_states.

A file of the format 0.0.5 stores units in these places:

• Each entry of unknowns carries the unit of its unknown as the string attribute unit (below).
• In the group layout, the entry of each unknown holds its unit as the dataset unit (below).
• The datasets of the grid faces in layout carry the attribute unit (below).
• The dataset time carries the attribute unit, which is "s".

No other entry carries a unit. The times and step lengths of solver_statistics and the time of the group checkpoint are in seconds as well.

Records

The settings group, the layout group and the record parts of a checkpoint are plain records (SettingsRecord) written as HDF5 objects:

• A record becomes a group with one entry per key.
• A list of records becomes a group with one subgroup per entry, named by the entry's name, and the dataset order with the names.
• A list of arrays becomes a group with the datasets 1, 2 and so on.
• A boolean becomes a byte with the attribute bool, read back as a boolean.
• Strings, numbers, arrays of numbers and lists of strings become datasets.

A record stores no unit with its numbers. The one exception is the root group layout (below), whose entries of the unknowns hold unit and whose datasets of grid faces carry the attribute unit.

The reader returns every group as a record, so a list of records reads back as a record keyed by the names and a list of arrays as one keyed "1", "2", …. read_settings returns the record of run_settings as written, type for type, and settings_hash of it equals the hash written.

The settings group

run_settings builds the record of a run from the describe records of its objects. Its sections:

The eight sections of the setup are SETUP_SECTIONS. code, environment, checkpoint, stop, log and segment stay outside the hashes, so the same setup has the same hash on another machine and another commit. The tables of prescribed are a section of their own, so the hash of system does not change with the time points. A term refers to its table by name (table_1, …), and a function used by several terms has one table.

The segment section holds index, start_time, saved_before, resumed, status, end_time, steps, stop_reason after a stop, checkpoint_written and attempts_before. saved_before counts the states saved before the segment, end_time is the time reached, and attempts_before counts the steps attempted before the segment. attempts_before is missing for a segment resumed from an output file. A resumed segment adds resumed_from (the file), resumed_from_segment, previous_commit, commit_changed, previous_setup_hash, settings_changed, and changed_sections when some section of the setup changed.

The layout group

The root group layout is the describe record of the UnknownLayout with its grids inline under grids, so it does not depend on the settings group. The record carries layout_version = 2. A reader rebuilds the layout from it and the kinetic grids with cylindrical_grid from the stored faces. A resume matches the unknowns by name with copy_unknowns!, which refuses a stored unknown on other momentum grids: the distribution is not remapped between grids.

Each entry of an unknown holds its unit, which the layout section of the settings does not hold, so the setup hash does not depend on the units. The datasets r_faces, p_faces and xi_faces carry the attribute unit: "m", "m_e c" and "1" for the dimensionless pitch.

Some entries are recorded only when they are set, and the reader takes a default where one is missing:

Two entries of the settings record follow the same rule: ions of the plasma_unknowns of a PlasmaFromUnknowns, absent for prescribed ions, and group of a Tolerance, absent for group 0.

Unknowns

The group unknowns has one entry per unknown, named by it, with the saved times as the last dimension:

The shapes are those Julia reads. A C-order reader such as h5py or h5dump sees the dimensions reversed, with time first.

The values are those of the state vector, in the units of each unknown (Units of the unknowns), not divided by its scale (a reference magnitude). The dataset time is in seconds, and its attribute unit is "s". In memory, unknown_view views one unknown of a saved state of a TimeSolution.

Each entry carries the unit of its unknown as the string attribute unit (unknown_unit), such as "V/m" or "m^-3". The group of an unknown on per-radius grids carries it, and so does each of its datasets radius_<ir>. The attribute is empty for an unknown without a unit. A file of the format 0.0.4 or older has no attribute unit, and its reader gives each unknown its default unit (Units of the unknowns). The reader of Runes2P takes the unit of an unknown from its entry in layout, and the attribute on its entry of unknowns is for other readers of the file. A unit in layout that is not an ASCII string without control characters, as another tool may write, is dropped with a warning, and the unknown takes its default unit.

Solver statistics and update states

The group solver_statistics has one dataset per field of SolverStatistics. Its time array has one entry per attempted step, rejected ones included, and is not the time of the saved states. The arrays of the Newton iterations have one entry per iteration and name their step by iteration_attempt. A field without entries has no dataset, and the reader takes it as empty. A field of one entry per attempted step that a file of an older format does not have reads as its default for every attempt: false, NaN, 0 or :unknown.

The group update_states has one dataset per interval update, named by the update, of shape (n, t), reversed for a C-order reader, with n the length of the update's state. Column t is the state of the update in force when saved state t was computed.

Checkpoint files

write_checkpoint writes the root of an output file with one state, and the group checkpoint:

With SparseLU the stepper record lets a resumed run repeat the steps of the uninterrupted one. The ordering and the matching of MKLPardiso are not stored, so a resume with it repeats the run within the Newton tolerance, and within round-off where the Newton iterations of the steps are the same. The preconditioner factors of GMRES with on-demand refresh are not stored either, so a resume with it repeats the run within the Newton tolerance. After either, the step lengths of a StepController may differ from those of the uninterrupted run. A state read from an output file has no stepper record, so its time must be a time point of the resumed run.

The units stay outside the setup hash, so a resume does not see a changed unit as a changed setup. It warns instead for each copied unknown whose unit stored in the file and unit of its declaration are both nonempty and differ, and it copies the values unchanged. A file of the format 0.0.4 or older stores no unit and gives no such warning.

Reading

• read_output returns the TimeSolution of an output file, or of the segments of a run joined. It refuses a checkpoint.
• read_settings returns the settings group alone.
• read_state returns one saved state of an output file, the last by default, or the state of a checkpoint, as a StoredState to resume from.
• read_checkpoint with the CheckpointSettings of a run reads its last checkpoint and falls back on the previous one. It returns nothing when neither file exists, and throws when files exist and none can be read, so a run with unreadable checkpoints stops instead of starting from the beginning and overwriting its files.

Joining segments

read_output(settings) joins the segment files that exist, in the order of their numbers (segment_files). Files given by name are joined in the order given. The join refuses files that are not the segments of one run, in these cases:

• A segment starts before the one given before it.
• The resumed_from_segment of a segment is not the index of the one before it, where both records hold them.
• The previous_setup_hash of a segment is not the setup_hash of the one before it, where both are known.
• A segment has other unknown names or another state length.
• A segment has saved states and other interval updates.
• The joined times are not strictly increasing.

A segment that went on after its last checkpoint is repeated from there by the following one. Its states after the start time of the following segment are dropped. Its statistics are cut at the count of attempts the following segment resumed from (attempts_before of both), or at the start time where a count is missing. The count matters because a rejected attempt is recorded at the time it aimed at, which may lie beyond the checkpoint. The joined solution carries the settings of the last file.

Its layout is that of the first segment, and each unknown takes the unit of the latest segment that stores one, the empty unit included. A first segment of the format 0.0.4 or older then does not hide the units of the later segments. Where two segments store different nonempty units for one unknown, the join warns with the unknown, the two units and the two files, and it joins the values unchanged.

The higher segments of an earlier run under the same name and with the same setup pass these checks. This is why a run refuses to start while segment files of its name with higher numbers than its segment exist (CheckpointSettings).