# Observability How the core is shaped so that future step-level tooling - an interactive debugger, live visualization, deterministic replay, time travel - can attach to it without rework. ADR-0012 makes these constraints binding; this document is the detail. The tooling itself is out of scope for now; everything below is a constraint on code being written anyway, not a feature. Read alongside `docs/architecture.md`. ADR-0002 (literal Appendix D port), ADR-0003 (pure core with effects), and ADR-0005 (full configuration, interned indexes) are the load-bearing decisions; this page is about not fusing away the seams they create. ## Vocabulary - **Microstep**: one round of the Appendix D inner loop - select a transition set for the current event (or eventless), exit states, execute content, enter states. - **Macrostep**: microsteps run to quiescence - the outer loop iteration that consumes one external event and drains eventless transitions and the internal queue until the configuration is stable. Step tooling wants to pause at both boundaries; visualization wants the trace of what happened inside each. ## Constraint 1: the microstep boundary is a resumable value Appendix D's `main_event_loop` runs microsteps to quiescence in a loop whose state (current event, internal queue position) is local. Ported literally as one recursive function, "paused between microsteps" would exist only on the call stack and a stepper could not stop there. Instead: - The machine_state struct reifies everything the spec holds in globals or loop variables: configuration, internal event queue, history values, datamodel, `running` flag, and the step counters from constraint 4. Any machine_state value is a complete, inspectable, resumable position. - `Statifier.Interpreter.microstep/1` - `microstep(machine_state) -> {machine_state, [effect]} | :quiescent` - is the unit of progress, and `Statifier.Interpreter.macrostep/1` is a fold over it. The fold function started private (`macrostep/2`) and is trivially publishable through `macrostep/1` - no fused loop needed dismantling. - This does not conflict with the literal port: the pseudocode's functions map one-to-one; only the *storage* of loop state moves onto the struct, and that deviation gets its mechanical-reason comment per ADR-0002. The payoff: a step debugger is `microstep/1` in a REPL; pausing, inspecting, and resuming need no support code in the core. ## Constraint 2: trace events are effects ADR-0003 already returns log entries as effect data. Trace extends the same vocabulary: structured effects emitted at the phase boundaries Appendix D names, so a debugger or visualizer is just another effect interpreter. Minimum trace vocabulary (shapes settled at implementation, the set is the commitment): | Trace effect | Emitted when | |---|---| | event dequeued | an external or internal event is selected for processing | | transitions selected | either selection function returns (includes the empty set) | | exit set | `compute_exit_set` result, before exiting | | content executed | a block of executable content ran (with its identity, constraint 3) | | entry set | `compute_entry_set` result, before entering | | macrostep stable | the configuration reached quiescence | | done | top-level final entry / `exit_interpreter` | | invoke pass | `run_invoke_pass` finished: the states walked and the invocations it started (``, spec 6.4) | | finalize/autoforward pass | `apply_invoke_passes` finished for one external event: which invocation(s) it ran `` for and which it autoforwarded the event to (spec 6.4/6.5) | The last two rows postdate the table above them - they arrived with `` support and are two more phase boundaries Appendix D itself names inside `mainEventLoop`, not a broadening of what counts as a phase boundary. The "Minimum trace vocabulary... the set is the commitment" line above is about the shapes being settled at implementation, not about the table being closed to boundaries Appendix D had not yet been ported far enough to need. Rules: - Emission is gated by an interpreter/machine option (working name `trace: true`); the untraced hot path allocates nothing for trace. - Trace effects carry the step counters (constraint 4) and identities (constraint 3), never bare structs that force tooling to re-derive context. - Trace effects are ordinary members of the effect list - same ordering guarantees, same delivery path. No side channel. - The ordering guarantee holds across batches too, not only within one: delivery order to a subscriber is non-decreasing in `(macrostep, round)` across the whole run, matching the order `Statifier.Replay` produces for the same recording. A mid-batch ADR-0039 re-entry keeps the enclosing macrostep and advances `round`, so its effects carry higher rounds than the outer batch's unsent tail; those effects are queued and drained after the batch that triggered them rather than delivered inline, which is what keeps arrival order monotone (ADR-0044 decision 1). That live-arrival guarantee is stronger than re-derivability, not a substitute for it: every effect carries the counter triple (ADR-0046), so a consumer holding a mixed stream whose arrival order was lost can still sort it back into `(macrostep, round)` order offline, including under `trace: false`. - "Either selection function" is `select_eventless_transitions/1` and `select_transitions/2` both, with no exception - including the terminal eventless probe that ends a macrostep, which is why the round reporting quiescence carries an effect list of its own rather than being a bare atom. - The "exit set" and "entry set" rows carry the *resulting* configuration (ADR-0005, full configuration, ancestors included), not only the `indexes` delta: `configuration` is the configuration as it stands after every state named by `indexes` has left (exit set) or been added (entry set). The table's "before exiting"/"before entering" wording still describes the phase boundary the payload is stamped against - the step counters keep coming from that boundary - but `configuration` is what the boundary produced, read after the mutation it names. This is what lets a consumer render the active configuration after every microstep without folding deltas or re-deriving `exit_interpreter`'s whole-configuration sweep; at `exit_interpreter`, the exit set's `configuration` is the empty set. ## Constraint 3: the Machine retains locations and identities The Document layer carries source locations and compiled expressions keep their source string alongside predicator's compiled envelope, which carries the span table with the instructions (`{:compiled, %Predicator.Compiled{}, source}`, per ADR-0014 item 2). The place this is lost by default is the compiler pass, because the runtime does not need locations. Keep them anyway: - States, transitions, and executable-content nodes on the Machine retain their source location. A visualizer highlighting the `` line currently executing needs nothing else. - Attribute-level spans are retained too, one step finer than the bullet above: `Statifier.Machine.State`, `Statifier.Machine.Transition` and `Statifier.Machine.Invoke` each carry their source node's `attribute_locations` map verbatim (`Statifier.Document`'s moduledoc holds the contract - value spans only, and a key exists only for an attribute the author wrote). The element-level sentence above is what a line-highlighting visualizer needs; a consumer with attribute-level hover targets on a ``'s `event`/`target`/`type` or a state's `id`/`initial` needs the map, and re-deriving it by re-parsing the source and joining Document nodes to Machine identities by location equality is sound only while the re-parsed bytes match what built the Machine, which nothing checks (st-9i5r). - SCXML transitions and executable content have no IDs, so the compiler assigns each a stable document-order index (per machine). Trace effects and error metadata reference these indexes; tooling maps them back to locations. - A `` element gets the same treatment, as a third document-order index: `d_index`, dense from 0, assigned by the compiler alongside `t_index` and `c_index` (`Statifier.Compiler`) and resolved back to its `%Statifier.Machine.Data{}` - carrying its own `location` and `value_location` - through `Statifier.Machine.data/2`. This completes an enumeration the compiler had already outgrown: `d_index` existed on `Machine.Data` and cited this ADR before this list named it (st-1xwh). - Locations and indexes are compile-time-immutable Machine data - no runtime cost beyond memory, no invalidation story. - Expression-level locations are in scope, not just SCXML element locations: compiled expressions retain predicator's span side table alongside the instructions, so an expression failure names the failing subexpression, not just the owning transition. Shape and sequencing are settled by ADR-0014 (spans, `on_unbound: :error`, table travels with the instructions). ## Constraint 4: steps are counted, causes are stamped - machine_state carries monotonic counters: macrostep number, microstep number within the macrostep, and round number within the macrostep's fold. They advance in exactly one place each, and `begin_macrostep/1` resets both child counters (`microstep` and `round`) when it advances `macrostep` (ADR-0020). - Every trace effect is stamped with the counters - `(macrostep, round)` is the ordering key for any timeline UI or log merge, and it advances on every round including those that run no microstep, so a fold that never reaches quiescence is still ordered and countable (ADR-0020). - A macrostep may carry more than one `Trace.MacrostepStable`: one per core drive that reached quiescence, since a session may re-enter the core mid-macrostep (ADR-0039). There is exactly one `MacrostepStable` per `(macrostep, round)`, and under ADR-0044 decision 1 the last one arriving within a macrostep is that macrostep's last quiescent point (ADR-0044 decision 3) - which is not always where the macrostep ends, since a macrostep that halts ends with `Trace.Done` instead, either after its final `MacrostepStable` or, when the halting drive is the only one, with no `MacrostepStable` of its own at all. - Internally raised events carry cause metadata: which transition or executable-content node (by constraint-3 identity) raised them, at which step and round. The first consumer is a better `error.execution` - "raised by the `` at line 42, transition 7, microstep 3, round 1" - which pays for the field before any debugger exists. ## Constraint 5: the pure queries stay callable The function-per-pseudocode-block structure of ADR-0002 is itself the debug API: `select_transitions(machine_state, event)` answers "what would happen if event X arrived" without stepping, and `compute_exit_set` / `compute_entry_set` preview a transition's effect on the configuration. The constraint is only: these functions take and return plain values (no hidden context), and they are not inlined or fused in a way that makes exposing them later a refactor. They may start private; `defdelegate` from a debug module later is the intended promotion path. ## Constraint 6: observe and record at the boundary `Statifier.Session` is *a* session boundary - the process that first hosted one, and the one every property below is stated against - but the property itself belongs to the logical session, not to the process. Two properties a session boundary preserves: - **Observation**: the session forwards its effect/trace stream to every subscriber pid as `{:statifier, session_id, {:effect, effect}}`, and a `:telemetry` bridge attaches at that same boundary. The authoritative reference for every `[:statifier, :session, ...]` event is now `Statifier.Telemetry` (ADR-0040, amended by ADR-0067): a caller-agnostic emitter called from wherever a driver steps a chart. `Statifier.Session.Telemetry` is the `driver: :session` facade over it - the nine functions 2.0.0 published, unchanged in arity, still the entry point for a `Statifier.Session` consumer. A driver stepping the pure interpreter directly, with no `Statifier.Session` process anywhere, emits the same contract by calling `Statifier.Telemetry` at its own step seam instead; see [docs/persistence.md](persistence.md) for what that loop looks like. Live tooling attaches at whichever boundary its driver runs. OpenTelemetry rides that contract from outside the repo: the bridge is the separate `opentelemetry_statifier` package (ADR-0062), and [docs/opentelemetry.md](opentelemetry.md) holds the span topology and propagation design it implements. `{:halted, :done | :cancelled | :budget_exhausted}` is the last message a session sends its subscribers for the run, so a consumer may treat it as end-of-stream (ADR-0044 decision 2) - constraint 2's cross-batch ordering sentence is what makes that true, since it guarantees no later-round effect can still be queued behind it. A pid that subscribes after `Statifier.Session.start_link/2` has already missed the initialize burst, and catches up by asking for the recording in the same call that subscribes it: `Statifier.Session.subscribe(server, pid, catch_up: true)` returns `{:ok, recording}`, and `Statifier.Replay.run/1` re-derives the missed prefix (ADR-0049). It requires `record: true`; nothing is retained on the session to answer it otherwise. What makes prefix and suffix meet exactly: between GenServer callbacks, `Statifier.Replay.run/1` over the session's current recording produces exactly the messages the session has notified so far - which generalizes the end-of-run equality `test/statifier/replay_round_trip_test.exs` asserts to every quiescent point. For a session started with `resume:`, the catch-up prefix begins at the resumed position rather than at the chart's initial configuration; see [docs/persistence.md](persistence.md#resuming-a-session). Observation is per session, and an invoke tree is a tree of sessions: `Statifier.Session.start_link/2`'s `:inherit_observers` (ADR-0050) starts each invoked child with the parent's `:trace` and subscribers so one attach at the root covers the whole tree, with each session's messages still carrying its own `session_id` in the envelope. `Statifier.Session.invocations/1` names the live children for an observer attaching to a tree already running, which necessarily misses each child's initialize burst. - **Replay**: because the core is pure and timers are effects, recording the external inputs (delivered events, timer firings, cancel markers) in the session's serialized input order at the session boundary makes a run reproducible. The recording has four inputs: (machine, initial data, external event log, `interpret/2` batches) - the fourth is each `Statifier.Session.interpret/2` call's effect list at its position in the session's serialized input order, and it is empty for a session never handed such a call, which keeps the familiar three-input tuple as the common case (ADR-0029). Keep the session's input path single and capturable - no side doors that inject events without crossing the recordable boundary (`Statifier.Session.Inbox`). The recorder is `Statifier.Session.Recording`, built when `Statifier.Session.start_link/2` is given `record: true` and read back with `Statifier.Session.recording/1`; note it cannot be a subscriber, because the effect stream does not distinguish core-derived effects (replay re-derives them) from `interpret/2`-injected ones (replay re-injects them). `Statifier.Replay` drives the pure core directly rather than a live session, and the recording carries ordinal order with no clock reading (ADR-0034). Each recorded entry that triggers a core drive also carries the `Statifier.Send.Routes.t()` route snapshot that drive was judged against (ADR-0048 decision 3), so `Statifier.Replay` re-supplies it rather than rebuilding one; this widens what an entry carries, and the set of recorded input kinds does not grow. `:invoke_types` (ADR-0051 decision 2) is recorded once as a normalized session option, beside `:routes`, rather than per entry: the registered `` set is fixed for the session's whole lifetime, so `Statifier.Session.Recording`'s `@normalized_opts` carries it and `Statifier.Replay` rebuilds the same snapshot for every drive. ## Non-goals (for now) - No stepper API, breakpoint model, debug protocol, or wire format. - No visualization or UI of any kind. - No trace persistence/rotation story - trace effects are handed to the effect interpreter and are its problem. - No performance work on tracing beyond the emission gate. ## Where the seams live | Seam | Where it lives | |---|---| | machine_state struct holds configuration, internal queue, history, datamodel, `running`, and step counters - no interpreter loop variable that is not reconstructible from the struct | `Statifier.MachineState` | | `microstep` step function exists; macrostep folds over it | `Statifier.Interpreter` | | trace effect types defined with the vocabulary above; emission gated | `Statifier.Effect`, `Statifier.Effect.Trace.*` | | compiler retains locations on states, transitions, executable content, and - on states, transitions and `` - the written-attribute spans of each (`attribute_locations`, carried verbatim) | `Statifier.Compiler`, `Statifier.Machine.State`/`Transition`/`Content`/`Invoke` | | compiled expressions carry their span table with the instructions (ADR-0014) | `Statifier.Compiler.Expressions.compile/3` stores `%Predicator.Compiled{}` whole | | compiler assigns document-order indexes to transitions, executable-content nodes, and `` elements (`t_index`/`c_index`/`d_index`, dense from 0) | `Statifier.Compiler` | | internally raised events carry cause metadata (identity + step) | `Statifier.Event.Cause`, `MachineState.raise_internal/4` | | Appendix D query functions take/return plain values, unfused | `Statifier.Interpreter.Selection`, `Statifier.Interpreter.ExitEntry`, `Statifier.Interpreter.Content` | | the session's live invocations are nameable from outside it, so an observer can walk the invoke tree | `Statifier.Session.invocations/1`, `Statifier.Session.Invocations.list/1` | | validator warnings retained on the compiled Machine (ADR-0033) | `Statifier.Validator.Warning`, `Statifier.Machine.warnings` | The last row is a seam of one link, not an omission: a validator finding is produced before a `Machine` exists, so none of the trace vocabulary above reaches it. There is no trace effect, no `Logger` call, and no `:telemetry` event for a validator finding - the return value `validate/2` hands back and the field `compile/1` stamps it onto *are* the channel, end to end.