Causal Lineage

Preserve the evidence connecting a decision to its inputs, permissions, actions, and observed outcomes.

By E. M. AbdullahPublished Updated

Causal Lineage

Preserve the evidence connecting a decision to its inputs, permissions, actions, and observed outcomes.

Context.

A customer disputes the closure of a support case. The team can find today's case record and yesterday's model response. Neither tells them which conversation revision was available, which rule permitted closure, or whether the customer replied while approval was pending.

Steps and States give the workflow its structure. Lineage preserves the relationships between those steps and their validated results. It also records failed attempts and proposals that never became accepted business state. Otherwise the system remembers only its successes.

Problem.

Ordinary diagnostic records often describe isolated activity: a request began, a model answered, a write succeeded. They may omit the exact inputs, expire quickly, or disappear under sampling. Joining them by timestamps guesses at dependencies when several requests overlap.

A useful decision history must distinguish what was available from what was actually supplied, what was proposed from what was authorised, and what was authorised from what happened. It must remain understandable after policies change and the people who built the workflow leave.

Forces.

  • Outcomes arrive late and may never arrive. Missing evidence must remain unknown rather than becoming an assumed success or failure.
  • Distributed services have separate owners, clocks, retention policies, and access rules. A single global sequence is usually unavailable.
  • Exact reconstruction needs retained payloads and versions; indiscriminate copying increases privacy exposure and storage cost.
  • Immutable history needs corrections, schema evolution, and controlled retention without pretending the earlier record never existed.
  • A dependency graph supports investigation. It does not establish that one business decision caused an observed outcome.

Pattern.

Give each workflow, step, attempt, and event a stable identity. Append records describing strategy selection, compiled context, model invocation, validation, approval, and effects. Connect them with explicit parent identifiers and typed relationships. Capture inputs at the point of use, including retrieved results and the versions of the rules that selected them.

Separate the event envelope from protected payloads. The envelope identifies the tenant, producing service, event type, schema version, dependencies, and payload references. Authorised investigators can resolve retained payloads; broad operational dashboards need not expose them. A digest checks integrity against retained bytes. It cannot reconstruct missing bytes or prove that their original source told the truth.

Use the term causal carefully. A consumed-input edge describes a system dependency. An observed-outcome edge associates a later measurement with the workflow. Estimating the effect of a strategy still requires an experiment or a defensible inference design. Model explanations are recorded claims, not privileged access to the model's internal reasoning.

The technical ancestor is Event Sourcing, described by Martin Fowler in 2005: preserving changes as events supports historical reconstruction. Full event sourcing of every business database is not a prerequisite for decision lineage. An append-only decision record can coexist with an existing transactional system if the relationship between its writes and its evidence is reliable.

Implementation.

This illustrative event records the validation of a case-closing proposal. Identifiers and digests are examples, not real evidence. The referenced objects must exist under independently enforced access controls.

event_id: support.ev-104
event_type: proposal.validated
schema_version: 2
tenant_id: tenant-7
producer: support-validator
workflow_id: support.wf-82
step_id: close-case
attempt_id: support.attempt-2
parents:
  - event_id: support.ev-103
    relation: validated_candidate
occurred_at: "2026-09-28T14:03:11Z"
recorded_at: "2026-09-28T14:03:12Z"
context:
  manifest_ref: evidence/context-82-v1
  digest: illustrative-context-digest
  case_revision: 19
strategy_version: close-case.v4
model_invocation_ref: support.ev-102
contract_version: CloseCaseProposal.v2
validator_version: support-rules.v8
policy_version: support.case-close.v3
candidate_ref: evidence/candidate-82-2
verdict: pending_review
reasons: [bulk_closure_flag]
effect_status: not_authorised
usage_ref: support.usage-102
retention_class: support-decision-evidence

The invocation event referenced here records the model identifier returned by the provider, requested configuration, prompt template revision, exact assembled input reference, raw response reference, usage, and completion status. Do not invent a model revision when the provider exposes only an alias. Record that reproducibility limit. The context manifest lists actual retrieved passages, their source versions, and any omissions relevant to the decision.

A later approval event points to this verdict and the exact candidate. An execution receipt then records whether the case actually closed. Keep rejected attempts and timeouts too, with an explicit retry relationship. The attempt identifier changes on a new invocation; an idempotency key for the same intended business effect does not.

Each service owns durable capture of its events. Authenticate producers, namespace identifiers, and deduplicate delivery by event identity. Persist the local business change and an outgoing event in one transaction where possible; a relay can deliver it repeatedly without creating new facts. For remote effects, record intent and reconcile the receipt. Expose unresolved parent references and delayed receipts as gaps.

Reconstruct a historical workflow using retained inputs and recorded responses with external writes disabled. Rebuilding a projection is replay. Calling a model again is a new attempt, even with the same prompt. Comparing a new strategy against old evidence is an evaluation run with its own identity. Never let an investigation resend customer messages.

When an outcome arrives, append its definition, measurement window, source, and link to the relevant decision. Record experiment assignment when one exists. Late corrections supersede earlier observations through explicit links; they do not overwrite them. Separate event time from recording time so an investigator can distinguish what happened from when the organisation learned about it.

Consequences.

The gain is a durable account of the workflow's evidence and authority. Engineers can compare versions, locate expensive retries, and distinguish a validation failure from an execution failure. Reviewers can state both what the record supports and what remains missing.

Durable capture adds latency. Payload storage, indexes, replicas, access logs, and historical readers cost money and operational effort. Cross-service ownership creates organisational friction over identifiers, retention, and incident responsibility. Set budgets using events per decision, average payload size, retention duration, and query load; a raw storage price is not a system cost.

Define retention and authorised deletion separately for metadata and sensitive payloads. Append correction or deletion records, preserve only permitted residual metadata, and mark reconstruction as incomplete when evidence expires. Restrict writers, protect archives, and test restoration. Calling a store immutable does not make privileged tampering impossible.

Anti-patterns.

The timestamp join. Nearby events are assumed to belong to the same decision. Carry explicit dependency identifiers across boundaries and make missing links visible.

The explanation as proof. A fluent rationale is treated as evidence of truth or internal reasoning. Retain it as a model claim and check its cited evidence independently.

The silent replay. Rebuilding history calls today's model or repeats an external write. Separate reconstruction from new evaluation and disable effect adapters during replay.

Test.

Choose a disputed decision from before the last policy change and reconstruct its supplied context, candidate, validation, authority, effect, and recorded cost using retained evidence. Follow its cross-service dependencies and identify any missing outcome explicitly. If this requires querying today's mutable source, trusting a generated explanation, or producing a fresh external effect, the lineage is incomplete.

Related.

Causal Lineage is the memory of Vertical AI Foundation and addresses the lost accountability in The AI Mainframe Trap. Control Plane vs. Execution Plane defines the scope it records; AI Airlocks contribute verdicts and approvals. Thin Boundary distributes ownership while preserving those shared evidence contracts.