# ID-T04 — Processing and derived data: completed example

Adrian Sutherland · Version 1.0 · Refreshed September 2026

© 2005–2026 Adrian Sutherland.
Source page: https://architectureportal.org/assets/information-data

Fictional teaching example. Proposed decisions and checks; actual trial results remain unknown.

## Decision and consumers

ID-PROC1 proposes a count of currently resolved cancellation cases by centre
at a stated cutoff. Centre means the centre responsible for the cancelled
course; an alternative elsewhere keeps that original attribution. The service
manager uses it with open-case information for
planning. It is a view of current state, not a count of all resolution events.
The service owner approves its meaning and required freshness.

## Inputs and transformation

Synthetic snapshot S1 has one current record per case: A/North/resolved,
B/North/open, C/South/resolved. Each case retains its identity and input version.
Rule RPT1 emits (centre,1) for each resolved case, groups by centre and sums.
Expected result: North=1, South=1. Known centres with no resolved case are
shown as zero; the centre list is a separately versioned input.

Retain the snapshot and centre-list identifiers, cutoff, RPT1 version and output
version. In a later S2, C is reopened: expected result North=1, South=0. This is
a proposed derived report separate from the authoritative case records.

## Layers and quality controls

One option is a medallion arrangement:

| Layer | Proposed contents and checks | Responsibility |
|---|---|---|
| Bronze | Retained received extracts, source/version identifiers and receipt log; check expected arrivals. | Ingestion owner records receipt and failures; role assignment pending. |
| Silver | Accepted current case versions with recognised states and valid references; reconcile repeated inputs and hold unresolved conflicts. | Data steward resolves definitions and coordinates correction with case owners. |
| Gold | RPT1 counts by centre with cutoff, links back to source records, freshness and any excluded or unresolved cases disclosed. Reconcile to the accepted input set. | Report owner maintains the output; service owner approves meaning and whether it is suitable for use. |

The operational case register remains authoritative for case facts. These
layers retain received versions and derived views. Source corrections propagate
through the affected layers with their rule versions recorded. An unresolved
conflict makes the report provisional until the service owner decides whether
the visible gap permits its intended use. Access, retention, removal and the
need to store each intermediate result remain open decisions.

## Options and operating choices

Compare a database query over a selected snapshot, a partitioned batch job and
an event-driven current-state view. The small example can be calculated directly.
No volume or throughput requirement for distributed execution has been established.
A nightly batch is an illustrative planning option; a live operational need
would require a different freshness decision. Real volume, arrival rate,
centre skew, platform, latency target and running cost remain unknown.

## Corrections and recovery

Repeating a run with the same snapshot, centre list and transformation version
produces the same counts. It replaces the corresponding output rather than
adding the counts again. A change to an input or the transformation version
creates a revised result linked to the earlier one. The report must
include zero for a centre whose last resolved case has reopened.

A streaming implementation would need event identity, ordering/version rules,
late-arrival treatment and corrections to earlier aggregates. Those details
remain to be designed. The report owner must agree access, input/output
retention and how corrections reach consumers. Recovery reprocesses identified
inputs and reconciles the output before acceptance.

## Checks and results

| Check | Expected result | Actual result |
|---|---|---|
| Process S1 under RPT1 | North=1, South=1. | Not run |
| Replay S1 | Same result, with no double counting. | Not run |
| Process S2 after C reopens | North=1, South=0; earlier output traceable. | Not run |
| Input has two conflicting current versions of one case | Hold for reconciliation before producing an accepted report. | Not run |
| An input arrives after the cutoff | Follow the agreed inclusion/revision policy and show output version. | Policy pending; not run |
| A source conflict prevents a case reaching silver | Report the unresolved case and mark the gold output provisional; record the service owner's use decision. | Not run |
| C is reopened after an accepted gold report | Trace the new received version through silver to the revised gold count; retain the previous report and its version links. | Not run |

These are paper expectations. A runnable pipeline, performance baseline and
measured benefit remain open. A real implementation would record freshness,
processing cost, reconciliation effort and the effect of later rule changes.

## Methodology configuration

ID-PROC1 could sit in a data-flow model, pipeline specification or data-product
record. At the service review, the service owner accepts meaning and freshness.
Delivery reviews check transformation and recovery; operation records delays
and corrections.

## History and open decisions

Version 1.0, September 2026; fictional proposed extension to the cancellation
example. The service owner decides report meaning and use. Operating targets,
implementation, late-arrival policy, permissions and retention require decisions
before use with real records.