A fast answer, an unresolved case
In the fictional cancellation example, a future case service could respond quickly while a learner still waits days for an agreed replacement. Technical response time would reveal little about that delay. The team would also need different observations to judge whether its architecture method helped it reach a useful design efficiently.
The Metrics aspect of A Simple Architectural Framework (ASAF) includes non-functional requirements: qualities such as availability, performance, recoverability and maintainability. It connects those requirements to observations and decisions throughout change and operation.
Three questions
| What is being assessed? | Examples of useful observations |
|---|---|
| Architecture and delivery method | Time to an accepted decision, waiting, review effort, rework and usefulness of handovers. |
| Resulting service or solution | Agreed resolutions, incorrect bookings, unauthorised access, recovery and effort to support or change the service. |
| Usefulness throughout the service’s life | Dated changes, retained responsibilities, replacement effort and reasons a solution remained useful or was retired. |
Define each measure before interpreting a number. Record its purpose, unit, start and end events, population, exclusions, observation period, source, owner and intended response. Keep a target separate from an observation and a proposed check separate from its result.
The assessment guide places these observations alongside questions about development, support, change and useful life.
Quality in a scenario
“Recoverable” becomes useful when the record states the disruption, affected work, acceptable loss or delay and how recovery will be checked. A maintainability scenario can ask how a different team changes a rule safely without rediscovering its meaning. A security requirement can ask whether a withdrawn partner account can still read cases.
For a summary produced by a machine-learning model, measure factual support and unacceptable omissions as well as response time and cost. Keep the versions of the inputs, instructions, model and evaluation method with the results.
Interpretation
Look at distributions and affected groups as well as averages. Identify missing observations, changes in workload and competing explanations. Repeated contact may reflect a confusing channel rather than more demand. Avoid using one measure to reward a behaviour that undermines another outcome.
To assess how long a solution remains useful, retain its history of use, change and retirement. The date of a design document does not show that history.
Methodology configuration
Your method may keep quality requirements in service objectives, acceptance criteria or design records. Use those names and connect the measures to their decisions. Methodology configuration explains how to map terms, records and reviews to your own approach.