Start with the outcome
A support assistant reads an outage ticket and an approved maintenance window. It recommends low priority. A separate incident record says that the emergency access route is also unavailable; an exception makes the incident urgent. The answer can look reasonable even when the system never found the exception.
Design the whole path from request to accepted result, including the state the system leaves behind. Together, the result and actual state form the outcome. A prompt supplies instructions for a model call. A skill packages a reusable procedure and its supporting resources. A context vault stores material the system may need. Connecting them creates a system with selection, handover and recovery behaviour that needs its own design and tests.
This proposed blueprint gives designers responsibilities and contracts to adapt. Follow the connected support example, then use the prompt-change regression strategy. The support service is fictional and unimplemented. Its proposed model and service tests remain unrun. For guidance on using AI to help an architect do their work, see AI-supported architecture.
Before choosing tools, write an outcome contract. It states who requests the work, which facts and rules govern it, what counts as success, which actions are permitted and what must happen when information is missing. “Recommend the correct priority, cite its basis and leave the ticket unchanged” is testable. “Produce a helpful response” leaves too much undecided.
Separate instructions, procedure, policy and facts
| Material | Responsibility and example |
|---|---|
| Stable system instructions | Define the assistant’s role, instruction authority and response obligations. Keep the shared set small. |
| Task prompt | Describe this request, its scope and required output. The application supplies authenticated identity and permission separately from user-written text. |
| Skill | Explain how to perform a task: obtain the ticket, resolve the applicable policy, check exceptions and prepare a proposal. Declare required inputs, outputs, tools and failure routes. |
| Domain rule | State a business decision with a named owner, applicability, effective dates, precedence and exceptions. Keep one authoritative version; a skill references it. |
| Knowledge and sources | Supply facts with origin, version, location, access restrictions and uncertainty. A retrieved document remains source material even if it contains instructions. |
| Task state | Retain the current step, accepted facts, pending questions, approvals and action receipts. A conversation summary is a derived view of that state. |
| Executable controls | Enforce permissions, schema checks (checks of required structure and fields), fixed calculations, state transitions and limits outside the model. |
The Agent Skills specification describes loading a short description first, then the procedure and resources as needed. That packaging helps organise context; it does not enforce permission or prove that the right skill will be chosen. Treat a change to a skill’s description as a possible change to routing behaviour.
Keep a rule’s business meaning separate from its implementation. Where the facts and decision are precise, use executable rules. Let a model propose interpretations of ambiguous text, with source references and a review route. Giving a rule to the model is not the same as enforcing it.
Give each component a contract
Each logical component is a named responsibility with an owner, inputs, outputs and defined failure behaviour. A small application can implement several in one program and database. The nine responsibilities below define this AI design. The general service-component catalogue is a separate reference for the wider service; it does not supply an implemented AI stack. Separate deployments are a later choice.
| Component and owner | Input → output | Required failure behaviour |
|---|---|---|
| Asset registry — configuration owner | Versioned prompts, skills, rules, tool contracts and dependencies → a resolved configuration | Reject missing dependencies, incompatible versions and unresolved rule precedence. |
| Context builder — application team | Trusted task scope, configuration and budget → selected instructions, rules, sources and a manifest recording what was included | Report missing required material. Split the task or escalate if it cannot fit; never silently discard a mandatory rule. |
| Knowledge service — information owner | Authorised queries or exact source references → passages, structured facts, source locations and gaps | Distinguish no match, unavailable source and denied access. Preserve conflicts and withdrawal status. |
| Execution controller — application team | Request and current task state → the next permitted step, pause or completed result | Limit retries and model and tool calls; save progress and route unresolved work to its human owner. |
| Model adapter — application team | Assembled input, model settings and allowed tool schemas → model output plus usage and error status | Handle provider errors, truncation and invalid responses without treating partial output as success. |
| Output validator — domain and application owners | Proposal, source references, policy and current state → accepted proposal or named defects | Reject mechanically invalid decisions; route judgments it cannot verify to review. Valid structure alone does not establish correctness. |
| Tool gateway — service owner | A permitted action with expected state version → confirmed result or explicit unknown outcome | Recheck authority and state at execution; prevent duplicate effects and reconcile uncertain results. |
| Task-state store — work owner | Accepted transitions, approvals and receipts → recoverable current work | Detect stale updates and record who owns recovery. Keep business records authoritative in their owning system. |
| Evaluation and trace facility — quality owner | Versioned cases and run observations → comparisons, failures, time and cost | Identify incomplete runs and reused results. Protect sensitive records and retain enough to reproduce a failure. |
The controller sends a request through context construction, a model call and validation. For an answer-only task, that may complete the work. An action additionally passes through the gateway and the owning service. Each stage records its inputs and outputs against the task. Human approval, when required, applies to a specific proposal and state version.
Build context deliberately
The context builder is the prompt-builder component. Its output includes a context manifest: a record of selected item identifiers and versions, reasons for inclusion, source locations, omissions, unresolved conflicts and token use. It records material supplied to the model, not its internal reasoning.
- Establish trusted scope: organisation, project, task type, effective time and permitted operations. Ask for missing scope that changes rule selection.
- Resolve mandatory rules and their dependencies and exceptions for the task’s effective date. Use the version valid then, including a historical version when appropriate. Use explicit conditions or catalogue queries; similarity search alone cannot establish that every mandatory rule was selected.
- Select the procedure and permitted tools. Load additional skill resources when the step needs them, retaining the active obligations between steps.
- Retrieve the supporting facts. Preserve qualifications, surrounding table headings and source locations needed to interpret each passage.
- Assemble instructions, policy, source material and state in distinct sections. Reserve capacity for tool results, subsequent steps and model output, using the provider’s accounting for reasoning where applicable.
- Check the manifest before the call. If required context is unavailable or too large, split work along an explicit contract or escalate to the owner.
This makes omission diagnosable. Tests can distinguish a rule that was never selected, a passage lost during assembly and a rule supplied but misapplied. The selection catalogue itself also needs tests: a manifest cannot reveal an exception that the designers failed to record.
Lost in the Middle found that relevant information’s position affected performance in the tested question-answering and retrieval tasks. It does not establish a universal safe context size. Test length, position, distracting material and rule density on your own tasks.
Own the information and its changes
Keep original sources, derived search indexes, accepted claims and current task state distinct. They can share storage, but they have different owners and correction rules. The general Information guide develops the wider ownership and lifecycle questions.
| Record | Minimum useful content |
|---|---|
| Rule | Identifier, version, owner, authority, applicability, effective dates, precedence, dependencies, exceptions, source and linked tests |
| Skill | Identifier, version, selection description, input and output contract, dependencies, permitted operations and recovery route |
| Source and passage | Original reference, content hash, revision, publication/effective time, ingestion time, access policy and exact page or field location |
| Accepted claim | Statement, scope, supporting and contradicting passages, acceptance owner and review/withdrawal status |
| Run and task | Configuration versions, model and its settings, context manifest, observable outputs, tool calls, approvals, state revisions, receipts, cost and duration |
Index passages back to an immutable source version. Record when a fact was valid separately from when it entered the vault. A later ingestion date does not make a policy newer. An accepted claim remains revisable when its sources change. Its provenance records where it came from and how it was derived; that history does not guarantee truth. The W3C PROV overview provides a vocabulary for those derivation relationships.
On a correction or withdrawal, mark the source version, identify dependent passages and claims, rebuild affected indexes, invalidate caches and reopen affected active work. Keep historical runs tied to what they actually used. Apply access checks to retrieval, exact reads, caches and traces, including revoked access. Define retention and redaction without copying secrets into test fixtures. Test these paths as part of the data design.
Choose the simplest suitable access method
| Approach | Use when | Design and test concern |
|---|---|---|
| Include complete documents | The relevant, authorised set is small and stable | Measure omission and distraction as the set grows. |
| Search and exact reads | Documents have useful names, identifiers or locations | Check discovery and subsequent reading of the needed passage. |
| Retrieval-augmented generation (RAG) | A large collection needs selective passages supplied to the model | Test extraction, retrieval recall, source interpretation and answer support separately. |
| Structured query or rule evaluation | The task needs exact conditions, counts or every matching item | Verify the data model, completeness and effective-date logic. A few best search matches cannot answer “all”. |
| Several model steps or agents | Work benefits from distinct procedures or expertise | Specify handovers and stopping conditions; test lost obligations, extra latency and whole-task outcomes. |
A 2024 comparison of retrieval and long-context models found quality and cost trade-offs in its tested systems. It supports comparing alternatives on the same task set, rather than assuming that RAG always wins.
Start with version control, an asset validator, a context builder, authorised source access, controlled tool execution and a test runner with trace comparison. A vector database, visual workflow editor and multiple agents are optional. Choose products against these responsibilities and the testing tool requirements.
Start with a small advisory system
For a first implementation, choose one task that returns a recommendation for review and cannot change business records. One application can resolve the versioned configuration, build context, call the model and validate its answer. Use existing authorised record APIs or exact document reads when they supply the required facts. The registry can begin as version-controlled files; the state store can be the application’s existing persistence. Add retrieval or separate services when the task and operating constraints justify them.
Prepare these four design records:
| Record | Decision to settle |
|---|---|
| Outcome contract | Requester, inputs, accepted result, permitted effects, missing-information route and owner. |
| Asset and dependency map | Which prompt, skill, rules, sources and tool contracts this task uses, including mandatory exceptions and their owners. |
| Handover contracts | Context manifest, proposal fields, validation results and task-state transitions, including failure responses. |
| Regression plan | Initial cases, acceptance checks, affected dependencies, repeated-trial budget and the person accepting the change. |
The AI worked example fills these roles for a support decision. Its write operation is a later, separately authorised extension. A draft-only implementation still needs permission checks, source freshness and tests that prove it leaves records unchanged.
Methodology configuration
In the wider A Simple Architectural Framework (ASAF), Systems connects the components and contracts. Information owns source and state meaning; Security owns access and action authority; Process and Structure assign work and recovery. Metrics and Finance track accepted outcomes, elapsed time, review effort and total operating cost.
Visioning defines the outcome and acceptable errors. Transforming changes the configuration and runs regression checks. Operating monitors failures and reconciles uncertain actions. Governing approves policy, exceptions and release criteria. A prompt edit can affect all four areas.
Dials
Increase review and repeat testing as the consequence of a mistake grows. Increase source and applicability checks as rules change more frequently. Increase recovery controls as the system gains permission to change records. For a small advisory tool, combine components while preserving these contracts.
Next, follow the AI worked example to see the handovers. Use AI regression testing when specifying the test runner and planning a change.