Start with a small advisory system
Begin with one task that returns a recommendation for someone to review. One application can select the versioned instructions, obtain the facts, call the model and check its answer.
Use the service’s existing application programming interfaces (APIs), which accept requests for records, or read known documents directly. A registry is a catalogue of versioned prompts, procedures and rules; it can begin as files in version control. The application can keep task progress in its existing storage.
Before choosing tools, write an outcome contract: a short statement of the request, accepted result, permitted actions and response to missing information. For this example, it requires a supported priority recommendation or a request for assessment, with the ticket unchanged. A general request for a helpful answer would not settle those decisions.
Prepare four records that make the design testable:
| Record | What to decide |
|---|---|
| Outcome contract | Who requests and accepts the work, what success means, and which effects are permitted |
| Asset and dependency map | Which prompts, skills, rules, exceptions, sources and tools the task needs, and who maintains them |
| Handover contracts | What each step receives and returns, including gaps, rejected proposals and recovery work |
| Test plan | Cases, acceptance checks, repeated trials, a budget and the person accepting the result |
The worked support example fills in these records. Its write operation is an optional extension. Even an advisory application needs permission checks and current sources; tests should check that the exercised tasks leave business records unchanged.
Separate instructions, procedure, policy and facts
| Material | Responsibility and example |
|---|---|
| Stable system instructions | Define the assistant’s role, instruction authority and response obligations. Keep the shared set small. |
| Task prompt | Describe this request, its scope and required output. The application supplies authenticated identity and permission separately from user-written text. |
| Skill | Explain how to perform a task: obtain the ticket, resolve the applicable policy, check exceptions and prepare a proposal. Declare required inputs, outputs, tools and failure routes. |
| Domain rule | State a business decision with a named owner, applicability, effective dates, precedence and exceptions. Keep one authoritative version; a skill references it. |
| Knowledge and sources | Supply facts with origin, version, location, access restrictions and uncertainty. A retrieved document remains source material even if it contains instructions. |
| Task state | Retain the current step, accepted facts, pending questions, approvals and action receipts. A conversation summary is a derived view of that state. |
| Executable controls | Enforce permissions, schema checks (checks of required structure and fields), fixed calculations, state transitions and limits outside the model. |
The Agent Skills specification describes loading a short description first, then the procedure and resources as needed. That packaging helps organise context; it does not enforce permission or prove that the right skill will be chosen. Treat a change to a skill’s description as a possible change to routing behaviour.
Keep a rule’s business meaning separate from its implementation. Where the facts and decision are precise, use executable rules. Let a model propose interpretations of ambiguous text, with source references and a review route. Giving a rule to the model is not the same as enforcing it.
Give each component a contract
Each logical component is a named responsibility with an owner, inputs, outputs and defined failure behaviour. A small application can implement several responsibilities in one program and database. The nine responsibilities below define this AI design. The general service-component catalogue is a separate reference for the wider service; it does not supply an implemented AI stack. Choose separate deployments only when operating needs justify them.
| Component and owner | Input → output | Required failure behaviour |
|---|---|---|
| Asset registry — configuration owner | Versioned prompts, skills, rules, tool contracts and dependencies → a resolved configuration | Reject missing dependencies, incompatible versions and unresolved rule precedence. |
| Context builder — application team | Trusted task scope, configuration and budget → selected instructions, rules, sources and a manifest recording what was included | Report missing required material. Split the task or escalate if it cannot fit; never silently discard a mandatory rule. |
| Knowledge service — information owner | Authorised queries or exact source references → passages, structured facts, source locations and gaps | Distinguish no match, unavailable source and denied access. Preserve conflicts and withdrawal status. |
| Execution controller — application team | Request and current task state → the next permitted step, pause or completed result | Limit retries and model and tool calls; save progress and route unresolved work to its human owner. |
| Model adapter — application team | Assembled input, model settings and allowed tool schemas → model output plus usage and error status | Handle provider errors, truncation and invalid responses without treating partial output as success. |
| Output validator — domain and application owners | Proposal, source references, policy and current state → accepted proposal or named defects | Reject mechanically invalid decisions; route judgements it cannot verify to review. Valid structure alone does not establish correctness. |
| Tool gateway — service owner | A permitted action with expected state version → confirmed result or explicit unknown outcome | Recheck authority and state at execution; prevent duplicate effects and reconcile uncertain results. |
| Task-state store — work owner | Accepted transitions, approvals and receipts → recoverable current work | Detect stale updates and record who owns recovery. Keep business records authoritative in their owning system. |
| Evaluation and trace facility — quality owner | Versioned cases and run observations → comparisons, failures, time and cost | Identify incomplete runs and reused results. Protect sensitive records and retain enough to reproduce a failure. |
In the diagram, the controller sends a request through context construction, a model call and validation. For an answer-only task, that may complete the work. An action additionally passes through the gateway and the owning service. Each stage records its inputs and outputs against the task. Human approval, when required, applies to a specific proposal and state version.
Choose the simplest suitable access method
| Approach | Use when | Design and test concern |
|---|---|---|
| Include complete documents | The relevant, authorised set is small and stable | Measure omission and distraction as the set grows. |
| Search and exact reads | Documents have useful names, identifiers or locations | Check discovery and subsequent reading of the needed passage. |
| Retrieval-augmented generation (RAG) | A large collection needs selective passages supplied to the model | Check extraction, how much required material retrieval finds, interpretation and answer support separately. |
| Structured query or rule evaluation | The task needs exact conditions, counts or every matching item | Verify the data model, completeness and effective-date logic. A few best search matches cannot answer “all”. |
| Several model steps or agents | Work benefits from distinct procedures or expertise | Specify handovers and stopping conditions; test lost obligations, extra latency and whole-task outcomes. |
A 2024 comparison of retrieval and long-context models found better average performance from sufficiently resourced long context in its tested systems, while retrieval cost less. Its failure analysis includes questions that need facts from several passages and questions whose wording does not name the information needed to answer them. One question asks for a singer’s nationality: the system must first identify the singer, then find that person’s nationality. Those examples explain why a relevant-looking search result may still be insufficient. See section 5.2 of the paper. Compare approaches on the work your service must perform; neither access method is a general accuracy guarantee.
Check whether existing tools can version the assets, validate their dependencies, build the model input, control source access and actions, and compare test records. Several responsibilities may share one application. Add separate tools where the service needs them. Use the testing tool checklist when comparing products.
Build context deliberately
The context builder is the prompt-builder component. Its output includes a context manifest: a record of selected item identifiers and versions, reasons for inclusion, source locations, omissions, unresolved conflicts and the amount of context used. Models process text in units called tokens; these matter when estimating capacity and cost. The manifest records material supplied to the model, not its internal reasoning.
- Establish trusted scope: organisation, project, task type, effective time and permitted operations. Ask for missing scope that changes rule selection.
- Resolve mandatory rules and their dependencies and exceptions for the task’s effective date. Use the version valid then, including a historical version when appropriate. Use explicit conditions or catalogue queries; similarity search alone cannot establish that every mandatory rule was selected.
- Select the procedure and permitted tools. Load additional skill resources when the step needs them, retaining the active obligations between steps.
- Retrieve the supporting facts. Preserve qualifications, surrounding table headings and source locations needed to interpret each passage.
- Assemble instructions, policy, source material and state in distinct sections. Reserve capacity for tool results, subsequent steps and model output, using the provider’s accounting for reasoning where applicable.
- Check the manifest before the call. If required context is unavailable or too large, split work along an explicit contract or escalate to the owner.
This makes omission diagnosable. Tests can distinguish a rule that was never selected, a passage lost during assembly and a rule supplied but misapplied. The policy owner must review the catalogue against the policies it represents, including applicability conditions and exceptions. Independently annotated cases test that selection. A manifest cannot reveal an exception that the designers failed to record.
In Lost in the Middle, section 2, researchers gave models a question and several documents, one containing the answer. They moved that document while keeping the question and required answer unchanged. Performance often fell when the answer was in the middle. A useful application here is to move the emergency-route exception between the beginning, middle and end of a test input and check that the decision remains correct. The study’s tested models and tasks do not establish a universal safe context size. Also vary distracting material and the number of rules to apply.
Loading skill resources only when needed can reduce the initial context. A July 2026 study of progressive disclosure compared reading books directly, using an index of passages, and adding another routing level. The benefit depended on the agent’s existing navigation and the size of the collection; the extra level did not improve the tested arrangements. For this blueprint, that suggests trying a simple index before adding hierarchy. The study concerns book questions, so a business workflow needs its own comparison.
Own the information and its changes
Keep original sources, derived search indexes, accepted claims and current task state distinct. They can share storage, but they have different owners and correction rules. The general Information guide develops the wider ownership and lifecycle questions.
| Record | Minimum useful content |
|---|---|
| Rule | Identifier, version, owner, authority, applicability, effective dates, precedence, dependencies, exceptions, source and linked tests |
| Skill | Identifier, version, selection description, input and output contract, dependencies, permitted operations and recovery route |
| Source and passage | Original reference, content fingerprint (hash), revision, publication/effective time, ingestion time, access policy and exact page or field location |
| Accepted claim | Statement, scope, supporting and contradicting passages, acceptance owner and review/withdrawal status |
| Run and task | Configuration versions, model and its settings, context manifest, observable outputs, tool calls, approvals, state revisions, receipts, cost and duration |
Index passages back to an immutable source version. Record when a fact was valid separately from when it entered the source store. A later ingestion date does not make a policy newer. An accepted claim remains revisable when its sources change. Its provenance records where it came from and how it was derived; that history does not guarantee truth. The W3C PROV overview provides a vocabulary for those derivation relationships.
On a correction or withdrawal, mark the source version, identify dependent passages and claims, rebuild affected indexes, invalidate caches and reopen affected active work. Keep historical runs tied to what they actually used. Apply access checks to retrieval, exact reads, caches and traces, including revoked access. Define retention and redaction without copying secrets into test fixtures. Test these paths as part of the data design.
Compare the design on representative tasks
Choose cases that reflect ordinary work and important exceptions. Agree what counts as an accepted result, which errors matter most and how much time, review effort and cost the task can justify. Include a straightforward solution without a model where structured data and fixed rules can already do the work. In the support example, code can calculate the priority; the model must justify its additional value through interpreting requests or explaining the result.
Compare complete documents, selected rules with exact reads, and selected rules with retrieval on the same tasks. Keep the sources, permissions, model settings and output allowance comparable. Measure missed rules and facts, incorrect decisions, unnecessary referrals, accepted outcomes and total cost. A smaller prompt is useful only if the resulting service still meets its needs.
For diagnosis, supply a human-selected set of all the required rules and facts. If that improves the answer, investigate selection and assembly. If it does not, investigate interpretation and the model step. This comparison helps locate a problem; it does not identify a unique cause or replace testing the whole workflow. The testing guide explains how to compare trials and check subsequent changes.
Methodology configuration
Use A Simple Architectural Framework (ASAF) to connect this design to the wider service. Systems examines the components and their contracts. Information asks what sources and state mean and who maintains them. Security examines access and authority. Process and Structure help assign work and recovery to people. Metrics and Finance help the team judge outcomes, elapsed time, review effort and operating cost.
Visioning defines the outcome and acceptable errors. Transforming changes the configuration and runs regression checks. Operating monitors failures and reconciles uncertain actions. Governing approves policy, exceptions and release criteria. A prompt edit can affect all four areas.
Dials
The delivery dials are choices about how the team works. Automation and checking should reflect the consequences of an error: more consequential decisions need closer review and repeated trials. Decision authority defines who may approve changes to records and who owns recovery. Cadence, the frequency of review, should account for how often policy and sources change.
A small advisory tool can combine components while retaining their responsibilities. Use the general methodology configuration guide to fit these reviews and records into your team’s existing method.
Next, follow the AI worked example to see the handovers. Use AI regression testing when specifying the test runner and planning a change.