Iron Brick AI Field Guide · Architecture ·
A demo needs a model and a prompt. A production AI system in an agency needs four layers working together, each with its own decisions and controls.
1. Orchestration
The orchestration layer decides what the system may do. Keep a registry of tools, each a governed API or MCP server with a clear specification and least-privilege scopes, and decide deliberately what the system remembers and for how long.
2. Retrieval
Most of an AI system's accuracy comes from retrieval. Chunk documents sensibly, combine keyword and semantic search, rerank results, and filter by metadata so users only retrieve what they are allowed to see. Retrieval should enforce the same access rules as the source systems.
3. Infrastructure
Containers, an orchestration platform, a serving layer, and model hosting, all inside the agency's authorization boundary. Plan GPU capacity and autoscaling early; they drive both cost and performance.
4. Observability and evaluation
Trace every request end to end, measure latency and cost, keep structured logs, and evaluate answer quality on a fixed test set before and after every change. These records are also the evidence assessors ask for.
Key takeaways
- Register every tool with a specification, owner, and least-privilege scopes.
- Make retrieval enforce the same access rules as the source systems.
- Host models inside the agency's authorized boundary.
- Evaluate quality on a fixed test set and keep traces and logs as evidence.