15 · Models that never leave the building
Agent systems running on infrastructure the institution controls, so customer data, positions and internal documents are never sent to a third-party model.
For a regulated institution the blocker on agentic tooling is rarely capability. It is that the useful work sits on data which cannot lawfully or commercially leave the estate: customer records, positions, case files, unreleased pricing. Sending that to a hosted model is a disclosure, and no amount of contractual assurance converts it back into a private one.
Running the models locally removes the disclosure instead of mitigating it. That changes the engineering problem from prompt design to systems design: model selection against real hardware budgets, quantisation and throughput, routing cheap work to small models and reserving the expensive one for the fraction that needs it, and an execution layer that can be audited afterwards because every action it took was recorded before it took it.
Common questions
Why run models locally instead of using a hosted API?
Because the work that is worth automating usually sits on data you are not permitted to send anywhere. Once the data cannot leave, the hosted option is not a cheaper version of the same thing, it is a different product that cannot do the job. Local inference also removes per-token cost from the unit economics, which changes what is worth automating at all.
What hardware does a local agent system actually need?
Far less than most teams assume, because most agentic work is not reasoning-hard. Routing classification, extraction and drafting to small quantised models and reserving a large model for genuine reasoning typically keeps a serious internal deployment inside a single well-specified machine. The sizing question is answered by profiling the real workload, not by picking a model first.
How do we stop an agent doing something it should not?
By making the action layer the control point, not the prompt. Permissions are enforced where the tool executes, not requested in the instructions, so a model that is confused or manipulated still cannot exceed them. Anything with an external effect, a payment, a message, a schema change, passes a human gate. A prompt is a suggestion; a server-side check is a rule.
Next capability
LLM evaluation harnesses
Bring us the hard part.
Forty-five minutes with the people who would actually run the build.