A self-service platform for internal AI agents
How developers ship internal AI agents from a template, inside approved models, a hard spend cap and encrypted audit logs, without touching the infrastructure.
Once people see what an AI agent can do, the requests arrive quickly. Can we have an assistant that answers questions from the wiki? Can something triage our alarms? Each request is reasonable. Each one also raises the same questions: which model, what data does it see, who pays, and what happens if it goes wrong?
Approving each agent one at a time doesn’t scale, and saying no just pushes people towards tools you can’t see. So at a non-profit organisation I built a self-service platform for internal AI agents on AWS. Developers start from a template repository and deploy inside spend caps and guardrails without touching the infrastructure.
What a developer gets
The template repository holds three things: the agent’s code, the infrastructure it runs on, and the pipeline that deploys it. A developer writes the agent’s logic and opens a pull request. The pipeline does the rest.
What they don’t have to think about is the important part. Every agent built from the template inherits the same guardrails:
- approved models only;
- a hard monthly cost cap;
- encrypted audit logs of every model call;
- a role that can only reach what that agent needs.
This is the same idea as any golden path: make the safe way the easy way, and most people will take it.
The guardrails
Approved models only
Agents use models from the Amazon Bedrock model catalog, so there are no separate AI vendor accounts or API keys to manage: access, billing and logging all sit in AWS. The platform approves a short list from that catalog. The agent’s role is only allowed to invoke the models on the list, so using an unapproved model fails at the AWS API, not in a code review:
{
"Effect": "Allow",
"Action": ["bedrock:InvokeModel", "bedrock:InvokeModelWithResponseStream"],
"Resource": "arn:aws:bedrock:ap-southeast-2::foundation-model/<approved-model-id>"
}The approved list lives in one shared variable that every agent’s infrastructure reads, so the policies are generated from it rather than written by hand. Adding a model is a single pull request to that variable, reviewed like any other change, and every agent picks it up on its next deploy. If agents use cross-Region inference profiles, the policy has to allow the inference profile as well as the models behind it.
A hard spend cap
Every agent has a monthly budget. AWS Budgets can run an action when a budget crosses a threshold, including applying an IAM policy or a service control policy, either automatically or after approval. Applying a policy that denies model calls turns a budget from an alert into a cap.
AWS Budgets works from billing data, which isn’t real time, so a budget action stops spend shortly after the threshold rather than at the exact cent. A custom Lambda function adds a second check: it runs every hour and watches each agent’s spend against its cap, so there are two independent ways for a runaway agent to be caught.
The cap is what makes saying yes easy. A new agent’s worst case is known before it ships.
Encrypted audit logs
Bedrock’s model invocation logging records the request, the response and metadata for every call, including the identity of the role that made it. It’s off by default. On this platform it’s switched on for the account and delivers to CloudWatch Logs, in a log group encrypted with a KMS key.
Because each agent has its own role, the logs answer “which agent sent what, and when” without any extra instrumentation. A CloudWatch Logs Insights query grouped by the caller’s identity shows usage per agent in seconds. That’s what turns an experiment into something you can explain to an auditor.
Least privilege
Each agent’s role can read only the data sources that agent needs. The wiki assistant can read the wiki; it can’t read customer data. If an agent misbehaves, its role limits how far the damage can reach.
Two agents on the platform
The wiki assistant
The internal wiki assistant runs on Amazon Bedrock with approved models only, a hard monthly cost cap and encrypted audit logs. It answers questions from the company wiki, so people find the right page without knowing where it lives.
The SRE bot
The SRE bot reads CloudWatch alarms and posts a “likely real” or “likely noise” call in Slack. Alarms are noisy, and the cost of noise is that people stop looking. A call with its reasons gives the on-call engineer a starting point instead of a bare alert.
The bot advises; people decide. For now it’s read-only: it doesn’t restart services or change anything. That made it easy to trust from day one, and easy to approve.
Read-only is a starting point, not a limit. Once the bot has shown over time that its calls hold up, the plan is to connect it with other bots so it can open Jira tickets or propose code changes. Each new capability will be earned the same way: proven first, then trusted with more.
The desktop side
Not every use of AI is an agent someone builds. The company-wide Claude Desktop rollout went through the managed Mac fleet, so people get an approved tool through the same device management as everything else, instead of signing up for whatever they find.
What I learned
- Guardrails belong in the template. If every team has to add them, some won’t.
- A spend cap makes yes easy. The question changes from “what could this cost?” to “is it useful?”
- Logs build trust. Being able to show exactly what an agent sent and received answers most security questions before they’re asked.
- Start with read-only, internal use cases. A wiki assistant and an alarm triage bot deliver value without being able to break anything.
- Advise before you automate. Let people see the agent’s judgement for a while before giving it the power to act, then add capabilities one at a time.
The platform isn’t really about AI. It’s the same pattern as any self-service platform: a template, a pipeline, and guardrails that come with them, so building the safe thing is also the fastest way to build anything.