AWS has published a Pluggable Agentic AI Framework, a seven-layer composable reference architecture for running multi-agent applications in production on Amazon Bedrock AgentCore. Each layer carries a distinct responsibility and a stable contract, while the listed services describe a reference implementation that companies may replace. The move addresses the gap between prototypes and systems that operate reliably, securely and cost-efficiently at scale.
How AWS structures production agent systems
The foundation is Data and Knowledge, which owns retrieval, embeddings and access to structured and unstructured data. It answers a query with the most relevant context for an agent, whether for a healthcare formulary, inventory or compliance rules. Example building blocks named in the framework include Amazon Bedrock Knowledge Base, Vector Engine for Amazon OpenSearch Serverless, Amazon S3 Vectors and Amazon DynamoDB.
Model and Inference are driven by configuration rather than hardwired logic, with smaller models for simple tasks and larger models for nuanced reasoning. A routing layer decouples agents from hosting environments and provider APIs through a consistent request-and-response contract. A proxy such as LiteLLM provides a common interface across providers, allowing teams to switch models after evaluation without rewriting application logic.
Orchestration separates straightforward work from complex cross-domain workflows. Single agents handle end-to-end tasks, while a supervisor delegates to specialists scoped to specific capabilities and tool sets. The framework warns against one agent doing everything, since growing context, latency and tool sprawl can reduce accuracy and create security breaches. Specialists are discovered through a registry and invoked via MCP and A2A, with Strands Agents, AgentCore harness, AWS Agent Registry and AgentCore Gateway cited as options.
What this means for operating agents at scale
For operating companies, the design reduces repeated scaffolding around routing, guardrails and observability by giving every agent a shared production base. Security and guardrails are built in from day one, covering prompt injection, tool misuse and memory poisoning. Controls include Amazon Bedrock Guardrails, AgentCore Identity and Policy, authentication of every request, scoped temporary credentials, masking of PHI and PII, audit trails, human approval gates and circuit breakers.
The application layer stays intentionally thin, presenting chat, API or embedded interfaces and passing requests to orchestration without business logic. A decision model at this boundary can score whether a request is in scope, needs a human or is urgent, returning probabilities for threshold-based handling. Observability then tracks logs, metrics, end-to-end traces, cost and token usage, time to first token, latency, error rates and spend, with Amazon CloudWatch, OpenTelemetry and CloudWatch Omni named as tools.
Agentic Operations will matter if agent quality drifts after release, since every production session is scored for groundedness, accuracy and safety with regression gates. AWS points to AgentCore Evaluations for scoring and AgentCore optimization for tuning, illustrated by a retail support implementation for order status and refund eligibility. A marker to watch is whether teams adopt registries, evaluations and guardrails as standard parts of deployment rather than add-ons after incidents.
