Skip to main content

Using Coding Agents with the AI Gateway

This guide explains how to route Claude Code, Codex, and custom coding agents through the AI Gateway, and how to govern the traffic they send.

A coding agent sends LLM requests on a developer's behalf, including tool calls, sub-agent requests, and large tool-definition payloads that a typical chat client doesn't produce. Routing that traffic through the AI Gateway applies the same model access rules, guards, and cost controls as any other AI Gateway route. Each agent has its own client-side behavior, including environment variables, header handling, and traffic the gateway can't inspect, that changes how those controls apply. This guide documents the setup, behavior, and known issues for each agent.

Each agent section has a Configure part and a Govern part, written for two roles:

  • API consumer: connects the agent to the gateway. Follow the Configure sections.
  • API publisher: controls what the agent can do through the gateway, including models, parameters, guards, and spend. Follow the Govern sections.

Overview​

The following table lists the supported agents and the AI Gateway middleware that serves each one.

AgentModelsAPIHub middlewareAuth
Claude CodeClaude models onlyAnthropic Messages API (also Bedrock, Vertex AI)Messages API / Bedrock Mantle / Google Agent PlatformANTHROPIC_AUTH_TOKEN (Bearer) or ANTHROPIC_API_KEY (x-api-key)
CodexOpenAI modelsOpenAI Responses APIResponses APIenv_key in config.toml (for example HUB_API_KEY)
Custom agents (LangChain, raw Anthropic Client SDK)Depends on the SDKMessages or Responses API, depending on the SDKMessages API or Responses API middleware, matching the SDKSDK-native

For other providers, such as Azure OpenAI, Mistral, or Gemini, use the Chat Completion middleware instead. Anthropic doesn't support routing Claude Code to non-Claude models through a gateway.

Claude Code​

Configure Claude Code​

Set the gateway as the Claude Code base URL and set the credential that Claude Code sends. Export the variables in the shell:

export ANTHROPIC_BASE_URL=https://<your-hub-ai-gateway-host>  # Replace with your gateway host
export ANTHROPIC_AUTH_TOKEN=<credential>  # Replace with the credential for your route

To persist the settings, add them to ~/.claude/settings.json instead:

{
  "env": {
    "ANTHROPIC_BASE_URL": "https://<your-hub-ai-gateway-host>",
    "ANTHROPIC_AUTH_TOKEN": "<credential>"
  }
}

Set ANTHROPIC_AUTH_TOKEN to the credential that matches how the route authenticates requests. The following options are listed from most to least preferred.

  • Authenticate at the gateway. Add an authentication middleware, such as API key or JWT, before the Messages API middleware on the route. Set ANTHROPIC_AUTH_TOKEN to a credential that the authentication middleware accepts. Claude Code sends the credential as Authorization: Bearer <credential>, and the gateway returns 401 for a missing or invalid credential. The Messages API middleware adds the provider key it holds to the upstream request. For the API key middleware, read the credential from the Authorization header with the Bearer scheme:

    spec:
      plugin:
        apiKey:
          keySource:
            header: Authorization  # Header that Claude Code sends
            headerAuthScheme: Bearer
          secretNonBase64Encoded: true
          secretValues:
            - urn:k8s:secret:gateway-keys:secret  # Replace with the secret that holds the hashed credentials

  • Forward the developer's provider key. Leave apiKey and authToken unset on the Messages API middleware, and set ANTHROPIC_AUTH_TOKEN to the developer's own provider key. The gateway forwards the key to the provider, and the provider authenticates the request. See Forward each client's own provider credential.

  • Skip authentication on a trusted network. On a route without an authentication middleware, the gateway doesn't check the credential. Set ANTHROPIC_AUTH_TOKEN to any placeholder value, for example unused. The route then uses the provider key configured on the Messages API middleware, and any client that can reach the route can use that key. Use this option only for a local test or another network you trust.

Clear ANTHROPIC_API_KEY

Claude Code gives ANTHROPIC_API_KEY precedence over ANTHROPIC_AUTH_TOKEN once the key is approved, so a leftover Anthropic key in a developer's shell replaces the credential you set. Requests still go to ANTHROPIC_BASE_URL, but with the wrong credential. When you set ANTHROPIC_AUTH_TOKEN, also run export ANTHROPIC_API_KEY="". In interactive mode, Claude Code asks you to approve the key before it takes over. See Claude Code's gateway documentation.

To verify the connection, run claude -p "Reply with OK". A reply confirms that the request reached the provider through the gateway. Next, apply access rules and guards as described in Govern Claude Code traffic.

Configure Claude Code for Amazon Bedrock​

For a route that forwards to Claude on Amazon Bedrock through the Bedrock Mantle middleware, use Claude Code's Mantle mode. Mantle mode sends only the beta headers that Bedrock accepts, and skips client-side AWS signing because the gateway signs the request. Export the variables:

export CLAUDE_CODE_USE_MANTLE=1
export CLAUDE_CODE_SKIP_MANTLE_AUTH=1
export AWS_REGION=<region>  # Replace with the AWS region of the route
export ANTHROPIC_BEDROCK_MANTLE_BASE_URL=https://<your-hub-ai-gateway-host>/anthropic  # Replace with your gateway host

End the base URL with /anthropic, because the Bedrock Mantle middleware serves the Messages API at /anthropic/v1/messages. Run claude -p "Reply with OK" to verify the connection, then continue with Govern Claude Code traffic. If Claude Code returns an error, see Troubleshooting Claude Code.

Configure Claude Code for Google Agent Platform​

For a route that uses the Google Agent Platform middleware, use Claude Code's Vertex mode. Vertex mode sends Google's request format, with the model in the URL path, and skips client-side Google authentication because the gateway adds the credential. Export the variables:

export CLAUDE_CODE_USE_VERTEX=1
export CLAUDE_CODE_SKIP_VERTEX_AUTH=1
export CLOUD_ML_REGION=<region>  # Replace with the Google Cloud region of the route
export ANTHROPIC_VERTEX_PROJECT_ID=<project-id>  # Replace with your Google Cloud project identifier
export ANTHROPIC_VERTEX_BASE_URL=https://<your-hub-ai-gateway-host>/v1  # Replace with your gateway host

End the base URL with /v1. Plain Messages mode (ANTHROPIC_BASE_URL) doesn't work on this route, because the request path carries no model and the middleware returns 400. See Model handling. Run claude -p "Reply with OK" to verify the connection, then continue with Govern Claude Code traffic.

Govern Claude Code traffic​

Attach Messages API (or Bedrock Mantle / Google Agent Platform for those upstreams) to the route that serves Claude Code, then add guard, cache, and rate-limit middlewares. These middlewares detect the Messages API format automatically. See Using AI Middlewares with Messages API.

Then apply the controls that your policy requires.

Forward each client's own provider credential. Leave apiKey and authToken unset on the Messages API middleware:

spec:
  plugin:
    messages-api:
      allowModelOverride: true  # No apiKey or authToken, so the client's credential is forwarded
      allowParamsOverride: true

Each developer or team then uses its own provider API key, which keeps usage and billing separate at the provider. The gateway still applies guards, limits, and observability to every request. The provider validates the forwarded credential, not the gateway. To use one provider key held by the gateway instead, set apiKey or authToken on the middleware, which then sets that header on every request.

Pin a model. Set model on the Messages API middleware and turn off the overrides:

spec:
  plugin:
    messages-api:
      apiKey: urn:k8s:secret:ai-keys:anthropic-api-key  # Replace with the secret that holds the provider key
      model: claude-haiku-4-5-20251001  # Model that every request uses
      allowModelOverride: false
      allowParamsOverride: false
      allowInlineSystemMessages: false
      params:
        maxTokens: 32000  # Replace with a value the pinned model supports

Every request then uses the pinned model, whatever model the developer selects in Claude Code. Set params.maxTokens to a value the pinned model supports, and keep allowInlineSystemMessages: false, because some models reject the max_tokens value and the inline system messages that Claude Code sends by default. See Troubleshooting Claude Code.

Group a session in traces. Set traces.conversationIdHeader to X-Claude-Code-Session-Id:

spec:
  plugin:
    messages-api:
      observability:
        traces:
          conversationIdHeader: X-Claude-Code-Session-Id  # Header that Claude Code sends on every request

The traces then group all requests of one Claude Code session under one conversation. Requests from a sub-agent carry the x-claude-code-agent-id header, which the gateway forwards but doesn't read, so you can't separate sub-agent traffic in traces. See Group requests from the same conversation.

Block sensitive content with request rules. Define the guard rules in request, not response:

spec:
  plugin:
    content-guard:
      engine:
        regex: {}
      request:
        rules:
          - reason: api_key_detected
            block: true
            entities:
              - 'sk-ant-[A-Za-z0-9_-]+'  # Block prompts that contain an Anthropic-style key

A response rule makes the gateway buffer the whole completion, so Claude Code receives a long response only after it is complete. Request rules don't delay the response.

After you apply the middlewares, run claude -p "Reply with OK" against the route to confirm that requests still succeed. If Claude Code returns an error, see Troubleshooting Claude Code.

Troubleshooting Claude Code​

Unexpected value for the anthropic-beta header

Error: 400 with Unexpected value(s) ... for the anthropic-beta header from Amazon Bedrock or Google Agent Platform.

Cause: In plain Messages mode (ANTHROPIC_BASE_URL), Claude Code sends beta feature headers that these upstreams don't accept. The error message lists the rejected values. The gateway forwards the headers unchanged.

Solution: On Amazon Bedrock, use Mantle mode as described in Configure Claude Code for Amazon Bedrock, or set CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS=1 on the client. On Google Agent Platform, use Vertex mode as described in Configure Claude Code for Google Agent Platform.

Claude Code returns a model error with a 404

Error: There's an issue with the selected model, with a 404 from the gateway.

Cause: The base URL doesn't end with the path the route matches, so Claude Code calls a path the route doesn't serve. Claude Code reports this error as a model error.

Solution: End ANTHROPIC_BEDROCK_MANTLE_BASE_URL with /anthropic for Bedrock Mantle, and ANTHROPIC_VERTEX_BASE_URL with /v1 for Google Agent Platform.

To publish the route under a different path, rewrite the path on the gateway instead. Add a path middleware such as AddPrefix or StripPrefix before the AI middleware on the route. For example, stripPrefix with /claude lets clients use https://<your-hub-ai-gateway-host>/claude as the base URL on a Messages API route.

max_tokens is larger than the model's maximum

Error: 400 with max_tokens: 128000 > 64000, which is the maximum allowed number of output tokens for the pinned model.

Cause: Claude Code requests a max_tokens value that is higher than the pinned model supports.

Solution: Set allowParamsOverride: false and a params.maxTokens that the pinned model supports. See Parameter override behavior.

role 'system' is not supported on this model

Error: 400 with role 'system' is not supported on this model.

Cause: Claude Code sends system instructions as entries inside messages, and some models reject them.

Solution: Set allowInlineSystemMessages: false on the Messages API middleware. See allowInlineSystemMessages.

Responses arrive all at once or time out

Problem: A long response reaches Claude Code only after it is complete, and a long generation can exceed Claude Code's stream timeout.

Cause: A Content Guard or LLM Guard response rule buffers the whole completion and returns it as a single chunk. Claude Code aborts a stream that stays silent for longer than its timeout, as described in its gateway protocol reference.

Solution: Use request-only rules instead of response rules on routes that serve Claude Code. See Content Guard streaming.

The model picker doesn't list gateway models

Problem: The /model picker in Claude Code doesn't list the models behind the gateway.

Cause: The Messages API route doesn't implement GET /v1/models, which Claude Code uses for model discovery when CLAUDE_CODE_ENABLE_GATEWAY_MODEL_DISCOVERY is set.

Solution: Set the model with the ANTHROPIC_MODEL environment variable or the --model option.

413 Request Entity Too Large

Error: 413 Request Entity Too Large from the gateway.

Cause: The request body is larger than the gateway limit, 1 MiB by default. Claude Code sends large request bodies, and sessions with many MCP servers or large tool definitions send more.

Solution: Raise hub.aigateway.maxRequestBodySize. See Request body size limits.

Fast mode reports a connectivity error

Problem: Fast mode reports a connectivity error while normal requests through the gateway succeed.

Cause: Fast mode and the WebFetch domain-safety check call api.anthropic.com directly, regardless of ANTHROPIC_BASE_URL. On a network that blocks direct egress to api.anthropic.com, these calls fail.

Solution: Allow egress to api.anthropic.com from the developer's machine. Requests that go through the gateway don't need it.

Codex​

Configure Codex​

Add the gateway as a model provider in ~/.codex/config.toml, and select it:

model_provider = "hub"

[model_providers.hub]
name = "Hub AI Gateway"
base_url = "https://<your-hub-ai-gateway-host>/v1"  # Replace with your gateway host
env_key = "HUB_API_KEY"
wire_api = "responses"

Then export the credential Codex reads from env_key:

export HUB_API_KEY=<credential>  # Replace with the credential for your route

End base_url with /v1, because Codex adds /responses to it.

Set HUB_API_KEY to the credential that matches how the route authenticates requests. The following options are listed from most to least preferred.

  • Authenticate at the gateway. Add an authentication middleware, such as API key or JWT, before the Responses API middleware on the route. Set HUB_API_KEY to a credential that the authentication middleware accepts. Configure the API key middleware to read the credential from the Authorization header with the Bearer scheme, as in Configure Claude Code. The Responses API middleware adds the token it holds to the upstream request.

  • Forward the developer's provider key. Leave token unset on the Responses API middleware, and set HUB_API_KEY to the developer's own provider key. The gateway forwards the key to the provider, and the provider authenticates the request.

  • Skip authentication on a trusted network. On a route without an authentication middleware, the gateway doesn't check the credential. Set HUB_API_KEY to any placeholder value. The route then uses the token configured on the Responses API middleware, and any client that can reach the route can use that token. Use this option only for a local test or another network you trust.

To verify the connection, run codex exec "Reply with OK". A reply confirms that the request reached the provider through the gateway. Next, apply access rules and guards as described in Govern Codex traffic.

Govern Codex traffic​

Attach the Responses API middleware to the route that serves Codex, then add guard, cache, and rate-limit middlewares. See Using AI Middlewares with Responses API.

Then apply the controls that your policy requires.

Forward each client's own provider credential. Leave token unset on the Responses API middleware. Each developer or team then uses its own provider API key, which keeps usage and billing separate. See Forward each client's own provider credential for the example and details.

Hide models from the model list. Add listPolicies to the Responses API middleware:

spec:
  plugin:
    responses-api:
      allowModelOverride: true
      listPolicies:
        - match: Prefix(`res.model.id`, `o1-`)  # Replace with the models to hide
          action: hide

Without a rule, GET /v1/models returns the provider's full model list for the configured key, so Codex shows models that your policy blocks. See Restrict and rename AI models by identity for per-identity rules.

After you apply the middlewares, run codex exec "Reply with OK" to confirm that requests still succeed. If Codex returns an error, see Troubleshooting Codex.

Codex sub-agent traffic​

The gateway can't inspect or govern Codex sub-agent traffic. When Codex spawns a sub-agent, the parent-to-child messages carry content.type: encrypted_content, and there's currently no way to disable this encryption on Codex's side. Hub passes this traffic through unmodified, because blocking it breaks sub-agents.

Guards and limits apply to the messages a user sends and receives through Codex, but not to the messages between a parent agent and its sub-agents. If your policy requires inspecting all agent traffic, account for this gap when you decide which Codex features developers can use.

Troubleshooting Codex​

Codex retries and fails to connect

Error: Reconnecting... 5/5, followed by an error, when Codex starts a request.

Cause: base_url doesn't end with /v1, so the request reaches the gateway as /responses.

Solution: End base_url in ~/.codex/config.toml with /v1. Alternatively, add an AddPrefix middleware with /v1 before the Responses API middleware, so the gateway adds the prefix.

Codex retries a blocked request and shows only a 403

Error: unexpected status 403 Forbidden, after Codex retries the request five times.

Cause: A guard denies the request with its default 403 status, and Codex treats a non-2xx status as a connection error instead of reading the deny message.

Solution: Set onDenyResponse on the guard to statusCode: 200 with a message. Codex then displays the message immediately. See Content Guard deny response configuration.

A Regex Guard blocks requests that don't contain the sensitive term

Problem: A guard blocks Codex requests that contain none of the terms in its rules.

Cause: An unanchored pattern such as (credit|card) also matches inside longer words such as wildcard. Codex requests include long instructions and tool definitions, which makes such a match more likely.

Solution: Anchor the pattern with word boundaries, for example (?i)\b(credit|card)\b. See Content Guard: Regex Engine.

net/http: abort Handler in debug logs

Problem: Debug logs contain net/http: abort Handler for a Codex request.

Cause: The streaming client disconnects right after the terminal event. The log line is harmless.

Solution: No action is needed. See Responses API Troubleshooting.

Custom agents (LangChain, Anthropic Client SDK)​

Custom agents call the Messages API or Responses API directly through their provider SDK, without the Claude Code or Codex CLI. Set the SDK's base_url (or api_base) to the gateway, then apply the Govern controls for the API that the SDK uses: Govern Claude Code traffic for the Messages API, and Govern Codex traffic for the Responses API.

For the LangChain Anthropic integration, set anthropic_api_url. The Anthropic SDK adds /v1/messages to the URL, so the value doesn't end with /v1:

from langchain_anthropic import ChatAnthropic

llm = ChatAnthropic(
    model="claude-sonnet-4-6",
    anthropic_api_url="https://<your-hub-ai-gateway-host>",  # Replace with your gateway host
    anthropic_api_key="<gateway-credential>",  # Replace with the credential for your route
)

Basic calls, streaming, and tool calls work through a Messages API route without extra settings. The gateway rewrites the model if the middleware pins one, and the credential rules for Claude Code apply. LangChain requests a small max_tokens and sends no inline system messages, so the max_tokens and role 'system' errors in Troubleshooting Claude Code don't occur.

For agents that use the Responses API, end base_url with /v1 and set use_responses_api=True in langchain-openai:

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="gpt-5-mini",
    base_url="https://<your-hub-ai-gateway-host>/v1",  # Replace with your gateway host
    api_key="<gateway-credential>",  # Replace with the credential for your route
    use_responses_api=True,
)

Basic calls, streaming, and tool calls work against a Responses API route.

Related content​