Skip to main content

Google Agent Platform

Early Access

This feature is currently in early access.

The Google Agent Platform middleware promotes any route to an Anthropic Messages API endpoint backed by Google Agent Platform's rawPredict API (formerly named Google Vertex AI). It builds on the Messages API middleware: requests use the same Anthropic Messages API payload format, and the middleware applies the same model and parameter governance, GenAI metrics, prompt caching, and system prompt controls. On top of that, it adds Google Cloud authentication so you can govern Claude models served through Google Agent Platform, useful for Claude Code and other coding agents, or custom agents built with frameworks such as LangChain.

Key Features and Benefits​

  • Google Cloud authentication: authenticate with a static bearer token, a Google service account key, or the default Google Cloud credential chain (for example, Workload Identity).
  • Credential type allow-list: restrict which Google credential types the middleware accepts when reading a service account key.
  • Token caching: cache OAuth access tokens in Redis so the middleware doesn't request a new token from Google on every request.
  • Messages API governance: lock or allow client overrides for model, temperature, topP, topK, and more, exactly as with the Messages API middleware.
  • Metrics: emits OpenTelemetry GenAI spans and counters, tagged with the gcp.vertex_ai provider.
  • Transparent to clients: works with the standard Anthropic SDKs pointed at the Google Agent Platform endpoint, so coding agents such as Claude Code can target Google Agent Platform through the gateway.

Requirements​

  • You must have AI Gateway enabled:

    helm upgrade traefik traefik/traefik -n traefik --wait \
      --reset-then-reuse-values \
      --set hub.aigateway.enabled=true

  • Define a Kubernetes ExternalName service pointing at the regional Google Agent Platform host for your project (<region>-aiplatform.googleapis.com). The Kubernetes CRD provider must allow ExternalName services (providers.kubernetesCRD.allowExternalNameServices=true).

  • Google Cloud credentials (a bearer token or a service account key) authorized to call the rawPredict API for the target Google Cloud project and model.

How It Works​

  1. Intercepts the request and applies Google Cloud authentication (see Authentication).
  2. Extracts the model from the request path instead of the request body. Google Agent Platform's Anthropic-compatible endpoint carries the model as a path segment, not a JSON field. See Model Handling.
  3. Applies governance by rewriting the model path segment or param fields when overrides are denied, identical to the Messages API middleware.
  4. Marks the request format as messagesAPI in the request context so other AI middlewares in the chain (Content Guard, LLM Guard, Semantic Cache, Token Rate Limit) automatically adapt to the Anthropic payload format.
  5. Forwards the request to the upstream Google Agent Platform host and records usage metrics from the response, tagged with the gcp.vertex_ai provider.
Provider name predates the rebrand

Google renamed Vertex AI to Google Agent Platform. The OpenTelemetry GenAI semantic convention attribute has not been updated to match, so metrics and traces still report gen_ai.provider.name = gcp.vertex_ai.

Authentication​

Google Agent Platform supports several authentication modes. authToken and credentials are mutually exclusive.

Omit authToken and credentials. The middleware resolves credentials from the default Google Cloud credential chain (for example, GKE Workload Identity or the metadata server), scoped to https://www.googleapis.com/auth/cloud-platform.

spec:
  plugin:
    google-agent-platform:
      anthropic:
        model: claude-opus-4-5

Service account key​

Provide credentials, either a URN referencing a Kubernetes Secret that holds the full service account JSON key, or a path to a key file mounted in the pod.

spec:
  plugin:
    google-agent-platform:
      credentials: urn:k8s:secret:gcp-creds:service-account.json
      anthropic:
        model: claude-opus-4-5

By default, the middleware only accepts service_account keys. Use allowedCredentialTypes to accept other Google credential types (for example, external_account for workload identity federation):

spec:
  plugin:
    google-agent-platform:
      credentials: urn:k8s:secret:gcp-creds:federation-config.json
      allowedCredentialTypes:
        - external_account
      anthropic:
        model: claude-opus-4-5

Static bearer token​

Provide a static bearer token with authToken. The middleware sets it as the Authorization: Bearer header on every request without contacting Google for a token.

spec:
  plugin:
    google-agent-platform:
      authToken: urn:k8s:secret:gcp-creds:bearer-token
      anthropic:
        model: claude-opus-4-5

Pass through the client's own authorization​

If the incoming request already carries an Authorization header and neither authToken nor credentials is configured, the middleware forwards it unchanged instead of resolving Google credentials itself. Use this when clients (for example, a Claude Code CLI configured with its own Vertex AI credentials) should authenticate directly with Google, and the gateway only governs and observes the traffic.

Caching Access Tokens in Redis​

By default, the middleware requests a new OAuth access token from Google on every request that uses credentials. Configure store to cache tokens in Redis, refreshed 30 seconds before they actually expire, reducing latency and the number of calls made to Google.

spec:
  plugin:
    google-agent-platform:
      credentials: urn:k8s:secret:gcp-creds:service-account.json
      store:
        keyPrefix: gap-tokens
        secret: urn:k8s:secret:gcp-creds:cache-encryption-key
        redis:
          endpoints:
            - redis.traefik-hub.svc.cluster.local:6379
      anthropic:
        model: claude-opus-4-5
store.secret length

secret must be 16, 24, or 32 bytes long (an AES-128, AES-192, or AES-256 key). The middleware fails to start otherwise.

Token caching only applies when credentials is set. It has no effect with authToken or with a passed-through client authorization, since neither of those involve the middleware fetching a token from Google.

Configuration Example​

apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: google-agent-platform
spec:
  plugin:
    google-agent-platform:
      credentials: urn:k8s:secret:gcp-creds:service-account.json
      store:
        keyPrefix: gap-tokens
        secret: urn:k8s:secret:gcp-creds:cache-encryption-key
        redis:
          endpoints:
            - redis.traefik-hub.svc.cluster.local:6379
      anthropic:
        model: claude-opus-4-5
        allowModelOverride: false
        allowParamsOverride: true
        params:
          temperature: 1
          maxTokens: 1024
Client base URL

Point your Anthropic client at the gateway, then use Google Agent Platform's path format so the middleware can extract the model: /v1/projects/<project>/locations/<region>/publishers/anthropic/models/<model>:rawPredict (or :streamRawPredict for streaming). The region in the path and in the ExternalName host must match.

Configuration Options​

FieldDescriptionRequiredDefault
authTokenURN of a Kubernetes Secret holding a static bearer token. Sets the Authorization: Bearer header on every request. Mutually exclusive with credentialsNo
credentialsURN of a Kubernetes Secret holding a Google service account JSON key, or a path to a key file mounted in the pod. Mutually exclusive with authTokenNo
allowedCredentialTypesGoogle credential types accepted when reading credentials (for example, service_account, external_account)No[service_account]
store.keyPrefixPrefix for the Redis keys used to cache access tokensNo
store.secretURN of a Kubernetes Secret holding the AES key used to encrypt cached tokens. Must be 16, 24, or 32 bytesConditional
store.redis.endpointsList of Redis endpoints (for example, redis:6379)Conditional
store.redis.usernameRedis username for authenticationNo
store.redis.passwordRedis password. Supports URN references (for example, urn:k8s:secret:redis:password)No
store.redis.databaseRedis database numberNo0
store.redis.tlsTLS configuration for secure Redis connectionsNo
store.redis.timeoutConnection timeout (for example, 5s)No
store.redis.clusterEnable Redis Cluster mode. Set to {} to enableNo
store.redis.sentinelRedis Sentinel configuration (masterSet, username, password)No
anthropicEmbedded Messages API configuration. See Nest governance fields under anthropic belowNo

store is required only if you configure token caching; see Caching Access Tokens in Redis.

Nest governance fields under anthropic​

Nest governance fields under anthropic

Model and parameter governance fields belong inside the anthropic block. Google Cloud authentication fields (authToken, credentials, allowedCredentialTypes, store) stay at the top level. anthropic.apiKey is ignored: authentication to Google Agent Platform always goes through authToken or credentials, never through the Anthropic X-Api-Key header.

The anthropic block accepts the same fields as the Messages API middleware:

  • model, allowModelOverride, allowParamsOverride
  • params.temperature, params.topP, params.topK, params.maxTokens, params.stopSequences, params.serviceTier, params.cacheControl
  • systemPrompt, allowInlineSystemMessages
  • metrics.addLabels.appId, metrics.addLabels.appName, metrics.addLabels.userId

Model Handling​

Unlike the Messages API and Bedrock Mantle middlewares, Google Agent Platform's endpoint carries the model in the URL path, not the request body:

POST /v1/projects/{PROJECT_ID}/locations/{LOCATION}/publishers/anthropic/models/{MODEL_ID}:rawPredict
POST /v1/projects/{PROJECT_ID}/locations/{LOCATION}/publishers/anthropic/models/{MODEL_ID}:streamRawPredict

The middleware always removes any model field from the request body before forwarding, and enforces governance against the path segment instead:

  • anthropic.allowModelOverride: true (default when anthropic.model is empty): the request path is forwarded unmodified. The client's chosen model in the path is used.
  • anthropic.allowModelOverride: false with anthropic.model set: the middleware rewrites the model segment of the request path to anthropic.model before forwarding, so requests reach the configured model even when the client's path names a different one.
  • If the request path has no model segment, the middleware returns 400 Bad Request.

If you omit the anthropic block entirely, the middleware skips governance altogether: it only injects Google Cloud authentication and forwards the request as-is.

Hiding the Upstream Path​

Google Agent Platform's endpoint requires the full /v1/projects/<project>/locations/<region>/publishers/anthropic/models/<model>:rawPredict path, which exposes your Google Cloud project and region to clients. Use a ReplacePathRegex middleware to expose a shorter public path instead, and let it rewrite the request to the full path before google-agent-platform sees it.

Rewrite a short public path to the full Google Agent Platform endpoint
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: gap-path-rewrite
spec:
  replacePathRegex:
    regex: ^/agent-platform/(.*)$
    replacement: /v1/projects/my-project/locations/us-east5/publishers/anthropic/models/$1:rawPredict
Middleware Order Matters

Place replacePathRegex (or any other path-rewrite middleware) before google-agent-platform in the chain. Google Agent Platform extracts the model from the request path it receives, so it only sees the rewritten path if the rewrite middleware already ran ahead of it. If the rewrite runs after it instead, Google Agent Platform reads the original client-facing path, which doesn't include the real project, location, and model segments it needs.

Metrics​

The middleware emits OpenTelemetry GenAI metrics when metrics are enabled. Spans and counters are tagged with gen_ai.provider.name = gcp.vertex_ai. In addition to standard input/output token counts, Anthropic-specific usage fields cache_creation_input_tokens and cache_read_input_tokens are recorded when prompt caching is active.

Prompt Caching, Streaming, and Body Size​

Google Agent Platform inherits the prompt caching, streaming, and request body size behavior of the Messages API middleware:

  • Prompt caching: configure anthropic.params.cacheControl.ttl (5m or 1h) to attach an Anthropic ephemeral cache_control marker to the last cacheable request block.
  • Streaming: use the :streamRawPredict path suffix instead of :rawPredict for streaming responses.
  • Request body size: governed by the global hub.aigateway.maxRequestBodySize Helm flag, shared by all AI Gateway middlewares. See Request-Body Limits & Model Matcher Costs.

Working with Other AI Middlewares​

When google-agent-platform is in the middleware chain, the AI Gateway propagates the Anthropic Messages API format to the other AI middlewares in the chain. Content Guard, LLM Guard, Semantic Cache, and Token Rate Limit detect the format and adapt their behavior without additional configuration.

Middleware Order Matters

Place the google-agent-platform middleware first in the chain among the AI kind. It sets the request format in the context, which the other AI middlewares rely on to detect the Anthropic format. Guards placed before it cannot detect the format.

For detailed examples of combining these middlewares, see the Using AI Middlewares with Messages API guide.

Troubleshooting​

authToken and credentialsPath can't be both defined

Problem: The middleware fails to start with authToken and credentialsPath can't be both defined.

Solution: Configure either authToken for a static bearer token or credentials for a service account key, but not both.

store.secret must be 16, 24, or 32 bytes

Problem: The middleware fails to start with store.secret must be 16, 24, or 32 bytes, got N.

Solution: Generate an AES key of exactly 16, 24, or 32 bytes and store it as the store.secret value.

<credential type> credential type not allowed

Problem: The middleware rejects a service account key with <type> credential type not allowed.

Solution: Add the credential type to allowedCredentialTypes, or provide a service_account key (the default accepted type).

400 Bad Request with no model in the path

Problem: Requests fail with 400 Bad Request and the logs show no model in request path.

Solution: Ensure the route's path matches Google Agent Platform's endpoint format, ending in /models/<model>:rawPredict or /models/<model>:streamRawPredict.

401 or 403 responses from Google

Problem: Receiving 401 Unauthorized or 403 Forbidden responses from Google Agent Platform.

Solution: Verify the service account (or the identity behind the default credential chain) has the IAM permissions to call aiplatform.googleapis.com for the target project, and that the region in the request path matches the ExternalName host's region.

Related Content​