Content Guard

The Content Guard middleware detects and processes sensitive or restricted content in requests and responses. In the AI Gateway, it lets you:

1. Block disallowed content (PII, secrets, policy-violating terms) in either direction.
2. Mask sensitive fragments, keeping UX intact while remaining compliant.

Two flavours exist:

Variant	Best for	Stream support	Notes
content-guard	Generic JSON / REST payloads	No	You choose the JSON paths.
chat-completion-content-guard	OpenAI-compatible chat traffic	Yes	Auto-detects chat schema; jsonPaths hidden.

Key Features and Benefits

• Prevent data leakage before prompts reach an LLM—or before completions reach users.
• Mask or block based on business policy.
• Custom entities through the detection engine (Presidio).

Requirements

• AI Gateway must be enabled:

  helm install traefik -n traefik --wait \
    --set hub.aigateway.enabled=true \
  traefik/traefik

• An external content detection engine is required. Currently, we support presidio as the content detection engine. Follow the presidio documentation to install and configure presidio as a service with Kubernetes.

• Configure the detection engine endpoint in the middleware definition.

How It Works

When the Content Guard middleware intercepts an HTTP request or response:

• Identify Relevant JSON Fields: You specify which parts of the JSON body to analyze for sensitive data. For example, you might target user messages in a request or AI-generated text in a response.

• Analyze Detected Text: The external engine (Presidio) checks whether the targeted text contains any specified “entities” (e.g., PERSON, EMAIL_ADDRESS, LOCATION). For a complete list of supported entities, see Presidio Supported Entities

• Block or Mask

  • Block: If you mark a rule to “block” and the engine finds a match, the middleware returns a 403 Forbidden, stopping the request or response.
  • Mask: If instead you specify a “mask,” the matched portions are replaced with a chosen character pattern (for example, using * or #), optionally revealing a few characters from the beginning or end.

• Custom Entities: In addition to built-in Presidio entities, you can define your own entity patterns. This can be helpful for organization-specific data formats like employee IDs or internal codes. For more details, please see the Custom Entities section.

Configuration Example

Below is an example demonstrating how to block certain content in requests and mask it in responses:

Middleware (API Gateway)

apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: content-guard
spec:
  plugin:
    content-guard:                 # generic variant
      engine:
        presidio:
          host: http://presidio
          language: en

      request:
        rules:
          # 1. Block if the payload leaks an e-mail address.
          - jsonPaths:
              - ".customer.email"
            block: true
            entities:
              - EMAIL_ADDRESS

          # 2. Mask phone numbers but let the request continue.
          - jsonPaths:
              - ".customer.phone"
            mask:
              char: "*"
              unmaskFromLeft: 2       # show first 2 chars
              unmaskFromRight: 2      # show last 2 chars
            entities:
              - PHONE_NUMBER
      response:
        rules:
          # 3. Block any response that still contains PII.
          - jsonPaths:
              - ".data[].ssn"
            block: true
            entities: [US_SSN]

          # 4. Mask locations (e.g., city names) in an array of results.
          - jsonPaths:
              - ".results[].location"
            mask:
              char: "#"
            entities:
              - LOCATION

Middleware (Chat Variant)

apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: content-guard-chat
spec:
  plugin:
    chat-completion-content-guard: # chat variant
      engine:
        presidio:
          host: http://presidio
          language: en

      response:
        # Mask email addresses in the response.
        rules:
          - mask:
              char: "#"
              unmaskFromRight: 3
            entities:
              - EMAIL_ADDRESS

Streaming

When you define a response.rules block, the middleware must inspect the entire completion. It buffers every SSE chunk, applies masking/block rules, then sends one aggregated response—format preserved, but real-time streaming is lost.

If you need live token updates, run chat-completion-content-guard only on the request (omit response.rules) or apply it to non-stream endpoints.

This design ensures the client that requested a stream gets a stream and not an unexpected response format that could break their application.

Configuation Options

Key	Description	Required	Default
engine.presidio.host	The base URL of your Presidio analyzer instance.	yes
engine.presidio.language	Language code used by Presidio for detection (for example, en).	yes
engine.presidio.entities	Optional list of entity types to detect (for example, PHONE_NUMBER, EMAIL_ADDRESS). If not set, Presidio uses all known entity types.	No
request.rules	An array of rule objects for incoming traffic. Each rule can specify jsonPaths, block, mask, and a list of entities.	No
request.rules[].jsonPaths	A list of gojq-style JSON paths to scan in the request (for example, ".messages[].content").	No
request.rules[].block	If true, any match triggers 403 Forbidden for the request.	No	false
request.rules[].mask	Defines how matched text is masked if block is false. This object can contain char, unmaskFromLeft, unmaskFromRight.	No
request.rules[].mask.char	Character used to replace matched text.	Yes (if using mask)	*
request.rules[].mask.unmaskFromLeft	Number of characters to leave unmasked at the start of a match.	No	0
request.rules[].mask.unmaskFromRight	Number of characters to leave unmasked at the end of a match.	No	0
request.rules[].entities	List of entities to look for. If omitted, Presidio attempts to detect all known entities. For a complete list of supported entities, see Presidio Supported Entities	No
response.rules	An array of rule objects for outgoing traffic. Each rule can also specify jsonPaths, block, mask, entities.	No
response.rules[].jsonPaths	A list of JSON paths to scan in the response body.	Yes (generic variant only)
response.rules[].block	If true, any match triggers 403 Forbidden for the response.	No	false
response.rules[].mask	Defines how matched text is masked if block is false. Can contain char, unmaskFromLeft, unmaskFromRight.	No
response.rules[].mask.char	Character used to replace matched text.	Yes (if using mask)	*
response.rules[].mask.unmaskFromLeft	Number of characters to leave unmasked at the start of a match.	No	0
response.rules[].mask.unmaskFromRight	Number of characters to leave unmasked at the end of a match.	No	0
response.rules[].entities	List of entities to detect. If omitted, Presidio attempts to detect all known entities. For a complete list of supported entities, see Presidio Supported Entities	No

note

jsonPaths is required only for the generic variant of the middleware. The chat variant ignores this field because the chat schema is auto-detected.

Custom Entities

You can define additional entities for Presidio to detect (such as specialized IDs or formats). These are typically added in Presidio’s configuration itself, or via its “custom analyzer” endpoints. Once added, you can reference them in the entities array like built-in types. For more details, please see the Presidio Custom Analyzer documentation.

Request vs. Response Rules

• Request Rules: Often used to block disallowed content before it ever arrives at the AI. For example, if a request includes personal information in .messages[].content, you can instantly return a 403 Forbidden.
• Response Rules: If your AI might return sensitive data (like phone numbers or addresses), you can mask that content before sending the response to the user. In unusual cases, you can also block the entire response.

Related Content

• Read the Chat Completion documentation.
• Read the Semantic Cache documentation.