InFlightReq

Limiting the Number of Simultaneous In-Flight Requests

InFlightReq

To proactively prevent services from being overwhelmed with high load, a limit on the number of simultaneous in-flight requests can be applied.

Configuration Examples

labels:
  - "traefik.http.middlewares.test-inflightreq.inflightreq.amount=10"
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: test-inflightreq
spec:
  inFlightReq:
    amount: 10
"labels": {
  "traefik.http.middlewares.test-inflightreq.inflightreq.amount": "10"
}
# Limiting to 10 simultaneous connections
labels:
  - "traefik.http.middlewares.test-inflightreq.inflightreq.amount=10"
# Limiting to 10 simultaneous connections
[http.middlewares]
  [http.middlewares.test-inflightreq.inFlightReq]
    amount = 10 
# Limiting to 10 simultaneous connections
http:
  middlewares:
    test-inflightreq:
      inFlightReq:
        amount: 10

Configuration Options

amount

The amount option defines the maximum amount of allowed simultaneous in-flight request. The middleware will return an HTTP 429 Too Many Requests if there are already amount requests in progress (based on the same sourceCriterion strategy).

labels:
  - "traefik.http.middlewares.test-inflightreq.inflightreq.amount=10"
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: test-inflightreq
spec:
  inFlightReq:
    amount: 10
"labels": {
  "traefik.http.middlewares.test-inflightreq.inflightreq.amount": "10"
}
# Limiting to 10 simultaneous connections
labels:
  - "traefik.http.middlewares.test-inflightreq.inflightreq.amount=10"
# Limiting to 10 simultaneous connections
[http.middlewares]
  [http.middlewares.test-inflightreq.inFlightReq]
    amount = 10 
# Limiting to 10 simultaneous connections
http:
  middlewares:
    test-inflightreq:
      inFlightReq:
        amount: 10

sourceCriterion

SourceCriterion defines what criterion is used to group requests as originating from a common source. The precedence order is ipStrategy, then requestHeaderName, then requestHost. If none are set, the default is to use the requestHost.

sourceCriterion.ipStrategy

The ipStrategy option defines two parameters that sets how Traefik will determine the client IP: depth, and excludedIPs.

ipStrategy.depth

The depth option tells Traefik to use the X-Forwarded-For header and take the IP located at the depth position (starting from the right).

  • If depth is greater than the total number of IPs in X-Forwarded-For, then the client IP will be empty.
  • depth is ignored if its value is lesser than or equal to 0.

Example of Depth & X-Forwarded-For

If depth was equal to 2, and the request X-Forwarded-For header was "10.0.0.1,11.0.0.1,12.0.0.1,13.0.0.1" then the "real" client IP would be "10.0.0.1" (at depth 4) but the IP used as the criterion would be "12.0.0.1" (depth=2).

X-Forwarded-For depth clientIP
"10.0.0.1,11.0.0.1,12.0.0.1,13.0.0.1" 1 "13.0.0.1"
"10.0.0.1,11.0.0.1,12.0.0.1,13.0.0.1" 3 "11.0.0.1"
"10.0.0.1,11.0.0.1,12.0.0.1,13.0.0.1" 5 "" 
labels:
  - "traefik.http.middlewares.test-inflightreq.inflightreq.sourcecriterion.ipstrategy.depth=2"
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: test-inflightreq
spec:
  inFlightReq:
    sourceCriterion:
      ipStrategy:
        depth: 2
labels:
  - "traefik.http.middlewares.test-inflightreq.inflightreq.sourcecriterion.ipstrategy.depth=2"
"labels": {
  "traefik.http.middlewares.test-inflightreq.inflightreq.sourcecriterion.ipstrategy.depth": "2"
}
[http.middlewares]
  [http.middlewares.test-inflightreq.inflightreq]
    [http.middlewares.test-inflightreq.inFlightReq.sourceCriterion.ipStrategy]
      depth = 2
http:
  middlewares:
    test-inflightreq:
      inFlightReq:
        sourceCriterion:
          ipStrategy:
            depth: 2
ipStrategy.excludedIPs

excludedIPs tells Traefik to scan the X-Forwarded-For header and pick the first IP not in the list.

If depth is specified, excludedIPs is ignored.

Example of ExcludedIPs & X-Forwarded-For

X-Forwarded-For excludedIPs clientIP
"10.0.0.1,11.0.0.1,12.0.0.1,13.0.0.1" "12.0.0.1,13.0.0.1" "11.0.0.1"
"10.0.0.1,11.0.0.1,12.0.0.1,13.0.0.1" "15.0.0.1,13.0.0.1" "12.0.0.1"
"10.0.0.1,11.0.0.1,12.0.0.1,13.0.0.1" "10.0.0.1,13.0.0.1" "12.0.0.1"
"10.0.0.1,11.0.0.1,12.0.0.1,13.0.0.1" "15.0.0.1,16.0.0.1" "13.0.0.1"
"10.0.0.1,11.0.0.1" "10.0.0.1,11.0.0.1" "" 
labels:
  - "traefik.http.middlewares.test-inflightreq.inflightreq.sourcecriterion.ipstrategy.excludedips=127.0.0.1/32, 192.168.1.7"
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: test-inflightreq
spec:
  inFlightReq:
    sourceCriterion:
      ipStrategy:
        excludedIPs:
        - 127.0.0.1/32
        - 192.168.1.7
"labels": {
  "traefik.http.middlewares.test-inflightreq.inflightreq.sourcecriterion.ipstrategy.excludedips": "127.0.0.1/32, 192.168.1.7"
}
labels:
  - "traefik.http.middlewares.test-inflightreq.inflightreq.sourcecriterion.ipstrategy.excludedips=127.0.0.1/32, 192.168.1.7"
[http.middlewares]
  [http.middlewares.test-inflightreq.inflightreq]
    [http.middlewares.test-inflightreq.inFlightReq.sourceCriterion.ipStrategy]
      excludedIPs = ["127.0.0.1/32", "192.168.1.7"]
http:
  middlewares:
    test-inflightreq:
      inFlightReq:
        sourceCriterion:
          ipStrategy:
            excludedIPs:
              - "127.0.0.1/32"
              - "192.168.1.7"

sourceCriterion.requestHeaderName

Requests having the same value for the given header are grouped as coming from the same source.

labels:
  - "traefik.http.middlewares.test-inflightreq.inflightreq.sourcecriterion.requestheadername=username"
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: test-inflightreq
spec:
  inFlightReq:
    sourceCriterion:
      requestHeaderName: username
labels:
  - "traefik.http.middlewares.test-inflightreq.inflightreq.sourcecriterion.requestheadername=username"
"labels": {
  "traefik.http.middlewares.test-inflightreq.inflightreq.sourcecriterion.requestheadername": "username"
}
[http.middlewares]
  [http.middlewares.test-inflightreq.inflightreq]
    [http.middlewares.test-inflightreq.inFlightReq.sourceCriterion]
      requestHeaderName = "username"
http:
  middlewares:
    test-inflightreq:
      inFlightReq:
        sourceCriterion:
          requestHeaderName: username

sourceCriterion.requestHost

Whether to consider the request host as the source.

labels:
  - "traefik.http.middlewares.test-inflightreq.inflightreq.sourcecriterion.requesthost=true"
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: test-inflightreq
spec:
  inFlightReq:
    sourceCriterion:
      requestHost: true
labels:
  - "traefik.http.middlewares.test-inflightreq.inflightreq.sourcecriterion.requesthost=true"
"labels": {
  "traefik.http.middlewares.test-inflightreq.inflightreq.sourcecriterion.requesthost": "true"
}
[http.middlewares]
  [http.middlewares.test-inflightreq.inflightreq]
    [http.middlewares.test-inflightreq.inFlightReq.sourceCriterion]
      requestHost = true
http:
  middlewares:
    test-inflightreq:
      inFlightReq:
        sourceCriterion:
          requestHost: true