Service
Service Load Balancer¶
The load balancers are able to load balance the requests between multiple instances of your programs.
Each service has a load-balancer, even if there is only one server to forward traffic to.
Configuration Example¶
http:
services:
my-service:
loadBalancer:
servers:
- url: "http://private-ip-server-1/"
weight: 2
preservePath: true
sticky:
cookie:
name: "sticky-cookie"
healthcheck:
path: "/health"
interval: "10s"
timeout: "3s"
passHostHeader: true
serversTransport: "customTransport@file"
responseForwarding:
flushInterval: "150ms"
[http.services]
[http.services.my-service.loadBalancer]
[[http.services.my-service.loadBalancer.servers]]
url = "http://private-ip-server-1/"
[http.services.my-service.loadBalancer.sticky.cookie]
name = "sticky-cookie"
[http.services.my-service.loadBalancer.healthcheck]
path = "/health"
interval = "10s"
timeout = "3s"
passHostHeader = true
serversTransport = "customTransport@file"
[http.services.my-service.loadBalancer.responseForwarding]
flushInterval = "150ms"
labels:
- "traefik.http.services.my-service.loadBalancer.servers[0].url=http://private-ip-server-1/"
- "traefik.http.services.my-service.loadBalancer.servers[0].weight=2"
- "traefik.http.services.my-service.loadBalancer.servers[0].preservePath=true"
- "traefik.http.services.my-service.loadBalancer.sticky.cookie.name=sticky-cookie"
- "traefik.http.services.my-service.loadBalancer.healthcheck.path=/health"
- "traefik.http.services.my-service.loadBalancer.healthcheck.interval=10s"
- "traefik.http.services.my-service.loadBalancer.healthcheck.timeout=3s"
- "traefik.http.services.my-service.loadBalancer.passHostHeader=true"
- "traefik.http.services.my-service.loadBalancer.serversTransport=customTransport@file"
- "traefik.http.services.my-service.loadBalancer.responseForwarding.flushInterval=150ms"
{
"Tags": [
"traefik.http.services.my-service.loadBalancer.servers[0].url=http://private-ip-server-1/",
"traefik.http.services.my-service.loadBalancer.servers[0].weight=2",
"traefik.http.services.my-service.loadBalancer.servers[0].preservePath=true",
"traefik.http.services.my-service.loadBalancer.sticky.cookie.name=sticky-cookie",
"traefik.http.services.my-service.loadBalancer.healthcheck.path=/health",
"traefik.http.services.my-service.loadBalancer.healthcheck.interval=10s",
"traefik.http.services.my-service.loadBalancer.healthcheck.timeout=3s",
"traefik.http.services.my-service.loadBalancer.passHostHeader=true",
"traefik.http.services.my-service.loadBalancer.serversTransport=customTransport@file",
"traefik.http.services.my-service.loadBalancer.responseForwarding.flushInterval=150ms"
]
}
Configuration Options¶
Field | Description | Required |
---|---|---|
servers |
Represents individual backend instances for your service | Yes |
sticky |
Defines a Set-Cookie header is set on the initial response to let the client know which server handles the first response. |
No |
healthcheck |
Configures health check to remove unhealthy servers from the load balancing rotation. | No |
passHostHeader |
Allows forwarding of the client Host header to server. By default, passHostHeader is true. |
No |
serversTransport |
Allows to reference an HTTP ServersTransport configuration for the communication between Traefik and your servers. If no serversTransport is specified, the default@internal will be used. |
No |
responseForwarding |
Configures how Traefik forwards the response from the backend server to the client. | No |
responseForwarding.FlushInterval |
Specifies the interval in between flushes to the client while copying the response body. It is a duration in milliseconds, defaulting to 100ms. A negative value means to flush immediately after each write to the client. The FlushInterval is ignored when ReverseProxy recognizes a response as a streaming response; for such responses, writes are flushed to the client immediately. |
No |
Servers¶
Servers represent individual backend instances for your service. The service loadBalancer servers
option lets you configure the list of instances that will handle incoming requests.
Configuration Options¶
Field | Description | Required |
---|---|---|
url |
Points to a specific instance. | Yes for File provider, No for Docker provider |
weight |
Allows for weighted load balancing on the servers. | No |
preservePath |
Allows to preserve the URL path. | No |
Health Check¶
The healthcheck
option configures health check to remove unhealthy servers from the load balancing rotation. Traefik will consider HTTP(s) servers healthy as long as they return a status code to the health check request (carried out every interval) between 2XX
and 3XX
, or matching the configured status. For gRPC servers, Traefik will consider them healthy as long as they return SERVING to gRPC health check v1 requests.
To propagate status changes (e.g. all servers of this service are down) upwards, HealthCheck must also be enabled on the parent(s) of this service.
Below are the available options for the health check mechanism:
Field | Description | Default | Required |
---|---|---|---|
path |
Defines the server URL path for the health check endpoint. | "" | Yes |
scheme |
Replaces the server URL scheme for the health check endpoint. | No | |
mode |
If defined to grpc , will use the gRPC health check protocol to probe the server. |
http | No |
hostname |
Defines the value of hostname in the Host header of the health check request. | "" | No |
port |
Replaces the server URL port for the health check endpoint. | No | |
interval |
Defines the frequency of the health check calls for healthy targets. | 30s | No |
unhealthyInterval |
Defines the frequency of the health check calls for unhealthy targets. When not defined, it defaults to the interval value. |
30s | No |
timeout |
Defines the maximum duration Traefik will wait for a health check request before considering the server unhealthy. | 5s | No |
headers |
Defines custom headers to be sent to the health check endpoint. | No | |
followRedirects |
Defines whether redirects should be followed during the health check calls. | true | No |
hostname |
Defines the value of hostname in the Host header of the health check request. | "" | No |
method |
Defines the HTTP method that will be used while connecting to the endpoint. | GET | No |
status |
Defines the expected HTTP status code of the response to the health check request. | No |
Sticky sessions¶
When sticky sessions are enabled, a Set-Cookie
header is set on the initial response to let the client know which server handles the first response.
On subsequent requests, to keep the session alive with the same server, the client should send the cookie with the value set.
Stickiness on multiple levels¶
When chaining or mixing load-balancers (e.g. a load-balancer of servers is one of the "children" of a load-balancer of services), for stickiness to work all the way, the option needs to be specified at all required levels. Which means the client needs to send a cookie with as many key/value pairs as there are sticky levels.
Stickiness & Unhealthy Servers¶
If the server specified in the cookie becomes unhealthy, the request will be forwarded to a new server (and the cookie will keep track of the new server).
Cookie Name¶
The default cookie name is an abbreviation of a sha1 (ex: `_1d52e`).
MaxAge¶
By default, the affinity cookie will never expire as the `MaxAge` option is set to zero.
This option indicates the number of seconds until the cookie expires.
When set to a negative number, the cookie expires immediately.
Secure & HTTPOnly & SameSite flags¶
By default, the affinity cookie is created without those flags.
One however can change that through configuration.
`SameSite` can be `none`, `lax`, `strict` or empty.
Domain¶
The Domain attribute of a cookie specifies the domain for which the cookie is valid.
By setting the Domain attribute, the cookie can be shared across subdomains (for example, a cookie set for example.com would be accessible to www.example.com, api.example.com, etc.). This is particularly useful in cases where sticky sessions span multiple subdomains, ensuring that the session is maintained even when the client interacts with different parts of the infrastructure.
Adding Stickiness -- Using the File Provider
## Dynamic configuration
http:
services:
my-service:
loadBalancer:
sticky:
cookie: {}
## Dynamic configuration
[http.services]
[http.services.my-service]
[http.services.my-service.loadBalancer.sticky.cookie]
Adding Stickiness with custom Options -- Using the File Provider
## Dynamic configuration
http:
services:
my-service:
loadBalancer:
sticky:
cookie:
name: my_sticky_cookie_name
secure: true
domain: mysite.site
httpOnly: true
## Dynamic configuration
[http.services]
[http.services.my-service]
[http.services.my-service.loadBalancer.sticky.cookie]
name = "my_sticky_cookie_name"
secure = true
httpOnly = true
domain = "mysite.site"
sameSite = "none"
Setting Stickiness on all the required levels -- Using the File Provider
## Dynamic configuration
http:
services:
wrr1:
weighted:
sticky:
cookie:
name: lvl1
services:
- name: whoami1
weight: 1
- name: whoami2
weight: 1
whoami1:
loadBalancer:
sticky:
cookie:
name: lvl2
servers:
- url: http://127.0.0.1:8081
- url: http://127.0.0.1:8082
whoami2:
loadBalancer:
sticky:
cookie:
name: lvl2
servers:
- url: http://127.0.0.1:8083
- url: http://127.0.0.1:8084
## Dynamic configuration
[http.services]
[http.services.wrr1]
[http.services.wrr1.weighted.sticky.cookie]
name = "lvl1"
[[http.services.wrr1.weighted.services]]
name = "whoami1"
weight = 1
[[http.services.wrr1.weighted.services]]
name = "whoami2"
weight = 1
[http.services.whoami1]
[http.services.whoami1.loadBalancer]
[http.services.whoami1.loadBalancer.sticky.cookie]
name = "lvl2"
[[http.services.whoami1.loadBalancer.servers]]
url = "http://127.0.0.1:8081"
[[http.services.whoami1.loadBalancer.servers]]
url = "http://127.0.0.1:8082"
[http.services.whoami2]
[http.services.whoami2.loadBalancer]
[http.services.whoami2.loadBalancer.sticky.cookie]
name = "lvl2"
[[http.services.whoami2.loadBalancer.servers]]
url = "http://127.0.0.1:8083"
[[http.services.whoami2.loadBalancer.servers]]
url = "http://127.0.0.1:8084"
To keep a session open with the same server, the client would then need to specify the two levels within the cookie for each request, e.g. with curl:
curl -b "lvl1=whoami1; lvl2=http://127.0.0.1:8081" http://localhost:8000
Weighted Round Robin (WRR)¶
The WRR is able to load balance the requests between multiple services based on weights.
This strategy is only available to load balance between services and not between servers.
Supported Providers
This strategy can be defined currently with the File or IngressRoute providers. To load balance between servers based on weights, the Load Balancer service should be used instead.
## Dynamic configuration
http:
services:
app:
weighted:
services:
- name: appv1
weight: 3
- name: appv2
weight: 1
appv1:
loadBalancer:
servers:
- url: "http://private-ip-server-1/"
appv2:
loadBalancer:
servers:
- url: "http://private-ip-server-2/"
## Dynamic configuration
[http.services]
[http.services.app]
[[http.services.app.weighted.services]]
name = "appv1"
weight = 3
[[http.services.app.weighted.services]]
name = "appv2"
weight = 1
[http.services.appv1]
[http.services.appv1.loadBalancer]
[[http.services.appv1.loadBalancer.servers]]
url = "http://private-ip-server-1/"
[http.services.appv2]
[http.services.appv2.loadBalancer]
[[http.services.appv2.loadBalancer.servers]]
url = "http://private-ip-server-2/"
Health Check¶
HealthCheck enables automatic self-healthcheck for this service, i.e. whenever one of its children is reported as down, this service becomes aware of it, and takes it into account (i.e. it ignores the down child) when running the load-balancing algorithm. In addition, if the parent of this service also has HealthCheck enabled, this service reports to its parent any status change.
Behavior
If HealthCheck is enabled for a given service and any of its descendants does not have it enabled, the creation of the service will fail.
HealthCheck on Weighted services can be defined currently only with the File provider.
## Dynamic configuration
http:
services:
app:
weighted:
healthCheck: {}
services:
- name: appv1
weight: 3
- name: appv2
weight: 1
appv1:
loadBalancer:
healthCheck:
path: /status
interval: 10s
timeout: 3s
servers:
- url: "http://private-ip-server-1/"
appv2:
loadBalancer:
healthCheck:
path: /status
interval: 10s
timeout: 3s
servers:
- url: "http://private-ip-server-2/"
## Dynamic configuration
[http.services]
[http.services.app]
[http.services.app.weighted.healthCheck]
[[http.services.app.weighted.services]]
name = "appv1"
weight = 3
[[http.services.app.weighted.services]]
name = "appv2"
weight = 1
[http.services.appv1]
[http.services.appv1.loadBalancer]
[http.services.appv1.loadBalancer.healthCheck]
path = "/health"
interval = "10s"
timeout = "3s"
[[http.services.appv1.loadBalancer.servers]]
url = "http://private-ip-server-1/"
[http.services.appv2]
[http.services.appv2.loadBalancer]
[http.services.appv2.loadBalancer.healthCheck]
path = "/health"
interval = "10s"
timeout = "3s"
[[http.services.appv2.loadBalancer.servers]]
url = "http://private-ip-server-2/"
P2C¶
Power of two choices algorithm is a load balancing strategy that selects two servers at random and chooses the one with the least number of active requests.
P2C Load Balancing -- Using the File Provider
## Dynamic configuration
http:
services:
my-service:
loadBalancer:
strategy: "p2c"
servers:
- url: "http://private-ip-server-1/"
- url: "http://private-ip-server-2/"
- url: "http://private-ip-server-3/"
## Dynamic configuration
[http.services]
[http.services.my-service.loadBalancer]
strategy = "p2c"
[[http.services.my-service.loadBalancer.servers]]
url = "http://private-ip-server-1/"
[[http.services.my-service.loadBalancer.servers]]
url = "http://private-ip-server-2/"
[[http.services.my-service.loadBalancer.servers]]
url = "http://private-ip-server-3/"
Mirroring¶
The mirroring is able to mirror requests sent to a service to other services. Please note that by default the whole request is buffered in memory while it is being mirrored. See the maxBodySize
option in the example below for how to modify this behaviour. You can also omit the request body by setting the mirrorBody
option to false.
Default behavior of percent
When configuring a mirror
service, if the percent
field is not set, it defaults to 0
, meaning no traffic will be sent to the mirror.
Supported Providers
This strategy can be defined currently with the File or IngressRoute providers.
## Dynamic configuration
http:
services:
mirrored-api:
mirroring:
service: appv1
# mirrorBody defines whether the request body should be mirrored.
# Default value is true.
mirrorBody: false
# maxBodySize is the maximum size allowed for the body of the request.
# If the body is larger, the request is not mirrored.
# Default value is -1, which means unlimited size.
maxBodySize: 1024
mirrors:
- name: appv2
# Percent defines the percentage of requests that should be mirrored.
# Default value is 0, which means no traffic will be sent to the mirror.
percent: 10
appv1:
loadBalancer:
servers:
- url: "http://private-ip-server-1/"
appv2:
loadBalancer:
servers:
- url: "http://private-ip-server-2/
## Dynamic configuration
[http.services]
[http.services.mirrored-api]
[http.services.mirrored-api.mirroring]
service = "appv1"
# maxBodySize is the maximum size in bytes allowed for the body of the request.
# If the body is larger, the request is not mirrored.
# Default value is -1, which means unlimited size.
maxBodySize = 1024
# mirrorBody defines whether the request body should be mirrored.
# Default value is true.
mirrorBody = false
[[http.services.mirrored-api.mirroring.mirrors]]
name = "appv2"
percent = 10
[http.services.appv1]
[http.services.appv1.loadBalancer]
[[http.services.appv1.loadBalancer.servers]]
url = "http://private-ip-server-1/"
[http.services.appv2]
[http.services.appv2.loadBalancer]
[[http.services.appv2.loadBalancer.servers]]
url = "http://private-ip-server-2/"
Health Check¶
HealthCheck enables automatic self-healthcheck for this service, i.e. if the main handler of the service becomes unreachable, the information is propagated upwards to its parent.
Behavior
If HealthCheck is enabled for a given service and any of its descendants does not have it enabled, the creation of the service will fail.
HealthCheck on Mirroring services can be defined currently only with the File provider.
## Dynamic configuration
http:
services:
mirrored-api:
mirroring:
healthCheck: {}
service: appv1
mirrors:
- name: appv2
percent: 10
appv1:
loadBalancer:
healthCheck:
path: /status
interval: 10s
timeout: 3s
servers:
- url: "http://private-ip-server-1/"
appv2:
loadBalancer:
servers:
- url: "http://private-ip-server-2/"
## Dynamic configuration
[http.services]
[http.services.mirrored-api]
[http.services.mirrored-api.mirroring]
service = "appv1"
[http.services.mirrored-api.mirroring.healthCheck]
[[http.services.mirrored-api.mirroring.mirrors]]
name = "appv2"
percent = 10
[http.services.appv1]
[http.services.appv1.loadBalancer]
[http.services.appv1.loadBalancer.healthCheck]
path = "/health"
interval = "10s"
timeout = "3s"
[[http.services.appv1.loadBalancer.servers]]
url = "http://private-ip-server-1/"
[http.services.appv2]
[http.services.appv2.loadBalancer]
[http.services.appv1.loadBalancer.healthCheck]
path = "/health"
interval = "10s"
timeout = "3s"
[[http.services.appv2.loadBalancer.servers]]
url = "http://private-ip-server-2/"
Failover¶
A failover service job is to forward all requests to a fallback service when the main service becomes unreachable.
Relation to HealthCheck
The failover service relies on the HealthCheck system to get notified when its main service becomes unreachable, which means HealthCheck needs to be enabled and functional on the main service. However, HealthCheck does not need to be enabled on the failover service itself for it to be functional. It is only required in order to propagate upwards the information when the failover itself becomes down (i.e. both its main and its fallback are down too).
Supported Provider
This strategy can currently only be defined with the File provider.
HealthCheck¶
HealthCheck enables automatic self-healthcheck for this service, i.e. if the main and the fallback services become unreachable, the information is propagated upwards to its parent.
Behavior
If HealthCheck is enabled for a given service and any of its descendants does not have it enabled, the creation of the service will fail.
HealthCheck on a Failover service can be defined currently only with the File provider.
## Dynamic configuration
http:
services:
app:
failover:
healthCheck: {}
service: main
fallback: backup
main:
loadBalancer:
healthCheck:
path: /status
interval: 10s
timeout: 3s
servers:
- url: "http://private-ip-server-1/"
backup:
loadBalancer:
healthCheck:
path: /status
interval: 10s
timeout: 3s
servers:
- url: "http://private-ip-server-2/"
## Dynamic configuration
[http.services]
[http.services.app]
[http.services.app.failover.healthCheck]
[http.services.app.failover]
service = "main"
fallback = "backup"
[http.services.main]
[http.services.main.loadBalancer]
[http.services.main.loadBalancer.healthCheck]
path = "/health"
interval = "10s"
timeout = "3s"
[[http.services.main.loadBalancer.servers]]
url = "http://private-ip-server-1/"
[http.services.backup]
[http.services.backup.loadBalancer]
[http.services.backup.loadBalancer.healthCheck]
path = "/health"
interval = "10s"
timeout = "3s"
[[http.services.backup.loadBalancer.servers]]
url = "http://private-ip-server-2/"