Installing Traefik Enterprise on Docker Swarm¶
This page guides you through the installation of Traefik Enterprise on Docker CE Swarm Mode.
Docker Knowledge
Assistance with configuring or setting up a Docker Swarm Mode cluster is not included in this guide. If you need more information about Docker and Swarm, start with the following resources:
Requirements¶
- A Docker Swarm cluster
- Supported versions:
17.06
to19.03
- At least 1 manager swarm node and 1 worker swarm node
- The
docker
CLI tool properly configured to communicate with the swarm cluster.teectl
relies on theDOCKER_HOST
,DOCKER_CERT_PATH
andDOCKER_TLS_VERIFY
environment variables to communicate with the cluster.
- Supported versions:
- Controller containers can reach
https://v4.license.containous.cloud
. - The
teectl
binary is installed, for cluster management
Installing on DockerCE¶
This guide provides a detailed explanation of the installation process. For a lighter approach, please refer to the getting started guide.
Generating the teectl
Configuration¶
To start the installation, teectl
must first be configured:
teectl setup --swarm --swarm.hosts="node1,node2,node3"
This command creates a new teectl
configuration under ~/.config/traefikee/default.yaml
and generates a bundle.zip
which carries a set of needed assets for the installation.
To further configure the teectl
setup, please refer to the reference documentation.
About the --swarm.hosts option
teectl
communicates with the cluster through the network. To do so it needs to know where the cluster API is served.
This option accepts a comma separated list of ip addresses or domain names that serve the API.
By default, the API is exposed by the controllers on the swarm routing mesh with the port 55055
, so the host to use
is potentially one or more node addresses of the swarm cluster.
Sensitive information
The generated bundle.zip
carries sensitive information and is required to generate installation manifests. It should be stored in a secure place.
Managing Multiple Clusters from a Single Workstation
To manage multiple clusters from a single workstation, please refer to the Managing Multiple Clusters guide
Writing the bundle at a different path
teectl setup
comes with an --output
option that specifies the output path in which to create the bundle.
Generating the Docker Compose Files¶
Traefik Enterprise is typically installed in its own stack. To create the stack resources, teectl
can generate docker compose v3 files for the controllers and proxies.
Controllers¶
# Generates a docker-compose file to deploy 3 controllers and store it in a file called controllers.yaml
teectl setup gen --license="${TRAEFIKEE_LICENSE}" --controllers=3 > controllers.yaml
To further configure the setup gen
command, please see the reference documentation.
Filesystem assets
The generated compose file refers to the bundle generated by teectl setup
. By default it is located at ./bundle.zip
but it can be configured using the --bundle
option.
Stateful instances
Controllers persist their state to a volume to ensure tolerance to container restart.
To be compatible with the greatest amount of environments, Traefik Enterprise relies by default on a local volume and assigns a replica to a specific node using a placement constraint.
Using global volumes
is possible. To do so please change the volume driver on the generated compose file before deploying.
One service per controller
The generated docker compose file defines one controller per service. As it might seem counterintuitive, it is actually done for multiple reasons:
- To operate properly, controllers need to maintain their network identifiers (network names in this case) across container restarts. This is the only way to do so in swarm at this moment.
- As explained above, controller instances are assigned to a node using a placement constraint. To do so it is required to have one service per instance.
Never scale a controller service
As explained above, controllers rely on one and only one instance per service.
Scaling a controller service would have an undefined behaviour and possibly severe consequences.
What is the api_proxy
service?
As mentioned above, teectl
communicates with the controllers using the port 55055
of the ingress network. However, one service is created per replica of the controllers so it is impossible to expose the port 55055 of all the controllers to the ingress network.
To achieve this, a small TCP proxy service is started alongside the controllers and exposes the port 55055
to the ingress network and load-balances all received connections between the controllers.
On DockerEE
To enable UCP, please refer to the DockerEE guide below.
Proxies¶
# Generates a docker-compose file to deploy 2 proxies.
teectl setup gen --license=${TRAEFIKEE_LICENSE} --proxies=2 > proxies.yaml
To further configure the setup gen
command, please see the reference documentation.
About networking
Two networks are declared in the generated compose files to guarantee network isolation:
- The
control
network is dedicated to the communication between the controllers and the proxies. - The
ingress
network is dedicated to the communication between proxies and application services.
All the services that need to be exposed have to be connected to the ingress
network and declared as external. An example is shown below:
# If the cluster is deployed under the stack `traefikee`
networks:
traefikee_ingress:
external: true
The swarm network discovery is also available to automatically connect proxies to networks where there are applications to expose.
Customizing the Docker Compose Files (Optional)¶
There is a section dedicated to customizing the compose file, with some common scenarios that requires it.
The following aspects should be taken in account:
- Memory/CPU Resevations and Limits: Both controllers and proxies come with enforced resource limits. The defaults are voluntarily kept large to fit most of use cases. They can be adjusted to better fit the targeted installation.
- Network Configuration and Exposed Ports: By default, the
proxies
service is configured to expose the ports80
and443
to the swarm ingress network. This setting has to be adjusted according to the target installation. If network performance is needed, please consider bypassing the routing mesh. - Volumes: Local volumes are being used by default in order to fit most installations. If a global volume driver is available on the targeted swarm cluster, it can be used instead. This would remove the placement constraints on the controllers, providing more flexibility. However, be careful with the I/O performance of these volumes as it is critical for the stability of the cluster.
- Placement constraints: By default, proxy services are configured to be deployed on workers only. If the workload is allowed on manager nodes or only certain swarm nodes are exposed to the internet then the placement constraint can be adjusted. However, unless a global volume is being used for the controller volumes, please do not remove the placement constraints on the controller services.
- Labels: To expose the dashboard, labels on the
proxies
service is required. However, please make sure not to remove thecom.containous.traefikee.component
label. - Environment variables: To use Datadog or ACME DNS Challenge for instance, environment variables should be configured either on proxies or controllers.
Rootless Image
A rootless image is available. For more information, please refer to this section.
Healthcheck
We don't provide a swarm compliant health check at this moment.
Deploying the Cluster¶
Deploy the controllers before the proxies
The Proxies docker compose files depends on resources that are created by the controllers once they are running. To be able to deploy proxies, please first deploy the controllers and make sure they are running.
Deploy the controllers and the proxies under the same stack
Docker Stack provides scoped resources on a stack.
If the controllers
and proxies
services are deployed under different stacks, they won't be able to reach each other.
Once the compose files are reviewed and ready, the next step is to deploy the cluster:
First, deploy the controllers:
docker stack deploy -c ./controllers.yaml traefikee
Creating secret traefikee_bundle
Creating network traefikee_control
Creating service traefikee_api-proxies
Creating service traefikee_controller-0
Creating service traefikee_controller-1
Creating service traefikee_controller-2
To monitor the controller boot process, use:
docker service ls
ID NAME MODE REPLICAS IMAGE PORTS
kdekcc86kf5s traefikee_api-proxies replicated 2/2 traefik/traefikee:v2.4.1 *:55055->55055/tcp
zs1q0yzs3ni7 traefikee_controller-0 replicated 1/1 traefik/traefikee:v2.4.1
06qn1eqb8e4a traefikee_controller-1 replicated 1/1 traefik/traefikee:v2.4.1
p1k7pwdyhwyc traefikee_controller-2 replicated 1/1 traefik/traefikee:v2.4.1
Then, once the controllers are ready (all replicas are running and stable), deploy the proxies:
docker stack deploy -c ./proxies.yaml traefikee
Creating network traefikee_ingress
Creating service traefikee_proxies
To check that the proxies are running:
docker service ls
ID NAME MODE REPLICAS IMAGE PORTS
[...]
p1k7pwdyhwyc traefikee_proxies replicated 2/2 traefik/traefikee:v2.4.1 *:80->80/tcp *:443->443/tcp
When all the pods are running, ensure that the cluster is properly installed using teectl get nodes
:
teectl get nodes
ID NAME STATUS ROLE
7tw8nppypruy0iqgfyqf2jvz5 controller-1 Ready Controller
i8lmlridgrihknvfpp9a4ckyo 78877d77d9 Ready Proxy
j8e9luok9ksoj35v80mjasies 7d9bf4w238 Ready Proxy
lufljo7vjyqptmsczfcxb4ljm controller-2 Ready Controller
rb74v1d548petedlqny2n76cm controller-0 Ready Controller (Leader)
Applying a Static Configuration¶
A cluster is created without any default configuration. To allow the controller to listen to a provider and proxies to manage incoming traffic, it is necessary to apply a static configuration.
The following example defines two entry-points (listening on 80
and 443
), and enables the docker provider with swarm mode enabled.
---
providers:
docker:
swarmMode: true
entryPoints:
web:
address: ":80"
websecure:
address: ":443"
[providers]
[providers.docker]
swarmMode=true
[entryPoints]
[entryPoints.web]
address = ":80"
[entryPoints.websecure]
address = ":443"
Apply the configuration using the following command:
teectl apply --file="./static.yaml"
teectl apply --file="./static.toml"
On DockerEE
To enable UCP, please refer to the DockerEE guide below.
At any time, it is possible to get the currently applied static configuration of a cluster using:
teectl get static-config
---
configuration:
global:
checkNewVersion: true
serversTransport:
maxIdleConnsPerHost: 200
entryPoints:
web:
address: :80
transport:
lifeCycle:
graceTimeOut: 10s
respondingTimeouts:
idleTimeout: 3m0s
forwardedHeaders: {}
websecure:
address: :443
transport:
lifeCycle:
graceTimeOut: 10s
respondingTimeouts:
idleTimeout: 3m0s
forwardedHeaders: {}
providers:
providersThrottleDuration: 2s
docker:
swarmmode: true
cluster:
cleanup:
gracePeriod: 1h0m0s
Deploying a Test Service¶
To validate your setup, it is possible to deploy a test application using the following docker compose file:
version: '3.4'
networks:
traefikee_ingress:
external: true
services:
whoami:
image: traefik/whoami:v1.6.1
deploy:
mode: replicated
replicas: 1
labels:
- "traefik.http.routers.whoami.rule=Path(`/whoami`)"
- "traefik.http.services.whoami.loadbalancer.server.port=80"
networks:
- traefikee_ingress
Then, access the application:
curl <your-cluster-hostname-or-ip>/whoami
Hostname: 06c13be38c61
IP: 127.0.0.1
IP: 172.17.0.3
RemoteAddr: 172.17.0.1:33684
GET / HTTP/1.1
Host: localhost
User-Agent: curl/7.68.0
Accept: */*
Congratulations! Your Traefik Enterprise cluster is ready.
What's next?
Now that the cluster is ready, we recommend reading the various operating guides to dive into the features that Traefik Enterprise provides.
We also recommend getting familiar with the various concepts of Traefik Enterprise.
Installing on DockerEE¶
Traefik Enterprise supports leveraging the Universal Control Plane to access the Docker API which brings the following benefits:
- Access control on the Docker API: UCP provides access control to the Docker API, improving the security of the installation.
- Workload on workers only: Using UCP removes the need of accessing the Docker socket, removing the constraint of executing controllers on managers.
To install Traefik Enterprise with UCP support enabled follow the DockerCE installation procedure above except the two following steps.
Generating the Controllers Docker Compose File¶
To enable UCP support, pass the --swarm.ucpbundle
option, which is a path to an UCP client bundle previously generated.
teectl setup gen --license="${TRAEFIKEE_LICENSE}" --controllers=3 --swarm.ucpbundle="./path/to/bundle.zip" > controllers.yaml
Allowing execution on managers
If it is needed to allow executing controllers on swarm managers, pass --swarm.allowmanagers
option to setup gen
.
Applying a Static Configuration¶
To support access to the Docker API through UCP, the docker provider has to be configured properly in the static configuration:
providers:
docker:
endpoint: "tcp://address-of-the-UCP-api:port"
swarmMode: true
tls:
ca: "/var/run/ucp/ca.pem"
key: "/var/run/ucp/key.pem"
cert: "/var/run/ucp/cert.pem"
[providers]
[providers.docker]
endpoint = "tcp://address-of-the-UCP-api:port"
swarmMode = true
[providers.docker.tls]
ca = "/var/run/ucp/ca.pem"
key = "/var/run/ucp/key.pem"
cert = "/var/run/ucp/cert.pem"