Installing Traefik Enterprise On-Premises¶
This installation guide is for users who want to install a cluster on-premises.
Requirements¶
In order to start a Traefik Enterprise cluster with 1 controller, 2 proxies and a plugin registry, ensure that:
- There are at least 4 Hosts/Virtual Machines
- Network stack allows communication on required ports
- Controller Host/VM can reach
https://v4.license.containous.cloud
- The
teectl
binary is installed, for cluster management
Networking
Only the hosts which run proxies have to be reachable from the internet. On Linux, root access is needed to bind to ports below 1024.
Default Traefik Enterprise paths
The default configration paths for traefikee
includes:
- /var/run/traefikee
- /var/lib/traefikee
- /var/run/secrets
Read and write access are required on then. See the teectl
referece for customization options.
Please refer to the documentation for more information
Multiple instances on the same Host/VM
Running multiple controllers or proxies on the same host is not recommended, and should be used only when upgrading a cluster for which there are no spare hosts.
Installation behind a proxy
In order to be able to install Traefik Enterprise behind a proxy, you must make sure that each Traefik Enterprise instance
has the proper HTTP_PROXY
and HTTPS_PROXY
environment variables defined.
Controller advertise address
The controller advertise address must be an external address reachable by the proxies. Do not use a dynamic address allocation feature, such as ":4242" under Linux, as it may propagate the wrong address to the proxies, causing a connection failure upon restart.
State Directory Management¶
By default, the statedir
path is /var/lib/traefikee
.
- The state directory's contents are used to recover a cluster's state in the event of a controller restart. It has to persist between two executions.
- In order to re-initialize a cluster, you have to delete the state directory before re-installing it. If this is not done, it will recover its previous state.
Install traefikee
¶
First, download traefikee
on each virtual machine, by using one of the following links:
traefikee
for Linux (Intel) 64 Bitstraefikee
for Linux (ARM) v6traefikee
for Linux (ARM) v7traefikee
for macOS (Intel/Apple) 64 Bitstraefikee
for Windows (Intel) 64 Bits
Check the integrity of the downloaded file
Use the sha256 checksums of the binaries:
# Compare this value to the one found in traefikee_checksums.txt
sha256sum ./traefikee_v2.4.21_linux_amd64.tar.gz
# Compare this value to the one found in traefikee_checksums.txt
shasum -a256 ./traefikee_v2.4.21_darwin_amd64.tar.gz
# Compare this value to the one found in traefikee_checksums.txt
Get-FileHash traefikee_v2.4.21_windows_amd64.zip -Algorithm SHA256
Extract the downloaded archive
Use the following commands to extract the archive:
# Compare this value to the one found in traefikee_checksums.txt
tar -zxvf traefikee_v2.4.21_linux_amd64.tar.gz
# Compare this value to the one found in traefikee_checksums.txt
tar -zxvf -a256 ./traefikee_v2.4.21_darwin_amd64.tar.gz
# Compare this value to the one found in traefikee_checksums.txt
Expand-Archive traefikee_v2.4.21_windows_amd64.zip
Copy the traefikee
binary to your PATH
or add its location to your environment ($PATH
or %PATH%
depending on your OS)
and make sure it's executable:
# Example with /usr/local/bin
# These command may need sudo rights
cp traefikee /usr/local/bin/traefikee
chmod a+x /usr/local/bin/traefikee
# Should print "/usr/local/bin/traefikee"
command -v traefikee
# Example with C:\Program Files
Copy-Item "traefikee.exe" -Destination "C:\Program Files\traefikee.exe"
# Should print "C:\Program Files\traefikee.exe"
where traefikee
You can now test your installation by executing traefikee
:
traefikee --help
traefikee Traefik Enterprise (Enterprise Edition) is a Cloud Native Edge Routing Platform based on Traefik Proxy,
a modern HTTP reverse proxy and load balancer made to deploy microservices with ease.
Complete documentation is available at https://doc.traefik.io.
Usage: traefikee [command] [resource] [flags]
Use "traefikee [command] --help" for help on any command.
Commands:
controller Start a new controller
proxy Start a new proxy
tokens Get tokens from the cluster
version Print version
Generating the teectl
Configuration¶
Install the teectl
management tool on a machine that has access to a controller node by following the instructions on this guide.
Once teectl
is installed, it is possible to generate a set of cluster credentials that enables remote access to the API:
teectl setup --onpremise.hosts="<ip1>,<ip2>,<ip3>"
The host IPs that will be used for controllers should also be set in the onpremise.hosts
option.
This command creates a new teectl
configuration under ~/.config/traefikee/default.yaml
and generates a bundle.zip
which carries a set of needed assets for the installation.
The bundle.zip
needs to be copied onto each controller host.
To further configure the teectl
setup, please refer to the reference documentation.
Connecting to a Controller Locally
It is possible to connect to a cluster by using the socket (default /var/run/traefikee/teectl.sock
) using the
socket
option on teectl
commands. teectl
must be used on the same host as a controller for this to work.
teectl get nodes --socket=/var/run/traefikee/teectl.sock
In this case, no bundle.zip
is needed to start the Controller and the above step can be skipped.
Generating a Pre-Shared Token¶
The Pre-Shared Token is used to secure communications between the controllers and the plugin registry.
Token Strength
As a security best practice, it is recommended that the token be at least 24 characters long made up of random characters.
A Pre-Shared Token can be generated using openssl
:
openssl rand -base64 32
l9RLmLbAwrW73GzbDJ6+T+KOQ7u0sQt/we6OOGj0AKU
Alternatively an online generator can be used:
- Open the random string generator web page.
- Set
Number of strings
to 1. - Set
Length
to 32. - Click the
Generate
button.
Start the Controller¶
Start a new controller by specifying the advertise address and Traefik Enterprise license:
traefikee controller --advertise="10.0.0.1:4242" --license="<your-license-key>" --api.bundle="<path-to>/bundle.zip" --plugin.url="https://plugins.mydomain" --plugin.token="<pre-shared-token>"
traefikee controller --advertise="10.0.0.1:4242" --license="<your-license-key>" --api.bundle="<path-to>/bundle.zip" --discovery.static.peers="10.0.0.1:4242,10.0.0.2:4242,10.0.0.3:4242" --plugin.url="https://plugins.mydomain" --plugin.token="<pre-shared-token>"
traefikee controller --advertise="10.0.0.1:4242" --license="<your-license-key>" --api.bundle="<path-to>/bundle.zip" --discovery.dns.domain="traefikee.mydomain" --discovery.dns.port="4242" --discovery.dns.bootstrapexpected=2 --plugin.url="https://plugins.mydomain" --plugin.token="<pre-shared-token>"
INFO[2020-01-20T15:00:35-03:00] Using static discovery to discover other controllers node=controller-0 role=controller
INFO[2020-01-20T15:00:35-03:00] Initializing a new node role=controller node=controller-0
INFO[2020-01-20T15:00:35-03:00] Discovering cluster node=controller-0 role=controller
INFO[2020-01-20T15:00:35-03:00] Found peers [10.0.0.1:4242] node=controller-0 role=controller
INFO[2020-01-20T15:00:35-03:00] Initial leader runs at 10.0.0.1:4242 node=controller-0 role=controller
INFO[2020-01-20T15:00:35-03:00] Initializing a new Traefik Enterprise cluster node=controller-0 role=controller
INFO[2020-01-20T15:00:35-03:00] API started role=controller node=controller-0
INFO[2020-01-20T15:00:35-03:00] Node started role=controller node=controller-0
INFO[2020-01-20T15:00:36-03:00] Node is ready node=controller-0 role=controller
INFO[2020-01-20T15:00:36-03:00] Running the cluster info controller service=provider node=controller-0 role=controller
INFO[2020-01-20T15:00:36-03:00] Running the cluster license controller with frequency "24h0m0s" node=controller-0 role=controller service=cleanup
INFO[2020-01-20T15:00:36-03:00] Running the cluster cleanup controller role=controller node=controller-0 service=cleanup
Node Name
By default Traefik Enterprise will use the hostname as the node. This behavior can be overridden by specifying the --name
flag on the controller
command.
Multi Controller with DNS
When using DNS as the discovery method it is a good practice to set the flag discovery.dns.bootstrapexpected
, as this will make the controllers wait until the specified number of nodes are reachable before bootstraping the cluster.
Get the Tokens¶
Get the generated proxy node token:
traefikee tokens | grep 'TRAEFIKEE_PROXY_TOKEN' | cut -d '=' -f2
Get the generated controller node token:
traefikee tokens | grep 'TRAEFIKEE_CONTROLLER_TOKEN' | cut -d '=' -f2
Share the Tokens¶
All proxies require a token to start. Any additional controller also require a token to start.
Sharing the token between controllers and proxies can be achieved using environment variables or a shared persistent storage.
To use environment variables, set the token value directly with the command argument --jointoken.value
.
To use a shared persistent storage solution, set the path to the target directory with --jointoken.file.path
on both proxies and controllers instead.
The remaining steps on this guide assumes the tokens are shared using environment variables named ${PROXY_NODE_TOKEN}
and ${CONTROLLER_NODE_TOKEN}
.
Start Additional Controllers (Multi Controller Only)¶
Start additional controllers by adding the join token flag:
traefikee controller --advertise="10.0.0.1:4242" --license="<your-license-key>" --api.bundle="<path-to>/bundle.zip" --plugin.url="https://plugins.mydomain" --plugin.token="<pre-shared-token>" --jointoken.value="${CONTROLLER_NODE_TOKEN}"
traefikee controller --advertise="10.0.0.1:4242" --license="<your-license-key>" --api.bundle="<path-to>/bundle.zip" --discovery.dns.domain="traefikee.mydomain" --discovery.dns.port="4242" --discovery.dns.bootstrapexpected=2 --plugin.url="https://plugins.mydomain" --plugin.token="<pre-shared-token>" --jointoken.value="${CONTROLLER_NODE_TOKEN}"
Start Proxies¶
Start a proxy by specifying the controller discovery method, static in this example, and the join token:
traefikee proxy --jointoken.value="$PROXY_NODE_TOKEN" --discovery.static.peers="10.0.0.1:4242"
traefikee proxy --jointoken.value="$PROXY_NODE_TOKEN" --discovery.dns.domain="traefikee.mydomain"
INFO[2020-01-20T15:09:47-03:00] Applying a new static configuration node=proxy-0 role=proxy module=server
INFO[2020-01-20T15:09:47-03:00] Using static discovery node=proxy-0 role=proxy peers="10.0.0.1:4242"
INFO[2020-01-20T15:09:47-03:00] Discovering controllers node=proxy-0 role=proxy
INFO[2020-01-20T15:09:47-03:00] Found controllers [10.0.0.1:4242] node=proxy-0 role=proxy
INFO[2020-01-20T15:09:47-03:00] Waiting for the join tokens to be available node=proxy-0 role=proxy
INFO[2020-01-20T15:09:47-03:00] Looking for a reachable peer role=proxy node=proxy-0
INFO[2020-01-20T15:09:47-03:00] Joining cluster role=proxy joined-peer="10.0.0.1:4242" node=proxy-0
INFO[2020-01-20T15:09:47-03:00] Node started node=proxy-0 role=proxy
INFO[2020-01-20T15:09:47-03:00] Node is ready node=proxy-0 role=proxy
Repeat the steps above for each proxy, then verify that the Traefik Enterprise installation was successful:
teectl get nodes
ID NAME STATUS ROLE
c5j53krue2avv77ajr8h5bcoz controller-0 Ready Controller (Leader)
3l5xt87fkc2ztlqlkwcpavuev proxy-0 Ready Proxy / Ingress
yjtz8kvnsgmqmuycup69vx180 proxy-1 Ready Proxy / Ingress
Start the Plugin Registry¶
Start the plugin registry by specifying the controller discovery method, static in this example, and the join token:
traefikee plugin-registry --token="<pre-shared-token>" --jointoken.value="$PROXY_NODE_TOKEN" --discovery.static.peers="10.0.0.1:4242"
Apply a Static Configuration¶
A Traefik Enterprise cluster is created without any default configuration. To allow the controller to listen to a provider and proxies to manage incoming traffic, it is necessary to apply a static configuration.
Below is an example of a static configuration:
File provider¶
entryPoints:
web:
address: ":80"
websecure:
address: ":443"
providers:
file:
filename: dynamic_conf.yml
[entryPoints]
[entryPoints.web]
address = ":80"
[entryPoints.websecure]
address = ":443"
[providers]
[providers.file]
filename = "/etc/traefikee/dynamic_conf.toml"
Important
When running multiple controllers, make sure to replicate the configuration file on all nodes at the same path.
Apply the configuration using the following command:
teectl apply --file=config.yaml
teectl apply --file=config.toml
Systemd (Linux only)¶
When installing under Linux, you can leverage systemd to improve the management of a Traefik Enterprise deployment while also allowing it to run as a daemon.
Another benefit is the ability to load a file of environment variables, which may contain your license key, node join token, and other sensitive information. This avoids setting variables on the host terminal or exposing the keys directly in the unit file.
VM/host preparation¶
It is always recommended to run system processes unprivileged. For that, a specific user and group must be created on each host/VM (shown here with example user and group IDs of 1500):
groupadd -g 1500 traefikee
useradd -g traefikee --no-user-group --home-dir="/opt/traefikee" --shell="/usr/sbin/nologin" --system --uid="1500" traefikee
OS Compatibility
This guide, including any example config files, was created using Debian Linux. There may be slight differences on other distributions.
Unit File Placement
Usually, systemd files can be placed under /etc/systemd/system
, but this may not be the case on all distributions.
Unit Install Target
The examples use network-online
as the target group dependency, but this may not be the same on distributions other than Debian. To find out what targets are actually available, run: systemctl list-units --type target
Controllers¶
Env File (optional)¶
Store the license key and the bind address for the controller into a new environment file.
Sensitive Information
Make sure this file has restricted permissions.
File permissions
We recommend giving this file restricted access as it contains sensitive information.
# Saved as /opt/traefikee/controller.env, update the bind address on each host according to your needs.
# Use an external address, reachable by the proxies.
CONTROLLER_BIND_ADDRESS="10.0.0.10:4242"
TRAEFIKEE_LICENSE="<my-license>"
CONTROLLER_BIND_ADDRESS="10.0.0.10:4242"
CONTROLLER_JOIN_TOKEN="<my-join-token>"
TRAEFIKEE_LICENSE="<my-license>"
Unit File¶
[Unit]
Description=Traefik Enterprise Controller
After=network-online.target
Wants=network-online.target systemd-networkd-wait-online.service
[Service]
EnvironmentFile=-/opt/traefikee/controller.env
Restart=on-abnormal
User=traefikee
Group=traefikee
ExecStart=/opt/traefikee/bin/traefikee controller --advertise=${CONTROLLER_BIND_ADDRESS} --license=${TRAEFIKEE_LICENSE} --api.socket=/opt/traefikee/run/teectl.sock --socket=/opt/traefikee/run/cluster.sock --statedir=/opt/traefikee/data --jointoken.file.path=/opt/traefikee/tokens --api.autocerts
PrivateTmp=true
PrivateDevices=false
ProtectHome=true
ProtectSystem=full
ReadWritePaths=/opt/traefikee
PermissionsStartOnly=true
ExecStartPre=/usr/bin/mkdir -p /opt/traefikee/run /opt/traefikee/data /opt/traefikee/tokens
ExecStartPre=-chown -R traefikee.traefikee /opt/traefikee
ExecStartPre=-chmod -R 700 /opt/traefikee
NoNewPrivileges=true
[Install]
WantedBy=multi-user.target
[Unit]
Description=Traefik Enterprise Controller
After=network-online.target
Wants=network-online.target systemd-networkd-wait-online.service
[Service]
EnvironmentFile=-/opt/traefikee/controller.env
Restart=on-abnormal
User=traefikee
Group=traefikee
ExecStart=/opt/traefikee/bin/traefikee controller --advertise=${CONTROLLER_BIND_ADDRESS} --license=${TRAEFIKEE_LICENSE} --api.socket=/opt/traefikee/run/teectl.sock --socket=/opt/traefikee/run/cluster.sock --statedir=/opt/traefikee/data --jointoken.value=${CONTROLLER_JOIN_TOKEN} --api.autocerts
PrivateTmp=true
PrivateDevices=false
ProtectHome=true
ProtectSystem=full
ReadWritePaths=/opt/traefikee
PermissionsStartOnly=true
ExecStartPre=/usr/bin/mkdir -p /opt/traefikee/run /opt/traefikee/data /opt/traefikee/tokens
ExecStartPre=-chown -R traefikee.traefikee /opt/traefikee
ExecStartPre=-chmod -R 700 /opt/traefikee
NoNewPrivileges=true
[Install]
WantedBy=multi-user.target
For this guide, the systemd unit below was saved as /etc/systemd/system/traefikee-controller.service
.
Systemd ExecStartPre paths
In some systemd configurations all ExecStartPre
commands must have an absolute path. If this is the case, please
update the Unit file to have absolute paths.
Proxies¶
Env File (optional)¶
Store the list of controller peers to connect to and the node join token into a new environment file.
File permissions
We recommend giving this file restricted access as it contains sensitive information.
# Saved as /opt/traefikee/proxies.env
CONTROLLER_PEERS="10.0.0.10:4242"
PROXY_NODE_TOKEN="<my-token>"
Unit File¶
[Unit]
Description=Traefik Enterprise proxy
After=network-online.target
Wants=network-online.target systemd-networkd-wait-online.service
[Service]
EnvironmentFile=-/opt/traefikee/traefikee.env
Restart=on-abnormal
User=traefikee
Group=traefikee
ExecStart=/opt/traefikee/bin/traefikee proxy --jointoken.value=${PROXY_NODE_TOKEN} --discovery.static.peers=${CONTROLLER_PEERS} --statedir=/opt/traefikee/data
PrivateTmp=true
PrivateDevices=false
ProtectHome=true
ProtectSystem=full
ReadWritePaths=/opt/traefikee
ExecStartPre=mkdir -p /opt/traefikee/data
; The following additional security directives only work with systemd v229 or later.
CapabilityBoundingSet=CAP_NET_BIND_SERVICE
AmbientCapabilities=CAP_NET_BIND_SERVICE
NoNewPrivileges=true
[Install]
WantedBy=multi-user.target
For this guide, the systemd unit below was saved as /etc/systemd/system/traefikee-proxy.service
.
Systemd ExecStartPre paths
In some systemd configurations all ExecStartPre
commands must have an absolute path. If this is the case, please
update the Unit file to have absolute paths.
Tip
For the first boot, the node join token must be retrieved from a controller and updated on each proxy node before starting them. This can be automated or handled manually as a one-time operation.
With everything in place, proceed to enable the units files on systemd with:
systemctl enable traefikee-controller
systemctl enable traefikee-proxy
Created symlink /etc/systemd/system/multi-user.target.wants/traefikee-controller.service → /etc/systemd/system/traefikee-controller.service.
Created symlink /etc/systemd/system/multi-user.target.wants/traefikee-proxy.service → /etc/systemd/system/traefikee-proxy.service.
Now both the controller and proxies should be manageable by systemd as usual:
systemctl start traefikee-controller
systemctl start traefikee-proxy
Remote Access Through teectl
¶
Once your cluster is ready, if you want to operate it remotely using the teectl
tool, you need to
generate credentials from your cluster using traefikee generate credentials
on one of your controllers and use teectl
to import the cluster credentials.
traefikee generate credentials --onpremise.hosts="<ip1>,<ip2>,<ip3>" --cluster=default --socket="/opt/traefikee/run/teectl.sock" > config.yaml
teectl cluster import --file="config.yaml"
You can now use teectl
to operate your cluster.
teectl get nodes
ID NAME STATUS ROLE
c5j53krue2avv77ajr8h5bcoz controller-0 Ready Controller (Leader)
3l5xt87fkc2ztlqlkwcpavuev proxy-0 Ready Proxy / Ingress
yjtz8kvnsgmqmuycup69vx180 proxy-1 Ready Proxy / Ingress
What's Next?
Now that the cluster is ready, we recommend reading the various operating guides to dive into all features that Traefik Enterprise provides.