Installing Traefik Enterprise on HashiCorp Nomad¶
This page guides you through the installation of Traefik Enterprise on HashiCorp Nomad.
Nomad Knowledge
Assistance with configuring or setting up a Nomad cluster is not included in this guide. If you need more information about Nomad, Consul, Vault, or any other components referenced in this guide, start with the following resources:
Requirements¶
- A Nomad cluster
- Version 0.12.7 or greater
- A persistent storage volume for the controllers
- The
nomadCLI tool properly configured to communicate with the cluster - Consul integration for service discovery
- Vault integration for secrets
- Controller containers can reach
https://v4.license.containous.cloud - The
teectlCLI tool, for cluster management
Installation¶
This guide will only focus on the Docker task driver, although it is also possible to run with Fork/Exec with a few changes that can be derived from our On-Premise guide
Nomad Preparation¶
Persistent Volume¶
Create a persistent volume, either by relying on a CSI storage plugin or by preselecting the nodes that will run controller tasks and binding a directory from the host as a volume.
For simplicity's sake, this guide will use the host volume approach. In this case, the directory must be present before you can expose it.
# Execute on each Nomad node that will run a controller instance
mkdir -p /opt/traefikeeAfter creating the directory, add a host_volume bind on the Nomad client configuration of each node:
client {
host_volume "traefikee-data" {
path = "/opt/traefikee"
}
}Service Discovery¶
To avoid specifying the Consul address manually on each Task network block, you can add it to the DNS resolution configuration on each Nomad host. Head to the DNS forwarding guide for Consul for more information.
License Secret¶
Create a secret in Hashicorp Vault with your license key:
vault kv put secret/traefikee/license license_key=$TRAEFIKEE_LICENSEGenerating the teectl Configuration¶
Install the teectl management tool on a machine with access to the nodes that will host controller instances by following the instructions in this guide.
Once teectl is installed, it is possible to generate a set of credentials that enables remote access to the cluster API:
# Replace ipX with the Nomad Client Node address that will host Traefik Enterprise controller instances
teectl setup --onpremise.hosts="<ip1>,<ip2>,<ip3>"This command creates a new teectl configuration under ~/.config/traefikee/default.yaml and generates a bundle.zip that carries a set of needed assets for the installation.
You must copy the bundle.zip file to the Nomad volume that will be used by the controllers.
Since teectl setup gen does not provide support for Nomad manifest files, you will need to write your own job definition with the necessary resources/stanzas and deploy a Traefik Enteprise cluster of one controller and two proxies.
Nomad Job Definition¶
# Saved as traefikee.json
{
"job": [
{
"traefikee": [
{
"datacenters": [
"dc1"
],
"group": [
{
"controllers": [
{
"count": 1,
"network": [
{
"port": [
{
"control": [
{
"static": 4242
}
]
},
{
"api": [
{
"static": 55055
}
]
}
]
}
],
"service": [
{
"check": [
{
"failures_before_critical": "2",
"interval": "30s",
"port": "control",
"timeout": "5s",
"type": "tcp"
}
],
"name": "traefikee-controllers",
"port": "control",
"task": "controllers"
}
],
"task": [
{
"controllers": [
{
"config": [
{
"args": [
"controller",
"--name=${NOMAD_ALLOC_NAME}",
"--advertise=${NOMAD_ADDR_control}",
"--discovery.static.peers=",
"--license=${TRAEFIKEE_LICENSE}",
"--statedir=/data",
"--jointoken.file.path=/data/tokens",
"--api.bundle=/data/bundle.zip",
"--socket=${NOMAD_TASK_DIR}/cluster.sock",
"--api.socket=${NOMAD_TASK_DIR}/api.sock",
"--log.level=",
"--log.filepath=",
"--log.format="
],
"cap_add": [
"NET_BIND_SERVICE"
],
"cap_drop": [
"ALL"
],
"image": "traefik/traefikee:v2.10.0",
"ports": [
"control",
"api"
]
}
],
"driver": "docker",
"resources": [
{
"cpu": 500,
"memory": 256
}
],
"template": [
{
"data": "TRAEFIKEE_LICENSE=\"{{'{{'}}with secret \"secret/traefikee/license\"{{'}}'}}{{'{{'}}.Data.data.license_key{{'}}'}}{{'{{'}}end{{'}}'}}\"\r\n",
"destination": "secrets/traefikee.env",
"env": true
}
],
"volume_mount": [
{
"destination": "/data",
"volume": "data"
}
]
}
]
}
],
"volume": [
{
"data": [
{
"source": "traefikee-data",
"type": "host"
}
]
}
]
}
]
},
{
"proxies": [
{
"count": 2,
"network": [
{
"mode": "host",
"port": [
{
"distributed": [
{
"static": 8484
}
]
},
{
"web": [
{}
]
},
{
"websecure": [
{}
]
}
]
}
],
"service": [
{
"name": "traefikee-proxies",
"port": "web",
"task": "proxies"
}
],
"task": [
{
"proxies": [
{
"config": [
{
"args": [
"proxy",
"--discovery.static.peers=traefikee-controllers:4242",
"--jointoken.value=${PROXY_JOIN_TOKEN}",
"--log.level=",
"--log.filepath=",
"--log.format="
],
"cap_add": [
"NET_BIND_SERVICE"
],
"cap_drop": [
"ALL"
],
"dns_servers": [
"10.5.0.2",
"${attr.unique.network.ip-address}",
"8.8.8.8"
],
"image": "traefik/traefikee:v2.10.0",
"ports": [
"web",
"websecure",
"distributed"
]
}
],
"driver": "docker",
"resources": [
{
"cpu": 500,
"memory": 256
}
],
"template": [
{
"data": "PROXY_JOIN_TOKEN=\"{{'{{'}}with secret \"secret/traefikee/tokens\"{{'}}'}}{{'{{'}}.Data.data.proxy_join{{'}}'}}{{'{{'}}end{{'}}'}}\"\r\n",
"destination": "secrets/traefikee.env",
"env": true
}
]
}
]
}
]
}
]
}
],
"type": "service",
"update": [
{
"max_parallel": 1,
"stagger": "30s"
}
]
}
]
}
]
}Customize the file to your needs then deploy:
nomad run traefikee.json==> Monitoring evaluation "0f10a2e6"
Evaluation triggered by job "traefikee"
Allocation "3713cdb9" created: node "23159a1c", group "controllers"
Allocation "8f330cfb" created: node "23159a1c", group "proxies"
Evaluation within deployment: "bc1568a8"
Allocation "3713cdb9" status changed: "pending" -> "running" (Tasks are running)
Evaluation status changed: "pending" -> "complete"
==> Evaluation "0f10a2e6" finished with status "complete"Check the status of the tasks:
nomad status traefikeeID = traefikee
Name = traefikee
Submit Date = 2020-11-16T15:11:05-03:00
Type = service
Priority = 50
Datacenters = dc1
Namespace = default
Status = running
Periodic = false
Parameterized = false
Summary
Task Group Queued Starting Running Failed Complete Lost
controllers 0 1 0 0 0 0
proxies 0 2 0 0 0 0
Latest Deployment
ID = bc1568a8
Status = running
Description = Deployment is running
Deployed
Task Group Desired Placed Healthy Unhealthy Progress Deadline
controllers 1 1 0 1 2020-11-16T18:21:05Z
proxies 2 1 0 0 2020-11-16T18:21:05Z
Allocations
ID Node ID Task Group Version Desired Status Created Modified
bddea080 713fa2e6 controllers 1 run pending 21s ago 18s ago
8f330cfb 23159a1c proxies 1 run pending 21s ago 18s agoYou should expect the Proxy instances to fail to start, staying on the "pending" status until you create the Vault secret containing the cluster join token.
Sharing the Proxy Join Token¶
Once the controller is up and running, you can fetch the proxy join token by running:
nomad alloc exec -i -t -task controllers bddea080 /traefikee tokens --socket local/cluster.sockexport TRAEFIKEE_CONTROLLER_TOKEN=5531644e5645744f4c5445744e47706f4e7a5a3261574a765a485274616a55345a57526f4d7a566f65444e6f4e544e70616d4a6d615459314e5464684f48526f4d6e6c735933466e4e33686a6348417459544e6c4e5451314e475a70616d4e304e484e796432466f4e445a796447707859673d3d3a303a97b42afa416f0df94d7c453bd4c55fcddd339ec4570068eac2c5cc5504c158d42521e9852615ab95049e2b3296b67c4a
export TRAEFIKEE_PROXY_TOKEN=5531644e5645744f4c5445744e47706f4e7a5a3261574a765a485274616a55345a57526f4d7a566f65444e6f4e544e70616d4a6d615459314e5464684f48526f4d6e6c735933466e4e33686a634841744e47397563484d31636a4d354d7a42354e5463785933557762546b774e7a55794f413d3d3a313a97b42afa416f0df94d7c453bd4c55fcddd339ec4570068eac2c5cc5504c158d42194b3059e6861761121966a44f964d3Create a Vault secret with the TRAEFIKEE_PROXY_TOKEN value:
vault kv put secret/traefikee/tokens proxy_join=$TRAEFIKEE_PROXY_TOKENOnce the secret is created, the proxy instances should start and join the cluster. Verify that the Traefik Enterprise installation was successful:
teectl get nodesID NAME STATUS ROLE
mmc2qofw8hkudf8vcrs1k8cnw controller-0 Ready Controller (Leader)
oncgmk54uuipcj7jnt0lhwwqy proxy-0 Ready Proxy
u122z00fgr2cqrp78jc5em0fi proxy-1 Ready ProxyWhat Next?
Now that the cluster is ready, you should read the various operating guides to dive into all the features that Traefik Enterprise provides.
License Monitoring¶
When a Traefik Enterprise controller starts for the first time, it checks the license validity.
If the license is valid, another check is done once every 24 hours.
If the controller can't communicate with the license server, a 72-day grace period starts to recover from this situation.
Once the grace period is over, the controller stops updating the proxies configuration.
Please look at the FAQ to know how to implement the license monitoring.