Skip to content

Installing Traefik Enterprise on HashiCorp Nomad

This page guides you through the installation of Traefik Enterprise on HashiCorp Nomad.

Nomad Knowledge

Assistance with configuring or setting up a Nomad cluster is not included in this guide. If you need more information about Nomad, Consul, Vault, or any other components referenced in this guide, start with the following resources:

Requirements

  • A Nomad cluster
    • Version 0.12.7 or greater
    • A persistent storage volume for the controllers
    • The nomad CLI tool properly configured to communicate with the cluster
    • Consul integration for service discovery
    • Vault integration for secrets
  • Controller containers can reach https://v4.license.containous.cloud
  • The teectl CLI tool, for cluster management

Installation

This guide will only focus on the Docker task driver, although it is also possible to run with Fork/Exec with a few changes that can be derived from our On-Premise guide

Nomad Preparation

Persistent Volume

Create a persistent volume, either by relying on a CSI storage plugin or by preselecting the nodes that will run controller tasks and binding a directory from the host as a volume.

For simplicity's sake, this guide will use the host volume approach. In this case, the directory must be present before you can expose it.

# Execute on each Nomad node that will run a controller instance
mkdir -p /opt/traefikee

After creating the directory, add a host_volume bind on the Nomad client configuration of each node:

client {
  host_volume "traefikee-data" {
    path = "/opt/traefikee"
  }
}

Service Discovery

To avoid specifying the Consul address manually on each Task network block, you can add it to the DNS resolution configuration on each Nomad host. Head to the DNS forwarding guide for Consul for more information.

License Secret

Create a secret in Hashicorp Vault with your license key:

vault kv put secret/traefikee/license license_key=$TRAEFIKEE_LICENSE

Generating the teectl Configuration

Install the teectl management tool on a machine with access to the nodes that will host controller instances by following the instructions in this guide.

Once teectl is installed, it is possible to generate a set of credentials that enables remote access to the cluster API:

# Replace ipX with the Nomad Client Node address that will host Traefik Enterprise controller instances
teectl setup --onpremise.hosts="<ip1>,<ip2>,<ip3>"

This command creates a new teectl configuration under ~/.config/traefikee/default.yaml and generates a bundle.zip that carries a set of needed assets for the installation.

You must copy the bundle.zip file to the Nomad volume that will be used by the controllers.

Since teectl setup gen does not provide support for Nomad manifest files, you will need to write your own job definition with the necessary resources/stanzas and deploy a Traefik Enteprise cluster of one controller and two proxies.

Nomad Job Definition

# Saved as traefikee.json
{
  "job": [
    {
      "traefikee": [
        {
          "datacenters": [
            "dc1"
          ],
          "group": [
            {
              "controllers": [
                {
                  "count": 1,
                  "network": [
                    {
                      "port": [
                        {
                          "control": [
                            {
                              "static": 4242
                            }
                          ]
                        },
                        {
                          "api": [
                            {
                              "static": 55055
                            }
                          ]
                        }
                      ]
                    }
                  ],
                  "service": [
                    {
                      "check": [
                        {
                          "failures_before_critical": "2",
                          "interval": "30s",
                          "port": "control",
                          "timeout": "5s",
                          "type": "tcp"
                        }
                      ],
                      "name": "traefikee-controllers",
                      "port": "control",
                      "task": "controllers"
                    }
                  ],
                  "task": [
                    {
                      "controllers": [
                        {
                          "config": [
                            {
                              "args": [
                                "controller",
                                "--name=${NOMAD_ALLOC_NAME}",
                                "--advertise=${NOMAD_ADDR_control}",
                                "--discovery.static.peers=",
                                "--license=${TRAEFIKEE_LICENSE}",
                                "--statedir=/data",
                                "--jointoken.file.path=/data/tokens",
                                "--api.bundle=/data/bundle.zip",
                                "--socket=${NOMAD_TASK_DIR}/cluster.sock",
                                "--api.socket=${NOMAD_TASK_DIR}/api.sock",
                                "--log.level=",
                                "--log.filepath=",
                                "--log.format="
                              ],
                              "cap_add": [
                                "NET_BIND_SERVICE"
                              ],
                              "cap_drop": [
                                "ALL"
                              ],
                              "image": "traefik/traefikee:v2.6.0",
                              "ports": [
                                "control",
                                "api"
                              ]
                            }
                          ],
                          "driver": "docker",
                          "resources": [
                            {
                              "cpu": 500,
                              "memory": 256
                            }
                          ],
                          "template": [
                            {
                              "data": "TRAEFIKEE_LICENSE=\"{{'{{'}}with secret \"secret/traefikee/license\"{{'}}'}}{{'{{'}}.Data.data.license_key{{'}}'}}{{'{{'}}end{{'}}'}}\"\r\n",
                              "destination": "secrets/traefikee.env",
                              "env": true
                            }
                          ],
                          "volume_mount": [
                            {
                              "destination": "/data",
                              "volume": "data"
                            }
                          ]
                        }
                      ]
                    }
                  ],
                  "volume": [
                    {
                      "data": [
                        {
                          "source": "traefikee-data",
                          "type": "host"
                        }
                      ]
                    }
                  ]
                }
              ]
            },
            {
              "proxies": [
                {
                  "count": 2,
                  "network": [
                    {
                      "mode": "host",
                      "port": [
                        {
                          "distributed": [
                            {
                              "static": 8484
                            }
                          ]
                        },
                        {
                          "web": [
                            {}
                          ]
                        },
                        {
                          "websecure": [
                            {}
                          ]
                        }
                      ]
                    }
                  ],
                  "service": [
                    {
                      "name": "traefikee-proxies",
                      "port": "web",
                      "task": "proxies"
                    }
                  ],
                  "task": [
                    {
                      "proxies": [
                        {
                          "config": [
                            {
                              "args": [
                                "proxy",
                                "--discovery.static.peers=traefikee-controllers:4242",
                                "--jointoken.value=${PROXY_JOIN_TOKEN}",
                                "--log.level=",
                                "--log.filepath=",
                                "--log.format="
                              ],
                              "cap_add": [
                                "NET_BIND_SERVICE"
                              ],
                              "cap_drop": [
                                "ALL"
                              ],
                              "dns_servers": [
                                "10.5.0.2",
                                "${attr.unique.network.ip-address}",
                                "8.8.8.8"
                              ],
                              "image": "traefik/traefikee:v2.6.0",
                              "ports": [
                                "web",
                                "websecure",
                                "distributed"
                              ]
                            }
                          ],
                          "driver": "docker",
                          "resources": [
                            {
                              "cpu": 500,
                              "memory": 256
                            }
                          ],
                          "template": [
                            {
                              "data": "PROXY_JOIN_TOKEN=\"{{'{{'}}with secret \"secret/traefikee/tokens\"{{'}}'}}{{'{{'}}.Data.data.proxy_join{{'}}'}}{{'{{'}}end{{'}}'}}\"\r\n",
                              "destination": "secrets/traefikee.env",
                              "env": true
                            }
                          ]
                        }
                      ]
                    }
                  ]
                }
              ]
            }
          ],
          "type": "service",
          "update": [
            {
              "max_parallel": 1,
              "stagger": "30s"
            }
          ]
        }
      ]
    }
  ]
}

Customize the file to your needs then deploy:

nomad run traefikee.json
==> Monitoring evaluation "0f10a2e6"
    Evaluation triggered by job "traefikee"
    Allocation "3713cdb9" created: node "23159a1c", group "controllers"
    Allocation "8f330cfb" created: node "23159a1c", group "proxies"
    Evaluation within deployment: "bc1568a8"
    Allocation "3713cdb9" status changed: "pending" -> "running" (Tasks are running)
    Evaluation status changed: "pending" -> "complete"
==> Evaluation "0f10a2e6" finished with status "complete"

Check the status of the tasks:

nomad status traefikee
ID            = traefikee
Name          = traefikee
Submit Date   = 2020-11-16T15:11:05-03:00
Type          = service
Priority      = 50
Datacenters   = dc1
Namespace     = default
Status        = running
Periodic      = false
Parameterized = false

Summary
Task Group   Queued  Starting  Running  Failed  Complete  Lost
controllers  0       1         0        0       0         0
proxies      0       2         0        0       0         0

Latest Deployment
ID          = bc1568a8
Status      = running
Description = Deployment is running

Deployed
Task Group   Desired  Placed  Healthy  Unhealthy  Progress Deadline
controllers  1        1       0        1          2020-11-16T18:21:05Z
proxies      2        1       0        0          2020-11-16T18:21:05Z

Allocations
ID        Node ID   Task Group   Version  Desired  Status    Created    Modified
bddea080  713fa2e6  controllers  1        run      pending   21s ago    18s ago
8f330cfb  23159a1c  proxies      1        run      pending   21s ago    18s ago

You should expect the Proxy instances to fail to start, staying on the "pending" status until you create the Vault secret containing the cluster join token.

Sharing the Proxy Join Token

Once the controller is up and running, you can fetch the proxy join token by running:

nomad alloc exec -i -t -task controllers bddea080 /traefikee tokens --socket local/cluster.sock
export TRAEFIKEE_CONTROLLER_TOKEN=5531644e5645744f4c5445744e47706f4e7a5a3261574a765a485274616a55345a57526f4d7a566f65444e6f4e544e70616d4a6d615459314e5464684f48526f4d6e6c735933466e4e33686a6348417459544e6c4e5451314e475a70616d4e304e484e796432466f4e445a796447707859673d3d3a303a97b42afa416f0df94d7c453bd4c55fcddd339ec4570068eac2c5cc5504c158d42521e9852615ab95049e2b3296b67c4a
export TRAEFIKEE_PROXY_TOKEN=5531644e5645744f4c5445744e47706f4e7a5a3261574a765a485274616a55345a57526f4d7a566f65444e6f4e544e70616d4a6d615459314e5464684f48526f4d6e6c735933466e4e33686a634841744e47397563484d31636a4d354d7a42354e5463785933557762546b774e7a55794f413d3d3a313a97b42afa416f0df94d7c453bd4c55fcddd339ec4570068eac2c5cc5504c158d42194b3059e6861761121966a44f964d3

Create a Vault secret with the TRAEFIKEE_PROXY_TOKEN value:

vault kv put secret/traefikee/tokens proxy_join=$TRAEFIKEE_PROXY_TOKEN

Once the secret is created, the proxy instances should start and join the cluster. Verify that the Traefik Enterprise installation was successful:

teectl get nodes
ID                         NAME           STATUS  ROLE
mmc2qofw8hkudf8vcrs1k8cnw  controller-0   Ready   Controller (Leader)
oncgmk54uuipcj7jnt0lhwwqy  proxy-0        Ready   Proxy
u122z00fgr2cqrp78jc5em0fi  proxy-1        Ready   Proxy

What Next?

Now that the cluster is ready, you should read the various operating guides to dive into all the features that Traefik Enterprise provides.