Advanced Installation of Traefik Enterprise Edition with One Control Node on Kubernetes with YAML

This installation guide is for experts who want to fine-tune their TraefikEE (Traefik Enterprise Edition) installation.

It covers how to install TraefikEE as a Kubernetes Ingress Controller using Kubernetes YAML files .

Kubernetes Knowledge

Assistance with configuring or setting up a Kubernetes cluster is not included in this guide. If you need more information about Kubernetes, start with the following resources:

• Kubernetes Concepts
• Kubernetes Basics
• Learning Kubernetes with tutorials
• Setting up Kubernetes

Requirements

• The traefikeectl tool installed
• A Kubernetes cluster:
  • Supported versions: 1.10, 1.11, 1.12 and 1.13
  • RBAC enabled (recommended),
  • Access to the Kubernetes API with kubectl, with the ability to create and manage namespaces and their resources.
• Control node pod can reach https://v3.license.containous.cloud:8443.

Create the Namespace

Create a namespace named traefikee to host the TraefikEE installation:

Command

kubectl apply -f https://s3.amazonaws.com/traefikee/examples/v1.0.7/kubernetes/one-cn/namespace.yaml

Expected Output

namespace/traefikee created

Prepare Role Based Access Control (RBAC)

Create the RBAC objects to allow TraefikEE pods to interact with the Kubernetes API:

Command

kubectl apply -f https://s3.amazonaws.com/traefikee/examples/v1.0.7/kubernetes/one-cn/rbac-singlecn.yaml

Expected Output

serviceaccount/traefikee created
clusterrole.rbac.authorization.k8s.io/traefikee-role created
clusterrolebinding.rbac.authorization.k8s.io/traefikee created

Note

The following Service Account is created:

• serviceaccount/traefikee, to access the Kubernetes API actions required by TraefikEE.

Create the Service

Create the services to allow network access to the TraefikEE cluster:

Command

kubectl apply -f https://s3.amazonaws.com/traefikee/examples/v1.0.7/kubernetes/one-cn/data-node-external-service.yaml
kubectl apply -f https://s3.amazonaws.com/traefikee/examples/v1.0.7/kubernetes/one-cn/control-node-internal-service.yaml
kubectl apply -f https://s3.amazonaws.com/traefikee/examples/v1.0.7/kubernetes/one-cn/cluster-internal-api-service.yaml

Expected Output

service "traefikee-lb" created
service "traefikee-control-nodes" created
service "traefikee-api" created

Note

The following services are created:

• traefikee-lb handles the traffic of the ingress rules for your applications.
• traefikee-control-nodes handles the traffic for control nodes internal communication.
• traefikee-api handles the traffic for TraefikEE's API internal usage such as metric collection or Web UI.

Create the Control Node

Installing behind a proxy

In order to install a behind proxy, you must define the HTTP_PROXY, HTTPS_PROXY and NO_PROXY environment variables for the control node. As the control node will make requests to the kubernetes API server, the NO_PROXY variable must contain its IP address. It can be found using kubectl get service kubernetes.

containers:
  - name: ...
    # [...]
    env:
      - name: HTTP_PROXY
        value: http://127.0.0.1:3129 # Your HTTP proxy URL
      - name: HTTPS_PROXY
        value: http://127.0.0.1:3129 # Your HTTPS proxy URL
      - name: NO_PROXY
        value: 100.64.0.1 # IP of the kubernetes service

Download the file control-node-statefulset.yaml:

Command

curl -sSLO \
  https://s3.amazonaws.com/traefikee/examples/v1.0.7/kubernetes/one-cn/control-node-statefulset.yaml

• Insert your license key in the file control-node-statefulset.yaml:

  • Search for line --licensekey=fakeLicense
  • Replace the value fakeLicense with your license key

• Save the file

Create a control node to initialize the cluster:

Command

kubectl apply -f ./control-node-statefulset.yaml

Expected Output

statefulset.apps/traefikee-control-node created

Note

Alternatively, you can use an environment variable, and set the license key without changing the file:

Command

export TRAEFIKEE_LICENSE_KEY="YOUR-LICENSE_KEY-VALUE"
curl -sSL https://s3.amazonaws.com/traefikee/examples/v1.0.7/kubernetes/one-cn/control-node-statefulset.yaml \
| sed "s/--licensekey=.*/--licensekey=${TRAEFIKEE_LICENSE_KEY}\"/g" \
| kubectl apply -f -

Connect traefikeectl to the New Cluster

About a minute after applying the control node statefulset, you should be able to connect to the cluster.

Configure traefikeectl to have access to the new cluster.

Command

traefikeectl connect --kubernetes

Expected Output

Retrieving TraefikEE Control credentials...ok
Removing cluster credentials from platform...ok
Credentials saved in "$HOME/.config/traefikee/traefikee", please make sure to keep them safe as they can never be retrieved again.
✔ Successfuly gained access to the cluster. You can now use other traefikeectl commands.

In case of an error

If traefikeectl connect displays the following error: unable to retrieve API credentials, it might be because the control node isn't done creating your cluster yet. Wait a few seconds and try again.

One-time operation

When running traefikeectl connect, your credentials will be retrieved and it will not be possible to do it again in the future without re-installing a TraefikEE cluster. Remember to keep your credentials safe!

Check if the bootstrap node started correctly by listing the cluster nodes.

The bootstrap node's role should be CONTROL NODE, and its availability should be ACTIVE:

Command

traefikeectl list-nodes

Expected Output

Name                                  Availability  Role          Leader
----                                  ------------  ----          ------
traefikee-control-node-0              ACTIVE        CONTROL NODE  YES

Create the Token Secret

Get the data node token generated by the bootstrap node, and put it into the environment variable DATA_NODE_TOKEN:

Command

export DATA_NODE_TOKEN="$(kubectl exec -t \
  --namespace=traefikee \
  $(kubectl get pods --namespace traefikee --selector=app=traefikee,component=control-nodes --output jsonpath="{.items[*].metadata.name}") \
  -- /traefikee env | grep 'DATA_NODE' | cut -d '"' -f2)"

Validate that the environment contains the data node token:

Command

printenv DATA_NODE_TOKEN

Expected Output

...6645376267354e51734d59336a457369712...

Download the file cluster-secret.yaml:

Command

curl -sSLO \
  https://s3.amazonaws.com/traefikee/examples/v1.0.7/kubernetes/one-cn/hooks/cluster-secret.yaml

From the content of the file cluster-secret.yaml, insert the value of the token, and apply the modified content to Kubernetes:

Command

cat cluster-secret.yaml \
  | sed "s/data-node:.*$/data-node: $DATA_NODE_TOKEN/" \
  | kubectl apply -f -

Expected Output

secret/traefikee-tokens configured

Ensure that the secret named traefikee-tokens, of type "Opaque" exists in Kubernetes:

Command

kubectl get secrets --namespace=traefikee

Expected Output

NAME                 TYPE      DATA      AGE
...
traefikee-tokens     Opaque    1         52s
...

Note

You can retrieve the base64-encoded value of the data node token stored in Kubernetes with the following command:

kubectl get secret --namespace=traefikee \
    traefikee-tokens --output="jsonpath={.data.data-node}"

Allow TraefikEE to Generate Token Secrets Automatically

When the TraefikEE bootstrap node is run in kubernetes, it will automatically attempt to connect to the Kubernetes API, and generate the token secret without requiring user interaction. It will create the secret in the same namespace as the bootstrap with the same traefikee-tokens name as in the guide above.

You may not have to manually generate these tokens and this secret if your RBAC for the bootstrap node allows secret creation.

Note

By allowing the bootstrap node to create this secret, you can reduce the number of steps needed to bootstrap your cluster.

Create Data Nodes

Create the data nodes, to handle your application traffic:

Command

kubectl apply -f https://s3.amazonaws.com/traefikee/examples/v1.0.7/kubernetes/one-cn/data-node-deployment.yaml

Expected Output

deployment.apps "traefikee-data-node" created

Verify that the 2 data nodes are running:

Command

kubectl get pod --namespace=traefikee --selector='app=traefikee' --selector='component=data-nodes'

Expected Output

NAME                                   READY     STATUS    RESTARTS   AGE
traefikee-data-node-867cc84788-j78qg   1/1       Running   0          50s
traefikee-data-node-867cc84788-s9qwm   1/1       Running   0          50s

Validate that the TraefikEE cluster lists the 2 data nodes as members, with an availability status of "ACTIVE":

Command

traefikeectl list-nodes

Expected Output

Name                                  Availability  Role          Leader
----                                  ------------  ----          ------
traefikee-control-node-0              ACTIVE        CONTROL NODE  YES
traefikee-data-node-867cc84788-j78qg  ACTIVE        DATA NODE
traefikee-data-node-867cc84788-j78qg  ACTIVE        DATA NODE

Backup your Installation

Don't forget to setup regular backups using the traefikeectl backup command. More information can be found in the backup and restore documentation.