Installing Traefik Enterprise on ECS Fargate

This page guides you through the installation of Traefik Enterprise on AWS ECS Fargate.

AWS Knowledge

Information about setting up AWS services is not included in this guide. If you want to know more about any of the components referenced in this guide, start with the following resources:

• Service Discovery
• Secret Manager
• Security Groups
• IAM Policies and Permissions
• CloudWatch

In this guide all examples surrounded by angle brackets <>, must be replaced accordingly.

Requirements

• An ECS Fargate cluster
  • The aws CLI tool properly configured to communicate with the cluster
  • Enough permissions on the ECS Fargate instance role to access any integrated AWS service
  • All required ports are open on the associated security group of the container instances
• Controller containers can reach https://v4.license.containous.cloud
• The teectl CLI tool, for cluster management

Installation

Although some external AWS resources are referenced in the logConfiguration and secrets sections, they are optional. You can just remove or customize them in the task definition. Otherwise, you must create those resources manually with the AWS Console or CLI tool which is not covered in this guide as well.

Since teectl setup gen does not provide support for ECS Fargate manifest files, this guide demonstrates how to write your own task and service definitions, using JSON syntax, to define the necessary resources and deploy a Traefik Enterprise cluster of one controller and two proxies.

Note that flags used in each task definitions are referenced here: Traefikee Command-Line Reference

IAM Roles and Policy

Traefik Enterprise requires the 2 different roles to run, one for the task execution and one when the task is running. Roles needs will map policies, so we gonna need to create the policies first.

Execution role

AWS already have a built in policy for the execution policy named AmazonECSTaskExecutionRolePolicy. We can then create the role and attach the policy to it. Let's create a trust relashionships definition in a file named trust-policy.json, and put it this content:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "",
            "Effect": "Allow",
            "Principal": {
                "Service": "ecs-tasks.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

We create the role that we could name ecsTaskExecutionRole with this aws cli command:

aws iam create-role --role-name ecsTaskExecutionRole --assume-role-policy-document file://trust-policy.json 

And we associate the policy:

aws iam attach-role-policy --policy-arn arn:aws:iam::aws:policy/AmazonECSTaskExecutionRolePolicy --role-name ecsTaskExecutionRole

Use of Secret

If you want to use secret to pass your license, and the token as environement variables you need to add a policy to the ecsTaskExecutionRole in order to let the task be able to read the secret.

Here's an example for the secret read policy, that you only need to associate with the execution role:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "secretsmanager:GetSecretValue"
      ],
      "Resource": [
        "<secret_arn>"
      ]
    }
  ]
}

Running role

As traefik need to access the ECS API for the service discovery, we will need to create a specific policy. To create the policy in AWS you could create a json file with the content below:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "TraefikECSReadAccess",
            "Effect": "Allow",
            "Action": [
                "ecs:ListClusters",
                "ecs:DescribeClusters",
                "ecs:ListTasks",
                "ecs:DescribeTasks",
                "ecs:DescribeContainerInstances",
                "ecs:DescribeTaskDefinition",
                "ec2:DescribeInstances",
                "logs:CreateLogGroup",
                "logs:CreateLogStream",
                "logs:PutLogEvents",
                "logs:DescribeLogStreams"
            ],
            "Resource": [
                "*"
            ]
        },
        {
            "Sid": "TraefikECSContainerExec",
            "Effect": "Allow",
            "Action": [
                "ssmmessages:CreateControlChannel",
                "ssmmessages:CreateDataChannel",
                "ssmmessages:OpenControlChannel",
                "ssmmessages:OpenDataChannel"
            ],
            "Resource": [
                "*"
            ]
        }
    ]
}

Then create the policy with this command(change the name of the policy and file accordingly):

aws iam create-policy --policy-name TraefikECSReadAccessRolePolicy --policy-document file://TraefikECSReadAccessRolePolicy.json

Apply the same process as above to create a new role and associate the policy.

Service Discovery

This deployment use AWS Service discovery to manage internal DNS name for Traefik Enterprise component.

As stated before the AWS Service discovery is documented by AWS and will not be described here in details, this section will just give you the command to create a Service Discovery namespace to deploy Traefik Enterprise.

Create a Service Discovery namespace

aws servicediscovery create-private-dns-namespace --name traefikee --vpc <vpc-id>

Create a Service Discovery Service for Traefik Enterprise controller

aws servicediscovery create-service --name controller-0 --dns-config 'NamespaceId="<ns-id>",DnsRecords=[{Type="A",TTL="60"}]' --health-check-custom-config FailureThreshold=1

Create a Service Discovery Service for Traefik Enterprise registry

aws servicediscovery create-service --name registry --dns-config 'NamespaceId="<ns-id>",DnsRecords=[{Type="A",TTL="60"}]' --health-check-custom-config FailureThreshold=1

In case of many controllers, you will need to set up new services for each controller name accordingly.

Use of Secret

If you want to use secret to pass your license, and the token as environement variables you need to add a policy to the ecsTaskExecutionRole in order to let the task be able to read the secret.

here's an example for this policy:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "secretsmanager:GetSecretValue"
      ],
      "Resource": [
        "<secret_arn>"
      ]
    }
  ]
}

Traefik Enterprise will look for specific environment variable during startup, including for its license key and cluster tokens as follows:

• TRAEFIKEE_LICENSE
• TRAEFIKEE_JOIN_TOKEN
• TRAEFIKEE_PLUGIN_TOKEN

However in ECS Fargate environment variables are not expanded at the container startup, which means you can't set command flags with environment variables. In the task definition in order to use the secret as environment varialbe we only will need to remove flags defining those values.

Plugin Registry Token

The plugin registry needs a token to secure its communications with the controller. This token is set on the controller and plugin registry task definition.

Here is an example of how to create it:

Command

openssl rand -base64 10

Expected output

MvXVeX3qDylxJQ==

Controllers

Mono controller installation

Controller Storage

Fargate by default assign non-persistent storage to the container, in order to keep state, certificate, and necessary data, if the controller container needs to be recreated you need to add an EFS volume as described on the task definition below.

Create a file named controller-task.json with the following task definition:

Without Secret

{
    "family": "traefikee-controller",
    "taskRoleArn": "arn:aws:iam::<your_account_id>:role/RoleTraefikECSReadAccess",
    "executionRoleArn": "arn:aws:iam::<your_account_id>:role/ecsTaskExecutionRole",
    "cpu": "512",
    "memory": "1024",
    "containerDefinitions": [
        {
            "name": "controller-0",
            "image": "traefik/traefikee:v2.10.0",
            "cpu": 500,
            "portMappings": [
                {
                    "containerPort": 55055,
                    "hostPort": 55055,
                    "protocol": "tcp"
                }
            ],
            "essential": true,
            "command": [
                "controller",
                "--name=controller-0",
                "--advertise=controller-0.traefikee:4242",
                "--license=<your_license>",
                "--statedir=/data/state",
                "--jointoken.file.path=/data/tokens",
                "--api.socket=/var/run/traefikee/teectl-controller-0.sock",
                "--socket=/var/run/traefikee/controller-0.sock",
                "--api.autocerts",
                "--plugin.url=https://registry.traefikee:443",
                "--plugin.token=<your_generated_plugin_token>"
            ],
            "linuxParameters": {
                "initProcessEnabled": true
            },
            "environment": [],
            "secrets": [],
            "volumesFrom": [],
            "dockerLabels": {
                "com.traefik.traefikee.component": "controller"
            },
            "mountPoints": [
                {
                  "readOnly": false,
                  "containerPath": "/data",
                  "sourceVolume": "traefikee-data"
                }
            ],
            "readonlyRootFilesystem": false,
            "privileged": false,
            "logConfiguration": {
                "logDriver": "awslogs",
                "options": {
                    "awslogs-group": "<your_cloud_watch_log_group",
                    "awslogs-region": "<your_aws_region>",
                    "awslogs-stream-prefix": "traefikee-controller"
                }
        }
    }
    ],
    "volumes": [
        {
          "name": "traefikee-data",
          "efsVolumeConfiguration": {
            "fileSystemId": "<controller_efs_id>",
            "transitEncryption": "DISABLED",
            "rootDirectory": "/"
          }
        }
    ],
    "networkMode": "awsvpc",
    "runtimePlatform": {
        "cpuArchitecture": "X86_64",
        "operatingSystemFamily": "LINUX"
    },

    "requiresCompatibilities": [
        "FARGATE"
    ]
}

With Secret

{
    "family": "traefikee-controller",
    "taskRoleArn": "arn:aws:iam::<your_account_id>:role/RoleTraefikECSReadAccess",
    "executionRoleArn": "arn:aws:iam::<your_account_id>:role/ecsTaskExecutionRole",
    "cpu": "512",
    "memory": "1024",
    "containerDefinitions": [
        {
            "name": "controller-0",
            "image": "traefik/traefikee:v2.10.0",
            "cpu": 500,
            "portMappings": [
                {
                    "containerPort": 55055,
                    "hostPort": 55055,
                    "protocol": "tcp"
                }
            ],
            "essential": true,
            "command": [
                "controller",
                "--name=controller-0",
                "--advertise=controller-0.traefikee:4242",
                "--statedir=/data/state",
                "--jointoken.file.path=/data/tokens",
                "--api.socket=/var/run/traefikee/teectl-controller-0.sock",
                "--socket=/var/run/traefikee/controller-0.sock",
                "--api.autocerts",
                "--plugin.url=https://registry.traefikee:443"
            ],
            "linuxParameters": {
                "initProcessEnabled": true
            },
            "environment": [],
            "secrets": [
                {
                    "name": "TRAEFIKEE_LICENSE",
                    "valueFrom": "arn:aws:secretsmanager:<your_aws_region>:<your_account_id>:secret:<your_secret_name>:<traefik_license>::"
                },
                {
                    "name": "TRAEFIKEE_PLUGIN_TOKEN",
                    "valueFrom": "arn:aws:secretsmanager:<your_aws_region>:<your_account_id>:secret:<your_secret_name>:<traefik_plugin_token_key>::"
                }
            ],
            "volumesFrom": [],
            "dockerLabels": {
                "com.traefik.traefikee.component": "controller"
            },
            "mountPoints": [
                {
                    "readOnly": false,
                    "containerPath": "/data",
                    "sourceVolume": "traefikee-data"
                }
            ],
            "readonlyRootFilesystem": false,
            "privileged": false,
            "logConfiguration": {
                "logDriver": "awslogs",
                "options": {
                    "awslogs-group": "<your_cloud_watch_log_group",
                    "awslogs-region": "<your_aws_region>",
                    "awslogs-stream-prefix": "traefikee-controller"
                }
            }
        }
    ],
    "volumes": [
        {
            "name": "traefikee-data",
            "efsVolumeConfiguration": {
                "fileSystemId": "<controller_efs_id>",
                "transitEncryption": "DISABLED",
                "rootDirectory": "/"
            }
        }
    ],
    "networkMode": "awsvpc",
    "runtimePlatform": {
        "cpuArchitecture": "X86_64",
        "operatingSystemFamily": "LINUX"
    },
    "requiresCompatibilities": [
        "FARGATE"
    ]
}

Customize the file, then register the task definition on ECS Fargate:

Command

aws ecs register-task-definition --cli-input-json file://controller-task.json --region <aws_region>

Now create a file named controller-svc.json with the service definition:

{
    "serviceName": "traefikee-controller",
    "taskDefinition": "traefikee-controller",
    "desiredCount": 1,
    "launchType": "FARGATE",
    "schedulingStrategy": "REPLICA",
    "serviceRegistries": [
        {
           "registryArn": "<service_discovery_controller_service_arn>"
        }
    ],
    "networkConfiguration": {
        "awsvpcConfiguration": {
            "subnets": [
                "<subnet_id_a>",
                "<subnet_id_b>",
                "<subnet_id_c>"
            ],
            "securityGroups": [
                "<security_group_id>"
            ],
            "assignPublicIp": "ENABLED"
        }
    }
}

Customize the file and deploy it to the ECS Fargate cluster:

aws ecs create-service --cli-input-json file://controller-svc.json --cluster <my_cluster> --region <aws_region> --enable-execute-command

Check the service status to ensure the service was started:

Command

aws ecs describe-services --services traefikee-controller --cluster <my_cluster> --region <aws_region>

Expected output

{
    "services": [
        {
            "status": "ACTIVE",
            "serviceRegistries": [],
            "pendingCount": 0,
            "launchType": "FARGATE",
            "enableECSManagedTags": true,
            "schedulingStrategy": "REPLICA",
            "loadBalancers": [],
            "placementConstraints": [
                {
                    "type": "distinctInstance"
                }
            ],
            "createdAt": 1605815767.081,
            "desiredCount": 1,
            "serviceName": "traefikee-controller",
            "clusterArn": "arn:aws:ecs:aws_region:aws_account_id:cluster/traefikee-ecs",
            "createdBy": "arn:aws:iam::aws_account_id:user/aws_user",
            "taskDefinition": "arn:aws:ecs:aws_region:aws_account_id:task-definition/traefikee-controller:7",
            "serviceArn": "arn:aws:ecs:aws_region:aws_account_id:service/traefikee-ecs/traefikee-controller",
            "propagateTags": "SERVICE",
            "deploymentConfiguration": {
                "maximumPercent": 200,
                "minimumHealthyPercent": 100
            },
            "deployments": [
                {
                    "status": "PRIMARY",
                    "pendingCount": 0,
                    "launchType": "EC2",
                    "createdAt": 1605815767.081,
                    "desiredCount": 1,
                    "taskDefinition": "arn:aws:ecs:aws_region:aws_account_id:task-definition/traefikee-controller:7",
                    "updatedAt": 1605815785.903,
                    "id": "ecs-svc/4291651818662396701",
                    "runningCount": 1
                }
            ],
            "events": [
                {
                    "message": "(service traefikee-controller) has reached a steady state.",
                    "id": "2ae7ae80-11bc-48fa-bd1c-ec6235521928",
                    "createdAt": 1605815785.907
                },
                {
                    "message": "(service traefikee-controller) has started 1 tasks: (task fe932a1e5690403eacd5d3b4f8221b18).",
                    "id": "901615c4-8e67-4ed2-91f6-b52fcc88d147",
                    "createdAt": 1605815774.564
                }
            ],
            "runningCount": 1,
            "placementStrategy": []
        }
    ],
    "failures": []
}

Wait until the controllers are up and running before proceeding to the next steps.

Get the Proxy Join Token

Fetch the proxy join token by connecting to the container instance and executing the following commands:

Command

# Get the tokens
aws ecs execute-command --cluster <my_cluster> --container controller-0 --task <task-arn> --interactive --command "/traefikee tokens --socket /var/run/traefikee/controller-0.sock"

Expected output

export TRAEFIKEE_CONTROLLER_TOKEN=5531644e5645744f4c5445744e47706f4e7a5a3261574a765a485274616a55345a57526f4d7a566f65444e6f4e544e70616d4a6d615459314e5464684f48526f4d6e6c735933466e4e33686a6348417459544e6c4e5451314e475a70616d4e304e484e796432466f4e445a796447707859673d3d3a303a97b42afa416f0df94d7c453bd4c55fcddd339ec4570068eac2c5cc5504c158d42521e9852615ab95049e2b3296b67c4a
export TRAEFIKEE_PROXY_TOKEN=5531644e5645744f4c5445744e47706f4e7a5a3261574a765a485274616a55345a57526f4d7a566f65444e6f4e544e70616d4a6d615459314e5464684f48526f4d6e6c735933466e4e33686a634841744e47397563484d31636a4d354d7a42354e5463785933557762546b774e7a55794f413d3d3a313a97b42afa416f0df94d7c453bd4c55fcddd339ec4570068eac2c5cc5504c158d42194b3059e6861761121966a44f964d3

Write down the proxy token as it is required for the next step, the proxies task definition.

Multi controller Installation

Controller Storage

Fargate by default assign non-persistent storage to the container, in order to keep state, certificate, and necessary data, if the controllers needs to be recreated you need to add an EFS volume for each controller as described on the tasks definitions below.

Note that multi controller should not be deployed without secret, each controller will need the a join token if it is recreated or restarted. The environment variable that will be checked by the controller at bootstrap is TRAEFIKEE_JOIN_TOKEN.

In order to manage each controller independently in case of maintenance, it is recommended to create each and every controller as separated tasks.

After deploying the first controller get the join token and update secret accordingly.

Here is the example of the configuration for a 3 nodes controller cluster:

Controller-0

{
    "family": "traefikee-controller-0",
    "taskRoleArn": "arn:aws:iam::<your_account_id>:role/RoleTraefikECSReadAccess",
    "executionRoleArn": "arn:aws:iam::<your_account_id>:role/ecsTaskExecutionRole",
    "cpu": "512",
    "memory": "1024",
    "containerDefinitions": [
        {
            "name": "controller-0",
            "image": "traefik/traefikee:v2.10.0",
            "cpu": 500,
            "portMappings": [
                {
                    "containerPort": 55055,
                    "hostPort": 55055,
                    "protocol": "tcp"
                }
            ],
            "essential": true,
            "command": [
                "controller",
                "--name=controller-0",
                "--advertise=controller-0.traefikee:4242",
                "--discovery.static.peers=controller-1.traefikee:4242,controller-2.traefikee:4242",
                "--statedir=/data/state",
                "--jointoken.file.path=/data/tokens",
                "--api.socket=/var/run/traefikee/teectl-controller-0.sock",
                "--socket=/var/run/traefikee/controller-0.sock",
                "--api.autocerts",
                "--plugin.url=https://registry.traefikee:443"
            ],
            "linuxParameters": {
                "initProcessEnabled": true
            },
            "environment": [],
            "secrets": [
                                {
                    "name": "TRAEFIKEE_LICENSE",
                    "valueFrom": "arn:aws:secretsmanager:<your_aws_region>:<your_account_id>:secret:<your_secret_name>:<traefik_license>::"
                },
                {
                    "name": "TRAEFIKEE_PLUGIN_TOKEN",
                    "valueFrom": "arn:aws:secretsmanager:<your_aws_region>:<your_account_id>:secret:<your_secret_name>:<traefik_plugin_token_key>::"
                },
                {
                    "name": "TRAEFIKEE_JOIN_TOKEN",
                    "valueFrom": "arn:aws:secretsmanager:<your_aws_region>:<your_account_id>:secret:<your_secret_name>:<traefik_controller_join_token_key>::"
                }
            ],
            "volumesFrom": [],
            "dockerLabels": {
                "com.traefik.traefikee.component": "controller-0"
            },
            "mountPoints": [
                {
                  "readOnly": false,
                  "containerPath": "/data",
                  "sourceVolume": "traefikee-data"
                }
            ],
            "readonlyRootFilesystem": false,
            "privileged": false,
            "logConfiguration": {
                "logDriver": "awslogs",
                "options": {
                    "awslogs-group": "<your_cloud_watch_log_group",
                    "awslogs-region": "<your_aws_region>",
                    "awslogs-stream-prefix": "traefikee-controller-0"
                }
        }
    }
    ],
    "volumes": [
        {
          "name": "traefikee-data",
          "efsVolumeConfiguration": {
            "fileSystemId": "<controller-0_efs_id>",
            "transitEncryption": "DISABLED",
            "rootDirectory": "/"
          }
        }
    ],
    "networkMode": "awsvpc",
    "runtimePlatform": {
        "cpuArchitecture": "X86_64",
        "operatingSystemFamily": "LINUX"
    },

    "requiresCompatibilities": [
        "FARGATE"
    ]
}

Controller-1

{
    "family": "traefikee-controller-1",
    "taskRoleArn": "arn:aws:iam::<your_account_id>:role/RoleTraefikECSReadAccess",
    "executionRoleArn": "arn:aws:iam::<your_account_id>:role/ecsTaskExecutionRole",
    "cpu": "512",
    "memory": "1024",
    "containerDefinitions": [
        {
            "name": "controller-1",
            "image": "traefik/traefikee:v2.10.0",
            "cpu": 500,
            "portMappings": [
                {
                    "containerPort": 55055,
                    "hostPort": 55055,
                    "protocol": "tcp"
                }
            ],
            "essential": true,
            "command": [
                "controller",
                "--name=controller-1",
                "--advertise=controller-1.traefikee:4242",
                "--discovery.static.peers=controller-0.traefikee:4242,controller-2.traefikee:4242",
                "--statedir=/data/state",
                "--jointoken.file.path=/data/tokens",
                "--api.socket=/var/run/traefikee/teectl-controller-1.sock",
                "--socket=/var/run/traefikee/controller-1.sock",
                "--api.autocerts",
                "--plugin.url=https://registry.traefikee:443"
            ],
            "linuxParameters": {
                "initProcessEnabled": true
            },
            "environment": [],
            "secrets": [
                                {
                    "name": "TRAEFIKEE_LICENSE",
                    "valueFrom": "arn:aws:secretsmanager:<your_aws_region>:<your_account_id>:secret:<your_secret_name>:<traefik_license>::"
                },
                {
                    "name": "TRAEFIKEE_PLUGIN_TOKEN",
                    "valueFrom": "arn:aws:secretsmanager:<your_aws_region>:<your_account_id>:secret:<your_secret_name>:<traefik_plugin_token_key>::"
                },
                {
                    "name": "TRAEFIKEE_JOIN_TOKEN",
                    "valueFrom": "arn:aws:secretsmanager:<your_aws_region>:<your_account_id>:secret:<your_secret_name>:<traefik_controller_join_token_key>::"
                }
            ],
            "volumesFrom": [],
            "dockerLabels": {
                "com.traefik.traefikee.component": "controller-1"
            },
            "mountPoints": [
                {
                  "readOnly": false,
                  "containerPath": "/data",
                  "sourceVolume": "traefikee-data"
                }
            ],
            "readonlyRootFilesystem": false,
            "privileged": false,
            "logConfiguration": {
                "logDriver": "awslogs",
                "options": {
                    "awslogs-group": "<your_cloud_watch_log_group",
                    "awslogs-region": "<your_aws_region>",
                    "awslogs-stream-prefix": "traefikee-controller-1"
                }
        }
    }
    ],
    "volumes": [
        {
          "name": "traefikee-data",
          "efsVolumeConfiguration": {
            "fileSystemId": "<controller-1_efs_id>",
            "transitEncryption": "DISABLED",
            "rootDirectory": "/"
          }
        }
    ],
    "networkMode": "awsvpc",
    "runtimePlatform": {
        "cpuArchitecture": "X86_64",
        "operatingSystemFamily": "LINUX"
    },

    "requiresCompatibilities": [
        "FARGATE"
    ]
}

Controller-2

{
    "family": "traefikee-controller-2",
    "taskRoleArn": "arn:aws:iam::<your_account_id>:role/RoleTraefikECSReadAccess",
    "executionRoleArn": "arn:aws:iam::<your_account_id>:role/ecsTaskExecutionRole",
    "cpu": "512",
    "memory": "1024",
    "containerDefinitions": [
        {
            "name": "controller-2",
            "image": "traefik/traefikee:v2.10.0",
            "cpu": 500,
            "portMappings": [
                {
                    "containerPort": 55055,
                    "hostPort": 55055,
                    "protocol": "tcp"
                }
            ],
            "essential": true,
            "command": [
                "controller",
                "--name=controller-2",
                "--advertise=controller-2.traefikee:4242",
                "--discovery.static.peers=controller-0.traefikee:4242,controller-1.traefikee:4242",
                "--statedir=/data/state",
                "--jointoken.file.path=/data/tokens",
                "--api.socket=/var/run/traefikee/teectl-controller-2.sock",
                "--socket=/var/run/traefikee/controller-2.sock",
                "--api.autocerts",
                "--plugin.url=https://registry.traefikee:443"
            ],
            "linuxParameters": {
                "initProcessEnabled": true
            },
            "environment": [],
            "secrets": [
                                {
                    "name": "TRAEFIKEE_LICENSE",
                    "valueFrom": "arn:aws:secretsmanager:<your_aws_region>:<your_account_id>:secret:<your_secret_name>:<traefik_license>::"
                },
                {
                    "name": "TRAEFIKEE_PLUGIN_TOKEN",
                    "valueFrom": "arn:aws:secretsmanager:<your_aws_region>:<your_account_id>:secret:<your_secret_name>:<traefik_plugin_token_key>::"
                },
                {
                    "name": "TRAEFIKEE_JOIN_TOKEN",
                    "valueFrom": "arn:aws:secretsmanager:<your_aws_region>:<your_account_id>:secret:<your_secret_name>:<traefik_controller_join_token_key>::"
                }
            ],
            "volumesFrom": [],
            "dockerLabels": {
                "com.traefik.traefikee.component": "controller-2"
            },
            "mountPoints": [
                {
                  "readOnly": false,
                  "containerPath": "/data",
                  "sourceVolume": "traefikee-data"
                }
            ],
            "readonlyRootFilesystem": false,
            "privileged": false,
            "logConfiguration": {
                "logDriver": "awslogs",
                "options": {
                    "awslogs-group": "<your_cloud_watch_log_group>",
                    "awslogs-region": "<your_aws_region>",
                    "awslogs-stream-prefix": "traefikee-controller-2"
                }
        }
    }
    ],
    "volumes": [
        {
          "name": "traefikee-data",
          "efsVolumeConfiguration": {
            "fileSystemId": "<controller-2_efs_id>",
            "transitEncryption": "DISABLED",
            "rootDirectory": "/"
          }
        }
    ],
    "networkMode": "awsvpc",
    "runtimePlatform": {
        "cpuArchitecture": "X86_64",
        "operatingSystemFamily": "LINUX"
    },
    "requiresCompatibilities": [
        "FARGATE"
    ]
}

Create a service for each task as described in the mono controller section and deploy.

Proxies

Create a file named proxies-task.json with the following task definition:

Without Secret

{
    "family": "traefikee-proxies",
    "taskRoleArn": "arn:aws:iam::<your_account_id>:role/RoleTraefikECSReadAccess",
    "executionRoleArn": "arn:aws:iam::<your_account_id>:role/ecsTaskExecutionRole",
    "cpu": "512",
    "memory": "1024",
    "containerDefinitions": [
        {
            "name": "proxy",
            "image": "traefik/traefikee:v2.10.0",
            "cpu": 500,
            "portMappings": [
                {
                    "containerPort": 80,
                    "hostPort": 80,
                    "protocol": "tcp"
                },
                {
                    "containerPort": 443,
                    "hostPort": 443,
                    "protocol": "tcp"
                },
                {
                    "containerPort": 8484,
                    "hostPort": 8484,
                    "protocol": "tcp"
                }
            ],
            "essential": true,
            "command": [
                "proxy",
                "--role=ingress",
                "--discovery.static.peers=controller-0.traefikee:4242",
                "--jointoken.value=<proxy_join_token>"
            ],
            "linuxParameters": {},
            "environment": [],
            "volumesFrom": [],
            "secrets": [],
            "dockerLabels": {
                "com.traefik.traefikee.component": "proxies"
            },
            "mountPoints": [],
            "readonlyRootFilesystem": true,
            "privileged": false,
            "logConfiguration": {
                "logDriver": "awslogs",
                "options": {
                    "awslogs-group": "<your_cloud_watch_log_group",
                    "awslogs-region": "<your_aws_region>",
                    "awslogs-stream-prefix": "traefikee-proxies"
                }
        }
    ],
    "networkMode": "awsvpc",
    "runtimePlatform": {
        "cpuArchitecture": "X86_64",
        "operatingSystemFamily": "LINUX"
    },

    "requiresCompatibilities": [
        "FARGATE"
    ]
}

With Secret

{
    "family": "traefikee-proxies",
    "taskRoleArn": "arn:aws:iam::<your_account_id>:role/RoleTraefikECSReadAccess",
    "executionRoleArn": "arn:aws:iam::<your_account_id>:role/ecsTaskExecutionRole",
    "cpu": "512",
    "memory": "1024",
    "containerDefinitions": [
        {
            "name": "proxy",
            "image": "traefik/traefikee:v2.10.0",
            "cpu": 500,
            "portMappings": [
                {
                    "containerPort": 80,
                    "hostPort": 80,
                    "protocol": "tcp"
                },
                {
                    "containerPort": 443,
                    "hostPort": 443,
                    "protocol": "tcp"
                },
                {
                    "containerPort": 8484,
                    "hostPort": 8484,
                    "protocol": "tcp"
                }
            ],
            "essential": true,
            "command": [
                "proxy",
                "--role=ingress",
                "--discovery.static.peers=controller-0.traefikee:4242"
            ],
            "linuxParameters": {},
            "environment": [],
            "volumesFrom": [],
            "secrets": [
                {
                    "name": "TRAEFIKEE_JOIN_TOKEN",
                    "valueFrom": "arn:aws:secretsmanager:<your_aws_region>:<your_account_id>:secret:<your_secret_name>:<traefik_proxy_join_token_key>::"
                }
            ],
            "dockerLabels": {
                "com.traefik.traefikee.component": "proxies"
            },
            "mountPoints": [],
            "readonlyRootFilesystem": true,
            "privileged": false,
            "logConfiguration": {
                "logDriver": "awslogs",
                "options": {
                    "awslogs-group": "<your_cloud_watch_log_group",
                    "awslogs-region": "<your_aws_region>",
                    "awslogs-stream-prefix": "traefikee-proxies"
                }
        }

        }
    ],
    "networkMode": "awsvpc",
    "runtimePlatform": {
        "cpuArchitecture": "X86_64",
        "operatingSystemFamily": "LINUX"
    },

    "requiresCompatibilities": [
        "FARGATE"
    ]
}

Proxies

In case of multi controller installation you should add all the controller in the static.discovery.peers field.

Customize the file then register the task definition in Fargate:

aws ecs register-task-definition --cli-input-json file://proxies-task.json --region <aws_region>

Now create a file named proxies-svc.json with the service definition:

{
    "serviceName": "traefikee-proxies",
    "taskDefinition": "traefikee-proxies",
    "desiredCount": 2,
    "launchType": "FARGATE",
    "schedulingStrategy": "REPLICA",
    "networkConfiguration": {
        "awsvpcConfiguration": {
            "subnets": [
                "<subnet_id_a>",
                "<subnet_id_b>",
                "<subnet_id_c>"
            ],
            "securityGroups": [
                "<security_group_id>"
            ],
            "assignPublicIp": "ENABLED"
        }
    }
}

Customize the file and deploy it to the Fargate cluster:

aws ecs create-service --cli-input-json file://proxies-svc.json --cluster <my_cluster> --region <aws_region>

Check the service status to ensure the service was started:

Command

aws ecs describe-services --services traefikee-proxies --cluster <my_cluster> --region <aws_region>

Expected output

{
    "services": [
        {
            "status": "ACTIVE",
            "serviceRegistries": [],
            "pendingCount": 0,
            "launchType": "EC2",
            "enableECSManagedTags": true,
            "schedulingStrategy": "REPLICA",
            "loadBalancers": [],
            "placementConstraints": [
                {
                    "type": "distinctInstance"
                }
            ],
            "createdAt": 1605879947.392,
            "desiredCount": 2,
            "serviceName": "traefikee-proxies",
            "clusterArn": "arn:aws:ecs:aws_region:aws_account_id:cluster/traefikee-ecs",
            "createdBy": "arn:aws:iam::aws_account_id:user/aws_user",
            "taskDefinition": "arn:aws:ecs:aws_region:aws_account_id:task-definition/traefikee-proxies:5",
            "serviceArn": "arn:aws:ecs:aws_region:aws_account_id:service/traefikee-ecs/traefikee-proxies",
            "propagateTags": "SERVICE",
            "deploymentConfiguration": {
                "maximumPercent": 200,
                "minimumHealthyPercent": 100
            },
            "deployments": [
                {
                    "status": "PRIMARY",
                    "pendingCount": 0,
                    "launchType": "EC2",
                    "createdAt": 1605880984.365,
                    "desiredCount": 2,
                    "taskDefinition": "arn:aws:ecs:aws_region:aws_account_id:task-definition/traefikee-proxies:5",
                    "updatedAt": 1605881208.064,
                    "id": "ecs-svc/4529007424825088897",
                    "runningCount": 2
                }
            ],
            "events": [
                {
                    "message": "(service traefikee-proxies) has reached a steady state.",
                    "id": "51e07037-3595-40fa-a394-c489561fcd11",
                    "createdAt": 1605881208.068
                },
                {
                    "message": "(service traefikee-proxies) has started 1 tasks: (task 11ad2458e5624330955dc97f6ce41d41).",
                    "id": "7e74652c-f88c-4e0f-b922-446ac8a7a114",
                    "createdAt": 1605881197.123
                },
                {
                    "message": "(service traefikee-proxies) has reached a steady state.",
                    "id": "2d857d41-658a-452b-b7d6-c1c3bb4a2944",
                    "createdAt": 1605881165.664
                },
                {
                    "message": "(service traefikee-proxies) has started 1 tasks: (task b94de86d3664413081c045ed25f9b326).",
                    "id": "74bb63a2-bbf8-45a2-bc22-a5a266b4289e",
                    "createdAt": 1605881153.114
                }
            ],
            "runningCount": 2,
            "placementStrategy": []
        }
    ],
    "failures": []
}

Plugin Registry

Create a file named registry-task.json with the following task definition:

Registry Storage

Fargate by default assign non-persistent storage to the container, in order to keep your custom plugins, if the registry container needs to be recreated you need to add an EFS volume as described on the task definition below.

Without Secret

{
  "family": "traefikee-registry",
  "taskRoleArn": "arn:aws:iam::<your_account_id>:role/RoleTraefikECSReadAccess",
  "executionRoleArn": "arn:aws:iam::<your_account_id>:role/ecsTaskExecutionRole",
  "cpu": "512",
  "memory": "1024",
  "containerDefinitions": [
      {
        "name" : "registry",
        "image": "traefik/traefikee:v2.10.0",
        "cpu": 500,
        "portMappings": [
          {
            "hostPort": 443,
            "protocol": "tcp",
            "containerPort": 443
          }
        ],
        "essential": true,
        "command": [
          "plugin-registry",
          "--name=registry",
          "--discovery.static.peers=controller-0.traefikee",
          "--plugindir=/var/lib/plugins",
          "--token=<your_generated_plugin_token>",
          "--jointoken.value=<proxy_join_token>"
        ],
        "linuxParameters": {},
        "environment": [],
        "volumesFrom": [],
        "secrets": [],
        "dockerLabels": {
            "com.traefik.traefikee.component": "registry"
        },
        "mountPoints": [
          {
            "readOnly": false,
            "containerPath": "/var/lib/plugins",
            "sourceVolume": "traefikee-plugins"
          }
        ],
        "readonlyRootFilesystem": false,
        "privileged": false,
        "logConfiguration": {
            "logDriver": "awslogs",
            "options": {
                "awslogs-group": "<your_cloud_watch_log_group",
                "awslogs-region": "<your_aws_region>",
                "awslogs-stream-prefix": "traefikee-registry"
            }
    }
      }
    ],
    "volumes": [
      {
        "name": "traefikee-plugins",
        "efsVolumeConfiguration": {
          "fileSystemId": "<registry_efs_id>",
          "transitEncryption": "DISABLED",
          "rootDirectory": "/"
        }
      }
  ],
  "networkMode": "awsvpc",
  "runtimePlatform": {
      "cpuArchitecture": "X86_64",
      "operatingSystemFamily": "LINUX"
  },

  "requiresCompatibilities": [
      "FARGATE"
  ]
}

With Secret

{
  "family": "traefikee-registry",
  "taskRoleArn": "arn:aws:iam::<your_account_id>:role/RoleTraefikECSReadAccess",
  "executionRoleArn": "arn:aws:iam::<your_account_id>:role/ecsTaskExecutionRole",
  "cpu": "512",
  "memory": "1024",
  "containerDefinitions": [
      {
        "name" : "registry",
        "image": "traefik/traefikee:v2.10.0",
        "cpu": 500,
        "portMappings": [
          {
            "hostPort": 443,
            "protocol": "tcp",
            "containerPort": 443
          }
        ],
        "essential": true,
        "command": [
          "plugin-registry",
          "--name=registry",
          "--discovery.dns.domain=controller-0.traefikee",
          "--plugindir=/var/lib/plugins"
        ],
        "linuxParameters": {},
        "environment": [],
        "volumesFrom": [],
        "secrets": [
                {
                    "name": "TRAEFIKEE_PLUGIN_TOKEN",
                    "valueFrom": "arn:aws:secretsmanager:<your_aws_region>:<your_account_id>:secret:<your_secret_name>:<traefik_plugin_token_key>::"
                },
                {
                    "name": "TRAEFIKEE_JOIN_TOKEN",
                    "valueFrom": "arn:aws:secretsmanager:<your_aws_region>:<your_account_id>:secret:<your_secret_name>:<traefik_proxy_join_token_key>::"
                }
        ],
        "dockerLabels": {
            "com.traefik.traefikee.component": "registry"
        },
        "mountPoints": [
          {
            "readOnly": false,
            "containerPath": "/var/lib/plugins",
            "sourceVolume": "traefikee-plugins"
          }
        ],
        "readonlyRootFilesystem": false,
        "privileged": false,
        "logConfiguration": {
            "logDriver": "awslogs",
            "options": {
                "awslogs-group": "<your_cloud_watch_log_group",
                "awslogs-region": "<your_aws_region>",
                "awslogs-stream-prefix": "traefikee-registry"
            }
    }
      }
    ],
    "volumes": [
      {
        "name": "traefikee-plugins",
        "efsVolumeConfiguration": {
          "fileSystemId": "<registry_efs_id>",
          "transitEncryption": "DISABLED",
          "rootDirectory": "/"
        }
      }
  ],
  "networkMode": "awsvpc",
  "runtimePlatform": {
      "cpuArchitecture": "X86_64",
      "operatingSystemFamily": "LINUX"
  },

  "requiresCompatibilities": [
      "FARGATE"
  ]
}

Registry with Multi controller

In case of multi controller installation, you should add all the controller in the static.discovery.peers field.

Customize the file, then register the task definition in Fargate:

aws ecs register-task-definition --cli-input-json file://registry-task.json --region <aws_region>

Now create a file named registry-svc.json with the service definition:

{
    "serviceName": "traefikee-registry",
    "taskDefinition": "traefikee-registry",
    "desiredCount": 1,
    "launchType": "FARGATE",
    "schedulingStrategy": "REPLICA",
    "serviceRegistries": [
        {
           "registryArn": "<service_discovery_registry_service_arn>"
        }
     ],
    "networkConfiguration": {
        "awsvpcConfiguration": {
            "subnets": [
                "<subnet_id_a>",
                "<subnet_id_b>",
                "<subnet_id_c>"
            ],
            "securityGroups": [
                "<security_group_id>"
            ],
            "assignPublicIp": "ENABLED"
        }
    }
}

Customize the file and deploy it to the Fargate cluster:

aws ecs create-service --cli-input-json file://registry-svc.json --cluster <my_cluster> --region <aws_region>

Check the service status to ensure the service was started:

Command

aws ecs describe-services --services registry --cluster <my_cluster> --region <aws_region>

Expected output

{
    "services": [
        {
            "serviceArn": "arn:aws:ecs:eu-north-1:114072598128:service/traefikee-ecs/traefikee-registry",
            "serviceName": "traefikee-registry",
            "clusterArn": "arn:aws:ecs:eu-north-1:114072598128:cluster/traefikee-ecs",
            "loadBalancers": [],
            "serviceRegistries": [],
            "status": "ACTIVE",
            "desiredCount": 1,
            "runningCount": 1,
            "pendingCount": 0,
            "launchType": "EC2",
            "taskDefinition": "arn:aws:ecs:eu-north-1:114072598128:task-definition/traefikee-registry:2",
            "deploymentConfiguration": {
                "deploymentCircuitBreaker": {
                    "enable": false,
                    "rollback": false
                },
                "maximumPercent": 200,
                "minimumHealthyPercent": 100
            },
            "deployments": [
                {
                    "id": "ecs-svc/5244500742569760246",
                    "status": "PRIMARY",
                    "taskDefinition": "arn:aws:ecs:eu-north-1:114072598128:task-definition/traefikee-registry:2",
                    "desiredCount": 1,
                    "pendingCount": 0,
                    "runningCount": 1,
                    "failedTasks": 0,
                    "createdAt": 1610097942.755,
                    "updatedAt": 1610097977.375,
                    "launchType": "EC2",
                    "rolloutState": "COMPLETED",
                    "rolloutStateReason": "ECS deployment ecs-svc/5244500742569760246 completed."
                }
            ],
            "events": [
                {
                    "id": "9b8f4549-e972-4506-848a-8050084f57f0",
                    "createdAt": 1610097977.381,
                    "message": "(service registry) has reached a steady state."
                },
                {
                    "id": "adee0f69-5ceb-4f11-8fd9-47effc1b39c2",
                    "createdAt": 1610097977.38,
                    "message": "(service registry) (deployment ecs-svc/5244500742569760246) deployment completed."
                },
                {
                    "id": "6b070094-8686-4ab7-9886-82c248203698",
                    "createdAt": 1610097966.485,
                    "message": "(service registry) has started 1 tasks: (task 65e57cd8504346da8a8d654917919c7b)."
                },
            ],
            "createdAt": 1610096466.659,
            "placementConstraints": [],
            "placementStrategy": [
                {
                    "type": "spread",
                    "field": "attribute:ecs.availability-zone"
                },
                {
                    "type": "spread",
                    "field": "instanceId"
                }
            ],
            "schedulingStrategy": "REPLICA",
            "createdBy": "arn:aws:iam::114072598128:user/xxx",
            "enableECSManagedTags": true,
            "propagateTags": "NONE"
        }
    ],
    "failures": []
}

Remote Access Through teectl

Once your cluster is ready, if you want to operate the cluster remotely using the teectl tool, you will need to generate credentials from your cluster using traefikee generate credentials on one of your controllers and use teectl to import the cluster credentials.

First connect to a container instance running a controller task, then run:

Command

# Get a teectl config with credentials
aws ecs execute-command --cluster <my_cluster> --container controller-0 --task <task-arn> --interactive --command "/traefikee generate credentials --cluster <my_cluster> --onpremise.hosts <controller-container-public-ip> --socket /var/run/teectl-controller-0.sock"

Expected output

cluster_name: <my_cluster>
tls:
  cert: |
    -----BEGIN CERTIFICATE-----
    MIIB+jCCAaCgAwIBAgIQKHgQX14LGetAsD8wSEta5jAKBggqhkjOPQQDAjAuMRUw
    EwYDVQQKEwxUcmFlZmlrIExhYnMxFTATBgNVBAMTDFRyYWVmaWtFRSBDQTAgFw0y
    MDExMTkyMDAwMDNaGA8yMTIwMTAyNjIwMDAwM1owFTETMBEGA1UEAxMKY2xpZW50
    LmVjczBZMBMGByqGSM49AgEGCCqGSM49AwEHA0IABKcFJkSGRxbfAnQx2JtYpEnA
    knHdqukFWF8Sbvht30Ge/EnBrTe37ilzJ0KlY11UQK/KlGPMoqVSrjAnqZz10nGj
    gbYwgbMwDgYDVR0PAQH/BAQDAgGGMBMGA1UdJQQMMAoGCCsGAQUFBwMCMAwGA1Ud
    EwEB/wQCMAAwKQYDVR0OBCIEIKh1ygHajrADFcxpfv4p/ff7FH4lDULi0uqcDp7j
    qaylMCsGA1UdIwQkMCKAILoCK8uf89Yveb/uZN+Vs+gfvz05Y6NHB2/hd3T6ix3T
    MCYGA1UdEQQfMB2CCmNsaWVudC5lY3OCCWxvY2FsaG9zdIcEfwAAATAKBggqhkjO
    PQQDAgNIADBFAiEAiXT09co8egGQ0Y6hJ/lgIzCGC0I9GiFfvrxCGKl15l8CICYY
    E6ig/k2D4coprOclVF5QVkqHCcx2EL0HA/zK05eI
    -----END CERTIFICATE-----
  key: |
    -----BEGIN EC PRIVATE KEY-----
    MHcCAQEEIC8CsJ/B115S+JtR1/l3ZQwKA3XdXt9zLqusF1VXc/KloAoGCCqGSM49
    AwEHoUQDQgAEpwUmRIZHFt8CdDHYm1ikScCScd2q6QVYXxJu+G3fQZ78ScGtN7fu
    KXMnQqVjXVRAr8qUY8yipVKuMCepnPXScQ==
    -----END EC PRIVATE KEY-----
  ca: |
    -----BEGIN CERTIFICATE-----
    MIIB2DCCAX2gAwIBAgIQDEHgiwDfaIX3wS1EHF800DAKBggqhkjOPQQDAjAuMRUw
    EwYDVQQKEwxUcmFlZmlrIExhYnMxFTATBgNVBAMTDFRyYWVmaWtFRSBDQTAgFw0y
    MDExMTkxOTU2MjBaGA8yMTIwMTAyNjE5NTYyMFowLjEVMBMGA1UEChMMVHJhZWZp
    ayBMYWJzMRUwEwYDVQQDEwxUcmFlZmlrRUUgQ0EwWTATBgcqhkjOPQIBBggqhkjO
    PQMBBwNCAATe+gM99l/nAAeNIy/kn8vbrcSNORhbWGBUp+j1CtL2ADqLYIel/acB
    B3ssLPnbAZLoKJefrQS/CNJOdZpRZBnfo3sweTAOBgNVHQ8BAf8EBAMCAYYwDwYD
    VR0TAQH/BAUwAwEB/zApBgNVHQ4EIgQgugIry5/z1i95v+5k35Wz6B+/PTljo0cH
    b+F3dPqLHdMwKwYDVR0jBCQwIoAgugIry5/z1i95v+5k35Wz6B+/PTljo0cHb+F3
    dPqLHdMwCgYIKoZIzj0EAwIDSQAwRgIhAMRjpKzzAZr5wT5IXMaBWv0OUvy6wxFn
    kyZApTzhcbr9AiEAtaJZ7DN+xLeA0/RUljK0uDKuZALix30VSx4U2aaYdc4=
    -----END CERTIFICATE-----
onPremise:
  hosts:
  - <controller-container-public-ip>
  port: 55055

Save the output of the last command above to a file named teectl-config.yaml, and then run:

teectl cluster import --file="teectl-config.yaml"

teectl cluster use --name <my_cluster>

You can now use teectl to operate your cluster.

Command

teectl get nodes

Expected output

ID                         NAME                                      STATUS  ROLE
hlx1b3gu8bb5n1lg8qtiy5nvv  controller-0                              Ready   Controller (Leader)
jm5wv9kdmp9imspqx39n300b3  ip-10-0-0-130.us-east-1.compute.internal  Ready   Proxy / Ingress
y0l2me7zjalidnzqf3fqanuxk  ip-10-0-0-178.us-east-1.compute.internal  Ready   Proxy / Ingress
odxyyk3l7pkwiab8rtwvhflgs  registry                                  Ready   Plugin Registry

Going further

Now that the cluster is ready, we recommend reading the various operating guides to dive into all the features that Traefik Enterprise provides.

License Monitoring

When a Traefik Enterprise controller starts for the first time, it checks the license validity.

If the license is valid, another check is done once every 24 hours.

If the controller can't communicate with the license server, a 72-day grace period starts to recover from this situation.

Once the grace period is over, the controller stops updating the proxies configuration.

Please look at the FAQ to know how to implement the license monitoring.