Installing Traefik Enterprise ECS¶

This page guides you through the installation of Traefik Enterprise on AWS Elastic Container Service.

AWS Knowledge

Assistance with configuring or setting up AWS services is not included in this guide. If you need more information about any of the components referenced in this guide, start with the following resources:

Requirements¶

An ECS cluster
- The aws CLI tool properly configured to communicate with the cluster
- Enough permissions on the ECS instance role to access any integrated AWS service
- All required ports are open on the associated security group of the container instances
- SSH access to your EC2 container instances
Controller containers can reach https://v4.license.containous.cloud
The teectl CLI tool, for cluster management

Installation¶

This guide only focuses on the EC2 launch type, it may be possible to run with the Fargate with some customization.

Altough some external AWS resources are referenced in the logConfiguration and secrets sections, they are optional and you can just remove or customize them in the task definition, otherwise you must create those resources manually with the AWS Console or CLI tool which is not covered in this guide as well.

Since teectl setup gen does not provide support for ECS manifest files, this guide demonstrates how to write your own task and service definitions, using JSON syntax, to define the necessary resources and deploy a Traefik Enteprise cluster of one controller and two proxies.

Plugin Registry Token¶

The plugin registry needs a token to secure its communications with the controller. This token is set on the controller and plugin registry task definition.

Here is an example of how to create it:

Command

openssl rand -base64 10

Expected output

MvXVeX3qDylxJQ==

Controllers¶

Create a file named controller-task.json with the following task definition:

{
    "family": "traefikee-controller",
    "taskRoleArn": "arn:aws:iam::aws_account_id:role/ecsTaskTraefikee",
    "executionRoleArn": "arn:aws:iam::aws_account_id:role/ecsTaskTraefikee",
    "networkMode": "host",
    "containerDefinitions": [        
        {
            "name": "controller-0",
            "image": "traefik/traefikee:v2.11.0",
            "cpu": 500,
            "memory": 1024,
            "memoryReservation": 256,
            "essential": true,
            "hostname": "traefikee-controller-0",
            "disableNetworking": false,
            "privileged": false,
            "readonlyRootFilesystem": true,
            "interactive": false,
            "pseudoTerminal": false,
            "secrets": [
                {
                    "name": "TRAEFIKEE_LICENSE",
                    "valueFrom": "arn:aws:secretsmanager:aws_region:aws_account_id:secret:traefikee-license-03vFNy:key::"
                }
            ],            
            "portMappings": [
                {
                    "containerPort": 55055,
                    "hostPort": 55055,
                    "protocol": "tcp"
                }
            ],            
            "command": [
                "controller",
                "--name=controller-0",
                "--advertise=controller-0:4242",
                "--discovery.static.peers=",
                "--license=$TRAEFIKEE_LICENSE",
                "--log.level=",
                "--log.filepath=",
                "--log.format=",
                "--statedir=/data/state",
                "--jointoken.file.path=/data/tokens",
                "--api.socket=/data/run/teectl-controller-0.sock",
                "--socket=/data/run/controller-0.sock",
                "--api.autocerts",
                "--plugin.url=https://[my-plugin-registry-ip]:443",
                "--plugin.token=[my-plugin-registry-token]"
            ],
            "mountPoints": [
                {
                    "sourceVolume": "traefikee-data",
                    "containerPath": "/data",
                    "readOnly": false
                }
            ],            
            "linuxParameters": {
                "capabilities": {
                    "add": [
                        "NET_BIND_SERVICE"
                    ],
                    "drop": [
                        "ALL"
                    ]
                }                
            },
            "logConfiguration": {
                "logDriver": "awslogs",
                "options": {
                    "awslogs-group": "traefikee",
                    "awslogs-region": "aws_region",
                    "awslogs-stream-prefix": "controller-0"
                }
            },            
            "dockerLabels": {
                "com.traefik.traefikee.component": "controllers"
            }            
        }
    ],
    "volumes": [
        {
            "name": "traefikee-data",       
            "dockerVolumeConfiguration": {
                "scope": "shared",
                "autoprovision": true,
                "driver": "local"                
            }            
        }
    ],
    "requiresCompatibilities": [
        "EC2"
    ]
}

Controller Storage

To keep things simple this guide is using a docker volume configuration with a local driver. This is fine for a single controller cluster but not suited for a multi controller cluster, because tasks would be spread across different ECS instances. Refer to the official ECS docs for all supported volume drivers.

Customize the file, then register the task definition on ECS:

Command

aws ecs register-task-definition --cli-input-json file://controller-task.json --region $AWS_REGION

Now create a file named controller-svc.json with the service definition:

{
    "serviceName": "traefikee-controllers",
    "taskDefinition": "traefikee-controller",
    "desiredCount": 1,
    "launchType": "EC2",    
    "enableECSManagedTags": true,
    "propagateTags": "SERVICE",
    "placementConstraints": [
        {
            "type": "distinctInstance"
        }
    ],
    "schedulingStrategy": "REPLICA",
    "deploymentController": {
        "type": "ECS"
    }    
}

Customize the file and deploy it to the ECS cluster:

aws ecs create-service --cli-input-json file://controller-svc.json --cluster mycluster --region $AWS_REGION

Check the service status to ensure the service was started:

Command

aws ecs describe-services --services traefikee-controllers --cluster mycluster --region $AWS_REGION

Expected output

{
    "services": [
        {
            "status": "ACTIVE",
            "serviceRegistries": [],
            "pendingCount": 0,
            "launchType": "EC2",
            "enableECSManagedTags": true,
            "schedulingStrategy": "REPLICA",
            "loadBalancers": [],
            "placementConstraints": [
                {
                    "type": "distinctInstance"
                }
            ],
            "createdAt": 1605815767.081,
            "desiredCount": 1,
            "serviceName": "traefikee-controllers",
            "clusterArn": "arn:aws:ecs:aws_region:aws_account_id:cluster/traefikee-ecs",
            "createdBy": "arn:aws:iam::aws_account_id:user/aws_user",
            "taskDefinition": "arn:aws:ecs:aws_region:aws_account_id:task-definition/traefikee-controller:7",
            "serviceArn": "arn:aws:ecs:aws_region:aws_account_id:service/traefikee-ecs/traefikee-controllers",
            "propagateTags": "SERVICE",
            "deploymentConfiguration": {
                "maximumPercent": 200,
                "minimumHealthyPercent": 100
            },
            "deployments": [
                {
                    "status": "PRIMARY",
                    "pendingCount": 0,
                    "launchType": "EC2",
                    "createdAt": 1605815767.081,
                    "desiredCount": 1,
                    "taskDefinition": "arn:aws:ecs:aws_region:aws_account_id:task-definition/traefikee-controller:7",
                    "updatedAt": 1605815785.903,
                    "id": "ecs-svc/4291651818662396701",
                    "runningCount": 1
                }
            ],
            "events": [
                {
                    "message": "(service traefikee-controllers) has reached a steady state.",
                    "id": "2ae7ae80-11bc-48fa-bd1c-ec6235521928",
                    "createdAt": 1605815785.907
                },
                {
                    "message": "(service traefikee-controllers) has started 1 tasks: (task fe932a1e5690403eacd5d3b4f8221b18).",
                    "id": "901615c4-8e67-4ed2-91f6-b52fcc88d147",
                    "createdAt": 1605815774.564
                }
            ],
            "runningCount": 1,
            "placementStrategy": []
        }
    ],
    "failures": []
}

Wait until the controllers are up and running before proceeding to the next steps.

Get the Proxy Join Token¶

Fetch the proxy join token by connecting to the container instance and executing the following commands:

Command

# Get the controller container id
CONTAINER_ID=$(docker ps | grep traefikee | cut -f 1 -d " ")
# Get the tokens
docker exec -it $CONTAINER_ID /traefikee tokens --socket /data/run/controller-0.sock

Expected output

export TRAEFIKEE_CONTROLLER_TOKEN=5531644e5645744f4c5445744e47706f4e7a5a3261574a765a485274616a55345a57526f4d7a566f65444e6f4e544e70616d4a6d615459314e5464684f48526f4d6e6c735933466e4e33686a6348417459544e6c4e5451314e475a70616d4e304e484e796432466f4e445a796447707859673d3d3a303a97b42afa416f0df94d7c453bd4c55fcddd339ec4570068eac2c5cc5504c158d42521e9852615ab95049e2b3296b67c4a
export TRAEFIKEE_PROXY_TOKEN=5531644e5645744f4c5445744e47706f4e7a5a3261574a765a485274616a55345a57526f4d7a566f65444e6f4e544e70616d4a6d615459314e5464684f48526f4d6e6c735933466e4e33686a634841744e47397563484d31636a4d354d7a42354e5463785933557762546b774e7a55794f413d3d3a313a97b42afa416f0df94d7c453bd4c55fcddd339ec4570068eac2c5cc5504c158d42194b3059e6861761121966a44f964d3

Write down the proxy token as it is required for the next step, the proxies task definition.

Proxies¶

Create a file named proxies-task.json with the following task definition:

{
    "family": "traefikee-proxies",
    "taskRoleArn": "arn:aws:iam::aws_account_id:role/ecsTaskTraefikee",
    "networkMode": "host",
    "containerDefinitions": [        
        {
            "name": "proxy",
            "image": "traefik/traefikee:v2.11.0",
            "disableNetworking": false,
            "privileged": false,
            "readonlyRootFilesystem": true,
            "interactive": true,
            "pseudoTerminal": true,
            "essential": true,
            "cpu": 500,
            "memory": 1024,
            "memoryReservation": 256,                
            "portMappings": [
                {
                    "containerPort": 80,
                    "hostPort": 80,
                    "protocol": "tcp"
                },
                {
                    "containerPort": 443,
                    "hostPort": 443,
                    "protocol": "tcp"
                },
                {
                    "containerPort": 8484,
                    "hostPort": 8484,
                    "protocol": "tcp"
                }
            ],            
            "command": [
                "traefikee",
                "proxy",
                "--role=ingress",
                "--discovery.static.peers=[my-controller-ip]:4242",
                "--jointoken.value=[my-proxy-join-token]",
                "--log.level=",
                "--log.filepath=",
                "--log.format="
            ],
            "linuxParameters": {
                "capabilities": {
                    "add": [
                        "NET_BIND_SERVICE"
                    ],
                    "drop": [
                        "ALL"
                    ]
                }                
            },
            "logConfiguration": {
                "logDriver": "awslogs",
                "options": {
                    "awslogs-group": "traefikee",
                    "awslogs-region": "aws_region",
                    "awslogs-stream-prefix": "proxies"
                }
            },
            "dockerLabels": {
                "com.containous.traefikee.component": "proxies"
            }            
        }
    ],
    "requiresCompatibilities": [
        "EC2"
    ]
}

Customize the file then register the task definition on ECS:

aws ecs register-task-definition --cli-input-json file://proxies-task.json --region $AWS_REGION

Now create a file named proxies-svc.json with the service definition:

{
    "serviceName": "traefikee-proxies",
    "taskDefinition": "traefikee-proxies",
    "desiredCount": 2,
    "launchType": "EC2",
    "enableECSManagedTags": true,
    "propagateTags": "SERVICE",
    "placementConstraints": [
        {
            "type": "distinctInstance"
        }
    ],
    "schedulingStrategy": "REPLICA",
    "deploymentController": {
        "type": "ECS"
    }
}

Customize the file and deploy it to the ECS cluster:

aws ecs create-service --cli-input-json file://proxies-svc.json --cluster mycluster --region $AWS_REGION

Check the service status to ensure the service was started:

Command

aws ecs describe-services --services traefikee-proxies --cluster mycluster --region $AWS_REGION

Expected output

{
    "services": [
        {
            "status": "ACTIVE",
            "serviceRegistries": [],
            "pendingCount": 0,
            "launchType": "EC2",
            "enableECSManagedTags": true,
            "schedulingStrategy": "REPLICA",
            "loadBalancers": [],
            "placementConstraints": [
                {
                    "type": "distinctInstance"
                }
            ],
            "createdAt": 1605879947.392,
            "desiredCount": 2,
            "serviceName": "traefikee-proxies",
            "clusterArn": "arn:aws:ecs:aws_region:aws_account_id:cluster/traefikee-ecs",
            "createdBy": "arn:aws:iam::aws_account_id:user/aws_user",
            "taskDefinition": "arn:aws:ecs:aws_region:aws_account_id:task-definition/traefikee-proxies:5",
            "serviceArn": "arn:aws:ecs:aws_region:aws_account_id:service/traefikee-ecs/traefikee-proxies",
            "propagateTags": "SERVICE",
            "deploymentConfiguration": {
                "maximumPercent": 200,
                "minimumHealthyPercent": 100
            },
            "deployments": [
                {
                    "status": "PRIMARY",
                    "pendingCount": 0,
                    "launchType": "EC2",
                    "createdAt": 1605880984.365,
                    "desiredCount": 2,
                    "taskDefinition": "arn:aws:ecs:aws_region:aws_account_id:task-definition/traefikee-proxies:5",
                    "updatedAt": 1605881208.064,
                    "id": "ecs-svc/4529007424825088897",
                    "runningCount": 2
                }
            ],
            "events": [
                {
                    "message": "(service traefikee-proxies) has reached a steady state.",
                    "id": "51e07037-3595-40fa-a394-c489561fcd11",
                    "createdAt": 1605881208.068
                },
                {
                    "message": "(service traefikee-proxies) has started 1 tasks: (task 11ad2458e5624330955dc97f6ce41d41).",
                    "id": "7e74652c-f88c-4e0f-b922-446ac8a7a114",
                    "createdAt": 1605881197.123
                },
                {
                    "message": "(service traefikee-proxies) has reached a steady state.",
                    "id": "2d857d41-658a-452b-b7d6-c1c3bb4a2944",
                    "createdAt": 1605881165.664
                },
                {
                    "message": "(service traefikee-proxies) has started 1 tasks: (task b94de86d3664413081c045ed25f9b326).",
                    "id": "74bb63a2-bbf8-45a2-bc22-a5a266b4289e",
                    "createdAt": 1605881153.114
                },                
            ],
            "runningCount": 2,
            "placementStrategy": []
        }
    ],
    "failures": []
}

Plugin Registry¶

Create a file named registry-task.json with the following task definition:

{
  "executionRoleArn": "arn:aws:iam::aws_account_id:role/ecsTaskExecutionRole",
  "containerDefinitions": [
    {
      "portMappings": [
        {
          "hostPort": 443,
          "protocol": "tcp",
          "containerPort": 443
        }
      ],
      "command": [
        "plugin-registry",
        "--name=registry",
        "--plugindir=/var/lib/plugins",
        "--token=[my-plugin-registry-token]",
        "--log.level=debug",
        "--log.filepath=",
        "--log.format=",
        "--discovery.static.peers=[my-controller-ip]:4242",
        "--jointoken.file.path=/data/tokens"
      ],
      "cpu": 500,
      "mountPoints": [
        {
          "readOnly": false,
          "containerPath": "/data",
          "sourceVolume": "traefikee-data"
        }
      ],
      "memory": 128,
      "image": "traefik/traefikee:v2.11.0",
      "name": "registry"
    }
  ],
  "taskRoleArn": "arn:aws:iam::aws_account_id:role/ecsTaskExecutionRole",
  "family": "registry",
  "requiresCompatibilities": [
    "EC2"
  ],
  "volumes": [
    {
      "name": "traefikee-data",
      "dockerVolumeConfiguration": {
        "autoprovision": false,
        "scope": "shared",
        "driver": "local"
      }
    }
  ]
}

Customize the file then register the task definition on ECS:

aws ecs register-task-definition --cli-input-json file://registry-task.json --region $AWS_REGION

Now create a file named registry-svc.json with the service definition:

{
    "serviceName": "registry",
    "taskDefinition": "registry",
    "desiredCount": 2,
    "launchType": "EC2",
    "enableECSManagedTags": true,
    "propagateTags": "SERVICE",
    "placementConstraints": [
        {
            "type": "distinctInstance"
        }
    ],
    "schedulingStrategy": "REPLICA",
    "deploymentController": {
        "type": "ECS"
    }
}

Customize the file and deploy it to the ECS cluster:

aws ecs create-service --cli-input-json file://registry-svc.json --cluster mycluster --region $AWS_REGION

Check the service status to ensure the service was started:

Command

aws ecs describe-services --services registry --cluster mycluster --region $AWS_REGION

Expected output

{
    "services": [
        {
            "serviceArn": "arn:aws:ecs:eu-north-1:114072598128:service/rc2/registry",
            "serviceName": "registry",
            "clusterArn": "arn:aws:ecs:eu-north-1:114072598128:cluster/rc2",
            "loadBalancers": [],
            "serviceRegistries": [],
            "status": "ACTIVE",
            "desiredCount": 1,
            "runningCount": 1,
            "pendingCount": 0,
            "launchType": "EC2",
            "taskDefinition": "arn:aws:ecs:eu-north-1:114072598128:task-definition/registry:2",
            "deploymentConfiguration": {
                "deploymentCircuitBreaker": {
                    "enable": false,
                    "rollback": false
                },
                "maximumPercent": 200,
                "minimumHealthyPercent": 100
            },
            "deployments": [
                {
                    "id": "ecs-svc/5244500742569760246",
                    "status": "PRIMARY",
                    "taskDefinition": "arn:aws:ecs:eu-north-1:114072598128:task-definition/registry:2",
                    "desiredCount": 1,
                    "pendingCount": 0,
                    "runningCount": 1,
                    "failedTasks": 0,
                    "createdAt": 1610097942.755,
                    "updatedAt": 1610097977.375,
                    "launchType": "EC2",
                    "rolloutState": "COMPLETED",
                    "rolloutStateReason": "ECS deployment ecs-svc/5244500742569760246 completed."
                }
            ],
            "events": [
                {
                    "id": "9b8f4549-e972-4506-848a-8050084f57f0",
                    "createdAt": 1610097977.381,
                    "message": "(service registry) has reached a steady state."
                },
                {
                    "id": "adee0f69-5ceb-4f11-8fd9-47effc1b39c2",
                    "createdAt": 1610097977.38,
                    "message": "(service registry) (deployment ecs-svc/5244500742569760246) deployment completed."
                },
                {
                    "id": "6b070094-8686-4ab7-9886-82c248203698",
                    "createdAt": 1610097966.485,
                    "message": "(service registry) has started 1 tasks: (task 65e57cd8504346da8a8d654917919c7b)."
                },
            ],
            "createdAt": 1610096466.659,
            "placementConstraints": [],
            "placementStrategy": [
                {
                    "type": "spread",
                    "field": "attribute:ecs.availability-zone"
                },
                {
                    "type": "spread",
                    "field": "instanceId"
                }
            ],
            "schedulingStrategy": "REPLICA",
            "createdBy": "arn:aws:iam::114072598128:user/xxx",
            "enableECSManagedTags": true,
            "propagateTags": "NONE"
        }
    ],
    "failures": []
}

Remote Access Through `teectl`¶

Once your cluster is ready, if you want to operate the cluster remotely using the teectl tool, you will need to generate credentials from your cluster using traefikee generate credentials on one of your controllers and use teectl to import the cluster credentials.

First connect to a container instance running a controller task, then run:

Command

# Get the controller container id
CONTAINER_ID=$(docker ps | grep traefikee | cut -f 1 -d " ")
# Get a teectl config with credentials
docker exec -it $CONTAINER_ID /traefikee generate c --onpremise.hosts="[Container-Instance-Public-IP]" --cluster ecs --socket /data/run/teectl-controller-0.sock

Expected output

cluster_name: default
tls:
  cert: |
    -----BEGIN CERTIFICATE-----
    MIIB+jCCAaCgAwIBAgIQKHgQX14LGetAsD8wSEta5jAKBggqhkjOPQQDAjAuMRUw
    EwYDVQQKEwxUcmFlZmlrIExhYnMxFTATBgNVBAMTDFRyYWVmaWtFRSBDQTAgFw0y
    MDExMTkyMDAwMDNaGA8yMTIwMTAyNjIwMDAwM1owFTETMBEGA1UEAxMKY2xpZW50
    LmVjczBZMBMGByqGSM49AgEGCCqGSM49AwEHA0IABKcFJkSGRxbfAnQx2JtYpEnA
    knHdqukFWF8Sbvht30Ge/EnBrTe37ilzJ0KlY11UQK/KlGPMoqVSrjAnqZz10nGj
    gbYwgbMwDgYDVR0PAQH/BAQDAgGGMBMGA1UdJQQMMAoGCCsGAQUFBwMCMAwGA1Ud
    EwEB/wQCMAAwKQYDVR0OBCIEIKh1ygHajrADFcxpfv4p/ff7FH4lDULi0uqcDp7j
    qaylMCsGA1UdIwQkMCKAILoCK8uf89Yveb/uZN+Vs+gfvz05Y6NHB2/hd3T6ix3T
    MCYGA1UdEQQfMB2CCmNsaWVudC5lY3OCCWxvY2FsaG9zdIcEfwAAATAKBggqhkjO
    PQQDAgNIADBFAiEAiXT09co8egGQ0Y6hJ/lgIzCGC0I9GiFfvrxCGKl15l8CICYY
    E6ig/k2D4coprOclVF5QVkqHCcx2EL0HA/zK05eI
    -----END CERTIFICATE-----
  key: |
    -----BEGIN EC PRIVATE KEY-----
    MHcCAQEEIC8CsJ/B115S+JtR1/l3ZQwKA3XdXt9zLqusF1VXc/KloAoGCCqGSM49
    AwEHoUQDQgAEpwUmRIZHFt8CdDHYm1ikScCScd2q6QVYXxJu+G3fQZ78ScGtN7fu
    KXMnQqVjXVRAr8qUY8yipVKuMCepnPXScQ==
    -----END EC PRIVATE KEY-----
  ca: |
    -----BEGIN CERTIFICATE-----
    MIIB2DCCAX2gAwIBAgIQDEHgiwDfaIX3wS1EHF800DAKBggqhkjOPQQDAjAuMRUw
    EwYDVQQKEwxUcmFlZmlrIExhYnMxFTATBgNVBAMTDFRyYWVmaWtFRSBDQTAgFw0y
    MDExMTkxOTU2MjBaGA8yMTIwMTAyNjE5NTYyMFowLjEVMBMGA1UEChMMVHJhZWZp
    ayBMYWJzMRUwEwYDVQQDEwxUcmFlZmlrRUUgQ0EwWTATBgcqhkjOPQIBBggqhkjO
    PQMBBwNCAATe+gM99l/nAAeNIy/kn8vbrcSNORhbWGBUp+j1CtL2ADqLYIel/acB
    B3ssLPnbAZLoKJefrQS/CNJOdZpRZBnfo3sweTAOBgNVHQ8BAf8EBAMCAYYwDwYD
    VR0TAQH/BAUwAwEB/zApBgNVHQ4EIgQgugIry5/z1i95v+5k35Wz6B+/PTljo0cH
    b+F3dPqLHdMwKwYDVR0jBCQwIoAgugIry5/z1i95v+5k35Wz6B+/PTljo0cHb+F3
    dPqLHdMwCgYIKoZIzj0EAwIDSQAwRgIhAMRjpKzzAZr5wT5IXMaBWv0OUvy6wxFn
    kyZApTzhcbr9AiEAtaJZ7DN+xLeA0/RUljK0uDKuZALix30VSx4U2aaYdc4=
    -----END CERTIFICATE-----
onPremise:
  hosts:
  - ec2-18-223-125-157.us-east-1.compute.amazonaws.com
  port: 55055

Save the output of the last command above to a file named teectl-config.yaml, then from your machine run:

teectl cluster import --file="teectl-config.yaml"

You can now use teectl to operate your cluster.

Command

teectl get nodes

Expected output

ID                         NAME                                      STATUS  ROLE
hlx1b3gu8bb5n1lg8qtiy5nvv  controller-0                              Ready   Controller (Leader)
jm5wv9kdmp9imspqx39n300b3  ip-10-0-0-130.us-east-1.compute.internal  Ready   Proxy / Ingress
y0l2me7zjalidnzqf3fqanuxk  ip-10-0-0-178.us-east-1.compute.internal  Ready   Proxy / Ingress

What's Next?

Learn more about the features that Traefik Enterprise provides.

License Monitoring¶

When a Traefik Enterprise controller starts for the first time, it checks the license validity.

If the license is valid, another check is done once every 24 hours.

If the controller can't communicate with the license server, a 72-day grace period starts to recover from this situation.

Once the grace period is over, the controller stops updating the proxies configuration.

Please look at the FAQ to know how to implement the license monitoring.

Installing Traefik Enterprise ECS¶

Requirements¶

Installation¶

Plugin Registry Token¶

Controllers¶

Get the Proxy Join Token¶

Proxies¶

Plugin Registry¶

Remote Access Through teectl¶

License Monitoring¶

Remote Access Through `teectl`¶