Installing Traefik Enterprise on ECS Fargate¶
This page guides you through the installation of Traefik Enterprise on AWS ECS Fargate.
AWS Knowledge
Information about setting up AWS services is not included in this guide. If you want to know more about any of the components referenced in this guide, start with the following resources:
In this guide all examples surrounded by angle brackets <>, must be replaced accordingly.
Requirements¶
- An ECS Fargate cluster
- The
awsCLI tool properly configured to communicate with the cluster - Enough permissions on the ECS Fargate instance role to access any integrated AWS service
- All required ports are open on the associated security group of the container instances
- The
- Controller containers can reach
https://v4.license.containous.cloud - The
teectlCLI tool, for cluster management
Installation¶
Although some external AWS resources are referenced in the logConfiguration and secrets sections, they are optional.
You can just remove or customize them in the task definition.
Otherwise, you must create those resources manually with the AWS Console or CLI tool which is not covered in this guide as well.
Since teectl setup gen does not provide support for ECS Fargate manifest files, this guide demonstrates how to write your own task and service definitions, using JSON syntax, to define the necessary resources and deploy a Traefik Enterprise cluster of one controller and two proxies.
Note that flags used in each task definitions are referenced here: Traefikee Command-Line Reference
IAM Roles and Policy¶
Traefik Enterprise requires the 2 different roles to run, one for the task execution and one when the task is running. Roles needs will map policies, so we gonna need to create the policies first.
Execution role¶
AWS already have a built in policy for the execution policy named AmazonECSTaskExecutionRolePolicy. We can then create the role and attach the policy to it. Let's create a trust relashionships definition in a file named trust-policy.json, and put it this content:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"Service": "ecs-tasks.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}We create the role that we could name ecsTaskExecutionRole with this aws cli command:
aws iam create-role --role-name ecsTaskExecutionRole --assume-role-policy-document file://trust-policy.json And we associate the policy:
aws iam attach-role-policy --policy-arn arn:aws:iam::aws:policy/AmazonECSTaskExecutionRolePolicy --role-name ecsTaskExecutionRoleUse of Secret¶
If you want to use secret to pass your license, and the token as environement variables you need to add a policy to the ecsTaskExecutionRole in order to let the task be able to read the secret.
Here's an example for the secret read policy, that you only need to associate with the execution role:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"secretsmanager:GetSecretValue"
],
"Resource": [
"<secret_arn>"
]
}
]
}Running role¶
As traefik need to access the ECS API for the service discovery, we will need to create a specific policy. To create the policy in AWS you could create a json file with the content below:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "TraefikECSReadAccess",
"Effect": "Allow",
"Action": [
"ecs:ListClusters",
"ecs:DescribeClusters",
"ecs:ListTasks",
"ecs:DescribeTasks",
"ecs:DescribeContainerInstances",
"ecs:DescribeTaskDefinition",
"ec2:DescribeInstances",
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents",
"logs:DescribeLogStreams"
],
"Resource": [
"*"
]
},
{
"Sid": "TraefikECSContainerExec",
"Effect": "Allow",
"Action": [
"ssmmessages:CreateControlChannel",
"ssmmessages:CreateDataChannel",
"ssmmessages:OpenControlChannel",
"ssmmessages:OpenDataChannel"
],
"Resource": [
"*"
]
}
]
}Then create the policy with this command(change the name of the policy and file accordingly):
aws iam create-policy --policy-name TraefikECSReadAccessRolePolicy --policy-document file://TraefikECSReadAccessRolePolicy.jsonApply the same process as above to create a new role and associate the policy.
Service Discovery¶
This deployment use AWS Service discovery to manage internal DNS name for Traefik Enterprise component.
As stated before the AWS Service discovery is documented by AWS and will not be described here in details, this section will just give you the command to create a Service Discovery namespace to deploy Traefik Enterprise.
Create a Service Discovery namespace¶
aws servicediscovery create-private-dns-namespace --name traefikee --vpc <vpc-id>Create a Service Discovery Service for Traefik Enterprise controller¶
aws servicediscovery create-service --name controller-0 --dns-config 'NamespaceId="<ns-id>",DnsRecords=[{Type="A",TTL="60"}]' --health-check-custom-config FailureThreshold=1Create a Service Discovery Service for Traefik Enterprise registry¶
aws servicediscovery create-service --name registry --dns-config 'NamespaceId="<ns-id>",DnsRecords=[{Type="A",TTL="60"}]' --health-check-custom-config FailureThreshold=1In case of many controllers, you will need to set up new services for each controller name accordingly.
Use of Secret¶
If you want to use secret to pass your license, and the token as environement variables you need to add a policy to the ecsTaskExecutionRole in order to let the task be able to read the secret.
here's an example for this policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"secretsmanager:GetSecretValue"
],
"Resource": [
"<secret_arn>"
]
}
]
}Traefik Enterprise will look for specific environment variable during startup, including for its license key and cluster tokens as follows:
- TRAEFIKEE_LICENSE
- TRAEFIKEE_JOIN_TOKEN
- TRAEFIKEE_PLUGIN_TOKEN
However in ECS Fargate environment variables are not expanded at the container startup, which means you can't set command flags with environment variables. In the task definition in order to use the secret as environment varialbe we only will need to remove flags defining those values.
Plugin Registry Token¶
The plugin registry needs a token to secure its communications with the controller. This token is set on the controller and plugin registry task definition.
Here is an example of how to create it:
openssl rand -base64 10MvXVeX3qDylxJQ==Controllers¶
Mono controller installation¶
Controller Storage
Fargate by default assign non-persistent storage to the container, in order to keep state, certificate, and necessary data, if the controller container needs to be recreated you need to add an EFS volume as described on the task definition below.
Create a file named controller-task.json with the following task definition:
{
"family": "traefikee-controller",
"taskRoleArn": "arn:aws:iam::<your_account_id>:role/RoleTraefikECSReadAccess",
"executionRoleArn": "arn:aws:iam::<your_account_id>:role/ecsTaskExecutionRole",
"cpu": "512",
"memory": "1024",
"containerDefinitions": [
{
"name": "controller-0",
"image": "traefik/traefikee:v2.11.0",
"cpu": 500,
"portMappings": [
{
"containerPort": 55055,
"hostPort": 55055,
"protocol": "tcp"
}
],
"essential": true,
"command": [
"controller",
"--name=controller-0",
"--advertise=controller-0.traefikee:4242",
"--license=<your_license>",
"--statedir=/data/state",
"--jointoken.file.path=/data/tokens",
"--api.socket=/var/run/traefikee/teectl-controller-0.sock",
"--socket=/var/run/traefikee/controller-0.sock",
"--api.autocerts",
"--plugin.url=https://registry.traefikee:443",
"--plugin.token=<your_generated_plugin_token>"
],
"linuxParameters": {
"initProcessEnabled": true
},
"environment": [],
"secrets": [],
"volumesFrom": [],
"dockerLabels": {
"com.traefik.traefikee.component": "controller"
},
"mountPoints": [
{
"readOnly": false,
"containerPath": "/data",
"sourceVolume": "traefikee-data"
}
],
"readonlyRootFilesystem": false,
"privileged": false,
"logConfiguration": {
"logDriver": "awslogs",
"options": {
"awslogs-group": "<your_cloud_watch_log_group",
"awslogs-region": "<your_aws_region>",
"awslogs-stream-prefix": "traefikee-controller"
}
}
}
],
"volumes": [
{
"name": "traefikee-data",
"efsVolumeConfiguration": {
"fileSystemId": "<controller_efs_id>",
"transitEncryption": "DISABLED",
"rootDirectory": "/"
}
}
],
"networkMode": "awsvpc",
"runtimePlatform": {
"cpuArchitecture": "X86_64",
"operatingSystemFamily": "LINUX"
},
"requiresCompatibilities": [
"FARGATE"
]
}{
"family": "traefikee-controller",
"taskRoleArn": "arn:aws:iam::<your_account_id>:role/RoleTraefikECSReadAccess",
"executionRoleArn": "arn:aws:iam::<your_account_id>:role/ecsTaskExecutionRole",
"cpu": "512",
"memory": "1024",
"containerDefinitions": [
{
"name": "controller-0",
"image": "traefik/traefikee:v2.11.0",
"cpu": 500,
"portMappings": [
{
"containerPort": 55055,
"hostPort": 55055,
"protocol": "tcp"
}
],
"essential": true,
"command": [
"controller",
"--name=controller-0",
"--advertise=controller-0.traefikee:4242",
"--statedir=/data/state",
"--jointoken.file.path=/data/tokens",
"--api.socket=/var/run/traefikee/teectl-controller-0.sock",
"--socket=/var/run/traefikee/controller-0.sock",
"--api.autocerts",
"--plugin.url=https://registry.traefikee:443"
],
"linuxParameters": {
"initProcessEnabled": true
},
"environment": [],
"secrets": [
{
"name": "TRAEFIKEE_LICENSE",
"valueFrom": "arn:aws:secretsmanager:<your_aws_region>:<your_account_id>:secret:<your_secret_name>:<traefik_license>::"
},
{
"name": "TRAEFIKEE_PLUGIN_TOKEN",
"valueFrom": "arn:aws:secretsmanager:<your_aws_region>:<your_account_id>:secret:<your_secret_name>:<traefik_plugin_token_key>::"
}
],
"volumesFrom": [],
"dockerLabels": {
"com.traefik.traefikee.component": "controller"
},
"mountPoints": [
{
"readOnly": false,
"containerPath": "/data",
"sourceVolume": "traefikee-data"
}
],
"readonlyRootFilesystem": false,
"privileged": false,
"logConfiguration": {
"logDriver": "awslogs",
"options": {
"awslogs-group": "<your_cloud_watch_log_group",
"awslogs-region": "<your_aws_region>",
"awslogs-stream-prefix": "traefikee-controller"
}
}
}
],
"volumes": [
{
"name": "traefikee-data",
"efsVolumeConfiguration": {
"fileSystemId": "<controller_efs_id>",
"transitEncryption": "DISABLED",
"rootDirectory": "/"
}
}
],
"networkMode": "awsvpc",
"runtimePlatform": {
"cpuArchitecture": "X86_64",
"operatingSystemFamily": "LINUX"
},
"requiresCompatibilities": [
"FARGATE"
]
}Customize the file, then register the task definition on ECS Fargate:
aws ecs register-task-definition --cli-input-json file://controller-task.json --region <aws_region>Now create a file named controller-svc.json with the service definition:
{
"serviceName": "traefikee-controller",
"taskDefinition": "traefikee-controller",
"desiredCount": 1,
"launchType": "FARGATE",
"schedulingStrategy": "REPLICA",
"serviceRegistries": [
{
"registryArn": "<service_discovery_controller_service_arn>"
}
],
"networkConfiguration": {
"awsvpcConfiguration": {
"subnets": [
"<subnet_id_a>",
"<subnet_id_b>",
"<subnet_id_c>"
],
"securityGroups": [
"<security_group_id>"
],
"assignPublicIp": "ENABLED"
}
}
}Customize the file and deploy it to the ECS Fargate cluster:
aws ecs create-service --cli-input-json file://controller-svc.json --cluster <my_cluster> --region <aws_region> --enable-execute-commandCheck the service status to ensure the service was started:
aws ecs describe-services --services traefikee-controller --cluster <my_cluster> --region <aws_region>{
"services": [
{
"status": "ACTIVE",
"serviceRegistries": [],
"pendingCount": 0,
"launchType": "FARGATE",
"enableECSManagedTags": true,
"schedulingStrategy": "REPLICA",
"loadBalancers": [],
"placementConstraints": [
{
"type": "distinctInstance"
}
],
"createdAt": 1605815767.081,
"desiredCount": 1,
"serviceName": "traefikee-controller",
"clusterArn": "arn:aws:ecs:aws_region:aws_account_id:cluster/traefikee-ecs",
"createdBy": "arn:aws:iam::aws_account_id:user/aws_user",
"taskDefinition": "arn:aws:ecs:aws_region:aws_account_id:task-definition/traefikee-controller:7",
"serviceArn": "arn:aws:ecs:aws_region:aws_account_id:service/traefikee-ecs/traefikee-controller",
"propagateTags": "SERVICE",
"deploymentConfiguration": {
"maximumPercent": 200,
"minimumHealthyPercent": 100
},
"deployments": [
{
"status": "PRIMARY",
"pendingCount": 0,
"launchType": "EC2",
"createdAt": 1605815767.081,
"desiredCount": 1,
"taskDefinition": "arn:aws:ecs:aws_region:aws_account_id:task-definition/traefikee-controller:7",
"updatedAt": 1605815785.903,
"id": "ecs-svc/4291651818662396701",
"runningCount": 1
}
],
"events": [
{
"message": "(service traefikee-controller) has reached a steady state.",
"id": "2ae7ae80-11bc-48fa-bd1c-ec6235521928",
"createdAt": 1605815785.907
},
{
"message": "(service traefikee-controller) has started 1 tasks: (task fe932a1e5690403eacd5d3b4f8221b18).",
"id": "901615c4-8e67-4ed2-91f6-b52fcc88d147",
"createdAt": 1605815774.564
}
],
"runningCount": 1,
"placementStrategy": []
}
],
"failures": []
}Wait until the controllers are up and running before proceeding to the next steps.
Get the Proxy Join Token¶
Fetch the proxy join token by connecting to the container instance and executing the following commands:
# Get the tokens
aws ecs execute-command --cluster <my_cluster> --container controller-0 --task <task-arn> --interactive --command "/traefikee tokens --socket /var/run/traefikee/controller-0.sock"export TRAEFIKEE_CONTROLLER_TOKEN=5531644e5645744f4c5445744e47706f4e7a5a3261574a765a485274616a55345a57526f4d7a566f65444e6f4e544e70616d4a6d615459314e5464684f48526f4d6e6c735933466e4e33686a6348417459544e6c4e5451314e475a70616d4e304e484e796432466f4e445a796447707859673d3d3a303a97b42afa416f0df94d7c453bd4c55fcddd339ec4570068eac2c5cc5504c158d42521e9852615ab95049e2b3296b67c4a
export TRAEFIKEE_PROXY_TOKEN=5531644e5645744f4c5445744e47706f4e7a5a3261574a765a485274616a55345a57526f4d7a566f65444e6f4e544e70616d4a6d615459314e5464684f48526f4d6e6c735933466e4e33686a634841744e47397563484d31636a4d354d7a42354e5463785933557762546b774e7a55794f413d3d3a313a97b42afa416f0df94d7c453bd4c55fcddd339ec4570068eac2c5cc5504c158d42194b3059e6861761121966a44f964d3Write down the proxy token as it is required for the next step, the proxies task definition.
Multi controller Installation¶
Controller Storage
Fargate by default assign non-persistent storage to the container, in order to keep state, certificate, and necessary data, if the controllers needs to be recreated you need to add an EFS volume for each controller as described on the tasks definitions below.
Note that multi controller should not be deployed without secret, each controller will need the a join token if it is recreated or restarted. The environment variable that will be checked by the controller at bootstrap is TRAEFIKEE_JOIN_TOKEN.
In order to manage each controller independently in case of maintenance, it is recommended to create each and every controller as separated tasks.
After deploying the first controller get the join token and update secret accordingly.
Here is the example of the configuration for a 3 nodes controller cluster:
{
"family": "traefikee-controller-0",
"taskRoleArn": "arn:aws:iam::<your_account_id>:role/RoleTraefikECSReadAccess",
"executionRoleArn": "arn:aws:iam::<your_account_id>:role/ecsTaskExecutionRole",
"cpu": "512",
"memory": "1024",
"containerDefinitions": [
{
"name": "controller-0",
"image": "traefik/traefikee:v2.11.0",
"cpu": 500,
"portMappings": [
{
"containerPort": 55055,
"hostPort": 55055,
"protocol": "tcp"
}
],
"essential": true,
"command": [
"controller",
"--name=controller-0",
"--advertise=controller-0.traefikee:4242",
"--discovery.static.peers=controller-1.traefikee:4242,controller-2.traefikee:4242",
"--statedir=/data/state",
"--jointoken.file.path=/data/tokens",
"--api.socket=/var/run/traefikee/teectl-controller-0.sock",
"--socket=/var/run/traefikee/controller-0.sock",
"--api.autocerts",
"--plugin.url=https://registry.traefikee:443"
],
"linuxParameters": {
"initProcessEnabled": true
},
"environment": [],
"secrets": [
{
"name": "TRAEFIKEE_LICENSE",
"valueFrom": "arn:aws:secretsmanager:<your_aws_region>:<your_account_id>:secret:<your_secret_name>:<traefik_license>::"
},
{
"name": "TRAEFIKEE_PLUGIN_TOKEN",
"valueFrom": "arn:aws:secretsmanager:<your_aws_region>:<your_account_id>:secret:<your_secret_name>:<traefik_plugin_token_key>::"
},
{
"name": "TRAEFIKEE_JOIN_TOKEN",
"valueFrom": "arn:aws:secretsmanager:<your_aws_region>:<your_account_id>:secret:<your_secret_name>:<traefik_controller_join_token_key>::"
}
],
"volumesFrom": [],
"dockerLabels": {
"com.traefik.traefikee.component": "controller-0"
},
"mountPoints": [
{
"readOnly": false,
"containerPath": "/data",
"sourceVolume": "traefikee-data"
}
],
"readonlyRootFilesystem": false,
"privileged": false,
"logConfiguration": {
"logDriver": "awslogs",
"options": {
"awslogs-group": "<your_cloud_watch_log_group",
"awslogs-region": "<your_aws_region>",
"awslogs-stream-prefix": "traefikee-controller-0"
}
}
}
],
"volumes": [
{
"name": "traefikee-data",
"efsVolumeConfiguration": {
"fileSystemId": "<controller-0_efs_id>",
"transitEncryption": "DISABLED",
"rootDirectory": "/"
}
}
],
"networkMode": "awsvpc",
"runtimePlatform": {
"cpuArchitecture": "X86_64",
"operatingSystemFamily": "LINUX"
},
"requiresCompatibilities": [
"FARGATE"
]
}{
"family": "traefikee-controller-1",
"taskRoleArn": "arn:aws:iam::<your_account_id>:role/RoleTraefikECSReadAccess",
"executionRoleArn": "arn:aws:iam::<your_account_id>:role/ecsTaskExecutionRole",
"cpu": "512",
"memory": "1024",
"containerDefinitions": [
{
"name": "controller-1",
"image": "traefik/traefikee:v2.11.0",
"cpu": 500,
"portMappings": [
{
"containerPort": 55055,
"hostPort": 55055,
"protocol": "tcp"
}
],
"essential": true,
"command": [
"controller",
"--name=controller-1",
"--advertise=controller-1.traefikee:4242",
"--discovery.static.peers=controller-0.traefikee:4242,controller-2.traefikee:4242",
"--statedir=/data/state",
"--jointoken.file.path=/data/tokens",
"--api.socket=/var/run/traefikee/teectl-controller-1.sock",
"--socket=/var/run/traefikee/controller-1.sock",
"--api.autocerts",
"--plugin.url=https://registry.traefikee:443"
],
"linuxParameters": {
"initProcessEnabled": true
},
"environment": [],
"secrets": [
{
"name": "TRAEFIKEE_LICENSE",
"valueFrom": "arn:aws:secretsmanager:<your_aws_region>:<your_account_id>:secret:<your_secret_name>:<traefik_license>::"
},
{
"name": "TRAEFIKEE_PLUGIN_TOKEN",
"valueFrom": "arn:aws:secretsmanager:<your_aws_region>:<your_account_id>:secret:<your_secret_name>:<traefik_plugin_token_key>::"
},
{
"name": "TRAEFIKEE_JOIN_TOKEN",
"valueFrom": "arn:aws:secretsmanager:<your_aws_region>:<your_account_id>:secret:<your_secret_name>:<traefik_controller_join_token_key>::"
}
],
"volumesFrom": [],
"dockerLabels": {
"com.traefik.traefikee.component": "controller-1"
},
"mountPoints": [
{
"readOnly": false,
"containerPath": "/data",
"sourceVolume": "traefikee-data"
}
],
"readonlyRootFilesystem": false,
"privileged": false,
"logConfiguration": {
"logDriver": "awslogs",
"options": {
"awslogs-group": "<your_cloud_watch_log_group",
"awslogs-region": "<your_aws_region>",
"awslogs-stream-prefix": "traefikee-controller-1"
}
}
}
],
"volumes": [
{
"name": "traefikee-data",
"efsVolumeConfiguration": {
"fileSystemId": "<controller-1_efs_id>",
"transitEncryption": "DISABLED",
"rootDirectory": "/"
}
}
],
"networkMode": "awsvpc",
"runtimePlatform": {
"cpuArchitecture": "X86_64",
"operatingSystemFamily": "LINUX"
},
"requiresCompatibilities": [
"FARGATE"
]
}{
"family": "traefikee-controller-2",
"taskRoleArn": "arn:aws:iam::<your_account_id>:role/RoleTraefikECSReadAccess",
"executionRoleArn": "arn:aws:iam::<your_account_id>:role/ecsTaskExecutionRole",
"cpu": "512",
"memory": "1024",
"containerDefinitions": [
{
"name": "controller-2",
"image": "traefik/traefikee:v2.11.0",
"cpu": 500,
"portMappings": [
{
"containerPort": 55055,
"hostPort": 55055,
"protocol": "tcp"
}
],
"essential": true,
"command": [
"controller",
"--name=controller-2",
"--advertise=controller-2.traefikee:4242",
"--discovery.static.peers=controller-0.traefikee:4242,controller-1.traefikee:4242",
"--statedir=/data/state",
"--jointoken.file.path=/data/tokens",
"--api.socket=/var/run/traefikee/teectl-controller-2.sock",
"--socket=/var/run/traefikee/controller-2.sock",
"--api.autocerts",
"--plugin.url=https://registry.traefikee:443"
],
"linuxParameters": {
"initProcessEnabled": true
},
"environment": [],
"secrets": [
{
"name": "TRAEFIKEE_LICENSE",
"valueFrom": "arn:aws:secretsmanager:<your_aws_region>:<your_account_id>:secret:<your_secret_name>:<traefik_license>::"
},
{
"name": "TRAEFIKEE_PLUGIN_TOKEN",
"valueFrom": "arn:aws:secretsmanager:<your_aws_region>:<your_account_id>:secret:<your_secret_name>:<traefik_plugin_token_key>::"
},
{
"name": "TRAEFIKEE_JOIN_TOKEN",
"valueFrom": "arn:aws:secretsmanager:<your_aws_region>:<your_account_id>:secret:<your_secret_name>:<traefik_controller_join_token_key>::"
}
],
"volumesFrom": [],
"dockerLabels": {
"com.traefik.traefikee.component": "controller-2"
},
"mountPoints": [
{
"readOnly": false,
"containerPath": "/data",
"sourceVolume": "traefikee-data"
}
],
"readonlyRootFilesystem": false,
"privileged": false,
"logConfiguration": {
"logDriver": "awslogs",
"options": {
"awslogs-group": "<your_cloud_watch_log_group>",
"awslogs-region": "<your_aws_region>",
"awslogs-stream-prefix": "traefikee-controller-2"
}
}
}
],
"volumes": [
{
"name": "traefikee-data",
"efsVolumeConfiguration": {
"fileSystemId": "<controller-2_efs_id>",
"transitEncryption": "DISABLED",
"rootDirectory": "/"
}
}
],
"networkMode": "awsvpc",
"runtimePlatform": {
"cpuArchitecture": "X86_64",
"operatingSystemFamily": "LINUX"
},
"requiresCompatibilities": [
"FARGATE"
]
}Create a service for each task as described in the mono controller section and deploy.
Proxies¶
Create a file named proxies-task.json with the following task definition:
{
"family": "traefikee-proxies",
"taskRoleArn": "arn:aws:iam::<your_account_id>:role/RoleTraefikECSReadAccess",
"executionRoleArn": "arn:aws:iam::<your_account_id>:role/ecsTaskExecutionRole",
"cpu": "512",
"memory": "1024",
"containerDefinitions": [
{
"name": "proxy",
"image": "traefik/traefikee:v2.11.0",
"cpu": 500,
"portMappings": [
{
"containerPort": 80,
"hostPort": 80,
"protocol": "tcp"
},
{
"containerPort": 443,
"hostPort": 443,
"protocol": "tcp"
},
{
"containerPort": 8484,
"hostPort": 8484,
"protocol": "tcp"
}
],
"essential": true,
"command": [
"proxy",
"--role=ingress",
"--discovery.static.peers=controller-0.traefikee:4242",
"--jointoken.value=<proxy_join_token>"
],
"linuxParameters": {},
"environment": [],
"volumesFrom": [],
"secrets": [],
"dockerLabels": {
"com.traefik.traefikee.component": "proxies"
},
"mountPoints": [],
"readonlyRootFilesystem": true,
"privileged": false,
"logConfiguration": {
"logDriver": "awslogs",
"options": {
"awslogs-group": "<your_cloud_watch_log_group",
"awslogs-region": "<your_aws_region>",
"awslogs-stream-prefix": "traefikee-proxies"
}
}
],
"networkMode": "awsvpc",
"runtimePlatform": {
"cpuArchitecture": "X86_64",
"operatingSystemFamily": "LINUX"
},
"requiresCompatibilities": [
"FARGATE"
]
}{
"family": "traefikee-proxies",
"taskRoleArn": "arn:aws:iam::<your_account_id>:role/RoleTraefikECSReadAccess",
"executionRoleArn": "arn:aws:iam::<your_account_id>:role/ecsTaskExecutionRole",
"cpu": "512",
"memory": "1024",
"containerDefinitions": [
{
"name": "proxy",
"image": "traefik/traefikee:v2.11.0",
"cpu": 500,
"portMappings": [
{
"containerPort": 80,
"hostPort": 80,
"protocol": "tcp"
},
{
"containerPort": 443,
"hostPort": 443,
"protocol": "tcp"
},
{
"containerPort": 8484,
"hostPort": 8484,
"protocol": "tcp"
}
],
"essential": true,
"command": [
"proxy",
"--role=ingress",
"--discovery.static.peers=controller-0.traefikee:4242"
],
"linuxParameters": {},
"environment": [],
"volumesFrom": [],
"secrets": [
{
"name": "TRAEFIKEE_JOIN_TOKEN",
"valueFrom": "arn:aws:secretsmanager:<your_aws_region>:<your_account_id>:secret:<your_secret_name>:<traefik_proxy_join_token_key>::"
}
],
"dockerLabels": {
"com.traefik.traefikee.component": "proxies"
},
"mountPoints": [],
"readonlyRootFilesystem": true,
"privileged": false,
"logConfiguration": {
"logDriver": "awslogs",
"options": {
"awslogs-group": "<your_cloud_watch_log_group",
"awslogs-region": "<your_aws_region>",
"awslogs-stream-prefix": "traefikee-proxies"
}
}
}
],
"networkMode": "awsvpc",
"runtimePlatform": {
"cpuArchitecture": "X86_64",
"operatingSystemFamily": "LINUX"
},
"requiresCompatibilities": [
"FARGATE"
]
}Proxies
In case of multi controller installation you should add all the controller in the static.discovery.peers field.
Customize the file then register the task definition in Fargate:
aws ecs register-task-definition --cli-input-json file://proxies-task.json --region <aws_region>Now create a file named proxies-svc.json with the service definition:
{
"serviceName": "traefikee-proxies",
"taskDefinition": "traefikee-proxies",
"desiredCount": 2,
"launchType": "FARGATE",
"schedulingStrategy": "REPLICA",
"networkConfiguration": {
"awsvpcConfiguration": {
"subnets": [
"<subnet_id_a>",
"<subnet_id_b>",
"<subnet_id_c>"
],
"securityGroups": [
"<security_group_id>"
],
"assignPublicIp": "ENABLED"
}
}
}Customize the file and deploy it to the Fargate cluster:
aws ecs create-service --cli-input-json file://proxies-svc.json --cluster <my_cluster> --region <aws_region>Check the service status to ensure the service was started:
aws ecs describe-services --services traefikee-proxies --cluster <my_cluster> --region <aws_region>{
"services": [
{
"status": "ACTIVE",
"serviceRegistries": [],
"pendingCount": 0,
"launchType": "EC2",
"enableECSManagedTags": true,
"schedulingStrategy": "REPLICA",
"loadBalancers": [],
"placementConstraints": [
{
"type": "distinctInstance"
}
],
"createdAt": 1605879947.392,
"desiredCount": 2,
"serviceName": "traefikee-proxies",
"clusterArn": "arn:aws:ecs:aws_region:aws_account_id:cluster/traefikee-ecs",
"createdBy": "arn:aws:iam::aws_account_id:user/aws_user",
"taskDefinition": "arn:aws:ecs:aws_region:aws_account_id:task-definition/traefikee-proxies:5",
"serviceArn": "arn:aws:ecs:aws_region:aws_account_id:service/traefikee-ecs/traefikee-proxies",
"propagateTags": "SERVICE",
"deploymentConfiguration": {
"maximumPercent": 200,
"minimumHealthyPercent": 100
},
"deployments": [
{
"status": "PRIMARY",
"pendingCount": 0,
"launchType": "EC2",
"createdAt": 1605880984.365,
"desiredCount": 2,
"taskDefinition": "arn:aws:ecs:aws_region:aws_account_id:task-definition/traefikee-proxies:5",
"updatedAt": 1605881208.064,
"id": "ecs-svc/4529007424825088897",
"runningCount": 2
}
],
"events": [
{
"message": "(service traefikee-proxies) has reached a steady state.",
"id": "51e07037-3595-40fa-a394-c489561fcd11",
"createdAt": 1605881208.068
},
{
"message": "(service traefikee-proxies) has started 1 tasks: (task 11ad2458e5624330955dc97f6ce41d41).",
"id": "7e74652c-f88c-4e0f-b922-446ac8a7a114",
"createdAt": 1605881197.123
},
{
"message": "(service traefikee-proxies) has reached a steady state.",
"id": "2d857d41-658a-452b-b7d6-c1c3bb4a2944",
"createdAt": 1605881165.664
},
{
"message": "(service traefikee-proxies) has started 1 tasks: (task b94de86d3664413081c045ed25f9b326).",
"id": "74bb63a2-bbf8-45a2-bc22-a5a266b4289e",
"createdAt": 1605881153.114
}
],
"runningCount": 2,
"placementStrategy": []
}
],
"failures": []
}Plugin Registry¶
Create a file named registry-task.json with the following task definition:
Registry Storage
Fargate by default assign non-persistent storage to the container, in order to keep your custom plugins, if the registry container needs to be recreated you need to add an EFS volume as described on the task definition below.
{
"family": "traefikee-registry",
"taskRoleArn": "arn:aws:iam::<your_account_id>:role/RoleTraefikECSReadAccess",
"executionRoleArn": "arn:aws:iam::<your_account_id>:role/ecsTaskExecutionRole",
"cpu": "512",
"memory": "1024",
"containerDefinitions": [
{
"name" : "registry",
"image": "traefik/traefikee:v2.11.0",
"cpu": 500,
"portMappings": [
{
"hostPort": 443,
"protocol": "tcp",
"containerPort": 443
}
],
"essential": true,
"command": [
"plugin-registry",
"--name=registry",
"--discovery.static.peers=controller-0.traefikee",
"--plugindir=/var/lib/plugins",
"--token=<your_generated_plugin_token>",
"--jointoken.value=<proxy_join_token>"
],
"linuxParameters": {},
"environment": [],
"volumesFrom": [],
"secrets": [],
"dockerLabels": {
"com.traefik.traefikee.component": "registry"
},
"mountPoints": [
{
"readOnly": false,
"containerPath": "/var/lib/plugins",
"sourceVolume": "traefikee-plugins"
}
],
"readonlyRootFilesystem": false,
"privileged": false,
"logConfiguration": {
"logDriver": "awslogs",
"options": {
"awslogs-group": "<your_cloud_watch_log_group",
"awslogs-region": "<your_aws_region>",
"awslogs-stream-prefix": "traefikee-registry"
}
}
}
],
"volumes": [
{
"name": "traefikee-plugins",
"efsVolumeConfiguration": {
"fileSystemId": "<registry_efs_id>",
"transitEncryption": "DISABLED",
"rootDirectory": "/"
}
}
],
"networkMode": "awsvpc",
"runtimePlatform": {
"cpuArchitecture": "X86_64",
"operatingSystemFamily": "LINUX"
},
"requiresCompatibilities": [
"FARGATE"
]
}{
"family": "traefikee-registry",
"taskRoleArn": "arn:aws:iam::<your_account_id>:role/RoleTraefikECSReadAccess",
"executionRoleArn": "arn:aws:iam::<your_account_id>:role/ecsTaskExecutionRole",
"cpu": "512",
"memory": "1024",
"containerDefinitions": [
{
"name" : "registry",
"image": "traefik/traefikee:v2.11.0",
"cpu": 500,
"portMappings": [
{
"hostPort": 443,
"protocol": "tcp",
"containerPort": 443
}
],
"essential": true,
"command": [
"plugin-registry",
"--name=registry",
"--discovery.dns.domain=controller-0.traefikee",
"--plugindir=/var/lib/plugins"
],
"linuxParameters": {},
"environment": [],
"volumesFrom": [],
"secrets": [
{
"name": "TRAEFIKEE_PLUGIN_TOKEN",
"valueFrom": "arn:aws:secretsmanager:<your_aws_region>:<your_account_id>:secret:<your_secret_name>:<traefik_plugin_token_key>::"
},
{
"name": "TRAEFIKEE_JOIN_TOKEN",
"valueFrom": "arn:aws:secretsmanager:<your_aws_region>:<your_account_id>:secret:<your_secret_name>:<traefik_proxy_join_token_key>::"
}
],
"dockerLabels": {
"com.traefik.traefikee.component": "registry"
},
"mountPoints": [
{
"readOnly": false,
"containerPath": "/var/lib/plugins",
"sourceVolume": "traefikee-plugins"
}
],
"readonlyRootFilesystem": false,
"privileged": false,
"logConfiguration": {
"logDriver": "awslogs",
"options": {
"awslogs-group": "<your_cloud_watch_log_group",
"awslogs-region": "<your_aws_region>",
"awslogs-stream-prefix": "traefikee-registry"
}
}
}
],
"volumes": [
{
"name": "traefikee-plugins",
"efsVolumeConfiguration": {
"fileSystemId": "<registry_efs_id>",
"transitEncryption": "DISABLED",
"rootDirectory": "/"
}
}
],
"networkMode": "awsvpc",
"runtimePlatform": {
"cpuArchitecture": "X86_64",
"operatingSystemFamily": "LINUX"
},
"requiresCompatibilities": [
"FARGATE"
]
}Registry with Multi controller
In case of multi controller installation, you should add all the controller in the static.discovery.peers field.
Customize the file, then register the task definition in Fargate:
aws ecs register-task-definition --cli-input-json file://registry-task.json --region <aws_region>Now create a file named registry-svc.json with the service definition:
{
"serviceName": "traefikee-registry",
"taskDefinition": "traefikee-registry",
"desiredCount": 1,
"launchType": "FARGATE",
"schedulingStrategy": "REPLICA",
"serviceRegistries": [
{
"registryArn": "<service_discovery_registry_service_arn>"
}
],
"networkConfiguration": {
"awsvpcConfiguration": {
"subnets": [
"<subnet_id_a>",
"<subnet_id_b>",
"<subnet_id_c>"
],
"securityGroups": [
"<security_group_id>"
],
"assignPublicIp": "ENABLED"
}
}
}Customize the file and deploy it to the Fargate cluster:
aws ecs create-service --cli-input-json file://registry-svc.json --cluster <my_cluster> --region <aws_region>Check the service status to ensure the service was started:
aws ecs describe-services --services registry --cluster <my_cluster> --region <aws_region>{
"services": [
{
"serviceArn": "arn:aws:ecs:eu-north-1:114072598128:service/traefikee-ecs/traefikee-registry",
"serviceName": "traefikee-registry",
"clusterArn": "arn:aws:ecs:eu-north-1:114072598128:cluster/traefikee-ecs",
"loadBalancers": [],
"serviceRegistries": [],
"status": "ACTIVE",
"desiredCount": 1,
"runningCount": 1,
"pendingCount": 0,
"launchType": "EC2",
"taskDefinition": "arn:aws:ecs:eu-north-1:114072598128:task-definition/traefikee-registry:2",
"deploymentConfiguration": {
"deploymentCircuitBreaker": {
"enable": false,
"rollback": false
},
"maximumPercent": 200,
"minimumHealthyPercent": 100
},
"deployments": [
{
"id": "ecs-svc/5244500742569760246",
"status": "PRIMARY",
"taskDefinition": "arn:aws:ecs:eu-north-1:114072598128:task-definition/traefikee-registry:2",
"desiredCount": 1,
"pendingCount": 0,
"runningCount": 1,
"failedTasks": 0,
"createdAt": 1610097942.755,
"updatedAt": 1610097977.375,
"launchType": "EC2",
"rolloutState": "COMPLETED",
"rolloutStateReason": "ECS deployment ecs-svc/5244500742569760246 completed."
}
],
"events": [
{
"id": "9b8f4549-e972-4506-848a-8050084f57f0",
"createdAt": 1610097977.381,
"message": "(service registry) has reached a steady state."
},
{
"id": "adee0f69-5ceb-4f11-8fd9-47effc1b39c2",
"createdAt": 1610097977.38,
"message": "(service registry) (deployment ecs-svc/5244500742569760246) deployment completed."
},
{
"id": "6b070094-8686-4ab7-9886-82c248203698",
"createdAt": 1610097966.485,
"message": "(service registry) has started 1 tasks: (task 65e57cd8504346da8a8d654917919c7b)."
},
],
"createdAt": 1610096466.659,
"placementConstraints": [],
"placementStrategy": [
{
"type": "spread",
"field": "attribute:ecs.availability-zone"
},
{
"type": "spread",
"field": "instanceId"
}
],
"schedulingStrategy": "REPLICA",
"createdBy": "arn:aws:iam::114072598128:user/xxx",
"enableECSManagedTags": true,
"propagateTags": "NONE"
}
],
"failures": []
}Remote Access Through teectl¶
Once your cluster is ready, if you want to operate the cluster remotely using the teectl tool, you will need to
generate credentials from your cluster using traefikee generate credentials on one of your controllers and use teectl to import the cluster credentials.
First connect to a container instance running a controller task, then run:
# Get a teectl config with credentials
aws ecs execute-command --cluster <my_cluster> --container controller-0 --task <task-arn> --interactive --command "/traefikee generate credentials --cluster <my_cluster> --onpremise.hosts <controller-container-public-ip> --socket /var/run/teectl-controller-0.sock"cluster_name: <my_cluster>
tls:
cert: |
-----BEGIN CERTIFICATE-----
MIIB+jCCAaCgAwIBAgIQKHgQX14LGetAsD8wSEta5jAKBggqhkjOPQQDAjAuMRUw
EwYDVQQKEwxUcmFlZmlrIExhYnMxFTATBgNVBAMTDFRyYWVmaWtFRSBDQTAgFw0y
MDExMTkyMDAwMDNaGA8yMTIwMTAyNjIwMDAwM1owFTETMBEGA1UEAxMKY2xpZW50
LmVjczBZMBMGByqGSM49AgEGCCqGSM49AwEHA0IABKcFJkSGRxbfAnQx2JtYpEnA
knHdqukFWF8Sbvht30Ge/EnBrTe37ilzJ0KlY11UQK/KlGPMoqVSrjAnqZz10nGj
gbYwgbMwDgYDVR0PAQH/BAQDAgGGMBMGA1UdJQQMMAoGCCsGAQUFBwMCMAwGA1Ud
EwEB/wQCMAAwKQYDVR0OBCIEIKh1ygHajrADFcxpfv4p/ff7FH4lDULi0uqcDp7j
qaylMCsGA1UdIwQkMCKAILoCK8uf89Yveb/uZN+Vs+gfvz05Y6NHB2/hd3T6ix3T
MCYGA1UdEQQfMB2CCmNsaWVudC5lY3OCCWxvY2FsaG9zdIcEfwAAATAKBggqhkjO
PQQDAgNIADBFAiEAiXT09co8egGQ0Y6hJ/lgIzCGC0I9GiFfvrxCGKl15l8CICYY
E6ig/k2D4coprOclVF5QVkqHCcx2EL0HA/zK05eI
-----END CERTIFICATE-----
key: |
-----BEGIN EC PRIVATE KEY-----
MHcCAQEEIC8CsJ/B115S+JtR1/l3ZQwKA3XdXt9zLqusF1VXc/KloAoGCCqGSM49
AwEHoUQDQgAEpwUmRIZHFt8CdDHYm1ikScCScd2q6QVYXxJu+G3fQZ78ScGtN7fu
KXMnQqVjXVRAr8qUY8yipVKuMCepnPXScQ==
-----END EC PRIVATE KEY-----
ca: |
-----BEGIN CERTIFICATE-----
MIIB2DCCAX2gAwIBAgIQDEHgiwDfaIX3wS1EHF800DAKBggqhkjOPQQDAjAuMRUw
EwYDVQQKEwxUcmFlZmlrIExhYnMxFTATBgNVBAMTDFRyYWVmaWtFRSBDQTAgFw0y
MDExMTkxOTU2MjBaGA8yMTIwMTAyNjE5NTYyMFowLjEVMBMGA1UEChMMVHJhZWZp
ayBMYWJzMRUwEwYDVQQDEwxUcmFlZmlrRUUgQ0EwWTATBgcqhkjOPQIBBggqhkjO
PQMBBwNCAATe+gM99l/nAAeNIy/kn8vbrcSNORhbWGBUp+j1CtL2ADqLYIel/acB
B3ssLPnbAZLoKJefrQS/CNJOdZpRZBnfo3sweTAOBgNVHQ8BAf8EBAMCAYYwDwYD
VR0TAQH/BAUwAwEB/zApBgNVHQ4EIgQgugIry5/z1i95v+5k35Wz6B+/PTljo0cH
b+F3dPqLHdMwKwYDVR0jBCQwIoAgugIry5/z1i95v+5k35Wz6B+/PTljo0cHb+F3
dPqLHdMwCgYIKoZIzj0EAwIDSQAwRgIhAMRjpKzzAZr5wT5IXMaBWv0OUvy6wxFn
kyZApTzhcbr9AiEAtaJZ7DN+xLeA0/RUljK0uDKuZALix30VSx4U2aaYdc4=
-----END CERTIFICATE-----
onPremise:
hosts:
- <controller-container-public-ip>
port: 55055Save the output of the last command above to a file named teectl-config.yaml, and then run:
teectl cluster import --file="teectl-config.yaml"teectl cluster use --name <my_cluster>You can now use teectl to operate your cluster.
teectl get nodesID NAME STATUS ROLE
hlx1b3gu8bb5n1lg8qtiy5nvv controller-0 Ready Controller (Leader)
jm5wv9kdmp9imspqx39n300b3 ip-10-0-0-130.us-east-1.compute.internal Ready Proxy / Ingress
y0l2me7zjalidnzqf3fqanuxk ip-10-0-0-178.us-east-1.compute.internal Ready Proxy / Ingress
odxyyk3l7pkwiab8rtwvhflgs registry Ready Plugin RegistryGoing further
Now that the cluster is ready, we recommend reading the various operating guides to dive into all the features that Traefik Enterprise provides.