Open Policy Agent (OPA) User Guide¶
Open Policy Agent (OPA) is an open source, general-purpose policy engine that can be used to enforce unified, context-aware access policies across the application stack. Policies are definied using a high-level, declarative rules language called Rego.
Traeifk Enterprise 2.4 and later includes an OPA Middleware that supports Rego policies. This guide demonstrates how to use this Middleware to enforce policies for services running on a Kubernetes cluster.
Prerequisites¶
To complete this tutorial, you will need a few things:
- A running Kubernetes cluster
- The
kubectl
binary installed and configured to connect to the cluster - Traefik Enterprise installed on the cluster, with a static configuration applied
- Optional: The installation manifest (as output by
teectl
during installation) saved asmanifest.yaml
Deploy a Demo Service¶
For purposes of this demonstration, you will run an instance of whoami
, a simple service that does nothing more than return some information about the requests it receives.
The file whoami-kubernetes.yaml
, below, creates a Kubernetes Deployment with a single instance of whoami
, an accompanying Kubernetes Service, and an IngressRoute that instructs Traefik Enterprise to expose it on the loopback interface at whoami.localhost
:
whoami-kubernetes.yaml
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: whoami
namespace: traefikee
spec:
replicas: 1
selector:
matchLabels:
app: whoami
template:
metadata:
labels:
app: whoami
spec:
containers:
- name: whoami
image: traefik/whoami
---
apiVersion: v1
kind: Service
metadata:
name: whoami
namespace: traefikee
labels:
app: whoami
spec:
type: ClusterIP
ports:
- port: 80
name: whoami
selector:
app: whoami
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: whoami
namespace: traefikee
spec:
entryPoints:
- web
routes:
- match: Host(`whoami.localhost`)
kind: Rule
services:
- name: whoami
namespace: traefikee
port: 80
You should deploy this and the other configuration files in this guide using kubectl
, following the pattern:
kubectl apply -f whoami-kubernetes.yaml
Once the service is deployed, you can verify that it's running and producing typical output:
curl -H Host:whoami.localhost http://localhost
Hostname: whoami-deployment-676886cb66-52h4n
IP: 127.0.0.1
IP: 10.1.0.123
RemoteAddr: 10.1.0.122:54010
GET / HTTP/1.1
Host: whoami.localhost
User-Agent: curl/7.64.1
Accept: */*
Accept-Encoding: gzip
X-Forwarded-For: 192.168.65.3
X-Forwarded-Host: whoami.localhost
X-Forwarded-Port: 80
X-Forwarded-Proto: http
X-Forwarded-Server: traefik-77fdb5c487-42cd6
X-Real-Ip: 192.168.65.3
Secure the Service with OPA¶
Enabling OPA-based access policies on your new service is a two-step process.
First, define an instance of the Traefik Enterprise OPA Middleware:
---
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
name: my-opa-plugin
namespace: traefikee
spec:
plugin:
opa:
allow: data.example.authz.allow
forwardHeaders:
Group: data.example.authz.group
policy: |
package example.authz
default allow = false
default group = ""
auth := split(input.headers.Authorization, " ")
#[header, payload, signature]
jwtDecode := io.jwt.decode(auth[1])
payload := jwtDecode[1]
allow {
payload["email"] == "[email protected]"
}
# if allow is true, group will hold payload["grp"] value
group = g {
allow
g = payload["grp"]
}
Notice that this configuration includes an OPA policy definition (in Rego). It defines two policies, each of which depends on the contents of a JSON Web Token (JWT) that must be passed with each request:
- Access is only granted if the JWT includes a specific email address in the
email
claim ([email protected]
). - If access is granted, the contents of the
grp
claim from the JWT will be forwarded as an additional header calledGroup
.
Next, to apply these policies to your whoami
service, update its IngressRoute to enable the OPA Middleware:
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: whoami
namespace: traefikee
spec:
entryPoints:
- web
routes:
- match: Host(`whoami.localhost`)
kind: Rule
services:
- name: whoami
namespace: traefikee
port: 80
middlewares:
- name: my-opa-plugin
If you compare this file to whoami-kubernetes.yaml
, above, you will see that it adds just two lines of configuration to the original IngressRoute.
Test the Service¶
Apply the above configuration and call the whoami
service, as before. You will see that without a JWT, access is now forbidden:
curl --verbose -H Host:whoami.localhost http://localhost
< HTTP/1.1 403 Forbidden
< Date: Thu, 12 Nov 2020 13:33:51 GMT
< Content-Length: 0
<
* Connection #0 to host whoami.localhost left intact
Now call the service with a properly formed JWT token and observe the results:
curl --verbose -H "Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyLCJlbWFpbCI6ImZvb0BleGFtcGxlLmNvbSIsImp0aSI6IjM1MWY4YTEzLTIyNGMtNGQwNy05ODZhLWYxZTQ2MjEzNTBjYiIsImV4cCI6MTYwODE5ODQ3OX0.X_N-hCn64Yz2JWMJTT5Qi7LB-9ylXL8-bi8lSOzWrnY" -H Host:whoami.localhost http://localhost
< HTTP/1.1 200 OK
< Content-Length: 686
< Content-Type: text/plain; charset=utf-8
< Date: Thu, 17 Dec 2020 12:54:46 GMT
<
Hostname: whoami-75d5976d8d-jl2p4
IP: 127.0.0.1
IP: 10.1.0.9
RemoteAddr: 10.1.0.7:47714
GET / HTTP/1.1
Host: whoami.localhost
User-Agent: curl/7.64.1
Accept: */*
Accept-Encoding: gzip
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyLCJlbWFpbCI6ImZvb0BleGFtcGxlLmNvbSIsImp0aSI6IjM1MWY4YTEzLTIyNGMtNGQwNy05ODZhLWYxZTQ2MjEzNTBjYiIsImV4cCI6MTYwODE5ODQ3OX0.X_N-hCn64Yz2JWMJTT5Qi7LB-9ylXL8-bi8lSOzWrnY
Group:
X-Forwarded-For: 192.168.65.3
X-Forwarded-Host: whoami.localhost
X-Forwarded-Port: 80
X-Forwarded-Proto: http
X-Forwarded-Server: default-proxy-f59689c97-kscbd
X-Real-Ip: 192.168.65.3
Because the JWT included the address [email protected]
in its payload, access is now granted.
Something is still missing, however.
The Group
header is still blank.
Try sending a new request with a JWT that includes the grp
payload, and you will see that the header is populated and forwarded, as specified in the OPA policy:
curl --verbose -H "Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyLCJlbWFpbCI6ImZvb0BleGFtcGxlLmNvbSIsImdycCI6Imdyb3VwMSIsImp0aSI6IjM1MWY4YTEzLTIyNGMtNGQwNy05ODZhLWYxZTQ2MjEzNTBjYiIsImV4cCI6MTYwODIxNDI4MH0.k6k9EyCeZQS34j6Ke3Ey7Gc-ZGWuHUusB3Xd_Ua7j7Y" -H Host:whoami.localhost http://localhost
< HTTP/1.1 200 OK
< Content-Length: 712
< Content-Type: text/plain; charset=utf-8
< Date: Thu, 17 Dec 2020 13:12:20 GMT
<
Hostname: whoami-75d5976d8d-jl2p4
IP: 127.0.0.1
IP: 10.1.0.9
RemoteAddr: 10.1.0.7:56266
GET / HTTP/1.1
Host: whoami.localhost
User-Agent: curl/7.64.1
Accept: */*
Accept-Encoding: gzip
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyLCJlbWFpbCI6ImZvb0BleGFtcGxlLmNvbSIsImdycCI6Imdyb3VwMSIsImp0aSI6IjM1MWY4YTEzLTIyNGMtNGQwNy05ODZhLWYxZTQ2MjEzNTBjYiIsImV4cCI6MTYwODIxNDI4MH0.k6k9EyCeZQS34j6Ke3Ey7Gc-ZGWuHUusB3Xd_Ua7j7Y
Group: group1
X-Forwarded-For: 192.168.65.3
X-Forwarded-Host: whoami.localhost
X-Forwarded-Port: 80
X-Forwarded-Proto: http
X-Forwarded-Server: default-proxy-f59689c97-kscbd
X-Real-Ip: 192.168.65.3
More About JWTs
The tools at JSONWebToken.io are among many online resources that can help you learn more about JWTs and explore the tokens provided with the examples in this guide.
Deploying Policies as Bundles¶
Traefik Enterprise also supports deploying OPA policies as bundles.
This section assumes you have Traefik Enterprise running on your cluster with a static configuration applied and the whoami
service deployed.
You will also need the example bundle file (opa.tar.gz).
You may also wish to have handy your Traefik Enterprise installation manifest (manifest.yaml
), as output by teectl
during installation.
Configure the Bundle¶
First, create a ConfigMap that holds the bundle:
kubectl create configmap -n traefikee --from-file=./opa.tar.gz opa-bundle
You must complete the above step before proceeding further.
Next, patch the Traefik Enterprise proxy deployment (default-proxy
) on your cluster to add the ConfigMap as a volume and mount it.
The file below provides the necessary patches:
spec:
template:
spec:
volumes:
- name: "opa-bundle"
configMap:
name: "opa-bundle"
containers:
- name: "default-proxy"
volumeMounts:
- name: "opa-bundle"
mountPath: "/var/lib/traefikee/opa"
Apply the patches from the command line:
kubectl patch deployment default-proxy --patch "$(cat opa-bundle-patch.yaml)" -n traefikee
kubectl patch deployment default-proxy --patch $(Get-Content opa-bundle-patch.yaml -Raw) -n traefikee
Updating Your Manifest
In addition to patching your running default-proxy
deployment, you may also wish to add the patches from opa-bundle-patch.yaml
to your manifest.yaml
, in case you wish to reinstall Traefik Enterprise in the future.
For guidance, consult proxy-bundle-example.yaml
, below, which shows a partial manifest.
The relevant sections are near the end of the file.
proxy-bundle-example.yaml
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: "default-proxy"
namespace: traefikee
labels:
app: traefikee
release: "default"
spec:
replicas: 1
selector:
matchLabels:
app: traefikee
release: "default"
component: proxies
template:
metadata:
labels:
app: traefikee
release: "default"
component: proxies
spec:
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
podAffinityTerm:
labelSelector:
matchExpressions:
- key: component
operator: In
values:
- proxies
topologyKey: "kubernetes.io/hostname"
terminationGracePeriodSeconds: 30
automountServiceAccountToken: false
initContainers:
- name: wait-dns
image: busybox:1.31.1
command: ['sh', '-c', 'until nslookup -type=a default-ctrl-svc.traefikee.svc.cluster.local; do echo waiting for published dns records; sleep 1; done;']
resources:
requests:
memory: "10Mi"
cpu: "100m"
limits:
memory: "100Mi"
cpu: "1000m"
containers:
- name: "default-proxy"
image: traefikee:latest
imagePullPolicy: IfNotPresent
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
securityContext:
readOnlyRootFilesystem: true
capabilities:
drop:
- ALL
add:
- NET_BIND_SERVICE
ports:
- containerPort: 8484
name: distributed
- containerPort: 80
name: http
- containerPort: 443
name: https
# readinessProbe:
# tcpSocket:
# port: http
# initialDelaySeconds: 2
# periodSeconds: 5
resources:
requests:
memory: "100Mi"
cpu: "100m"
limits:
memory: "4Gi"
cpu: "1000m"
volumeMounts:
- name: "default-proxy-data"
mountPath: "/var/lib/traefikee"
- name: "join-token"
mountPath: "/var/run/secrets"
# Add the "opa-bundle" VolumeMount here
- name: "opa-bundle"
mountPath: "/var/lib/traefikee/opa"
command:
- "/traefikee"
- "proxy"
- "--role=ingress"
- "--name=$(POD_NAME)"
- "--discovery.dns.domain=default-ctrl-svc.$(POD_NAMESPACE)"
- "--jointoken.file.path=/var/run/secrets"
- "--log.level="
- "--log.filepath="
- "--log.format="
volumes:
- name: "default-proxy-data"
emptyDir: {}
- name: "join-token"
secret:
secretName: "default-tokens"
# Add the "opa-bundle" ConfigMap as a volume here
- name: "opa-bundle"
configMap:
name: "opa-bundle"
Once the default-proxy
deployment is patched, you can define the IngressRoute and Middleware that will enable OPA policy enforcement for your whoami
service:
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: whoami-opa-bundle
namespace: traefikee
spec:
entryPoints:
- web
routes:
- match: Host(`whoami-bundle.localhost`)
kind: Rule
services:
- name: whoami
namespace: traefikee
port: 80
middlewares:
- name: opa-bundle
---
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
name: opa-bundle
namespace: traefikee
spec:
plugin:
opa:
allow: data.example.authz.allow
bundlePath: /var/lib/traefikee/opa/opa.tar.gz
Test the Bundle¶
As with the previous example, when you call the whoami
service without supplying a JWT, access is denied:
curl --verbose -H Host:whoami-bundle.localhost localhost
< HTTP/1.1 403 Forbidden
< Date: Thu, 17 Dec 2020 18:49:53 GMT
< Content-Length: 0
<
* Connection #0 to host localhost left intact
* Closing connection 0
Supplying a properly formed JWT with the request, on the other hand, produces the expected whoami
output:
curl --verbose -H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyLCJlbWFpbCI6ImZvb0BleGFtcGxlLmNvbSIsImdycCI6Imdyb3VwMSJ9.fWnRCDbqpBMyS6uRAyQYLPBpiz9YRQgmAGt2mo-NvK8" -H Host:whoami-bundle.localhost http://localhost
< HTTP/1.1 200 OK
< Content-Length: 713
< Content-Type: text/plain; charset=utf-8
< Date: Thu, 17 Dec 2020 18:39:09 GMT
<
Hostname: whoami-75d5976d8d-sfgsj
IP: 127.0.0.1
IP: 10.1.0.9
RemoteAddr: 10.1.0.10:39064
GET / HTTP/1.1
Host: whoami-bundle.localhost
User-Agent: curl/7.64.1
Accept: */*
Accept-Encoding: gzip
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyLCJlbWFpbCI6ImZvb0BleGFtcGxlLmNvbSIsImdycCI6Imdyb3VwMSIsImp0aSI6IjM1MWY4YTEzLTIyNGMtNGQwNy05ODZhLWYxZTQ2MjEzNTBjYiIsImV4cCI6MTYwODIxNDI4MH0.k6k9EyCeZQS34j6Ke3Ey7Gc-ZGWuHUusB3Xd_Ua7j7Y
X-Forwarded-For: 192.168.65.3
X-Forwarded-Host: whoami-bundle.localhost
X-Forwarded-Port: 80
X-Forwarded-Proto: http
X-Forwarded-Server: default-proxy-6ddc545cbb-w6g2x
X-Real-Ip: 192.168.65.3
For more information on crafting OPA policies, consult the project documentation.