Skip to main content

Nutanix Prism Central Setup Guide

This guide walks you through setting up Traefik Hub to automatically discover and route traffic to virtual machines running on Nutanix AHV infrastructure using the Nutanix Prism Central provider.

What You'll Accomplish​

By the end of this guide, you will have:

  • Configured a Nutanix Prism Central service account for Traefik Hub
  • Created categories in Prism Central to organize your VMs
  • Enabled automatic service discovery for your Nutanix VMs
  • Verified that Traefik Hub can route traffic to your discovered services

Prerequisites​

Before you begin, ensure you have:

  • Nutanix Prism Central with administrative access
  • Traefik Hub installed and running (see the installation guides)
  • At least 2 VMs running in Nutanix for testing purposes
  • kubectl configured (if using Kubernetes deployment) or access to Traefik Hub configuration
  • Flow Virtual Networking enabled (optional, only needed for VPC filtering)

Step 1: Create a Service Account in Prism Central​

First, you'll need to create a dedicated service account for Traefik Hub with appropriate permissions.

Create the Service Account​

  1. Log in to your Nutanix Prism Central web console
  2. Click the Application Switcher and select Admin Center
  3. From the navigation bar, select Settings > IAM > Identities
  4. On the top right, click Service Accounts
  5. In the service accounts list, select Add New Service Account
  6. Configure the service account:
    • Name: traefik-hub-discovery
    • Description: Service account for Traefik Hub VM discovery (optional)
  7. Click Save & Create Keys

Generate an API Key​

  1. On creating the service account, you will be redirected to the service account details page. On the top right, click Add New Key
  2. Configure the API key:
    • Key Type: API Key
    • Name: traefik-hub-key
    • Key Expiration: Set as needed (optional)
  3. Click Generate Key
  4. Copy the generated API key immediately
Save Your API Key

The API key is displayed only once during creation. You cannot retrieve it later. If you lose the API key, you must generate a new one.

Assign Required Permissions​

After creating the service account & API key, you need to grant it permissions to read VMs, Categories, and Subnets (if using VPC filtering).

  1. From the navigation bar, select Settings > IAM > Authorization Policies
  2. Click Create Authorization Policy
  3. Configure the authorization policy:
    • Edit the autogenerated name to traefik-hub-read-only
    • Role: Select Prism Viewer
    • Entities: Select Full Access or configure specific clusters/VMs
    • Identities: Select service account then search for and select the traefik-hub-discovery service account from the dropdown
  4. Click Save
Custom Roles

For more restrictive permissions, you can create a custom role with only the following permissions:

  • View Virtual Machine - Required to read VM configurations
  • View Category Mapping - Required to read category assignments
  • View Name Category - Required to read category keys
  • View Value Category - Required to read category values
  • View Subnet - Required if using VPC filtering
  • View VPC - Required if using VPC filtering

However, using the built-in Prism Viewer role is simpler and provides all necessary read permissions.

Step 2: Identify VPC UUIDs (Optional)​

If you want to restrict service discovery to specific Virtual Private Clouds, you'll need to identify their UUIDs. Skip this step if you want to discover all VMs regardless of VPC. Discovering VMs across all VPCs can be useful for load balancing between highly-available distributed applications spanning multiple environments.

Find VPC UUIDs​

  1. Click the Application Switcher and select Admin Center
  2. From the navigation bar, select Infrastructure
  3. From the navigation bar, select Network & Security > VPCs
  4. Identify the VPC(s) you want to use for service discovery
  5. Select the VPC name to view details
  6. Once on the VPC details page, the UUID is displayed in the URL as the entity ID. Copy the UUID.

Step 3: Create Service Name Category in Prism Central​

Categories are key-value pairs that act as metadata for your VMs. Traefik Hub uses the TraefikServiceName category to map VMs to services defined in your base configuration file.

Create TraefikServiceName Category​

  1. Click the Application Switcher
  2. Select Admin Center (if not already there)
  3. From the navigation bar, select Categories
  4. On the top left, click New Category
  5. Configure the category:
    • Category/Key Name: TraefikServiceName
    • Add the following value (you can add more later):
      • whoami - For test service
  6. Click Save

Prism Central category creation interface

Category Reusability

Categories are global resources in Prism Central. Once created, they can be reused across all VMs. You don't need to recreate categories for each VM.

Service Properties

Service properties like port, health checks, and load balancing strategy are NOT configured via categories. They are defined in a base configuration file that you'll create in the next step.

Step 4: Create Base Configuration File​

Create a base configuration file that defines your services with their properties (port, health checks, load balancing strategy, etc.). The Nutanix provider will discover VM IPs and merge them with these service definitions.

Create the Configuration File​

Create a file named nutanix-base-config.yaml:

http:
  routers:
    whoami-route:
      rule: "Host(`whoami.example.com`)"
      entryPoints:
        - web
      service: whoami

  services:
    whoami:
      loadBalancer:
        scheme: http
        port: "80"
        strategy: p2c
        healthCheck:
          path: /health
          interval: 10s
          timeout: 3s

This configuration defines:

  • Router: Routes traffic from whoami.example.com to the whoami service
  • Service: Defines the whoami service with:
    • Port 80 where the application listens
    • p2c load balancing strategy (Power of Two Choices)
    • Health checks at /health every 10 seconds
File Location

Place this file where Traefik Hub can access it. Common locations:

  • Local deployment: /etc/traefik/nutanix-base-config.yaml
  • Kubernetes: Mount as ConfigMap or use a persistent volume

Step 5: Assign Category to VMs​

Now assign the TraefikServiceName category to your VMs to map them to the whoami service you defined in the base configuration.

Assign Category to Test VMs​

For this example, we'll configure two VMs to form the whoami service.

  1. Click the Application Switcher
  2. Select Infrastructure
  3. From the navigation bar, click Compute > VMs
  4. Select the VM (e.g., whoami-vm-01 or whoami-vm-02)
  5. On the VM summary tab, select More > Other Actions > Manage Categories from the top menu bar
  6. Assign the category:
    • TraefikServiceName: whoami
  7. Click Save
  8. Ensure the VM is powered ON
  9. Repeat for each VM in the service

Prism Central VM category assignment interface showing multiple categories assigned to a VM

Service Name Matching

The category value (whoami) must exactly match the service name in your base configuration file. If they don't match, the VM will not be added as a backend server.

Verify VM Configuration​

Confirm that your VMs are properly configured:

  1. From the Infrastructure application, navigate to Compute > VMs
  2. For each VM, verify:
    • Power State: ON
    • IP Address: Assigned and visible in the VM list
    • Categories: Shows TraefikServiceName: whoami

Step 6: Configure Traefik Hub Provider​

Now configure Traefik Hub to connect to your Nutanix Prism Central environment.

Configuration for Helm Deployment​

If you're using Helm to deploy Traefik Hub, update your values.yaml file:

providers:
  nutanixPrismCentral:
    enabled: true
    endpoint: https://prism-central.example.com
    apiKey: your-api-key-from-step-1
    filename: /etc/traefik/nutanix-base-config.yaml
    pollInterval: 30s
    pollTimeout: 5s
    serviceNameCategoryKey: TraefikServiceName
    allowedVPCs:
      - uuid: <your-vpc-uuid-from-step-2>  # Replace with your VPC UUID from Step 2
    tls:
      insecureSkipVerify: true  # Only for testing with self-signed certificates

Deploy or upgrade your Traefik Hub installation:

helm upgrade --install traefik-hub traefik/traefik-hub \
  --reset-then-reuse-values \
  --namespace traefik \
  --values values.yaml

Configuration for Static Config File​

If you're using a install configuration file, add the provider configuration:

# traefik-config.yaml
providers:
  nutanixPrismCentral:
    endpoint: https://prism-central.example.com
    apiKey: your-api-key-from-step-1
    filename: /etc/traefik/nutanix-base-config.yaml
    pollInterval: 30s
    pollTimeout: 5s
    serviceNameCategoryKey: TraefikServiceName
    tls:
      insecureSkipVerify: true

Restart Traefik Hub to apply the configuration.

Production TLS Configuration

The tls.insecureSkipVerify: true setting should only be used in testing environments with self-signed certificates. For production:

  • Remove insecureSkipVerify or set it to false
  • Configure proper TLS with tls.ca pointing to your CA certificate
  • Use valid certificates signed by a trusted CA

Step 6: Verify Service Discovery​

After configuring the provider, verify that Traefik Hub is successfully discovering your VMs.

Check Traefik Hub Logs​

Verify the provider is running and discovering services:

# View Traefik Hub logs
kubectl logs -n traefik -l app.kubernetes.io/name=traefik-hub --tail=100 -f

Check Traefik Hub Dashboard​

If you have the Traefik Hub dashboard enabled:

  1. Navigate to the dashboard (typically http://localhost:8080/dashboard/)
  2. Click HTTP Services in the navigation
  3. Look for services with the @nutanixprismcentral suffix
  4. Verify the service shows the correct number of servers

Traefik Hub dashboard showing the discovered whoami@nutanixprismcentral service with 2 backend servers

Step 7: Create Routing Rules​

Now that your services are discovered, create routing rules to direct traffic to them.

Create an IngressRoute (Kubernetes)​

If you're using Kubernetes with Traefik CRDs:

apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: whoami-route
  namespace: default
spec:
  entryPoints:
    - web
  routes:
    - match: Host(`whoami.example.com`)
      kind: Rule
      services:
        - name: whoami@nutanixprismcentral
          port: 8080

Apply the IngressRoute:

kubectl apply -f whoami-route.yaml

Create a File Provider Route​

If you're using the File provider for routing:

# routing-config.yaml
http:
  routers:
    whoami-router:
      rule: "Host(`whoami.example.com`)"
      service: whoami@nutanixprismcentral
      entryPoints:
        - web

Update your Traefik configuration to load this file.

Cross-Provider References

When referencing services from the Nutanix provider, always use the full service name with the @nutanixprismcentral suffix. This tells Traefik which provider manages the service.

Step 8: Test the Setup​

Verify that traffic is being routed to your discovered VMs.

Test with curl​

# Test the route
curl -H "Host: whoami.example.com" http://your-traefik-ip/

# Or if you have DNS configured
curl http://whoami.example.com/

Expected output (from one of your VMs):

Hostname: poeut
IP: 192.168.1.10
RemoteAddr: 10.0.0.5:45678
GET / HTTP/1.1
Host: whoami.example.com
...

Test Load Balancing​

Make multiple requests to verify P2C load balancing is working:

# Make 10 requests and check which backend responds
for i in {1..10}; do
  curl -s -H "Host: whoami.example.com" http://your-traefik-ip/ | grep "Hostname"
done

You should see requests distributed between your VMs:

Hostname: whoami-vm-01
Hostname: whoami-vm-02
Hostname: whoami-vm-01
Hostname: whoami-vm-02
...

Test from Browser​

Configure your local hosts file or DNS:

# Add to /etc/hosts (Linux/Mac) or C:\Windows\System32\drivers\etc\hosts (Windows)
your-traefik-ip whoami.example.com

Open your browser and navigate to http://whoami.example.com/. You should see the response from one of your VMs.

Troubleshooting​

No Services Discovered

Symptoms: No services appear in Traefik Hub, API returns empty results.

Possible Causes:

  • API key is incorrect or lacks permissions
  • VMs are powered OFF
  • VMs don't have TraefikServiceName category assigned
  • Endpoint URL is incorrect

Solutions:

  1. Check Traefik Hub logs for authentication errors:

    kubectl logs -n traefik -l app.kubernetes.io/name=traefik-hub | grep -i error

  2. Verify API key and permissions in Prism Central

  3. Ensure VMs are powered ON

  4. Verify categories are assigned correctly to VMs

  5. Test connectivity to Prism Central:

    curl -k https://prism-central.example.com/api/vmm/v4.2/ahv/config/vms
VMs Not Filtered by VPC

Symptoms: VMs from other VPCs are being discovered despite allowedVPCs configuration.

Possible Causes:

  • VPC UUID is incorrect
  • Flow Virtual Networking is not enabled
  • VMs are not attached to subnets in the specified VPC

Solutions:

  1. Verify the VPC UUID is correct:

    • Application Switcher > Infrastructure > Network & Security > VPCs > Select VPC > Subnet > Update
    • Copy the exact UUID including hyphens

  2. Check if Flow Virtual Networking is enabled in Prism Central

  3. Verify VMs' network configuration:

    • Ensure VM NICs are attached to subnets in the allowed VPC

  4. Test subnet filtering:

    # Check which subnets belong to your VPC
    curl -k -H "Authorization: Bearer $API_KEY" \
      "https://prism-central.example.com/api/networking/v4.2/config/subnets" | \
      jq '.data[] | select(.vpcReference == "your-vpc-uuid")'
Strategy Conflict Errors

Symptoms: Service appears in logs but is excluded from configuration. Log shows "Strategy conflict detected".

Possible Causes:

  • VMs in the same service have different TraefikLoadBalancerStrategy values

Solutions:

  1. Review category assignments for all VMs in the service
  2. Ensure all VMs have either:
    • The same strategy value (e.g., all use p2c)
    • OR no strategy category at all (uses default)
  3. Fix category assignments in Prism Central
  4. Wait for the next polling interval or restart Traefik Hub
Wrong IP Address Selected

Symptoms: Traefik routes to wrong IP or unreachable IP.

Possible Causes:

  • VM has multiple NICs and wrong NIC is selected
  • VPC filtering is selecting unexpected NICs
  • IPv6 is selected when IPv4 is expected

Solutions:

  1. Check VM NIC configuration in Prism Central
  2. If using VPC filtering, ensure the correct NICs are in allowed VPCs
  3. Review IP selection logic in the provider documentation
  4. Check Traefik Hub logs for IP selection details
TLS Certificate Errors

Symptoms: Logs show TLS handshake failures, certificate verification errors.

Possible Causes:

  • Self-signed certificate without insecureSkipVerify
  • Incorrect CA certificate configuration
  • Certificate name mismatch

Solutions:

  1. For testing with self-signed certificates:

    tls:
      insecureSkipVerify: true

  2. For production, provide the CA certificate:

    tls:
      ca: /path/to/prism-ca.pem

  3. Ensure the endpoint URL matches the certificate Common Name or SAN

  4. Test TLS connection:

    openssl s_client -connect prism-central.example.com:443 -showcerts
Slow Service Discovery

Symptoms: Changes to VMs take a long time to appear in Traefik.

Possible Causes:

  • pollInterval is too high
  • Prism Central API is slow

Solutions:

  1. Reduce pollInterval for faster updates:

    pollInterval: 15s  # Update every 15 seconds instead of 30s

  2. Monitor Prism Central API performance

  3. Check network latency between Traefik Hub and Prism Central

Service Excluded Due to No Valid IP

Symptoms: Logs show "Skipping VM without valid IP address".

Possible Causes:

  • VM NIC has no IP address assigned
  • VPC filtering excludes all NICs
  • Network configuration is incomplete

Solutions:

  1. Check VM network configuration in Prism Central
  2. Ensure VM has obtained an IP address (DHCP or static)
  3. Verify VPC filtering configuration is not too restrictive
  4. Check that VM's subnet has proper IP address management configured

Additional Resources​