Skip to content

Deploy Customer-Managed Prefect Cloud with Helm

Overview

This guide helps your platform teams with three key tasks: pulling Customer-Managed Prefect Cloud images, deploying the Kubernetes manifests via Helm, and upgrading instances.

Pre-requisites

Review the minimum infrastructure requirements before proceeding.

Pull Customer-Managed Docker images

Docker Overview

We use Docker to ship our Customer-Managed server and Customer-Managed UI applications in a portable manner.

You can read more about Docker and its benefits in Docker's documentation.

Pull and store Customer-Managed images

There are two Prefect-maintained Docker images relevant to Customer-Managed that need to be pulled and made available to your Kubernetes cluster:

  • server
  • ui

These images are stored in a private GCP Artifact Registry and can be pulled using the GCP service account token provided to your team.

Steps:

  1. Authenticate to the Google Cloud Artifact Registry

    gcloud auth login --cred-file=<provided-service-account-key>.json
gcloud auth configure-docker us-docker.pkg.dev

    docker login -u _json_key --password-stdin https://us-docker.pkg.dev < <provided-service-account-key>.json

  2. Pull down the images (latest versions can be found in releases)

    # server
docker pull us-docker.pkg.dev/prefect-prd-customer-managed/cloud/server:<server-version-tag>

# ui
docker pull us-docker.pkg.dev/prefect-prd-customer-managed/cloud/ui:<ui-version-tag>

  3. You can now tag and store these images in an internal registry.

Using a repository proxy

If your platform routes pulls through a proxy (for example, Artifactory or Nexus) instead of direct access to us-docker.pkg.dev, use the remote repository credentials configured for that proxy endpoint.

In some proxy setups for private Google Artifact Registry, you can configure the following credentials on the proxy:

  • username: _json_key_base64
  • password: base64-encoded service account key JSON (single line)

For more detailed instructions, see:

Pull and store the Customer-Managed Helm Charts

Helm Overview

  • Helm is an open-source package manager for Kubernetes that simplifies the deployment, management, and lifecycle operations of applications within Kubernetes clusters.
  • Helm is used to coordinate the deployment of Customer-Managed Kubernetes resources as well as upgrades as they become available.
  • Most Kubernetes continuous delivery systems support deploying Helm charts natively such as FluxCD, ArgoCD, Jenkins-X, GitLab CI/CD, and Octopus.

You can read more about Helm and its use cases in Helm's introduction

Pull and store the Prefect-Istio and Customer-Managed Helm Charts

There are two distinct Helm charts provided by Prefect that are required in deploying Customer-Managed in your Kubernetes cluster:

  1. The Prefect Istio Helm chart, which deploys the required Istio custom resource definitions and the istiod deployment to handle traffic routing and Istio sidecar creation.
  2. The Customer-Managed Helm chart, which deploys the database migration jobs, secrets, configmaps, deployments, and other resources needed for the Customer-Managed application to function correctly.

To pull the images from the registry:

  1. Authenticate against the Google Cloud Artifact Registry
gcloud auth login --cred-file=<provided-service-account-key>.json
gcloud auth print-access-token | helm registry login -u oauth2accesstoken --password-stdin https://us-docker.pkg.dev

helm registry login -u _json_key --password-stdin https://us-docker.pkg.dev < <provided-service-account-key>.json
  1. Pull down the Customer-Managed and Prefect Istio Helm charts (latest versions can be found in releases)
helm pull oci://us-docker.pkg.dev/prefect-prd-customer-managed/cloud/charts/prefect-istiod --version <latest-helm-chart-version>

helm pull oci://us-docker.pkg.dev/prefect-prd-customer-managed/cloud/charts/prefect-cloud --version <latest-helm-chart-version>

This step will pull down a .tgz file which can then be either pushed to your Helm repository. (such as Artifactory) or templated and deployed directly (examples in later sections).

Helm through a repository proxy

If your organization mirrors OCI artifacts behind a remote repository endpoint, run helm registry login against that endpoint with the credentials provisioned by your platform team.

See the note above (Using a repository proxy) for configuring Helm authentication to the Google Artifact Registry on the proxy.

Install Customer-Managed via Helm

Helm values overview

Customer-Managed is a highly configurable Kubernetes manifest stack that supports different customer requirements for authentication, scale, and other operational concerns. These configurations are managed through Helm values files, which contain configuration required for a successful installation, including fine-grained configurations for each service deployed by the Helm chart.

Viewing available configuration values

You can quickly view all available configuration values using helm show values:

# for a local chart
helm show values <chart-name.tgz>

# or from the registry at a specific version
helm show values oci://<registry-chart-name> --version <chart-version>

For a more detailed review of the values file, extract the chart locally:

  1. Pull the chart:
helm pull oci://us-docker.pkg.dev/prefect-prd-customer-managed/cloud/charts/<chart_name> --version <version_number>
  1. Extract the chart:
tar -xvf <chart_name>-<chart_version>.tgz
  1. View the values file:
cd <chart_name>/
cat values.yaml

This values.yaml file contains all configurable values with their default settings, along with comments and links to upstream documentation for subcharts.

Finding Available Versions

See the Releases documentation for available chart versions and release notes.

Example: Istio Chart Values Structure

When you extract the prefect-istiod chart, the values.yaml file will have a structure similar to this:

prefectNamespace: prefect

global: {}

# Values based on Istiod Helm chart (https://github.com/istio/istio/blob/1.25.5/manifests/charts/istio-cni/values.yaml)
istioCNI:
  enabled: false
  # platform: openshift

# Values based on Istiod Helm chart (https://github.com/istio/istio/blob/1.25.5/manifests/charts/base/values.yaml)
istioBase:
  enabled: true

# Values based on Istiod Helm chart (https://github.com/istio/istio/blob/1.25.5/manifests/charts/istio-control/istio-discovery/values.yaml)
istioDiscovery:
  enabled: true
  meshConfig: {}
  pilot:
    configMap: false
    resources:
      requests:
        cpu: 100m
        memory: 375Mi
      limits:
        cpu: 180m
        memory: 600Mi
    traceSampling: 80.0
    autoscaleEnabled: true
    autoscaleMin: 3
    autoscaleMax: 10
    replicaCount: 1
    rollingMaxSurge: 100%
    rollingMaxUnavailable: 25%
  telemetry:
    enabled: true
    v2:
      stackdriver:
        enabled: false
        logging: false
        monitoring: false
      prometheus:
        enabled: true
  global:
    defaultPodDisruptionBudget:
      enabled: true
    logAsJson: true
    logging:
      level: default:warn
    istiod:
      enableAnalysis: true
    proxy:
      # Log level for proxy, applies to gateways and sidecars
      logLevel: warning
      resources:
        requests:
          cpu: 35m
          memory: 70Mi
        limits:
          cpu: 100m
          memory: 140Mi

Key sections:

  • prefectNamespace: Namespace where Customer-Managed will be deployed
  • istioCNI: CNI plugin (required for OpenShift and similar platforms)
  • istioBase: Base Istio CRDs and cluster resources
  • istioDiscovery: Istio control plane configuration
    • pilot: Resource limits, autoscaling, replicas
    • telemetry: Metrics collection (Prometheus/Stackdriver)
    • global.proxy: Default sidecar settings for all services

Working with subcharts

The Customer-Managed Helm chart includes subcharts for dependent services such as embedded PostgreSQL and Istio components. This modular approach allows fine-grained control over each component's configuration.

Subchart value organization:

  • Common subchart values for typical deployments are exposed directly in the main values.yaml file
  • Advanced subchart values can be found by following documentation links provided in the main values.yaml file
  • Each subchart section in values.yaml includes a comment with a link to the upstream subchart's full values documentation

Overriding subchart values:

Override subchart values in your values-override.yaml using the subchart name as a key:

# Example: Customizing Istio pilot resources
istioDiscovery:
  pilot:
    resources:
      requests:
        cpu: 200m
        memory: 500Mi

Creating a values override file

Once you've reviewed the available values, create a values-override.yaml file containing only the values you want to customize. This file will be passed to Helm during installation using the -f flag:

helm install customer-managed prefect-cloud-<version>.tgz \
  -n customer-managed \
  -f values-override.yaml

Best Practice

Only override values you need to change. The defaults are production-tested. Keep your values-override.yaml in version control to track configuration changes over time.

Minimum installation values

This section demonstrates a minimum installation of Customer-Managed by configuring a values-override.yaml file. This example assumes you are providing your own ingress/load balancer, using adfs for authentication, and have already provisioned the Kubernetes cluster, external databases, and external Redis instances.

For detailed information about all supported authentication providers and their configuration options, see the Authentication Methods reference.

# values-override.yaml
global:
  frontendUrl: https://prefect-ui.company.com
  backendUrl: https://prefect-api.company.com
  instanceName: customer-managed
  ownerEmail: test@somecompany.io # Note this is the only user that will initially be able to log into Customer-Managed
  backend:
    app:
      image:
        repository: us-docker.pkg.dev/prefect-prd-customer-managed/cloud/server # change to your internal repository
        tag: 2025.10.202511042343
  authProvider:
    type: adfs
    adfs:
      clientID: "adfsId"
      clientSecret: "adfsSecret"
      clientAuthority: "adfsauthority.microsoft.com"
      clientVerifySSL: false
db:
  use_embedded: false
  serviceuser: # User will be created as part of the initial setup
    password: "secretPassword"
  superuser: # Should have been manually created already
    password: "secretSuperuserPassword"
  host:
    primary: 10.0.0.1 # Reachable primary db ip address
    replica: 10.0.0.2 # Reachable replica db ip address
redis:
  use_embedded: false
  cache:
    host: 10.0.0.3 # Reachable cache redis instance address
    password: "" # password if auth is required for your redis instance
  work:
    host: 10.0.0.4 # reachable work redis instance address
    password: ""
  streams:
    host: 10.0.0.5 # reachable streams redis instance address
    password: ""
ui:
  cloud2:
    image:
      repository: us-docker.pkg.dev/prefect-prd-customer-managed/cloud/ui # change to your internal repository
      tag: 2025.1023.154520

You can template this configuration using Helm to preview the Kubernetes manifests generated by this chart:

helm template prefect-cloud-<version>.tgz -n <intended-deployed-namespace> -f values-override.yaml > manifests.yaml

Deploy Customer-Managed

With a valid values override file created, you are ready to deploy the Helm charts.

NOTE: On a first install of Customer-Managed, you must install the Prefect Istio Helm chart first.

Deploy the Prefect Istio Helm chart

  1. Create and label the istio-system namespace:

    kubectl create namespace istio-system
kubectl label namespace istio-system istio-injection=enabled

  2. Install the chart (example using locally pulled .tgz chart):

    helm install istiod prefect-istiod-<version>.tgz

  3. Verify istiod is up and healthy:

    kubectl get pods -n istio-system

Deploy the Customer-Managed Helm chart

  1. Create and label the intended namespace where the Customer-Managed instance will be deployed:
kubectl create namespace customer-managed
kubectl label namespace customer-managed istio-injection=enabled
  1. Install the chart referencing the values-override.yaml file created above and the namespace where it will be deployed:
helm install customer-managed prefect-cloud-<version>.tgz -n customer-managed -f values-override.yaml
  1. Services will begin to start and may initially crash until database migrations are complete. You can monitor the database migrations:
kubectl logs -n <customer-managed-namespace> prefect-db-migrations
  1. Services should eventually show as healthy. You will need to attach your ingress/load balancer to the prefect-istio-gateway deployment if you did not set up the ingress via the Helm chart.
  2. Once the UI is available and the backend is healthy, the owner can log into Customer-Managed for the first time.

Upgrade Customer-Managed

Update a UI or Server image

To upgrade to a patched image of Customer-Managed, unless otherwise noted in the release notes:

  1. Pull the latest image(s) from the Customer-Managed Google Artifact Registry
  2. Push the latest image(s) to your internal registry, if you have one
  3. Update the values-override.yaml file with the new image tags:
global:
  backend:
    app:
      image:
        repository: us-docker.pkg.dev/prefect-prd-customer-managed/cloud/server # change to your internal repository
        tag: <new_tag>
ui:
  cloud2:
    image:
      repository: us-docker.pkg.dev/prefect-prd-customer-managed/cloud/ui # change to your internal repository
      tag: <new_tag>
  1. Run a Helm upgrade:
helm upgrade customer-managed prefect-cloud-<version>.tgz -n customer-managed -f values-override.yaml

Upgrade a Helm Chart with breaking changes

A quarterly chart release is typically distributed with every major release of Customer-Managed. This chart may deploy new services or fix bugs in existing services. The chart may also contain breaking changes that need to be addressed prior to installation in your Customer-Managed environment.

  1. Put Customer-Managed into maintenance mode
  2. Identify the latest chart versions and any potential breaking changes in the upgrade guide
  3. Pull down and store any required images if the latest image has changed
  4. Pull down the latest Helm charts and either store locally or push to your internal Helm chart repository
  5. Update your existing values-override.yaml file with any required changes
  6. Skip this step if there are no breaking changes for the latest chart release
  7. Run a Helm upgrade referencing the new chart version:
helm upgrade istiod prefect-istiod-<new_version>.tgz

helm upgrade customer-managed prefect-cloud-<version>.tgz -n customer-managed -f values-override.yaml
  1. Verify health of istiod pods and/or the database migrations pod
  2. Verify functionality of your Customer-Managed instance