Faq

What happens while the API is in maintenance mode?¶

When the API is in maintenance mode, it will:

Return a 200 on GET /load-balancer/health-check
Return a 503 to any other request, with a specified Retry-After header value

How do I put the API into maintenance mode for scheduled or unscheduled maintenance?¶

Set serviceMesh.maintenanceMode.enabled=true in the Helm chart values and then run helm upgrade to apply the changes. In Istio modes, this updates the Istio VirtualService to return retryable 503 error codes. In Traefik-only mode, this activates high-priority Traefik routers that return 503 with Prefect-Maintenance: true and Retry-After: 1200 headers.

Set to false and run another helm upgrade to take the application out of maintenance mode.

Is Customer-Managed FEDRAMP compliant?¶

Prefect uses services that are compliant with FEDRAMP requirements.

What are the options for Backups / Disaster Recovery?¶

Backup options would be cloud native using your desired cloud vendor. As the product is Customer-Managed, backup options would be available through database replication and snapshots (RDS / Aurora, PostgreSQL, etc.)

For Redis backup strategies and recommendations, see the Redis section of Infrastructure Services.

Sensitive data was accidentally stored - how can I remove it?¶

Task and flow run logs are stored in the Events PostgreSQL database and can be deleted directly. Run these queries against the Events database, substituting your account ID, workspace ID, and run ID:

DELETE FROM events
WHERE account = '<account-id>'
AND workspace = '<workspace-id>'
AND event = 'prefect.log.write'
AND occurred BETWEEN '<start-time>' AND '<end-time>'
-- narrow the occurred range if possible for performance
AND resource_id = 'prefect.flow-run/task-run.<run-id>';

-- for flow runs, also delete related events
DELETE FROM events
WHERE account = '<account-id>'
AND workspace = '<workspace-id>'
AND event = 'prefect.log.write'
AND occurred BETWEEN '<start-time>' AND '<end-time>'
-- narrow the occurred range if possible for performance
AND related_resource_ids && '{prefect.flow-run.<run-id>}'::text[];

Why do I get an `SSL: CERTIFICATE_VERIFY_FAILED` error when connecting to the API?¶

If your API is served with a certificate signed by an internal (non-system) certificate authority, the Prefect client may fail to connect with an error like:

httpx.ConnectError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate

The Prefect client ships with the certifi CA bundle and does not trust internal CAs by default (see SSL Termination).

To fix this, point the client at a CA bundle that includes your internal certificate authority by setting PREFECT_API_SSL_CERT_FILE to the path of the bundle in each client and worker environment:

export PREFECT_API_SSL_CERT_FILE=/path/to/ca-bundle.crt

The file should be a PEM-formatted bundle containing your internal CA certificate (and any intermediate certificates).

As a last resort, you can bypass certificate verification entirely by setting PREFECT_API_TLS_INSECURE_SKIP_VERIFY to True. This disables SSL verification and leaves connections vulnerable to man-in-the-middle attacks, so it is not recommended — prefer providing a CA bundle with PREFECT_API_SSL_CERT_FILE.

How can I modify log retention?¶

The default retention period set in PostgreSQL is determined by the environment variable PREFECT_CLOUD_EVENTS_POSTGRES_MAX_RETENTION_PERIOD which defaults to 30 days.

Is all data encrypted in transit? At rest?¶

Yes - all data is encrypted between the Prefect clients (both user clients and the Prefect workers) through TLS. Intra-cluster encryption in transit is supported if a customer chooses to implement in their environment. All Redis connections can be configured via TLS. All data at rest (database) can be configured for encryption through the customers cloud vendor via cloud managed or Customer-Managed keys.

Can images be modified from base?¶

Depending on the requirements for modification. Injecting / re-building images to include security certificates for communication is supported. Replacing packages, altering entrypoints, and further modifications are not supported.

How are logs collected and stored?¶

Prefect logs are at the granularity that the customer defines in their flow code (DEBUG through INFO). Our applications log request access logs from our webservers as well as job consumer activity. No customer information is exposed in such logs. No database queries are logged. All other logs such as cluster, database, load balancer logs would be at the customers discretion through their cloud provider.

Is there a patching cycle for the application? How often is it reviewed?¶

The CVE SLA for Prefect Customer-Managed matches our own internal cadence for Prefect Cloud. A risk matrix determines the risk, severity, and impact of the CVE, and is remediated in a period appropriate for the risk.

What regions / countries are supported?¶

Customer-Managed can be brought to any region or country that has the required infrastructure.

What is your SLA response time?¶

9x5 with 1 hour response time.

Does Prefect track my usage?¶

No telemetry is collected for Prefect Customer-Managed offerings.

Faq