Skip to content

Events Account Partitions

Overview

The Events service stores events and event_resources in PostgreSQL using hash partitioning on the account UUID. Customer-Managed Prefect installs created with chart 1.2.x and later use a single account partition (PREFECT_CLOUD_EVENTS_POSTGRES_NUMBER_OF_ACCOUNT_PARTITIONS=1), which is the right value for the single-account model that Customer-Managed deployments use.

Some installations created before 2024 were provisioned with 32 account partitions. These installs continue to function, but partman creates and manages 32 sets of daily sub-partitions instead of one, which can add unnecessary metadata churn and (in some workloads) create observable performance overhead.

How the partition count setting is used

The number of account partitions is governed by PREFECT_CLOUD_EVENTS_POSTGRES_NUMBER_OF_ACCOUNT_PARTITIONS (the postgres_number_of_account_partitions setting on the events service). The default is 1.

Runtime services (partman, ladler, triggers, events-background) discover the actual account partitions from PostgreSQL via pg_inherits rather than reading the setting, so a legacy 32-partition install continues to work correctly even when the setting is left at the default of 1. In practice, you do not need to override this setting just to keep an existing legacy install running.

The setting is consulted when partitions are being created — primarily the initial events / event_resources create-table migrations, and the fallback path in get_all_account_partitions when no partitions exist yet. In the Helm chart, the events-db-migrations container hardcodes this value to "1" and does not read global.backend.extraEnv, so the standard chart install always creates schemas at a single partition. Recreating a legacy multi-partition schema through Helm is not supported and would require patching the migration job template directly.

For an existing legacy install, no override is needed on runtime services — pg_inherits discovery handles the mismatch — and the right long-term path is to consolidate down to a single partition (see below).

Consolidating Legacy Partitions

A CLI command is available to reduce a legacy 32-partition install down to a single account partition. This is intended only for installs that are experiencing measurable performance issues attributable to the extra partitions.

Risk of Data Loss

This command modifies the schema of the events and event_resources tables and drops partitions. Although the procedure is designed to be safe in the supported single-account case, there is always a risk of data loss when restructuring partitioned tables.

Do not run this command unless you are seeing performance issues from the extra partitions. Engage Prefect support before running it, take a backup of the events database first, and exercise the run in a non- production environment.

Prerequisites

  • Single-account state. The command supports the single-account Customer-Managed model only. If multiple account partitions contain data, the preflight check will abort.
  • Scale down writers. ladler, triggers (reactive and proactive), and partman must be scaled to zero replicas before running. The preflight check inspects pg_stat_activity and refuses to proceed if any non-idle sessions are connected to the events database.
  • Database ownership. The command must run as the events database owner (or a role with USAGE on the owner) so it can ALTER TABLE on events and event_resources.
  • Backup. Take a fresh backup of the events database before running.

Running the command

Run the CLI from inside a pod that has the Customer-Managed source image, following the same approach used in the rollback procedure.

First, do a dry-run to display the consolidation plan without making any changes:

prefect-cloud events database consolidate-account-partitions --dry-run

The output will show, for each root table, which partition will be kept as the active one, how many partitions will be dropped, and the size of the active partition.

When you are ready to apply the change:

prefect-cloud events database consolidate-account-partitions

You will be prompted to confirm before the command modifies the schema. Pass --yes (or -y) to skip the prompt for non-interactive runs.

If the events tables are already at one partition the command exits successfully with Already consolidated; nothing to do.

After consolidation

Once the command completes, scale ladler, triggers, and partman back up to their previous replica counts.