You know that feeling when someone posts a Slack message saying "backup starting, expect slowness for the next 6 hours" and half the team silently groans? Yeah. That's been the Postgres backup reality for teams running databases measured in terabytes, not gigabytes.
PlanetScale recently published their approach to parallelizing Postgres backups, and it's worth breaking down. Not because it's some never-before-seen trick, but because the pattern is genuinely practical. It takes something most teams treat as a black box and makes it something you can reason about, tune, and actually fix.
Let's get into what they're doing and how you can steal the ideas.
The Problem With Traditional Postgres Backups
Most Postgres backup setups follow the same playbook. You run pg_dump or pg_basebackup against the primary. It reads everything sequentially. One table at a time. One file at a time. For a 50GB database, this is fine. You barely notice.
But when you hit 500GB or 2TB? That sequential read pattern turns into a wall clock problem. Your backup takes 4, 6, 8 hours. During that window, I/O is saturated, replication lag spikes, and your on-call engineer is refreshing Grafana dashboards wondering if they should cancel the whole thing.
The core issue isn't that Postgres is slow at dumping data. It's that the default tooling treats the database as one big blob. No parallelism. No prioritization. No awareness of which tables are hot and which haven't been touched in weeks.
What PlanetScale Actually Did
Their approach splits the problem into pieces that can run concurrently. Instead of one monolithic backup job crawling through every table in sequence, they parallelize at the table level. Each worker grabs a set of tables and backs them up independently.
The architecture is straightforward:
-
Discovery phase. Query
pg_classandpg_stat_user_tablesto get the full list of tables, their sizes, and how frequently they change. This metadata drives everything downstream. -
Scheduling phase. Assign tables to workers. Large tables get their own worker. Small tables get batched together. The goal is roughly equal work per worker so nobody finishes early while one giant table is still chugging along.
-
Parallel execution phase. Each worker runs its assigned table dumps concurrently. They can write to the same backup destination (S3, GCS, whatever) without conflicts because each table writes to its own file path.
-
Metadata assembly phase. Once all workers finish, you assemble a manifest that lists every file, its checksum, and the restore order. This is what makes the backup actually usable later.
That's the core pattern. Four stages, but the magic is in steps 2 and 3.
Why Table-Level Parallelism Works
There's a reason you parallelize at the table level and not, say, at the row level within a single table. Table boundaries are natural. Each table is an independent storage unit in Postgres. Backing up Table A doesn't interfere with backing up Table B, as long as you're reading (not writing) and you've handled your MVCC snapshot correctly.
Row-level parallelism inside one table gets complicated fast. You need to figure out range boundaries, handle TOAST data, deal with index bloat affecting scan performance. Table-level parallelism sidesteps all of that. It's coarser-grained, but it's clean.
And for most workloads, the distribution of table sizes follows a long-tail pattern. You might have 200 tables, but 5 of them hold 80% of the data. If you give those 5 large tables their own dedicated workers and batch the remaining 195 tables across a few more workers, you get solid parallelization without over-engineering things.
The Snapshot Problem
Here's where it gets tricky. If you're backing up tables in parallel at different times, you can't just start dumping each table whenever its worker is ready. You need a consistent snapshot. Table A's backup and Table B's backup need to reflect the same logical point in time. Otherwise your restore gives you a database where some tables are from 2:00 AM and others are from 2:45 AM.
PlanetScale handles this using Postgres snapshots (via pg_export_snapshot() or by starting all transactions from a single point). The idea:
- Start a transaction on the primary and export the snapshot ID.
- Each worker connects and sets its transaction to that snapshot via
SET TRANSACTION SNAPSHOT. - Now every worker reads from the same logical point in time, even though they're running concurrently.
This works well, but it does hold open a snapshot for the duration of the backup. That means Postgres can't vacuum dead tuples older than your snapshot. For a 2-hour backup window on a high-write database, that's something to monitor. VACUUM lag during the backup can lead to table bloat.
The practical mitigation: run your backups during low-write periods. If that's not possible (and for many SaaS workloads, there's no such thing as low-write), keep your backup parallelism high enough that the window shrinks significantly. Eight parallel workers finishing in 45 minutes is a lot less painful for vacuum pressure than one sequential job running for 6 hours.
Applying This Pattern Yourself
You don't need PlanetScale's infrastructure to do this. Here's a rough framework you can build with standard tooling and a weekend of scripting.
Step 1: Build a table inventory.
SELECT
schemaname,
relname,
pg_size_pretty(pg_total_relation_size(relid)) AS total_size,
pg_total_relation_size(relid) AS raw_size,
n_live_tup,
last_vacuum
FROM pg_stat_user_tables
JOIN pg_class ON pg_class.oid = relid
WHERE schemaname = 'public'
ORDER BY raw_size DESC;This gives you the data you need to make scheduling decisions.
Step 2: Define your worker pool.
Pick a parallelism level. Eight workers is a good starting point for a database with a few terabytes. You can tune this based on available I/O, CPU, and how much replication lag you can tolerate.
Write a scheduler that assigns tables to workers. The simplest heuristic: sort tables by size descending. Assign the largest table to worker 1, next largest to worker 2, and so on, cycling back when you reach your worker limit. More sophisticated versions use a bin-packing algorithm to balance total bytes per worker.
Step 3: Export and share the snapshot.
BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
SELECT pg_export_snapshot();This returns a snapshot ID like 00000003-0000000B-1. Pass it to every worker.
Step 4: Each worker dumps its assigned tables.
SET TRANSACTION SNAPSHOT '00000003-0000000B-1';Then run pg_dump -table=your_table or use the COPY command directly. Write output to a shared location with clear naming conventions.
Step 5: Assemble the manifest.
After all workers finish, generate a JSON manifest:
{
"snapshot_id": "00000003-0000000B-1",
"started_at": "2026-07-15T02:00:00Z",
"completed_at": "2026-07-15T02:43:00Z",
"tables": [
{"schema": "public", "table": "events", "file": "events.sql.gz", "size": 234881024, "sha256": "a1b2c3..."},
{"schema": "public", "table": "users", "file": "users.sql.gz", "size": 89123456, "sha256": "d4e5f6..."}
]
}Store this alongside your backups. Without it, restoring becomes a guessing game.
What About pg_basebackup?
pg_basebackup is the standard physical backup tool. It backs up the entire data directory as files. It's great for point-in-time recovery when combined with WAL archiving. But it's inherently sequential. You're copying files from the filesystem level, and those files are one-per-relation only internally.
Tools like pgBackRest have added parallelism to physical backups, and it's worth looking at if your team needs PITR and doesn't want to roll custom logical backup tooling. pgBackRest lets you configure process-max to control parallelism, and it handles the snapshot consistency problem internally.
The tradeoff: physical backups (pg_basebackup, pgBackRest) give you the full data directory and are faster for full restores. Logical backups (pg_dump, the parallel pattern above) give you per-table granularity, which is useful for selective restores, migrations, and dev environment seeding.
For most teams, the answer isn't either/or. Use physical backups for your disaster recovery strategy and parallel logical backups for operational flexibility.
Practical Gotchas Before You Ship This
A few things that bite people:
Replication slots. If you're using logical replication, your backup snapshot can hold the replication slot open. This prevents WAL cleanup. Monitor pg_replication_slots during backups.
Large objects. Postgres large objects (lo) aren't part of normal table dumps. If your app uses them, you need a separate strategy.
Extensions and custom types. pg_dump handles these, but when you split tables across workers, each worker's dump needs access to the same extension definitions. Make sure your restore process installs extensions first.
Permissions and ownership. Parallel dumps create files as the connecting user. If different tables have different owners and you're dumping as a superuser, ownership info gets preserved in the dump. Just double-check during restore testing.
Compression. Gzip your output. The CPU cost is almost always worth it. On a typical text-heavy Postgres database, gzip compression ratios of 5:1 to 10:1 are common. A 1TB backup becomes 100-200GB. That's the difference between a 2-hour upload and a 20-minute upload.
The Real Win
The point of all this isn't just "backups go faster." It's that backups stop being a source of operational anxiety. When your backup window drops from 6 hours to 45 minutes, you stop dreading it. You stop skipping backups "just this once" because the impact window is too long. You can run backups more frequently, which means your RPO improves.
For teams running Postgres at scale, parallelizing your backup strategy is one of the highest-leverage infrastructure improvements you can make. The pattern isn't complicated. The tools exist. The only real barrier is sitting down and wiring it together.
