Database Lab 0.2.0, a tool for fast cloning of large PostgreSQL databases to build
of temporary full-size Postgres databases. It is helpful in various cases:
1) Various testing, including using full-size databases in CI tests.
2) Verification of DB schema changes (database migrations), detecting potentially
dangerous changes, before they reach production.
3) SQL performance troubleshooting: a database clone provided by Database Lab
has execution plans identical to the original database, therefore "EXPLAIN (BUFFERS,
ANALYZE") will show the same plan structure and the same amount of data involved.
Verifying various SQL optimization ideas becomes simple (see also: Joe bot, working
on top of Database Lab
https://gitlab.com/postgres-ai/joe/).
Characteristics and features:
- Open Source, writing in Go.
- All components are running in Docker containers. Custom Postgres containers are
supported (extensions, additional tools, or even modified PostgreSQL binaries).
- Works well both on-premise and in clouds.
- Thin provisioning in seconds thanks to copy-on-write (CoW) provided by ZFS filesystem
and special methodology of preparing PostgreSQL database snapshots.
- Any types of original source databases are supported: on-premise, clouds (including
managed clouds like RDS – in this case, only dump/restore should be used for initial
thick cloning), any OS, FS. ZFS is supposed to be used only on Database Lab servers.
- Unlimited size of databases (Postgres database size is unlimited, ZFS volume can be
up to 2128 bytes, or 256 trillion yobibytes).
- Supported PostgreSQL versions: 9.6, 10, 11, and 12.
- Thin cloning takes only a few seconds, regardless of the database size.
- REST API.
- Client CLI.
- Automated deletion of clones after the specified amount of minutes of inactivity (configurable).
- Protection from deletion, to disable automated and accidental deletions.
- Continuously updated original copy of data is supported (based on "restore_command").
- Multiple snapshots to allow provisioning of various versions of the database.
