Re: In-order pg_dump (or in-order COPY TO) - Mailing list pgsql-general

From Ron Johnson
Subject Re: In-order pg_dump (or in-order COPY TO)
Date
Msg-id CANzqJaDGUPKM2h74wFb2DOgbKPnAe+c352eMTvehQ8D+vfxidA@mail.gmail.com
Whole thread Raw
In response to In-order pg_dump (or in-order COPY TO)  (Dimitrios Apostolou <jimis@gmx.net>)
Responses Re: In-order pg_dump (or in-order COPY TO)
List pgsql-general
On Tue, Aug 26, 2025 at 3:44 PM Dimitrios Apostolou <jimis@gmx.net> wrote:
Hello list,

I am storing dumps of a database (pg_dump custom format) in a
de-duplicating backup server. Each dump is many terabytes in size, so
deduplication is very important. And de-duplication itself is based on
rolling checksums which is pretty flexible, it can compensate for blocks
moving by some offset.

This might be a silly question, but why are you using -Fc to create multi-TB dumps instead of -Fd?  If nothing else, the parallelization granted by -Fd will greatly speed up the dumps.

--
Death to <Redacted>, and butter sauce.
Don't boil me, I'm still alive.
<Redacted> lobster!

pgsql-general by date:

Previous
From: "David G. Johnston"
Date:
Subject: Re: In-order pg_dump (or in-order COPY TO)
Next
From: Tom Lane
Date:
Subject: Re: In-order pg_dump (or in-order COPY TO)