Hi,
Please find attached a patch optimizing pg_dump's tar format (-Ft) when writing to a seekable file. The diff here is limited to src/bin/pg_dump/pg_backup_tar.c.
Currently, every TOC entry in the tar-format dump goes through a temporary file: data is written to a tmpfile, then on close the tmpfile is seeked to determine its length, the tar header is written, and the entire tmpfile gets copied to the tar output. We end up writing the data twice: once to the tmpfile and once to the final tar file.
The patch adds a "direct-write" mode for seekable outputs. Instead of using a tmpfile, we write a placeholder tar header (with length 0) directly to the tar output, stream the data after it, then seek back to rewrite the header with the actual length. This should cut the I/O in half for the data path.
The tmpfile path is preserved as a fallback for three cases:
1. Output is not seekable (stdout/pipe)
2. Another member is already being written directly (guard against interleaving)
3. We're in the LO section, where the blob TOC file stays open while individual blob data files are written and closed inside it
On a test 500K-row database (~255MB, 184MB dump file), pg_dump -Ft time goes down from about 1.42s (master) to 1.22s (patched). The percent improvement is a bit less for larger databases: dump time goes down from 10.24s (master) to 9.34s (patched) for a database about 10x as large.
A benchmark script (bench_tar_direct_write.sh) is included for reproducing some of the performance testing I did.
Thanks,
Josh
