Re: [PING] [PATCH v2] parallel pg_restore: avoid disk seeks when jumping short distance forward - Mailing list pgsql-hackers

From Tom Lane
Subject Re: [PING] [PATCH v2] parallel pg_restore: avoid disk seeks when jumping short distance forward
Date
Msg-id 845486.1760402167@sss.pgh.pa.us
Whole thread Raw
In response to Re: [PING] [PATCH v2] parallel pg_restore: avoid disk seeks when jumping short distance forward  (Chao Li <li.evan.chao@gmail.com>)
Responses Re: [PING] [PATCH v2] parallel pg_restore: avoid disk seeks when jumping short distance forward
List pgsql-hackers
Chao Li <li.evan.chao@gmail.com> writes:
> I tested DEFAULT_IO_BUFFER_SIZE with 4K, 32K, 64K, 128K and 256K. Looks like increasing the buffer size doesn’t
improvethe performance significantly. Actually, with the buffer size 64K, 128K and 256K, the test results are very
close.I tested both with lz4 and none compression. I am not suggesting tuning the buffer size. These data are only for
yourreference. 

Yeah, I would not expect straight pg_dump/pg_restore performance
to vary very much once the buffer size gets above not-too-many KB.
The thing we are really interested in here is how fast pg_restore
can skip over unwanted table data in a large archive file, and that
I believe should be pretty sensitive to block size.

You could measure that without getting into the complexities of
parallel restore if you make a custom-format dump of a few large
tables that does not have offset data in it, and then seeing how
fast is selective restore of just the last table.

            regards, tom lane



pgsql-hackers by date:

Previous
From: Chao Li
Date:
Subject: Re: Fixed a typo in comment in compress_lz4.c
Next
From: Chao Li
Date:
Subject: Re: [PING] [PATCH v2] parallel pg_restore: avoid disk seeks when jumping short distance forward