Scott Carey <scott@richrelevance.com> writes:
> On 7/30/09 2:53 PM, "Tom Lane" <tgl@sss.pgh.pa.us> wrote:
>> Scott Carey <scott@richrelevance.com> writes:
>>> Gzip does have some quirky performance behavior depending on the chunk size
>>> of data you stream into it.
>>
>> Can you enlarge on that comment? I'm not sure that pg_dump is aware
>> that there's anything to worry about there.
> For whatever reason, some other internals of gzip tend to perform much
> better if submitting say, 4k or 8k or 16k chunks rather than little bits at
> a time. But I'm sure some of that also depends on what library you're using
> since they all vary somewhat.
AFAIK there is only one widely-used implementation of zlib, and it
hasn't changed much in a long time.
I did some tracing and verified that pg_dump passes data to deflate()
one table row at a time. I'm not sure about the performance
implications of that, but it does seem like it might be something to
look into.
regards, tom lane
