On Wed, Sep 23, 2020 at 11:29 AM Andres Freund <andres@anarazel.de> wrote:
> I wonder what the effect of logging WAL records as huge as this (~256kb)
> is on concurrent sessions. I think it's possible that logging 32 pages
> at once would cause latency increases for concurrent OLTP-ish
> writes. And that a smaller batch size would reduce that, while still
> providing most of the speedup.
Something to consider, but I cannot see any speedup, and so have no
way of evaluating the idea right now.
> > It doesn't seem to make any difference on my machine, which has an
> > NVME SSD (a Samsung 970 Pro). This is quite a fast SSD, though the
> > sync time isn't exceptional.
>
> Yea, they are surprisingly slow at syncing, somewhat disappointing for
> the upper end of the consumer oriented devices.
I hesitate to call anything about this SSD disappointing, since
overall it performs extremely well -- especially when you consider the
price tag.
Detachable storage is a trend that's here to stay, so the storage
latency is still probably a lot lower than what you'll see on many
serious production systems. For better or worse.
> Really should replace WAL compression with lz4 (or possibly zstd).
Yeah. WAL compression is generally a good idea, and we should probably
find a way to enable it by default (in the absence of some better way
of dealing with the FPI bottleneck, at least). I had no idea that
compression could hurt this much with index builds until now, though.
To be clear: the *entire* index build takes 3 times more wall clock
time, start to finish -- if I drilled down to the portion of the index
build that actually writes WAL then it would be an even greater
bottleneck.
I know that we've tested different compression methods in the past,
but perhaps index build performance was overlooked. My guess is that
the compression algorithm matters a lot less with pure OLTP workloads.
Also, parallel CREATE INDEX may be a bit of an outlier here. Even
still, it's a very important outlier.
--
Peter Geoghegan
