Hi Andrey, all,
I propose some changes, and I'm attaching a new version:
I renamed the GUCs as clog_buffers etc (no "_slru_"). I fixed some
copy/paste mistakes where the different GUCs were mixed up. I made
some changes to the .conf.sample. I rewrote the documentation so that
it states the correct unit and defaults, and refers to the
subdirectories that are cached by these buffers instead of trying to
give a new definition of each of the SLRUs.
Do you like those changes?
Some things I thought about but didn't change:
I'm not entirely sure if we should use the internal and historical
names well known to hackers (CLOG), or the visible directory names (I
mean, we could use pg_xact_buffers instead of clog_buffers). I am not
sure why these GUCs need to be PGDLLIMPORT, but I see that NBuffers is
like that.
I wanted to do some very simple smoke testing of CLOG sizes on my
local development machine:
pgbench -i -s1000 postgres
pgbench -t4000000 -c8 -j8 -Mprepared postgres
I disabled autovacuum after running that just to be sure it wouldn't
interfere with my experiment:
alter table pgbench_accounts set (autovacuum_enabled = off);
Then I shut the cluster down and made a copy, so I could do some
repeated experiments from the same initial conditions each time. At
this point I had 30 files 0000-001E under pg_xact, holding 256kB = ~1
million transactions each. It'd take ~960 buffers to cache it all.
So how long does VACUUM FREEZE pgbench_accounts take?
I tested with just the 0001 patch, and also with the 0002 patch
(improved version, attached):
clog_buffers=128: 0001=2:28.499, 0002=2:17:891
clog_buffers=1024: 0001=1:38.485, 0002=1:29.701
I'm sure the speedup of the 0002 patch can be amplified by increasing
the number of transactions referenced in the table OR number of
clog_buffers, considering that the linear search produces
O(transactions * clog_buffers) work. That was 32M transactions and
8MB of CLOG, but I bet if you double both of those numbers once or
twice things start to get hot. I don't see why you shouldn't be able
to opt to cache literally all of CLOG if you want (something like 50MB
assuming default autovacuum_freeze_max_age, scale to taste, up to
512MB for the theoretical maximum useful value).
I'm not saying the 0002 patch is bug-free yet though, it's a bit finickity.
