Thread: pgsql: Avoid incrementing the CommandCounter when
Log Message:
-----------
Avoid incrementing the CommandCounter when CommandCounterIncrement is called
but no database changes have been made since the last CommandCounterIncrement.
This should result in a significant improvement in the number of "commands"
that can typically be performed within a transaction before hitting the 2^32
CommandId size limit. In particular this buys back (and more) the possible
adverse consequences of my previous patch to fix plan caching behavior.
The implementation requires tracking whether the current CommandCounter
value has been "used" to mark any tuples. CommandCounter values stored into
snapshots are presumed not to be used for this purpose. This requires some
small executor changes, since the executor used to conflate the curcid of
the snapshot it was using with the command ID to mark output tuples with.
Separating these concepts allows some small simplifications in executor APIs.
Something for the TODO list: look into having CommandCounterIncrement not do
AcceptInvalidationMessages. It seems fairly bogus to be doing it there,
but exactly where to do it instead isn't clear, and I'm disinclined to mess
with asynchronous behavior during late beta.
Modified Files:
--------------
pgsql/contrib/pgrowlocks:
pgrowlocks.c (r1.7 -> r1.8)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/contrib/pgrowlocks/pgrowlocks.c?r1=1.7&r2=1.8)
pgsql/src/backend/access/heap:
heapam.c (r1.245 -> r1.246)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/heap/heapam.c?r1=1.245&r2=1.246)
tuptoaster.c (r1.79 -> r1.80)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/heap/tuptoaster.c?r1=1.79&r2=1.80)
pgsql/src/backend/access/transam:
xact.c (r1.253 -> r1.254)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/transam/xact.c?r1=1.253&r2=1.254)
pgsql/src/backend/commands:
async.c (r1.136 -> r1.137)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/commands/async.c?r1=1.136&r2=1.137)
copy.c (r1.288 -> r1.289)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/commands/copy.c?r1=1.288&r2=1.289)
explain.c (r1.167 -> r1.168)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/commands/explain.c?r1=1.167&r2=1.168)
trigger.c (r1.224 -> r1.225)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/commands/trigger.c?r1=1.224&r2=1.225)
pgsql/src/backend/executor:
execMain.c (r1.300 -> r1.301)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/executor/execMain.c?r1=1.300&r2=1.301)
execUtils.c (r1.152 -> r1.153)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/executor/execUtils.c?r1=1.152&r2=1.153)
spi.c (r1.185 -> r1.186)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/executor/spi.c?r1=1.185&r2=1.186)
pgsql/src/backend/storage/ipc:
procarray.c (r1.37 -> r1.38)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/storage/ipc/procarray.c?r1=1.37&r2=1.38)
pgsql/src/backend/utils/cache:
inval.c (r1.81 -> r1.82)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/cache/inval.c?r1=1.81&r2=1.82)
pgsql/src/backend/utils/time:
tqual.c (r1.107 -> r1.108)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/time/tqual.c?r1=1.107&r2=1.108)
pgsql/src/include/access:
xact.h (r1.91 -> r1.92)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/include/access/xact.h?r1=1.91&r2=1.92)
pgsql/src/include/commands:
trigger.h (r1.63 -> r1.64)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/include/commands/trigger.h?r1=1.63&r2=1.64)
pgsql/src/include/executor:
executor.h (r1.144 -> r1.145)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/include/executor/executor.h?r1=1.144&r2=1.145)
pgsql/src/include/nodes:
execnodes.h (r1.181 -> r1.182)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/include/nodes/execnodes.h?r1=1.181&r2=1.182)
pgsql/src/test/regress/expected:
combocid.out (r1.1 -> r1.2)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/test/regress/expected/combocid.out?r1=1.1&r2=1.2)
pgsql/src/test/regress/sql:
combocid.sql (r1.1 -> r1.2)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/test/regress/sql/combocid.sql?r1=1.1&r2=1.2)
Added to TODO: * Consider if CommandCounterIncrement() can avoid its AcceptInvalidationMessages() call http://archives.postgresql.org/pgsql-committers/2007-11/msg00585.php --------------------------------------------------------------------------- Tom Lane wrote: > Log Message: > ----------- > Avoid incrementing the CommandCounter when CommandCounterIncrement is called > but no database changes have been made since the last CommandCounterIncrement. > This should result in a significant improvement in the number of "commands" > that can typically be performed within a transaction before hitting the 2^32 > CommandId size limit. In particular this buys back (and more) the possible > adverse consequences of my previous patch to fix plan caching behavior. > > The implementation requires tracking whether the current CommandCounter > value has been "used" to mark any tuples. CommandCounter values stored into > snapshots are presumed not to be used for this purpose. This requires some > small executor changes, since the executor used to conflate the curcid of > the snapshot it was using with the command ID to mark output tuples with. > Separating these concepts allows some small simplifications in executor APIs. > > Something for the TODO list: look into having CommandCounterIncrement not do > AcceptInvalidationMessages. It seems fairly bogus to be doing it there, > but exactly where to do it instead isn't clear, and I'm disinclined to mess > with asynchronous behavior during late beta. > > Modified Files: > -------------- > pgsql/contrib/pgrowlocks: > pgrowlocks.c (r1.7 -> r1.8) > (http://developer.postgresql.org/cvsweb.cgi/pgsql/contrib/pgrowlocks/pgrowlocks.c?r1=1.7&r2=1.8) > pgsql/src/backend/access/heap: > heapam.c (r1.245 -> r1.246) > (http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/heap/heapam.c?r1=1.245&r2=1.246) > tuptoaster.c (r1.79 -> r1.80) > (http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/heap/tuptoaster.c?r1=1.79&r2=1.80) > pgsql/src/backend/access/transam: > xact.c (r1.253 -> r1.254) > (http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/transam/xact.c?r1=1.253&r2=1.254) > pgsql/src/backend/commands: > async.c (r1.136 -> r1.137) > (http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/commands/async.c?r1=1.136&r2=1.137) > copy.c (r1.288 -> r1.289) > (http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/commands/copy.c?r1=1.288&r2=1.289) > explain.c (r1.167 -> r1.168) > (http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/commands/explain.c?r1=1.167&r2=1.168) > trigger.c (r1.224 -> r1.225) > (http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/commands/trigger.c?r1=1.224&r2=1.225) > pgsql/src/backend/executor: > execMain.c (r1.300 -> r1.301) > (http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/executor/execMain.c?r1=1.300&r2=1.301) > execUtils.c (r1.152 -> r1.153) > (http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/executor/execUtils.c?r1=1.152&r2=1.153) > spi.c (r1.185 -> r1.186) > (http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/executor/spi.c?r1=1.185&r2=1.186) > pgsql/src/backend/storage/ipc: > procarray.c (r1.37 -> r1.38) > (http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/storage/ipc/procarray.c?r1=1.37&r2=1.38) > pgsql/src/backend/utils/cache: > inval.c (r1.81 -> r1.82) > (http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/cache/inval.c?r1=1.81&r2=1.82) > pgsql/src/backend/utils/time: > tqual.c (r1.107 -> r1.108) > (http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/time/tqual.c?r1=1.107&r2=1.108) > pgsql/src/include/access: > xact.h (r1.91 -> r1.92) > (http://developer.postgresql.org/cvsweb.cgi/pgsql/src/include/access/xact.h?r1=1.91&r2=1.92) > pgsql/src/include/commands: > trigger.h (r1.63 -> r1.64) > (http://developer.postgresql.org/cvsweb.cgi/pgsql/src/include/commands/trigger.h?r1=1.63&r2=1.64) > pgsql/src/include/executor: > executor.h (r1.144 -> r1.145) > (http://developer.postgresql.org/cvsweb.cgi/pgsql/src/include/executor/executor.h?r1=1.144&r2=1.145) > pgsql/src/include/nodes: > execnodes.h (r1.181 -> r1.182) > (http://developer.postgresql.org/cvsweb.cgi/pgsql/src/include/nodes/execnodes.h?r1=1.181&r2=1.182) > pgsql/src/test/regress/expected: > combocid.out (r1.1 -> r1.2) > (http://developer.postgresql.org/cvsweb.cgi/pgsql/src/test/regress/expected/combocid.out?r1=1.1&r2=1.2) > pgsql/src/test/regress/sql: > combocid.sql (r1.1 -> r1.2) > (http://developer.postgresql.org/cvsweb.cgi/pgsql/src/test/regress/sql/combocid.sql?r1=1.1&r2=1.2) > > ---------------------------(end of broadcast)--------------------------- > TIP 6: explain analyze is your friend -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://postgres.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +