Proposed Resource Manager Changes - Mailing list pgsql-hackers

I would like to make a simple change to Resource Manager code to
introduce a plugin that can redefine or extend Rmgrs.

Providing an rmgr plugin in this way will also allow:

* filtering/control over apply of certain WAL records. We might decide
to skip all but certain tables, indexes, tablespaces or databases,
allowing a partial rebuild facility during PITR.

* ability to build a WAL debug utility that can stop/start/inspect WAL
records as we move thru WAL during recovery, or selectively output
specific record types or debug messages every N records. Would allow a
WAL debugger client GUI to be built into pgadmin, for example.

* dynamic addition of new index types, since we will be able to snap-in
any new index types via plugins, then insert into pg_am.

* additional actions alongside existing recovery actions. For example,
pre-processing of directories for new tablespace creation.

* new WAL records to allow issuing processing instructions to the
standby server with user defined meaning: "activate enterprise device
#72", "coordinate with user software x", "send SNMP trap 1157, to allow
us to measure delay between when it was sent on primary and when it was
processed on standby".

We could do each of the above in different ways, though it seems most
straightforward to provide a plugin that allows both extension and
redefinition of the RmgrTable.

Proposed way recognises that there is room for up to 255 rmgrs, since
the RmgrId is a single byte field on the WAL record.

* redefine RmgrTable as fixed size array of 255 elements that is
malloc'd into Startup process at beginning of StartupXlog()
* first few entries are "fixed" and we reserve first 32 elements for
future use by Postgres Core.
* values 33-127 are available by centrally managed registration to help
avoid conflicts in Postgres projects on pgfoundry
* values 128+ are user defined
* startup sets up fixed rmgrs, then calls plugin if it exists to modify
and/or add new rmgrs
* a new option to define behaviour if we receive an unhandled rmgrid.
Current behaviour is to declare this an invalid WAL record
* we might also check rmgrids when we enter XLogInsert() to ensure
everything written can be read if we crash, not sure whether people will
think that is overkill or essential (I'd say essential, but people may
fear performance problems).

Sample plugin showing filtering of WAL records for a specific databaseid
would be provided with patch.

(and yes, I see it will fall to me to document all of these new possible
types of plugin, so we have a new chapter on Server Extensibility).

Your thoughts?

-- Simon Riggs           www.2ndQuadrant.comPostgreSQL Training, Services and Support