WIP: write flat files at startup, enforce XID wrap limit - Mailing list pgsql-patches

I am about to commit the attached patch to HEAD; I'm posting it
principally for discussion about whether anyone cares to risk
applying it to REL8_0_STABLE as well.

The patch creates a new file src/backend/utils/init/flatfiles.c
to manage the "flat file" copies of pg_shadow and pg_group, as
well as a new flat file copy of pg_database.  The bulk of the
code in this file was formerly in commands/user.c.  (The patch
would be physically smaller if I hadn't moved code around, but
I thought this was an important improvement of modularity.)

The flat-files-writing code is now invoked in the database startup
process and in standalone backends, just after WAL replay is finished.
This guarantees that the flat files are properly synced with the
database contents in crash recovery situations.  This fixes the
problem Michael Klatt reported here:
http://archives.postgresql.org/pgsql-admin/2005-02/msg00031.php

It's only a partial fix unfortunately since it only handles rewriting
pg_pwd not pg_group.  I know how to make it work for pg_group in
all cases except an out-of-line-toasted grolist value, which is
probably a sufficient solution for 8.0.*.  (Hopefully the planned
pg_role rewrite will cure this limitation for 8.1 and beyond.)
However, what I have to do to make that happen is to change the
format of the pg_pwd and pg_group flat files: group membership has
to be recorded in terms of user sysids not user names, so that we
can avoid doing SysCache lookups in write_group_file.  That's a
significant additional chunk of work and I thought it would be
better to handle it as a separate patch.

The code to write a flat copy of pg_database is intended to allow us
to get rid of GetRawDatabaseInfo (hooray), but that goal can't be
achieved without an initdb to add a trigger to pg_database (else we
can't trust the flat file fully); so that part certainly isn't a
candidate for back-patching.  The reason for creating the file now
is that I've made the same routine also compute the oldest datfrozenxid
present in pg_database, which is exactly the information needed to
enforce a transaction ID limit to prevent XID wraparound (see recent
flamewar in pghackers).  So the patch also implements that.  I've
set it up to start warning at 10M transactions before wrap, and reject
further transactions at 1M transactions before wrap.  The only way out
of the latter condition is to shut down the postmaster and run a
standalone backend to do the needed VACUUMs (we only warn, not reject,
in a standalone backend).

I still have to document this stuff before committing, but the code
is all here.

Comments?  I personally think that this is too large a patch to risk
committing into the 8.0 branch, but I suppose it depends on your
level of concern about the XID wrap hazard.  (The patch does appear
to apply cleanly to the 8.0 branch, but I've not tested it there.)
I'd definitely not want to try to back-patch it to before 8.0.

            regards, tom lane