pg_stat_statements temporary file - Mailing list pgsql-hackers

From Magnus Hagander
Subject pg_stat_statements temporary file
Date
Msg-id CABUevEwxJxnGTOG+yUT5qcdG4iP=HZFVP8vMN6Z3BEGCdY4P4g@mail.gmail.com
Whole thread Raw
Responses Re: pg_stat_statements temporary file
List pgsql-hackers
If pg_stat_statements is set to store it's data across restarts, it
stores it in global/pg_stat_statements.stat. This causes some
interesting things to happen in combination with a base backup -
namely, if you take a base backup *after* you have restarted th
emaster, the slave will get a snapshot of whatever was in the
temporary file at the time of the restart. This is quite unpredictable
- particularly in relation to a slave where it gets a snapshot from
the last restart, not from the base backup, after which it diverges.
AFAICT, it also has the property that if the server crashes, it will
reload the latest snapshot - not reset to 0 or anything like that.

Finally, if the server were to crash *while* the file is being
written, it will get a corrupt file (I haven't tested this part, but
it's rather obvious from the code). I'm pretty sure this could lead to
a situation where the database wouldn't restart.

Fixing the last part is easy - we need to write the file to a
temporary file and then rename() it into place, like we do with the
stats collectors file.

Fixing the first one, I can think of a few things:

1) unlink() the file after we've read it.
2) forcibly exclude the file from base backups taken with
pg_basebackup. We'd still have the problem when it comes to backups
taken manually.
3) avoid loading the file on a standby (that wouldn't fix the similar
problem on the master of course)

And perhaps some other solution I haven't thought of?

If we want to go with option 2, we have another problem - it's in the
global directory. But the name is dependent on what's in a contrib
module. Meaning we'd have to teach core postgresql about a contrib
modules filename, which pretty much breaks the abstraction layer.
Perhaps a better choice here would be to create another directory
under the data directory that is always excluded from base backup, and
store it there? That would also work for third party modules that core
can never learn about...

In general, should a contrib module really store data in the global/
directory? Seems pretty ugly to me...

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/


pgsql-hackers by date:

Previous
From: Florian Pflug
Date:
Subject: Re: Changing the concept of a DATABASE
Next
From: Kohei KaiGai
Date:
Subject: Re: [RFC] Interface of Row Level Security