On 27.11.2014 00:21, Tomas Vondra wrote:
> On 27.11.2014 00:01, Maxim Boguk wrote:
>>
>>
>> On Thu, Nov 27, 2014 at 1:26 AM, Tomas Vondra <tv@fuzzy.cz
>> <mailto:tv@fuzzy.cz>> wrote:
>>
>> On 26.11.2014 23:06, Maxim Boguk wrote:
>> >
>> >
>> > I'm not sure I understand what you mean.
>> >
>> > The timestamp *IS* updated on every write - see this:
>> >
>> > https://github.com/postgres/postgres/blob/REL9_2_STABLE/src/backend/postmaster/pgstat.c#L3469
>> >
>> >
>> > I mean that usually 99.9% writes goes through mmap, which update the
>> > pgstat.tmp file timestamp (but not update last_statwrite value).
>>
>> I don't follow. Can you explain what you mean by the mmap? I don't see
>> any mmap stuff in pgstat.c, nor I remember dealing with mmap in my
>> patches in this area ...
>>
>>
>> Hm I had been wrong. I seen mmap call on strace after every statistic
>> collector file rewrite and I thought that all writes to that file goes
>> through memory mapped structure instead of full file writes every time
>> when someone need fresh data.
>>
>> However, point still open - there are no reason to write a new file
>> from scratch if no activity had been received from stat socket since
>> the last write (the same point valid for per-database stat files in
>> 9.3+).
>
> Well, nothing is perfect and patches are always welcome ;-)
>
> For us, the per-db split was enough - it lowered the CPU and I/O load to
> the point that adding such 'is dirty' flag would make absolutely no
> difference. I do have it on my TODO list, but it's not very high.
FWIW, I got curious and checked why we decided not to implement this
while reworking the stats in 9.3, as keeping an is_dirty flag seems as a
rather straightforward and simple optimization.
Turns out it's actually considerably more complex, because one of the
sources of statistics updates are (surprise surprise) autovacuum
workers. So the whole flow may look like this (in >= 9.3):
1) launcher requests a fresh stats file (dbs list only)
2) worker is started for a particular DB (by launcher)
3) the worker requests a stats file (with details for the DB)
4) the worker vacuums the DB - scans pg_class etc., updates the
table stats (autovac timestamps, blocks/tuples read ...)
5) at worker shutdown, the stats are sent back to postmaster
So this dirties the stats file (written in 3) for the database, making
the file obsolete.
Now, there are two choices - you may immediately flush the changes to
disk (but maybe you'll receive more changes before the next autovacuum
round happens), or you wait and only write the file if requested (which
is what we do now).
So the file *IS* dirty, and I don't see any obvious solution to this :-(
Tomas
