Thread: bgwriter stats

Attached is a new version of the bgwriter stats patch. Per previous
discussion, now uses the stats system only. Introduces a new stats message
for bgwriter, and also introduces a "global stats" part of the stats file
for statistics not bound to a database or table.

I've included a couple of more counters per ideas from Greg Smith in his
logging patch.

Unless there are objections, I'll go ahead and apply this version of the
patch (once the documentation is written, of course).

//Magnus

Magnus Hagander <magnus@hagander.net> writes:
> Attached is a new version of the bgwriter stats patch. Per previous
> discussion, now uses the stats system only. Introduces a new stats message
> for bgwriter, and also introduces a "global stats" part of the stats file
> for statistics not bound to a database or table.

It looks sane in a quick once-over, but the comments need some love --- too
many typos and misspellings for my taste.

            regards, tom lane

Tom Lane wrote:
> Magnus Hagander <magnus@hagander.net> writes:
>> Attached is a new version of the bgwriter stats patch. Per previous
>> discussion, now uses the stats system only. Introduces a new stats message
>> for bgwriter, and also introduces a "global stats" part of the stats file
>> for statistics not bound to a database or table.
>
> It looks sane in a quick once-over, but the comments need some love --- too
> many typos and misspellings for my taste.

Ok. I'll be sure to work them over before I commit. Thanks for the quick
check.

//Magnus

On Thu, 29 Mar 2007, Magnus Hagander wrote:

> I've included a couple of more counters per ideas from Greg Smith in his
> logging patch.

I just submitted a patch that logs the remaining things of value from my
original that couldn't be derived from the information you're collecting.
Between them I'm happy that a lot of previously hidden performance issues
can now be monitored--not necessarily easily, but it's at least possible.

I made one small change to your code in there as well I wanted to
highlight here.  You updated the buffers written by checkpoints one at a
time as they wrote out.  When I tried to develop something that monitored
pg_stat_bgwriter looking for when checkpoints happened, this made it
difficult to answer the important question "how many buffers did the last
checkpoint write?" just from the stats structure because I assumed it's
possible to get a view in the middle of the checkpoint.  It took watching
both the total and the checkpoint count, and that was hard to work with.

I modified things so that the checkpoint buffers written number gets
updated in one shot when the buffer flush is done.  No partial results,
much easier to monitor:  when the buffers_checkpoint value changes, the
difference from the previous value is what the last checkpoint wrote.  I
needed that total anyway which is why I just slipped it into the other
patch.

--
* Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD

Greg Smith wrote:
> On Thu, 29 Mar 2007, Magnus Hagander wrote:
>
>> I've included a couple of more counters per ideas from Greg Smith in his
>> logging patch.
>
> I just submitted a patch that logs the remaining things of value from my
> original that couldn't be derived from the information you're
> collecting. Between them I'm happy that a lot of previously hidden
> performance issues can now be monitored--not necessarily easily, but
> it's at least possible.
>
> I made one small change to your code in there as well I wanted to
> highlight here.  You updated the buffers written by checkpoints one at a
> time as they wrote out.  When I tried to develop something that
> monitored pg_stat_bgwriter looking for when checkpoints happened, this
> made it difficult to answer the important question "how many buffers did
> the last checkpoint write?" just from the stats structure because I
> assumed it's possible to get a view in the middle of the checkpoint.  It
> took watching both the total and the checkpoint count, and that was hard
> to work with.

Uh, what? ;)

The data in pg_stat_bgwriter certainly doesn't update *during* a
checkpoint, if that's what you're saying. It is collected one by one in
the global struct in the bgwriter, but it's only sent off to the stats
collector once the checkpoint has completed (it's sent at the end of the
loop in BackgroundWriterMain). And it's the data that's in the stats
collector that you're looking at with pg_stat_bgwriter.

//Magnus

On Sun, 1 Apr 2007, Magnus Hagander wrote:

> The data in pg_stat_bgwriter certainly doesn't update *during* a
> checkpoint, if that's what you're saying.

Scratch previous message, replace with "dude ur code rulez!"

I was working on the assumption it was possible to get a partial result
because I haven't had enough time track all the code paths involved to
prove otherwise. Glad to hear it was never an issue.  Doesn't change what
I submitted though.

--
* Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD