Re: [TODO] Track number of files ready to be archived in pg_stat_archiver - Mailing list pgsql-hackers

From Michael Paquier
Subject Re: [TODO] Track number of files ready to be archived in pg_stat_archiver
Date
Msg-id CAB7nPqTYK+Dj7oTTsx32tQNYHmGEFbEP4CKd9ZFOL3+AzEQsow@mail.gmail.com
Whole thread Raw
In response to Re: [TODO] Track number of files ready to be archived in pg_stat_archiver  ("Brightwell, Adam" <adam.brightwell@crunchydatasolutions.com>)
Responses Re: [TODO] Track number of files ready to be archived in pg_stat_archiver  (Michael Paquier <michael.paquier@gmail.com>)
Re: [TODO] Track number of files ready to be archived in pg_stat_archiver  (Julien Rouhaud <julien.rouhaud@dalibo.com>)
List pgsql-hackers
On Wed, Oct 22, 2014 at 12:50 AM, Brightwell, Adam
<adam.brightwell@crunchydatasolutions.com> wrote:
> Though, I would think that the general desire would be to keep the patch
> relevant ONLY to the necessary changes.  I would not qualify making those
> types of changes as relevant, IMHO.  I do think this is potential for
> cleanup, however, I would suspect that would be best done in a separate
> patch.  But again, I'd defer to a committer whether such changes are even
> necessary/acceptable.

I have been looking at this patch, and I think that it is a mistake to
count the .ready files present in archive_status when calling
pg_stat_get_archiver(). If there are many files waiting to be
archived, this penalizes the run time of this function, and the
application behind relying on those results, not to mention that
actually the loop used to count the .ready files is a copy of what is
in pgarch.c. Hence I think that we should simply count them in
pgarch_readyXlog, and then return a value back to
pgarch_ArchiverCopyLoop, value that could be decremented by 1 each
time a file is successfully archived to keep the stats as precise as
possible, and let the information know useful information when
archiver process is within a single loop process of
pgarch_ArchiverCopyLoop. This way, we just need to extend
PgStat_MsgArchiver with a new counter to track this number.

The attached patch, based on v2 sent previously, does so. Thoughts?
--
Michael

Attachment

pgsql-hackers by date:

Previous
From: Michael Paquier
Date:
Subject: Re: [TODO] Track number of files ready to be archived in pg_stat_archiver
Next
From: Jeff Davis
Date:
Subject: Re: 9.5: Better memory accounting, towards memory-bounded HashAgg