Re: log_checkpoint's "WAL file(s) added" is misleading to the point of uselessness - Mailing list pgsql-hackers

From Bossart, Nathan
Subject Re: log_checkpoint's "WAL file(s) added" is misleading to the point of uselessness
Date
Msg-id 446C88D7-2B94-4043-A4D7-132210D18069@amazon.com
Whole thread Raw
In response to Re: log_checkpoint's "WAL file(s) added" is misleading to the point of uselessness  (Fujii Masao <masao.fujii@oss.nttdata.com>)
Responses Re: log_checkpoint's "WAL file(s) added" is misleading to the point of uselessness  (Fujii Masao <masao.fujii@oss.nttdata.com>)
Re: log_checkpoint's "WAL file(s) added" is misleading to the point of uselessness  (Andres Freund <andres@anarazel.de>)
List pgsql-hackers
On 7/24/21, 8:10 PM, "Fujii Masao" <masao.fujii@oss.nttdata.com> wrote:
> On 2021/07/25 7:50, Andres Freund wrote:
>> Hi,
>>
>> I've been repeatedly confused by the the number of WAL files supposedly
>> added. Even when 100s of new WAL files are created the relevant portion
>> of log_checkpoints will only ever list zero or one added WAL file.
>>
>> The reason for that is that CheckpointStats.ckpt_segs_added is only
>> incremented in PreallocXlogFiles(). Which has the following comment:
>>   * XXX this is currently extremely conservative, since it forces only one
>>   * future log segment to exist, and even that only if we are 75% done with
>>   * the current one.  This is only appropriate for very low-WAL-volume systems.
>>
>> Whereas in real workloads WAL files are almost exclusively created via
>> XLogWrite()->XLogFileInit().
>>
>> I think we should consider just removing that field. Or, even better, show
>> something accurate instead.
>
> +1 to show something accurate instead.
>
> It's also worth showing them in monitoring stats view like pg_stat_wal?

+1.  I was confused by this when working on a WAL pre-allocation
patch [0].  Perhaps it could be replaced by a new parameter and a new
field in pg_stat_wal.  How about something like log_wal_init_interval,
where the value is the minimum amount of time between reporting the
number of WAL segments created since the last report?

Nathan

[0] https://www.postgresql.org/message-id/flat/20201225200953.jjkrytlrzojbndh5@alap3.anarazel.de


pgsql-hackers by date:

Previous
From: Tom Lane
Date:
Subject: Re: Removing "long int"-related limit on hash table sizes
Next
From: Stephen Frost
Date:
Subject: Re: Delegating superuser tasks to new security roles (Was: Granting control of SUSET gucs to non-superusers)