Thread: Doc about how to set max_wal_senders when setting minimal wal_level
Hi, hackers When I try to change wal_level to minimal and restart the database, it complains max_wal_senders > 0. 2022-03-03 10:10:16.938 CST [6389] FATAL: WAL streaming (max_wal_senders > 0) requires wal_level "replica" or "logical" However, the documentation about wal_level [1] doesn't mentation this. How about adding a sentence to describe how to set max_wal_senders when setting wal_level to minimal? [1] https://www.postgresql.org/docs/devel/runtime-config-wal.html -- Regrads, Japin Li. ChengDu WenWu Information Technology Co.,Ltd.
Attachment
Re: Doc about how to set max_wal_senders when setting minimal wal_level
From
"David G. Johnston"
Date:
On Wed, Mar 2, 2022 at 7:44 PM Japin Li <japinli@hotmail.com> wrote:
Hi, hackers
When I try to change wal_level to minimal and restart the database, it complains
max_wal_senders > 0.
2022-03-03 10:10:16.938 CST [6389] FATAL: WAL streaming (max_wal_senders > 0) requires wal_level "replica" or "logical"
However, the documentation about wal_level [1] doesn't mentation this.
How about adding a sentence to describe how to set max_wal_senders when
setting wal_level to minimal?
[1] https://www.postgresql.org/docs/devel/runtime-config-wal.html
I would suggest a wording more like:
"A precondition for using minimal WAL is to disable WAL archiving and streaming replication by setting max_wal_senders to 0, and archive_mode to off."
While accurate, the phrase "you must set" just doesn't feel right to me. I also don't like how the proposed sentence (either one) is added separately as opposed to being included in the immediately preceding paragraph. While this limited patch is probably sufficient I would suggest trying to work out a slightly larger patch the improves the wording on the entire existing paragraph while incorporating the reference to max_wal_senders.
Note, we seem to be missing the documentation of the default setting for archive_mode.
In addition, max_wal_senders could also be changed, adding a sentence like:
"If setting max_wal_senders to 0 consider also reducing the amount of WAL produced by changing wal_level to minimal."
At least insofar as core is concerned without a wal sender the additional wal of replica is not actually able to be leveraged as pg_basebackup will not work (at noted in its own description).
David J.
On Thu, 03 Mar 2022 at 11:25, David G. Johnston <david.g.johnston@gmail.com> wrote: > I would suggest a wording more like: > > "A precondition for using minimal WAL is to disable WAL archiving and > streaming replication by setting max_wal_senders to 0, and archive_mode to > off." > > While accurate, the phrase "you must set" just doesn't feel right to me. I > also don't like how the proposed sentence (either one) is added separately > as opposed to being included in the immediately preceding paragraph. While > this limited patch is probably sufficient I would suggest trying to work > out a slightly larger patch the improves the wording on the entire existing > paragraph while incorporating the reference to max_wal_senders. > Thanks for your review. Modified as your suggestion. > Note, we seem to be missing the documentation of the default setting for > archive_mode. > Add the default value for archive_mode. > In addition, max_wal_senders could also be changed, adding a sentence like: > > "If setting max_wal_senders to 0 consider also reducing the amount of WAL > produced by changing wal_level to minimal." > Modified. -- Regrads, Japin Li. ChengDu WenWu Information Technology Co.,Ltd.
Attachment
On Thu, 03 Mar 2022 at 12:10, Japin Li <japinli@hotmail.com> wrote: > On Thu, 03 Mar 2022 at 11:25, David G. Johnston <david.g.johnston@gmail.com> wrote: >> I would suggest a wording more like: >> >> "A precondition for using minimal WAL is to disable WAL archiving and >> streaming replication by setting max_wal_senders to 0, and archive_mode to >> off." >> >> While accurate, the phrase "you must set" just doesn't feel right to me. I >> also don't like how the proposed sentence (either one) is added separately >> as opposed to being included in the immediately preceding paragraph. While >> this limited patch is probably sufficient I would suggest trying to work >> out a slightly larger patch the improves the wording on the entire existing >> paragraph while incorporating the reference to max_wal_senders. >> > > Thanks for your review. Modified as your suggestion. > >> Note, we seem to be missing the documentation of the default setting for >> archive_mode. >> > > Add the default value for archive_mode. > >> In addition, max_wal_senders could also be changed, adding a sentence like: >> >> "If setting max_wal_senders to 0 consider also reducing the amount of WAL >> produced by changing wal_level to minimal." >> > > Modified. Attach v3 patch to fix missing close varname tag. -- Regrads, Japin Li. ChengDu WenWu Information Technology Co.,Ltd.
Attachment
Re: Doc about how to set max_wal_senders when setting minimal wal_level
From
Kyotaro Horiguchi
Date:
At Fri, 04 Mar 2022 12:18:29 +0800, Japin Li <japinli@hotmail.com> wrote in > > On Thu, 03 Mar 2022 at 12:10, Japin Li <japinli@hotmail.com> wrote: > > Attach v3 patch to fix missing close varname tag. + A precondition for using minimal WAL is to disable WAL archiving and + streaming replication by setting <xref linkend="guc-max-wal-senders"/> + to <literal>0</literal>, and <varname>archive_mode</varname> + to <literal>off</literal>. It is a bit odd that the features to stop and the corresponding GUCs are written irrespectively. It would be better they're in the same order. + servers. If setting <varname>max_wal_senders</varname> to + <literal>0</literal> consider also reducing the amount of WAL produced + by changing <varname>wal_level</varname> to <literal>minimal</literal>. Those who anively follow this suggestion may bump into failure when arhive_mode is on. Thus archive_mode is also worth referred to. But, anyway, IMHO, it is mere a performance tips that is not necessarily required in this section, or even in this documentaiotn. Addtion to that, if we write this for max_wal_senders, archive_mode will deserve the similar tips but I think it is too verbose. In short, I think I would not add that description at all. regards. -- Kyotaro Horiguchi NTT Open Source Software Center
On Fri, 04 Mar 2022 at 14:05, Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote:
> At Fri, 04 Mar 2022 12:18:29 +0800, Japin Li <japinli@hotmail.com> wrote in
>>
>> On Thu, 03 Mar 2022 at 12:10, Japin Li <japinli@hotmail.com> wrote:
>>
>> Attach v3 patch to fix missing close varname tag.
>
> + A precondition for using minimal WAL is to disable WAL archiving and
> + streaming replication by setting <xref linkend="guc-max-wal-senders"/>
> + to <literal>0</literal>, and <varname>archive_mode</varname>
> + to <literal>off</literal>.
>
> It is a bit odd that the features to stop and the corresponding GUCs
> are written irrespectively. It would be better they're in the same
> order.
>
Thanks for your review. Modified.
> + servers. If setting <varname>max_wal_senders</varname> to
> + <literal>0</literal> consider also reducing the amount of WAL produced
> + by changing <varname>wal_level</varname> to <literal>minimal</literal>.
>
> Those who anively follow this suggestion may bump into failure when
> arhive_mode is on. Thus archive_mode is also worth referred to. But,
> anyway, IMHO, it is mere a performance tips that is not necessarily
> required in this section, or even in this documentaiotn. Addtion to
> that, if we write this for max_wal_senders, archive_mode will deserve
> the similar tips but I think it is too verbose. In short, I think I
> would not add that description at all.
>
It already has a tip about wal_level for archive_mode [1], IIUC.
archive_mode cannot be enabled when wal_level is set to minimal.
[1] https://www.postgresql.org/docs/devel/runtime-config-wal.html#GUC-ARCHIVE-MODE
--
Regrads,
Japin Li.
ChengDu WenWu Information Technology Co.,Ltd.
Attachment
Re: Doc about how to set max_wal_senders when setting minimal wal_level
From
"David G. Johnston"
Date:
On Thu, Mar 3, 2022 at 11:05 PM Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote:
But,
anyway, IMHO, it is mere a performance tips that is not necessarily
required in this section, or even in this documentaiotn. Addtion to
that, if we write this for max_wal_senders, archive_mode will deserve
the similar tips but I think it is too verbose. In short, I think I
would not add that description at all.
I wrote it as a performance tip but it is documenting that when set to 0 no features of the server require more information than is captured in the minimal wal. That fact seems worthy of noting. Even at the cost of a bit of verbosity. These features interact with each other and that interaction should be adequately described. While subjective, this dynamic seems to warrant inclusion.
David J.
Re: Doc about how to set max_wal_senders when setting minimal wal_level
From
"David G. Johnston"
Date:
On Fri, Mar 4, 2022 at 2:49 AM Japin Li <japinli@hotmail.com> wrote:
Thanks for your review. Modified.
Works for me. I have some additional sparks of ideas but nothing that need hold this up.
David J.
Japin Li <japinli@hotmail.com> writes:
> [ v4-wal-level-documentation.patch ]
Hm, I don't care for the wording here:
+ A precondition for using minimal WAL is to disable WAL archiving and
+ streaming replication by setting <varname>archive_mode</varname> to
+ <literal>off</literal>, and <xref linkend="guc-max-wal-senders"/> to
+ <literal>0</literal>.
"Precondition" is an overly fancy word that makes things less clear
not more so. Does it mean that setting wal_level = minimal will fail
if you don't do these other things, or does it just mean that you
won't be getting the absolute minimum WAL volume? If the former,
I think it'd be better to say something like "To set wal_level to minimal,
you must also set [these variables], which has the effect of disabling
both WAL archiving and streaming replication."
+ servers. If setting <varname>max_wal_senders</varname> to
+ <literal>0</literal> consider also reducing the amount of WAL produced
+ by changing <varname>wal_level</varname> to <literal>minimal</literal>.
I don't think this is great advice. It will encourage people to use
wal_level = minimal even if they have other requirements that weigh
against it. If they feel that their system is producing too much
WAL, I doubt they'll have a hard time finding the wal_level knob.
TBH, I think the real problem with the docs in this area is that
the first para about wal_level is disjointed and unclear; we ought
to nuke and rewrite that. In particular it fails to provide adequate
context about what "logical decoding" means. I think possibly what
it needs to say is that replica mode supports *physical* replication
but if you want to use *logical* replication you need logical mode;
if you need neither, and are not doing WAL archiving either, you
can get away with minimal mode.
regards, tom lane
On Tue, Jul 5, 2022 at 08:02:33PM -0400, Tom Lane wrote: > "Precondition" is an overly fancy word that makes things less clear > not more so. Does it mean that setting wal_level = minimal will fail > if you don't do these other things, or does it just mean that you > won't be getting the absolute minimum WAL volume? If the former, > I think it'd be better to say something like "To set wal_level to minimal, > you must also set [these variables], which has the effect of disabling > both WAL archiving and streaming replication." I have created the attached patch to try to improve this text. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
Attachment
Sorry for the late reply. On Wed, 06 Jul 2022 at 08:02, Tom Lane <tgl@sss.pgh.pa.us> wrote: > Japin Li <japinli@hotmail.com> writes: >> [ v4-wal-level-documentation.patch ] > > Hm, I don't care for the wording here: > > + A precondition for using minimal WAL is to disable WAL archiving and > + streaming replication by setting <varname>archive_mode</varname> to > + <literal>off</literal>, and <xref linkend="guc-max-wal-senders"/> to > + <literal>0</literal>. > > "Precondition" is an overly fancy word that makes things less clear > not more so. Does it mean that setting wal_level = minimal will fail > if you don't do these other things, or does it just mean that you > won't be getting the absolute minimum WAL volume? If the former, > I think it'd be better to say something like "To set wal_level to minimal, > you must also set [these variables], which has the effect of disabling > both WAL archiving and streaming replication." Yeah, it's the former case. > > + servers. If setting <varname>max_wal_senders</varname> to > + <literal>0</literal> consider also reducing the amount of WAL produced > + by changing <varname>wal_level</varname> to <literal>minimal</literal>. > > I don't think this is great advice. It will encourage people to use > wal_level = minimal even if they have other requirements that weigh > against it. If they feel that their system is producing too much > WAL, I doubt they'll have a hard time finding the wal_level knob. > Agreed. It isn't good advice. We can remove the suggestion. -- Regrads, Japin Li. ChengDu WenWu Information Technology Co.,Ltd.
On Fri, 15 Jul 2022 at 08:49, Bruce Momjian <bruce@momjian.us> wrote: > On Tue, Jul 5, 2022 at 08:02:33PM -0400, Tom Lane wrote: >> "Precondition" is an overly fancy word that makes things less clear >> not more so. Does it mean that setting wal_level = minimal will fail >> if you don't do these other things, or does it just mean that you >> won't be getting the absolute minimum WAL volume? If the former, >> I think it'd be better to say something like "To set wal_level to minimal, >> you must also set [these variables], which has the effect of disabling >> both WAL archiving and streaming replication." > > I have created the attached patch to try to improve this text. IMO we can add the following sentence for wal_level description, since if wal_level = minimal and max_wal_senders > 0, we cannot start the database. To set wal_level to minimal, you must also set max_wal_senders to 0, which has the effect of disabling both WAL archiving and streaming replication. -- Regrads, Japin Li. ChengDu WenWu Information Technology Co.,Ltd.
Re: Doc about how to set max_wal_senders when setting minimal wal_level
From
"David G. Johnston"
Date:
On Fri, Jul 15, 2022 at 6:27 AM Japin Li <japinli@hotmail.com> wrote:
>
> + servers. If setting <varname>max_wal_senders</varname> to
> + <literal>0</literal> consider also reducing the amount of WAL produced
> + by changing <varname>wal_level</varname> to <literal>minimal</literal>.
>
> I don't think this is great advice. It will encourage people to use
> wal_level = minimal even if they have other requirements that weigh
> against it. If they feel that their system is producing too much
> WAL, I doubt they'll have a hard time finding the wal_level knob.
>
Agreed. It isn't good advice. We can remove the suggestion.
Yeah, I wrote that thinking that max_wal_senders being set to 0 implied archive_mode = off, but that isn't the case.
David J.
On Fri, Jul 15, 2022 at 09:29:20PM +0800, Japin Li wrote: > > On Fri, 15 Jul 2022 at 08:49, Bruce Momjian <bruce@momjian.us> wrote: > > On Tue, Jul 5, 2022 at 08:02:33PM -0400, Tom Lane wrote: > >> "Precondition" is an overly fancy word that makes things less clear > >> not more so. Does it mean that setting wal_level = minimal will fail > >> if you don't do these other things, or does it just mean that you > >> won't be getting the absolute minimum WAL volume? If the former, > >> I think it'd be better to say something like "To set wal_level to minimal, > >> you must also set [these variables], which has the effect of disabling > >> both WAL archiving and streaming replication." > > > > I have created the attached patch to try to improve this text. > > IMO we can add the following sentence for wal_level description, since > if wal_level = minimal and max_wal_senders > 0, we cannot start the database. > > To set wal_level to minimal, you must also set max_wal_senders to 0, > which has the effect of disabling both WAL archiving and streaming > replication. Okay, text added in the attached patch. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
Attachment
On Tue, 19 Jul 2022 at 03:58, Bruce Momjian <bruce@momjian.us> wrote: > On Fri, Jul 15, 2022 at 09:29:20PM +0800, Japin Li wrote: >> >> On Fri, 15 Jul 2022 at 08:49, Bruce Momjian <bruce@momjian.us> wrote: >> > On Tue, Jul 5, 2022 at 08:02:33PM -0400, Tom Lane wrote: >> >> "Precondition" is an overly fancy word that makes things less clear >> >> not more so. Does it mean that setting wal_level = minimal will fail >> >> if you don't do these other things, or does it just mean that you >> >> won't be getting the absolute minimum WAL volume? If the former, >> >> I think it'd be better to say something like "To set wal_level to minimal, >> >> you must also set [these variables], which has the effect of disabling >> >> both WAL archiving and streaming replication." >> > >> > I have created the attached patch to try to improve this text. >> >> IMO we can add the following sentence for wal_level description, since >> if wal_level = minimal and max_wal_senders > 0, we cannot start the database. >> >> To set wal_level to minimal, you must also set max_wal_senders to 0, >> which has the effect of disabling both WAL archiving and streaming >> replication. > > Okay, text added in the attached patch. Thanks for updating the patch! LGTM. -- Regrads, Japin Li. ChengDu WenWu Information Technology Co.,Ltd.
Re: Doc about how to set max_wal_senders when setting minimal wal_level
From
"David G. Johnston"
Date:
On Mon, Jul 18, 2022 at 6:27 PM Japin Li <japinli@hotmail.com> wrote:
On Tue, 19 Jul 2022 at 03:58, Bruce Momjian <bruce@momjian.us> wrote:
> On Fri, Jul 15, 2022 at 09:29:20PM +0800, Japin Li wrote:
>>
>> On Fri, 15 Jul 2022 at 08:49, Bruce Momjian <bruce@momjian.us> wrote:
>> > On Tue, Jul 5, 2022 at 08:02:33PM -0400, Tom Lane wrote:
>> >> "Precondition" is an overly fancy word that makes things less clear
>> >> not more so. Does it mean that setting wal_level = minimal will fail
>> >> if you don't do these other things, or does it just mean that you
>> >> won't be getting the absolute minimum WAL volume? If the former,
>> >> I think it'd be better to say something like "To set wal_level to minimal,
>> >> you must also set [these variables], which has the effect of disabling
>> >> both WAL archiving and streaming replication."
>> >
>> > I have created the attached patch to try to improve this text.
>>
>> IMO we can add the following sentence for wal_level description, since
>> if wal_level = minimal and max_wal_senders > 0, we cannot start the database.
>>
>> To set wal_level to minimal, you must also set max_wal_senders to 0,
>> which has the effect of disabling both WAL archiving and streaming
>> replication.
>
> Okay, text added in the attached patch.
Thanks for updating the patch! LGTM.
+0.90
Consider changing:
"makes any base backups taken before this unusable"
to:
"makes existing base backups unusable"
As I try to justify this, though, it isn't quite true, maybe:
"makes point-in-time recovery, using existing base backups, unable to replay future WAL."
David J.
On Mon, Jul 18, 2022 at 07:39:55PM -0700, David G. Johnston wrote: > On Mon, Jul 18, 2022 at 6:27 PM Japin Li <japinli@hotmail.com> wrote: > > > +0.90 > > Consider changing: > > "makes any base backups taken before this unusable" > > to: > > "makes existing base backups unusable" > > As I try to justify this, though, it isn't quite true, maybe: > > "makes point-in-time recovery, using existing base backups, unable to replay > future WAL." I went with simpler wording. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
Attachment
Re: Doc about how to set max_wal_senders when setting minimal wal_level
From
"David G. Johnston"
Date:
On Mon, Jul 18, 2022 at 8:16 PM Bruce Momjian <bruce@momjian.us> wrote:
On Mon, Jul 18, 2022 at 07:39:55PM -0700, David G. Johnston wrote:
> On Mon, Jul 18, 2022 at 6:27 PM Japin Li <japinli@hotmail.com> wrote:
>
>
> +0.90
>
> Consider changing:
>
> "makes any base backups taken before this unusable"
>
> to:
>
> "makes existing base backups unusable"
>
> As I try to justify this, though, it isn't quite true, maybe:
>
> "makes point-in-time recovery, using existing base backups, unable to replay
> future WAL."
I went with simpler wording.
+1
Thanks!
David J.
On Mon, Jul 18, 2022 at 11:16:15PM -0400, Bruce Momjian wrote: > On Mon, Jul 18, 2022 at 07:39:55PM -0700, David G. Johnston wrote: > > On Mon, Jul 18, 2022 at 6:27 PM Japin Li <japinli@hotmail.com> wrote: > > > > > > +0.90 > > > > Consider changing: > > > > "makes any base backups taken before this unusable" > > > > to: > > > > "makes existing base backups unusable" > > > > As I try to justify this, though, it isn't quite true, maybe: > > > > "makes point-in-time recovery, using existing base backups, unable to replay > > future WAL." > > I went with simpler wording. Patch applied back to PG 14, and partial to PG 13. Docs before that were too different to be safe. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson