Thread: Doc about how to set max_wal_senders when setting minimal wal_level

Doc about how to set max_wal_senders when setting minimal wal_level

From
Japin Li
Date:
Hi, hackers

When I try to change wal_level to minimal and restart the database, it complains
max_wal_senders > 0.

2022-03-03 10:10:16.938 CST [6389] FATAL:  WAL streaming (max_wal_senders > 0) requires wal_level "replica" or
"logical"

However, the documentation about wal_level [1] doesn't mentation this.
How about adding a sentence to describe how to set max_wal_senders when
setting wal_level to minimal?

[1] https://www.postgresql.org/docs/devel/runtime-config-wal.html

-- 
Regrads,
Japin Li.
ChengDu WenWu Information Technology Co.,Ltd.


Attachment

Re: Doc about how to set max_wal_senders when setting minimal wal_level

From
"David G. Johnston"
Date:
On Wed, Mar 2, 2022 at 7:44 PM Japin Li <japinli@hotmail.com> wrote:

Hi, hackers

When I try to change wal_level to minimal and restart the database, it complains
max_wal_senders > 0.

2022-03-03 10:10:16.938 CST [6389] FATAL:  WAL streaming (max_wal_senders > 0) requires wal_level "replica" or "logical"

However, the documentation about wal_level [1] doesn't mentation this.
How about adding a sentence to describe how to set max_wal_senders when
setting wal_level to minimal?

[1] https://www.postgresql.org/docs/devel/runtime-config-wal.html


I would suggest a wording more like:

"A precondition for using minimal WAL is to disable WAL archiving and streaming replication by setting max_wal_senders to 0, and archive_mode to off."

While accurate, the phrase "you must set" just doesn't feel right to me.  I also don't like how the proposed sentence (either one) is added separately as opposed to being included in the immediately preceding paragraph.  While this limited patch is probably sufficient I would suggest trying to work out a slightly larger patch the improves the wording on the entire existing paragraph while incorporating the reference to max_wal_senders.

Note, we seem to be missing the documentation of the default setting for archive_mode.

In addition, max_wal_senders could also be changed, adding a sentence like:

"If setting max_wal_senders to 0 consider also reducing the amount of WAL produced by changing wal_level to minimal."

At least insofar as core is concerned without a wal sender the additional wal of replica is not actually able to be leveraged as pg_basebackup will not work (at noted in its own description).

David J.

On Thu, 03 Mar 2022 at 11:25, David G. Johnston <david.g.johnston@gmail.com> wrote:
> I would suggest a wording more like:
>
> "A precondition for using minimal WAL is to disable WAL archiving and
> streaming replication by setting max_wal_senders to 0, and archive_mode to
> off."
>
> While accurate, the phrase "you must set" just doesn't feel right to me.  I
> also don't like how the proposed sentence (either one) is added separately
> as opposed to being included in the immediately preceding paragraph.  While
> this limited patch is probably sufficient I would suggest trying to work
> out a slightly larger patch the improves the wording on the entire existing
> paragraph while incorporating the reference to max_wal_senders.
>

Thanks for your review.  Modified as your suggestion.

> Note, we seem to be missing the documentation of the default setting for
> archive_mode.
>

Add the default value for archive_mode.

> In addition, max_wal_senders could also be changed, adding a sentence like:
>
> "If setting max_wal_senders to 0 consider also reducing the amount of WAL
> produced by changing wal_level to minimal."
>

Modified.

-- 
Regrads,
Japin Li.
ChengDu WenWu Information Technology Co.,Ltd.


Attachment
On Thu, 03 Mar 2022 at 12:10, Japin Li <japinli@hotmail.com> wrote:
> On Thu, 03 Mar 2022 at 11:25, David G. Johnston <david.g.johnston@gmail.com> wrote:
>> I would suggest a wording more like:
>>
>> "A precondition for using minimal WAL is to disable WAL archiving and
>> streaming replication by setting max_wal_senders to 0, and archive_mode to
>> off."
>>
>> While accurate, the phrase "you must set" just doesn't feel right to me.  I
>> also don't like how the proposed sentence (either one) is added separately
>> as opposed to being included in the immediately preceding paragraph.  While
>> this limited patch is probably sufficient I would suggest trying to work
>> out a slightly larger patch the improves the wording on the entire existing
>> paragraph while incorporating the reference to max_wal_senders.
>>
>
> Thanks for your review.  Modified as your suggestion.
>
>> Note, we seem to be missing the documentation of the default setting for
>> archive_mode.
>>
>
> Add the default value for archive_mode.
>
>> In addition, max_wal_senders could also be changed, adding a sentence like:
>>
>> "If setting max_wal_senders to 0 consider also reducing the amount of WAL
>> produced by changing wal_level to minimal."
>>
>
> Modified.

Attach v3 patch to fix missing close varname tag.

-- 
Regrads,
Japin Li.
ChengDu WenWu Information Technology Co.,Ltd.


Attachment

Re: Doc about how to set max_wal_senders when setting minimal wal_level

From
Kyotaro Horiguchi
Date:
At Fri, 04 Mar 2022 12:18:29 +0800, Japin Li <japinli@hotmail.com> wrote in 
> 
> On Thu, 03 Mar 2022 at 12:10, Japin Li <japinli@hotmail.com> wrote:
>
> Attach v3 patch to fix missing close varname tag.

+        A precondition for using minimal WAL is to disable WAL archiving and
+        streaming replication by setting <xref linkend="guc-max-wal-senders"/>
+        to <literal>0</literal>, and <varname>archive_mode</varname>
+        to <literal>off</literal>.

It is a bit odd that the features to stop and the corresponding GUCs
are written irrespectively. It would be better they're in the same
order.

+        servers.  If setting <varname>max_wal_senders</varname> to
+        <literal>0</literal> consider also reducing the amount of WAL produced
+        by changing <varname>wal_level</varname> to <literal>minimal</literal>.

Those who anively follow this suggestion may bump into failure when
arhive_mode is on.  Thus archive_mode is also worth referred to.  But,
anyway, IMHO, it is mere a performance tips that is not necessarily
required in this section, or even in this documentaiotn.  Addtion to
that, if we write this for max_wal_senders, archive_mode will deserve
the similar tips but I think it is too verbose.  In short, I think I
would not add that description at all.

regards.

-- 
Kyotaro Horiguchi
NTT Open Source Software Center



On Fri, 04 Mar 2022 at 14:05, Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote:
> At Fri, 04 Mar 2022 12:18:29 +0800, Japin Li <japinli@hotmail.com> wrote in 
>> 
>> On Thu, 03 Mar 2022 at 12:10, Japin Li <japinli@hotmail.com> wrote:
>>
>> Attach v3 patch to fix missing close varname tag.
>
> +        A precondition for using minimal WAL is to disable WAL archiving and
> +        streaming replication by setting <xref linkend="guc-max-wal-senders"/>
> +        to <literal>0</literal>, and <varname>archive_mode</varname>
> +        to <literal>off</literal>.
>
> It is a bit odd that the features to stop and the corresponding GUCs
> are written irrespectively. It would be better they're in the same
> order.
>

Thanks for your review.  Modified.

> +        servers.  If setting <varname>max_wal_senders</varname> to
> +        <literal>0</literal> consider also reducing the amount of WAL produced
> +        by changing <varname>wal_level</varname> to <literal>minimal</literal>.
>
> Those who anively follow this suggestion may bump into failure when
> arhive_mode is on.  Thus archive_mode is also worth referred to.  But,
> anyway, IMHO, it is mere a performance tips that is not necessarily
> required in this section, or even in this documentaiotn.  Addtion to
> that, if we write this for max_wal_senders, archive_mode will deserve
> the similar tips but I think it is too verbose.  In short, I think I
> would not add that description at all.
>

It already has a tip about wal_level for archive_mode [1], IIUC.

    archive_mode cannot be enabled when wal_level is set to minimal.


[1] https://www.postgresql.org/docs/devel/runtime-config-wal.html#GUC-ARCHIVE-MODE

-- 
Regrads,
Japin Li.
ChengDu WenWu Information Technology Co.,Ltd.


Attachment

Re: Doc about how to set max_wal_senders when setting minimal wal_level

From
"David G. Johnston"
Date:
On Thu, Mar 3, 2022 at 11:05 PM Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote:
But,
anyway, IMHO, it is mere a performance tips that is not necessarily
required in this section, or even in this documentaiotn.  Addtion to
that, if we write this for max_wal_senders, archive_mode will deserve
the similar tips but I think it is too verbose.  In short, I think I
would not add that description at all.


I wrote it as a performance tip but it is documenting that when set to 0 no features of the server require more information than is captured in the minimal wal.  That fact seems worthy of noting.  Even at the cost of a bit of verbosity.  These features interact with each other and that interaction should be adequately described.  While subjective, this dynamic seems to warrant inclusion.

David J.

Re: Doc about how to set max_wal_senders when setting minimal wal_level

From
"David G. Johnston"
Date:
On Fri, Mar 4, 2022 at 2:49 AM Japin Li <japinli@hotmail.com> wrote:
Thanks for your review.  Modified.

Works for me.  I have some additional sparks of ideas but nothing that need hold this up.

David J.

Japin Li <japinli@hotmail.com> writes:
> [ v4-wal-level-documentation.patch ]

Hm, I don't care for the wording here:

+        A precondition for using minimal WAL is to disable WAL archiving and
+        streaming replication by setting <varname>archive_mode</varname> to
+        <literal>off</literal>, and <xref linkend="guc-max-wal-senders"/> to
+        <literal>0</literal>.

"Precondition" is an overly fancy word that makes things less clear
not more so.  Does it mean that setting wal_level = minimal will fail
if you don't do these other things, or does it just mean that you
won't be getting the absolute minimum WAL volume?  If the former,
I think it'd be better to say something like "To set wal_level to minimal,
you must also set [these variables], which has the effect of disabling
both WAL archiving and streaming replication."

+        servers.  If setting <varname>max_wal_senders</varname> to
+        <literal>0</literal> consider also reducing the amount of WAL produced
+        by changing <varname>wal_level</varname> to <literal>minimal</literal>.

I don't think this is great advice.  It will encourage people to use
wal_level = minimal even if they have other requirements that weigh
against it.  If they feel that their system is producing too much
WAL, I doubt they'll have a hard time finding the wal_level knob.

TBH, I think the real problem with the docs in this area is that
the first para about wal_level is disjointed and unclear; we ought
to nuke and rewrite that.  In particular it fails to provide adequate
context about what "logical decoding" means.  I think possibly what
it needs to say is that replica mode supports *physical* replication
but if you want to use *logical* replication you need logical mode;
if you need neither, and are not doing WAL archiving either, you
can get away with minimal mode.

            regards, tom lane



Re: Doc about how to set max_wal_senders when setting minimal wal_level

From
Bruce Momjian
Date:
On Tue, Jul  5, 2022 at 08:02:33PM -0400, Tom Lane wrote:
> "Precondition" is an overly fancy word that makes things less clear
> not more so.  Does it mean that setting wal_level = minimal will fail
> if you don't do these other things, or does it just mean that you
> won't be getting the absolute minimum WAL volume?  If the former,
> I think it'd be better to say something like "To set wal_level to minimal,
> you must also set [these variables], which has the effect of disabling
> both WAL archiving and streaming replication."

I have created the attached patch to try to improve this text.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Indecision is a decision.  Inaction is an action.  Mark Batterson


Attachment
Sorry for the late reply.

On Wed, 06 Jul 2022 at 08:02, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Japin Li <japinli@hotmail.com> writes:
>> [ v4-wal-level-documentation.patch ]
>
> Hm, I don't care for the wording here:
>
> +        A precondition for using minimal WAL is to disable WAL archiving and
> +        streaming replication by setting <varname>archive_mode</varname> to
> +        <literal>off</literal>, and <xref linkend="guc-max-wal-senders"/> to
> +        <literal>0</literal>.
>
> "Precondition" is an overly fancy word that makes things less clear
> not more so.  Does it mean that setting wal_level = minimal will fail
> if you don't do these other things, or does it just mean that you
> won't be getting the absolute minimum WAL volume?  If the former,
> I think it'd be better to say something like "To set wal_level to minimal,
> you must also set [these variables], which has the effect of disabling
> both WAL archiving and streaming replication."

Yeah, it's the former case.

>
> +        servers.  If setting <varname>max_wal_senders</varname> to
> +        <literal>0</literal> consider also reducing the amount of WAL produced
> +        by changing <varname>wal_level</varname> to <literal>minimal</literal>.
>
> I don't think this is great advice.  It will encourage people to use
> wal_level = minimal even if they have other requirements that weigh
> against it.  If they feel that their system is producing too much
> WAL, I doubt they'll have a hard time finding the wal_level knob.
>

Agreed. It isn't good advice.  We can remove the suggestion.


--
Regrads,
Japin Li.
ChengDu WenWu Information Technology Co.,Ltd.



On Fri, 15 Jul 2022 at 08:49, Bruce Momjian <bruce@momjian.us> wrote:
> On Tue, Jul  5, 2022 at 08:02:33PM -0400, Tom Lane wrote:
>> "Precondition" is an overly fancy word that makes things less clear
>> not more so.  Does it mean that setting wal_level = minimal will fail
>> if you don't do these other things, or does it just mean that you
>> won't be getting the absolute minimum WAL volume?  If the former,
>> I think it'd be better to say something like "To set wal_level to minimal,
>> you must also set [these variables], which has the effect of disabling
>> both WAL archiving and streaming replication."
>
> I have created the attached patch to try to improve this text.

IMO we can add the following sentence for wal_level description, since
if wal_level = minimal and max_wal_senders > 0, we cannot start the database.

To set wal_level to minimal, you must also set max_wal_senders to 0,
which has the effect of disabling both WAL archiving and streaming
replication.

--
Regrads,
Japin Li.
ChengDu WenWu Information Technology Co.,Ltd.



Re: Doc about how to set max_wal_senders when setting minimal wal_level

From
"David G. Johnston"
Date:
On Fri, Jul 15, 2022 at 6:27 AM Japin Li <japinli@hotmail.com> wrote:

>
> +        servers.  If setting <varname>max_wal_senders</varname> to
> +        <literal>0</literal> consider also reducing the amount of WAL produced
> +        by changing <varname>wal_level</varname> to <literal>minimal</literal>.
>
> I don't think this is great advice.  It will encourage people to use
> wal_level = minimal even if they have other requirements that weigh
> against it.  If they feel that their system is producing too much
> WAL, I doubt they'll have a hard time finding the wal_level knob.
>

Agreed. It isn't good advice.  We can remove the suggestion.


Yeah, I wrote that thinking that max_wal_senders being set to 0 implied archive_mode = off, but that isn't the case.

David J.

Re: Doc about how to set max_wal_senders when setting minimal wal_level

From
Bruce Momjian
Date:
On Fri, Jul 15, 2022 at 09:29:20PM +0800, Japin Li wrote:
> 
> On Fri, 15 Jul 2022 at 08:49, Bruce Momjian <bruce@momjian.us> wrote:
> > On Tue, Jul  5, 2022 at 08:02:33PM -0400, Tom Lane wrote:
> >> "Precondition" is an overly fancy word that makes things less clear
> >> not more so.  Does it mean that setting wal_level = minimal will fail
> >> if you don't do these other things, or does it just mean that you
> >> won't be getting the absolute minimum WAL volume?  If the former,
> >> I think it'd be better to say something like "To set wal_level to minimal,
> >> you must also set [these variables], which has the effect of disabling
> >> both WAL archiving and streaming replication."
> >
> > I have created the attached patch to try to improve this text.
> 
> IMO we can add the following sentence for wal_level description, since
> if wal_level = minimal and max_wal_senders > 0, we cannot start the database.
> 
> To set wal_level to minimal, you must also set max_wal_senders to 0,
> which has the effect of disabling both WAL archiving and streaming
> replication.

Okay, text added in the attached patch.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Indecision is a decision.  Inaction is an action.  Mark Batterson


Attachment
On Tue, 19 Jul 2022 at 03:58, Bruce Momjian <bruce@momjian.us> wrote:
> On Fri, Jul 15, 2022 at 09:29:20PM +0800, Japin Li wrote:
>> 
>> On Fri, 15 Jul 2022 at 08:49, Bruce Momjian <bruce@momjian.us> wrote:
>> > On Tue, Jul  5, 2022 at 08:02:33PM -0400, Tom Lane wrote:
>> >> "Precondition" is an overly fancy word that makes things less clear
>> >> not more so.  Does it mean that setting wal_level = minimal will fail
>> >> if you don't do these other things, or does it just mean that you
>> >> won't be getting the absolute minimum WAL volume?  If the former,
>> >> I think it'd be better to say something like "To set wal_level to minimal,
>> >> you must also set [these variables], which has the effect of disabling
>> >> both WAL archiving and streaming replication."
>> >
>> > I have created the attached patch to try to improve this text.
>> 
>> IMO we can add the following sentence for wal_level description, since
>> if wal_level = minimal and max_wal_senders > 0, we cannot start the database.
>> 
>> To set wal_level to minimal, you must also set max_wal_senders to 0,
>> which has the effect of disabling both WAL archiving and streaming
>> replication.
>
> Okay, text added in the attached patch.

Thanks for updating the patch! LGTM.

-- 
Regrads,
Japin Li.
ChengDu WenWu Information Technology Co.,Ltd.



Re: Doc about how to set max_wal_senders when setting minimal wal_level

From
"David G. Johnston"
Date:
On Mon, Jul 18, 2022 at 6:27 PM Japin Li <japinli@hotmail.com> wrote:

On Tue, 19 Jul 2022 at 03:58, Bruce Momjian <bruce@momjian.us> wrote:
> On Fri, Jul 15, 2022 at 09:29:20PM +0800, Japin Li wrote:
>>
>> On Fri, 15 Jul 2022 at 08:49, Bruce Momjian <bruce@momjian.us> wrote:
>> > On Tue, Jul  5, 2022 at 08:02:33PM -0400, Tom Lane wrote:
>> >> "Precondition" is an overly fancy word that makes things less clear
>> >> not more so.  Does it mean that setting wal_level = minimal will fail
>> >> if you don't do these other things, or does it just mean that you
>> >> won't be getting the absolute minimum WAL volume?  If the former,
>> >> I think it'd be better to say something like "To set wal_level to minimal,
>> >> you must also set [these variables], which has the effect of disabling
>> >> both WAL archiving and streaming replication."
>> >
>> > I have created the attached patch to try to improve this text.
>>
>> IMO we can add the following sentence for wal_level description, since
>> if wal_level = minimal and max_wal_senders > 0, we cannot start the database.
>>
>> To set wal_level to minimal, you must also set max_wal_senders to 0,
>> which has the effect of disabling both WAL archiving and streaming
>> replication.
>
> Okay, text added in the attached patch.

Thanks for updating the patch! LGTM.


+0.90

Consider changing:

"makes any base backups taken before this unusable"

to:

"makes existing base backups unusable"

As I try to justify this, though, it isn't quite true, maybe:

"makes point-in-time recovery, using existing base backups, unable to replay future WAL."

David J.

Re: Doc about how to set max_wal_senders when setting minimal wal_level

From
Bruce Momjian
Date:
On Mon, Jul 18, 2022 at 07:39:55PM -0700, David G. Johnston wrote:
> On Mon, Jul 18, 2022 at 6:27 PM Japin Li <japinli@hotmail.com> wrote:
> 
> 
> +0.90
> 
> Consider changing:
> 
> "makes any base backups taken before this unusable"
> 
> to:
> 
> "makes existing base backups unusable"
> 
> As I try to justify this, though, it isn't quite true, maybe:
> 
> "makes point-in-time recovery, using existing base backups, unable to replay
> future WAL."

I went with simpler wording.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Indecision is a decision.  Inaction is an action.  Mark Batterson


Attachment

Re: Doc about how to set max_wal_senders when setting minimal wal_level

From
"David G. Johnston"
Date:
On Mon, Jul 18, 2022 at 8:16 PM Bruce Momjian <bruce@momjian.us> wrote:
On Mon, Jul 18, 2022 at 07:39:55PM -0700, David G. Johnston wrote:
> On Mon, Jul 18, 2022 at 6:27 PM Japin Li <japinli@hotmail.com> wrote:
>
>
> +0.90
>
> Consider changing:
>
> "makes any base backups taken before this unusable"
>
> to:
>
> "makes existing base backups unusable"
>
> As I try to justify this, though, it isn't quite true, maybe:
>
> "makes point-in-time recovery, using existing base backups, unable to replay
> future WAL."

I went with simpler wording.


+1

Thanks!

David J.

Re: Doc about how to set max_wal_senders when setting minimal wal_level

From
Bruce Momjian
Date:
On Mon, Jul 18, 2022 at 11:16:15PM -0400, Bruce Momjian wrote:
> On Mon, Jul 18, 2022 at 07:39:55PM -0700, David G. Johnston wrote:
> > On Mon, Jul 18, 2022 at 6:27 PM Japin Li <japinli@hotmail.com> wrote:
> > 
> > 
> > +0.90
> > 
> > Consider changing:
> > 
> > "makes any base backups taken before this unusable"
> > 
> > to:
> > 
> > "makes existing base backups unusable"
> > 
> > As I try to justify this, though, it isn't quite true, maybe:
> > 
> > "makes point-in-time recovery, using existing base backups, unable to replay
> > future WAL."
> 
> I went with simpler wording.

Patch applied back to PG 14, and partial to PG 13.  Docs before that
were too different to be safe.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Indecision is a decision.  Inaction is an action.  Mark Batterson