Re: Server won't start with fallback setting by initdb. - Mailing list pgsql-hackers

I wrote:
> Therefore, the condition that actually ought to be getting enforced here
> is either "ReservedBackends + max_wal_senders < MaxConnections", or
> "ReservedBackends + max_wal_senders <= MaxConnections", depending on
> whether you think it's appropriate to require at least one not-reserved-
> for-superusers connection slot to remain available if the walsenders
> slots are fully populated.

I propose the first attached patch to do that.  (I failed to resist the
temptation to copy-edit some nearby docs and comments, too.)

> My proposal is to default max_wal_senders to perhaps 3, and leave
> initdb's logic alone.

... and then the second attached patch to do that.

I noticed that a lot of our TAP tests are setting max_wal_senders
and max_replication_slots to random values around 4 or 5.  Probably
we could drop all that now, and let them just use the defaults.
I've not done that here, except for adjusting 010_pg_basebackup.pl
which would fail for no very good reason with minimal max_connections.

            regards, tom lane

diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 259a2d8..3a8fc7d 100644
*** a/doc/src/sgml/config.sgml
--- b/doc/src/sgml/config.sgml
*************** include_dir 'conf.d'
*** 696,703 ****

         <para>
          The default value is three connections. The value must be less
!         than the value of <varname>max_connections</varname>. This
!         parameter can only be set at server start.
         </para>
        </listitem>
       </varlistentry>
--- 696,704 ----

         <para>
          The default value is three connections. The value must be less
!         than <varname>max_connections</varname> minus
!         <xref linkend="guc-max-wal-senders"/>.
!         This parameter can only be set at server start.
         </para>
        </listitem>
       </varlistentry>
*************** include_dir 'conf.d'
*** 2982,2994 ****
          maximum number of simultaneously running WAL sender
          processes). The default is 10. The value 0 means replication is
          disabled. WAL sender processes count towards the total number
!         of connections, so the parameter cannot be set higher than
!         <xref linkend="guc-max-connections"/>.  Abrupt streaming client
!         disconnection might cause an orphaned connection slot until
          a timeout is reached, so this parameter should be set slightly
          higher than the maximum number of expected clients so disconnected
          clients can immediately reconnect.  This parameter can only
!         be set at server start. <varname>wal_level</varname> must be set to
          <literal>replica</literal> or higher to allow connections from standby
          servers.
         </para>
--- 2983,2998 ----
          maximum number of simultaneously running WAL sender
          processes). The default is 10. The value 0 means replication is
          disabled. WAL sender processes count towards the total number
!         of connections, so this parameter's value must be less than
!         <xref linkend="guc-max-connections"/> minus
!         <xref linkend="guc-superuser-reserved-connections"/>.
!         Abrupt streaming client disconnection might leave an orphaned
!         connection slot behind until
          a timeout is reached, so this parameter should be set slightly
          higher than the maximum number of expected clients so disconnected
          clients can immediately reconnect.  This parameter can only
!         be set at server start.
!         Also, <varname>wal_level</varname> must be set to
          <literal>replica</literal> or higher to allow connections from standby
          servers.
         </para>
*************** include_dir 'conf.d'
*** 3007,3016 ****
           (see <xref linkend="streaming-replication-slots"/>) that the server
           can support. The default is 10.  This parameter can only be set at
           server start.
!          <varname>wal_level</varname> must be set
!          to <literal>replica</literal> or higher to allow replication slots to
!          be used. Setting it to a lower value than the number of currently
           existing replication slots will prevent the server from starting.
          </para>
         </listitem>
        </varlistentry>
--- 3011,3021 ----
           (see <xref linkend="streaming-replication-slots"/>) that the server
           can support. The default is 10.  This parameter can only be set at
           server start.
!          Setting it to a lower value than the number of currently
           existing replication slots will prevent the server from starting.
+          Also, <varname>wal_level</varname> must be set
+          to <literal>replica</literal> or higher to allow replication slots to
+          be used.
          </para>
         </listitem>
        </varlistentry>
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index f3ddf82..660f318 100644
*** a/src/backend/postmaster/postmaster.c
--- b/src/backend/postmaster/postmaster.c
*************** char       *ListenAddresses;
*** 202,210 ****

  /*
   * ReservedBackends is the number of backends reserved for superuser use.
!  * This number is taken out of the pool size given by MaxBackends so
   * number of backend slots available to non-superusers is
!  * (MaxBackends - ReservedBackends).  Note what this really means is
   * "if there are <= ReservedBackends connections available, only superusers
   * can make new connections" --- pre-existing superuser connections don't
   * count against the limit.
--- 202,210 ----

  /*
   * ReservedBackends is the number of backends reserved for superuser use.
!  * This number is taken out of the pool size given by MaxConnections so
   * number of backend slots available to non-superusers is
!  * (MaxConnections - ReservedBackends).  Note what this really means is
   * "if there are <= ReservedBackends connections available, only superusers
   * can make new connections" --- pre-existing superuser connections don't
   * count against the limit.
*************** PostmasterMain(int argc, char *argv[])
*** 882,895 ****
      /*
       * Check for invalid combinations of GUC settings.
       */
!     if (ReservedBackends >= MaxConnections)
!     {
!         write_stderr("%s: superuser_reserved_connections must be less than max_connections\n", progname);
!         ExitPostmaster(1);
!     }
!     if (max_wal_senders >= MaxConnections)
      {
!         write_stderr("%s: max_wal_senders must be less than max_connections\n", progname);
          ExitPostmaster(1);
      }
      if (XLogArchiveMode > ARCHIVE_MODE_OFF && wal_level == WAL_LEVEL_MINIMAL)
--- 882,892 ----
      /*
       * Check for invalid combinations of GUC settings.
       */
!     if (ReservedBackends + max_wal_senders >= MaxConnections)
      {
!         write_stderr("%s: superuser_reserved_connections (%d) plus max_wal_senders (%d) must be less than
max_connections(%d)\n", 
!                      progname,
!                      ReservedBackends, max_wal_senders, MaxConnections);
          ExitPostmaster(1);
      }
      if (XLogArchiveMode > ARCHIVE_MODE_OFF && wal_level == WAL_LEVEL_MINIMAL)
diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c
index 4846289..d8f45b3 100644
*** a/src/backend/utils/init/postinit.c
--- b/src/backend/utils/init/postinit.c
*************** InitPostgres(const char *in_dbname, Oid
*** 778,784 ****
      }

      /*
!      * The last few connections slots are reserved for superusers. Although
       * replication connections currently require superuser privileges, we
       * don't allow them to consume the reserved slots, which are intended for
       * interactive use.
--- 778,784 ----
      }

      /*
!      * The last few connection slots are reserved for superusers.  Although
       * replication connections currently require superuser privileges, we
       * don't allow them to consume the reserved slots, which are intended for
       * interactive use.
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 1db7845..d91ba05 100644
*** a/src/backend/utils/misc/guc.c
--- b/src/backend/utils/misc/guc.c
*************** static struct config_int ConfigureNamesI
*** 1873,1878 ****
--- 1873,1879 ----
      },

      {
+         /* see max_connections and max_wal_senders */
          {"superuser_reserved_connections", PGC_POSTMASTER, CONN_AUTH_SETTINGS,
              gettext_noop("Sets the number of connection slots reserved for superusers."),
              NULL
*************** static struct config_int ConfigureNamesI
*** 2375,2381 ****
      },

      {
!         /* see max_connections */
          {"max_wal_senders", PGC_POSTMASTER, REPLICATION_SENDING,
              gettext_noop("Sets the maximum number of simultaneously running WAL sender processes."),
              NULL
--- 2376,2382 ----
      },

      {
!         /* see max_connections and superuser_reserved_connections */
          {"max_wal_senders", PGC_POSTMASTER, REPLICATION_SENDING,
              gettext_noop("Sets the maximum number of simultaneously running WAL sender processes."),
              NULL
*************** static struct config_int ConfigureNamesI
*** 2386,2392 ****
      },

      {
!         /* see max_connections */
          {"max_replication_slots", PGC_POSTMASTER, REPLICATION_SENDING,
              gettext_noop("Sets the maximum number of simultaneously defined replication slots."),
              NULL
--- 2387,2393 ----
      },

      {
!         /* see max_wal_senders */
          {"max_replication_slots", PGC_POSTMASTER, REPLICATION_SENDING,
              gettext_noop("Sets the maximum number of simultaneously defined replication slots."),
              NULL
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 3a8fc7d..db43caf 100644
*** a/doc/src/sgml/config.sgml
--- b/doc/src/sgml/config.sgml
*************** include_dir 'conf.d'
*** 2981,2987 ****
          Specifies the maximum number of concurrent connections from
          standby servers or streaming base backup clients (i.e., the
          maximum number of simultaneously running WAL sender
!         processes). The default is 10. The value 0 means replication is
          disabled. WAL sender processes count towards the total number
          of connections, so this parameter's value must be less than
          <xref linkend="guc-max-connections"/> minus
--- 2981,2987 ----
          Specifies the maximum number of concurrent connections from
          standby servers or streaming base backup clients (i.e., the
          maximum number of simultaneously running WAL sender
!         processes). The default is 3. The value 0 means replication is
          disabled. WAL sender processes count towards the total number
          of connections, so this parameter's value must be less than
          <xref linkend="guc-max-connections"/> minus
*************** include_dir 'conf.d'
*** 3009,3015 ****
          <para>
           Specifies the maximum number of replication slots
           (see <xref linkend="streaming-replication-slots"/>) that the server
!          can support. The default is 10.  This parameter can only be set at
           server start.
           Setting it to a lower value than the number of currently
           existing replication slots will prevent the server from starting.
--- 3009,3015 ----
          <para>
           Specifies the maximum number of replication slots
           (see <xref linkend="streaming-replication-slots"/>) that the server
!          can support. The default is 3.  This parameter can only be set at
           server start.
           Setting it to a lower value than the number of currently
           existing replication slots will prevent the server from starting.
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index d91ba05..85507d5 100644
*** a/src/backend/utils/misc/guc.c
--- b/src/backend/utils/misc/guc.c
*************** static struct config_int ConfigureNamesI
*** 2382,2388 ****
              NULL
          },
          &max_wal_senders,
!         10, 0, MAX_BACKENDS,
          NULL, NULL, NULL
      },

--- 2382,2388 ----
              NULL
          },
          &max_wal_senders,
!         3, 0, MAX_BACKENDS,
          NULL, NULL, NULL
      },

*************** static struct config_int ConfigureNamesI
*** 2393,2399 ****
              NULL
          },
          &max_replication_slots,
!         10, 0, MAX_BACKENDS /* XXX? */ ,
          NULL, NULL, NULL
      },

--- 2393,2399 ----
              NULL
          },
          &max_replication_slots,
!         3, 0, MAX_BACKENDS,
          NULL, NULL, NULL
      },

diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 3927292..15f257b 100644
*** a/src/backend/utils/misc/postgresql.conf.sample
--- b/src/backend/utils/misc/postgresql.conf.sample
***************
*** 234,245 ****

  # Set these on the master and on any standby that will send replication data.

! #max_wal_senders = 10        # max number of walsender processes
                  # (change requires restart)
  #wal_keep_segments = 0        # in logfile segments; 0 disables
  #wal_sender_timeout = 60s    # in milliseconds; 0 disables

! #max_replication_slots = 10    # max number of replication slots
                  # (change requires restart)
  #track_commit_timestamp = off    # collect timestamp of transaction commit
                  # (change requires restart)
--- 234,245 ----

  # Set these on the master and on any standby that will send replication data.

! #max_wal_senders = 3        # max number of walsender processes
                  # (change requires restart)
  #wal_keep_segments = 0        # in logfile segments; 0 disables
  #wal_sender_timeout = 60s    # in milliseconds; 0 disables

! #max_replication_slots = 3    # max number of replication slots
                  # (change requires restart)
  #track_commit_timestamp = off    # collect timestamp of transaction commit
                  # (change requires restart)
diff --git a/src/bin/pg_basebackup/t/010_pg_basebackup.pl b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
index cdf4f5b..29cd928 100644
*** a/src/bin/pg_basebackup/t/010_pg_basebackup.pl
--- b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
*************** $node->command_fails([ 'pg_basebackup',
*** 45,52 ****
  ok(-d "$tempdir/backup", 'backup directory was created and left behind');

  open my $conf, '>>', "$pgdata/postgresql.conf";
! print $conf "max_replication_slots = 10\n";
! print $conf "max_wal_senders = 10\n";
  print $conf "wal_level = replica\n";
  close $conf;
  $node->restart;
--- 45,52 ----
  ok(-d "$tempdir/backup", 'backup directory was created and left behind');

  open my $conf, '>>', "$pgdata/postgresql.conf";
! print $conf "max_replication_slots = 5\n";
! print $conf "max_wal_senders = 5\n";
  print $conf "wal_level = replica\n";
  close $conf;
  $node->restart;