Re: recovery_connections cannot start (was Re: master in standby mode croaks) - Mailing list pgsql-hackers
| From | Heikki Linnakangas |
|---|---|
| Subject | Re: recovery_connections cannot start (was Re: master in standby mode croaks) |
| Date | |
| Msg-id | 4BD581A6.60602@enterprisedb.com Whole thread Raw |
| In response to | Re: recovery_connections cannot start (was Re: master in standby mode croaks) (Tom Lane <tgl@sss.pgh.pa.us>) |
| Responses |
Re: recovery_connections cannot start (was Re: master in
standby mode croaks)
(Robert Haas <robertmhaas@gmail.com>)
|
| List | pgsql-hackers |
Tom Lane wrote: > Personally I agree with your objection to "crash" but not with the > objection to "standby". Maybe this would be appropriate: > > wal_mode = minimal | archive | hot_standby Ok, here's a patch implementing this proposal. It adds a new wal_mode setting, leaving archive_mode as it is. If you try to enable archive_mode when wal_mode is 'minimal', you get a warning and archive_mode is silently ignored. Likewise streaming replication connections are not allowed if wal_mode is 'minimal'. recovery_connections now does nothing in the master. A bit more bikeshedding before I commit this: * Should an invalid combination throw an ERROR and refuse to start, instead of just warning? * How about naming the parameter wal_level instead of wal_mode? That would better convey that the higher levels add stuff on top of the lower levels, instead of having different modes that are somehow mutually exclusive. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com diff --git a/doc/src/sgml/backup.sgml b/doc/src/sgml/backup.sgml index eb5765a..6c6a504 100644 --- a/doc/src/sgml/backup.sgml +++ b/doc/src/sgml/backup.sgml @@ -689,8 +689,7 @@ archive_command = 'test ! -f /mnt/server/archivedir/%f && cp %p /mnt/ser </para> <para> - When <varname>archive_mode</> is <literal>off</> and <xref - linkend="guc-max-wal-senders"> is zero some SQL commands + When <varname>wal_mode</> is <literal>minimal</> some SQL commands are optimized to avoid WAL logging, as described in <xref linkend="populate-pitr">. If archiving or streaming replication were turned on during execution of one of these statements, WAL would not diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml index c5692ba..63ca749 100644 --- a/doc/src/sgml/config.sgml +++ b/doc/src/sgml/config.sgml @@ -1353,6 +1353,43 @@ SET ENABLE_SEQSCAN TO OFF; <title>Settings</title> <variablelist> + <varlistentry id="guc-wal-mode" xreflabel="wal_mode"> + <term><varname>wal_mode</varname> (<type>enum</type>)</term> + <indexterm> + <primary><varname>wal_mode</> configuration parameter</primary> + </indexterm> + <listitem> + <para> + <varname>wal_mode</> determines how much information is written + to the WAL. The default value is <literal>minimal</>, which writes + only minimal information needed to recover from a crash or immediate + shutdown. <literal>archive</> adds logging required for WAL archiving, + and <literal>hot_standby</> further adds extra information about + running transactions required to run read-only queries on a standby + server. + This parameter can only be set at server start. + </para> + <para> + In <literal>minimal</> mode, WAL-logging of some bulk operations, like + <command>CREATE INDEX</>, <command>CLUSTER</> and <command>COPY</> on + a table that was created or truncated in the same transaction can be + safely skipped, which can make those operations much faster, but + minimal WAL does not contain enough information to reconstruct the + data from a base backup and the WAL logs, so at least + <literal>archive</> level must be used to enable WAL archiving + (<xref linkend="guc-archive-mode">) and streaming replication. See + also <xref linkend="populate-pitr">. + </para> + <para> + In <literal>hot_standby</> mode, the same information is logged as + in <literal>archive</> mode, plus information needed to reconstruct + the status of running transactions from the WAL. To enable read-only + queries on a standby server, <varname>wal_mode</> must be set to + <literal>hot_standby</> on the primary. + </para> + </listitem> + </varlistentry> + <varlistentry id="guc-fsync" xreflabel="fsync"> <indexterm> <primary><varname>fsync</> configuration parameter</primary> @@ -1726,7 +1763,9 @@ SET ENABLE_SEQSCAN TO OFF; <varname>archive_mode</> and <varname>archive_command</> are separate variables so that <varname>archive_command</> can be changed without leaving archiving mode. - This parameter can only be set at server start. + This parameter can only be set at server start. It is ignored + unless <varname>wal_mode</> is set to <literal>archive</> or + <literal>hot_standby</>. </para> </listitem> </varlistentry> @@ -1884,16 +1923,14 @@ SET ENABLE_SEQSCAN TO OFF; </indexterm> <listitem> <para> - Parameter has two roles. During recovery, specifies whether or not - you can connect and run queries to enable <xref linkend="hot-standby">. - During normal running, specifies whether additional information is written - to WAL to allow recovery connections on a standby server that reads - WAL data generated by this server. The default value is + During recovery, specifies whether or not you can connect and run + queries to enable <xref linkend="hot-standby">. The default value is <literal>on</literal>. It is thought that there is little measurable difference in performance from using this feature, so feedback is welcome if any production impacts are noticeable. It is likely that this parameter will be removed in later releases. - This parameter can only be set at server start. + This parameter can only be set at server start. It is ignored when + not in standby mode. </para> </listitem> </varlistentry> diff --git a/doc/src/sgml/high-availability.sgml b/doc/src/sgml/high-availability.sgml index d69f2ea..7fa0817 100644 --- a/doc/src/sgml/high-availability.sgml +++ b/doc/src/sgml/high-availability.sgml @@ -1589,9 +1589,9 @@ LOG: database system is ready to accept read only connections </programlisting> Consistency information is recorded once per checkpoint on the primary, as long - as <varname>recovery_connections</> is enabled on the primary. It is not possible + as <varname>wal_mode</> is set to <literal>hot_standby</> on the primary. It is not possible to enable recovery connections on the standby when reading WAL written during the - period that <varname>recovery_connections</> was disabled on the primary. + period that <varname>wal_mode</> was not set to <literal>hot_standby</> on the primary. Reaching a consistent state can also be delayed in the presence of both of these conditions: @@ -1838,7 +1838,7 @@ LOG: database system is ready to accept read only connections </para> <para> - On the primary, parameters <varname>recovery_connections</> and + On the primary, parameters <varname>wal_mode</> and <varname>vacuum_defer_cleanup_age</> can be used. <varname>max_standby_delay</> has no effect if set on the primary. </para> diff --git a/doc/src/sgml/perform.sgml b/doc/src/sgml/perform.sgml index b00e69f..a493348 100644 --- a/doc/src/sgml/perform.sgml +++ b/doc/src/sgml/perform.sgml @@ -835,10 +835,9 @@ SELECT * FROM x, y, a, b, c WHERE something AND somethingelse; <command>TRUNCATE</command> command. In such cases no WAL needs to be written, because in case of an error, the files containing the newly loaded data will be removed anyway. - However, this consideration does not apply when - <xref linkend="guc-archive-mode"> is on or streaming replication - is allowed (i.e., <xref linkend="guc-max-wal-senders"> is more - than or equal to one), as all commands must write WAL in that case. + However, this consideration only applies when + <xref linkend="guc-wal-mode"> is <literal>minimal</> as all commands + must write WAL otherwise. </para> </sect2> @@ -910,18 +909,16 @@ SELECT * FROM x, y, a, b, c WHERE something AND somethingelse; </sect2> <sect2 id="populate-pitr"> - <title>Turn off <varname>archive_mode</varname> and streaming replication</title> + <title>Disable WAL archival and streaming replication</title> <para> When loading large amounts of data into an installation that uses WAL archiving or streaming replication, you might want to disable - archiving (turn off the <xref linkend="guc-archive-mode"> - configuration variable) and replication (zero the - <xref linkend="guc-max-wal-senders"> configuration variable) - while loading. It might be + archiving and replication by setting the <xref linkend="guc-wal-mode"> + configuration variable to <literal>minimal</> while loading. It might be faster to take a new base backup after the load has completed than to process a large amount of incremental WAL data. - But note that changing either of these variables requires + But note that changing <varname>wal_mode</> requires a server restart. </para> @@ -929,10 +926,9 @@ SELECT * FROM x, y, a, b, c WHERE something AND somethingelse; Aside from avoiding the time for the archiver or WAL sender to process the WAL data, doing this will actually make certain commands faster, because they - are designed not to write WAL at all if <varname>archive_mode</varname> - is off and <varname>max_wal_senders</varname> is zero. (They can - guarantee crash safety more cheaply by doing an - <function>fsync</> at the end than by writing WAL.) + are designed not to write WAL at all if <varname>wal_mode</varname> + is <literal>minimal</>. (They can guarantee crash safety more cheaply + by doing an <function>fsync</> at the end than by writing WAL.) This applies to the following commands: <itemizedlist> <listitem> @@ -1015,9 +1011,10 @@ SELECT * FROM x, y, a, b, c WHERE something AND somethingelse; <listitem> <para> If using WAL archiving, consider disabling it during the restore. - To do that, turn off <varname>archive_mode</varname> before loading the - dump script, and afterwards turn it back on - and take a fresh base backup. + To do that, set <varname>wal_mode</varname> to <literal>minimal</> + before loading the dump script, and afterwards set it back to + <literal>archive</> or <literal>hot_standby</> and take a fresh + base backup. </para> </listitem> <listitem> diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 7647f4e..0879b05 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -76,6 +76,7 @@ int MaxStandbyDelay = 30; bool fullPageWrites = true; bool log_checkpoints = false; int sync_method = DEFAULT_SYNC_METHOD; +int wal_mode = WAL_MODE_MINIMAL; #ifdef WAL_DEBUG bool XLOG_DEBUG = false; @@ -97,6 +98,13 @@ bool XLOG_DEBUG = false; /* * GUC support */ +const struct config_enum_entry wal_mode_options[] = { + {"minimal", WAL_MODE_MINIMAL, false}, + {"archive", WAL_MODE_ARCHIVE, false}, + {"hot_standby", WAL_MODE_HOT_STANDBY, false}, + {NULL, 0, false} +}; + const struct config_enum_entry sync_method_options[] = { {"fsync", SYNC_METHOD_FSYNC, false}, #ifdef HAVE_FSYNC_WRITETHROUGH diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c index 47f71bd..eeab0f3 100644 --- a/src/backend/postmaster/postmaster.c +++ b/src/backend/postmaster/postmaster.c @@ -728,6 +728,12 @@ PostmasterMain(int argc, char *argv[]) write_stderr("%s: superuser_reserved_connections must be less than max_connections\n", progname); ExitPostmaster(1); } + if (XLogArchiveMode && wal_mode == WAL_MODE_MINIMAL) + ereport(WARNING, + (errmsg("archive_mode ignored because wal_mode is 'minimal'"))); + if (max_wal_senders > 0 && wal_mode == WAL_MODE_MINIMAL) + ereport(WARNING, + (errmsg("WAL streaming connections not allowed because wal_mode is 'minimal'"))); /* * Other one-time internal sanity checks can go here, if they are fast. diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 3838665..35bc772 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -253,6 +253,24 @@ WalSndHandshake(void) { StringInfoData buf; + /* + * Check that we're logging enough information in the + * WAL for log-shipping. + * + * NOTE: This only checks the current value of + * wal_mode. Even if the current setting is not + * 'minimal', there can be old WAL in the pg_xlog + * directory that was created with 'minimal'. + * So this is not bulletproof, the purpose is + * just to give a user-friendly error message that + * hints how to configure the system correctly. + */ + if (wal_mode == WAL_MODE_MINIMAL) + ereport(FATAL, + (errcode(ERRCODE_CANNOT_CONNECT_NOW), + errmsg("standby connections not allowed because wal_mode='minimal'"))); + + /* Send a CopyOutResponse message, and start streaming */ pq_beginmessage(&buf, 'H'); pq_sendbyte(&buf, 0); diff --git a/src/backend/storage/ipc/standby.c b/src/backend/storage/ipc/standby.c index 51753d6..a92a874 100644 --- a/src/backend/storage/ipc/standby.c +++ b/src/backend/storage/ipc/standby.c @@ -256,7 +256,7 @@ ResolveRecoveryConflictWithSnapshot(TransactionId latestRemovedXid, RelFileNode */ if (!TransactionIdIsValid(latestRemovedXid)) { - elog(DEBUG1, "Invalid latestremovexXid reported, using latestcompletedxid instead"); + elog(DEBUG1, "invalid latestremovexXid reported, using latestcompletedxid instead"); LWLockAcquire(ProcArrayLock, LW_SHARED); latestRemovedXid = ShmemVariableCache->latestCompletedXid; diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c index 2434cc0..2fb4090 100644 --- a/src/backend/utils/misc/guc.c +++ b/src/backend/utils/misc/guc.c @@ -340,6 +340,7 @@ static const struct config_enum_entry constraint_exclusion_options[] = { /* * Options for enum values stored in other modules */ +extern const struct config_enum_entry wal_mode_options[]; extern const struct config_enum_entry sync_method_options[]; /* @@ -2785,6 +2786,15 @@ static struct config_enum ConfigureNamesEnum[] = }, { + {"wal_mode", PGC_POSTMASTER, WAL_SETTINGS, + gettext_noop("Set the level of information written to the WAL."), + NULL + }, + &wal_mode, + WAL_MODE_MINIMAL, wal_mode_options, NULL + }, + + { {"wal_sync_method", PGC_SIGHUP, WAL_SETTINGS, gettext_noop("Selects the method used for forcing WAL updates to disk."), NULL @@ -7862,7 +7872,7 @@ pg_timezone_abbrev_initialize(void) static const char * show_archive_command(void) { - if (XLogArchiveMode) + if (XLogArchivingActive()) return XLogArchiveCommand; else return "(disabled)"; diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index 92763eb..c9ee77c 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -150,6 +150,7 @@ # - Settings - +#wal_mode = minimal # minimal, archive, or hot_standby #fsync = on # turns forced synchronization on or off #synchronous_commit = on # immediate fsync at commit #wal_sync_method = fsync # the default is the first option diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h index 6bfc7d5..fa209a5 100644 --- a/src/include/access/xlog.h +++ b/src/include/access/xlog.h @@ -196,23 +196,23 @@ extern bool log_checkpoints; extern bool XLogRequestRecoveryConnections; extern int MaxStandbyDelay; -#define XLogArchivingActive() (XLogArchiveMode) -#define XLogArchiveCommandSet() (XLogArchiveCommand[0] != '\0') +/* WAL modes */ +#define WAL_MODE_MINIMAL 0 +#define WAL_MODE_ARCHIVE 1 +#define WAL_MODE_HOT_STANDBY 2 +extern int wal_mode; -/* - * This is in walsender.c, but declared here so that we don't need to include - * walsender.h in all files that check XLogIsNeeded() - */ -extern int max_wal_senders; +#define XLogArchivingActive() (XLogArchiveMode && wal_mode >= WAL_MODE_ARCHIVE) +#define XLogArchiveCommandSet() (XLogArchiveCommand[0] != '\0') /* - * Is WAL-logging necessary? We need to log an XLOG record iff either - * WAL archiving is enabled or XLOG streaming is allowed. + * Is WAL-logging necessary for archival or log-shipping, or can we skip + * WAL-logging if we fsync() the data before committing instead? */ -#define XLogIsNeeded() (XLogArchivingActive() || (max_wal_senders > 0)) +#define XLogIsNeeded() (wal_mode >= WAL_MODE_ARCHIVE) /* Do we need to WAL-log information required only for Hot Standby? */ -#define XLogStandbyInfoActive() (XLogRequestRecoveryConnections && XLogIsNeeded()) +#define XLogStandbyInfoActive() (wal_mode >= WAL_MODE_HOT_STANDBY) #ifdef WAL_DEBUG extern bool XLOG_DEBUG; diff --git a/src/include/replication/walsender.h b/src/include/replication/walsender.h index 6ad40a9..db64c88 100644 --- a/src/include/replication/walsender.h +++ b/src/include/replication/walsender.h @@ -39,6 +39,7 @@ extern bool am_walsender; /* user-settable parameters */ extern int WalSndDelay; +extern int max_wal_senders; extern int WalSenderMain(void); extern void WalSndSignals(void);
pgsql-hackers by date: