Thread: pg_archive_bypass
Hi,
I tend to consider it a bug that there's no known way under windows to
use the same trick as under Unix by using '/usr/bin/true' as your
archive command. And this Unix trick itself does feel like a hack.
Also I'd very much like to be able to recommend (even if not change the
official defaults) to setup wal_level to archive, archive_mode=on and
archive_command=pg_archive_bypass, so that the day you have a HA budget
ain't the day you're going to restart the server to enable the fault
tolerance settings…
So please find attached a very simple "let's see about it" patch to
implement an internal archive_command that just returns true and is
called pg_archive_bypass. It's missing documentation, which I'll provide
if needed (meaning there's some will to consider applying such a patch).
Regards,
--
dim
*** a/src/backend/postmaster/pgarch.c
--- b/src/backend/postmaster/pgarch.c
***************
*** 475,481 **** pgarch_ArchiverCopyLoop(void)
/*
* pgarch_archiveXlog
*
! * Invokes system(3) to copy one archive file to wherever it should go
*
* Returns true if successful
*/
--- 475,485 ----
/*
* pgarch_archiveXlog
*
! * Invokes system(3) to copy one archive file to wherever it should go, or
! * just return true if set to the internal command "pg_archive_bypass". That
! * allows windows users to easily have the "/usr/bin/true", and that allows
! * for setting up a archiving only when you need it by changing the command
! * and issuing a reload.
*
* Returns true if successful
*/
***************
*** 490,495 **** pgarch_archiveXlog(char *xlog)
--- 494,509 ----
const char *sp;
int rc;
+ /*
+ * If the command is "pg_archive_bypass", just return true
+ */
+ if( strcmp(XLogArchiveCommand, "pg_archive_bypass") == 0 )
+ {
+ ereport(DEBUG3,
+ (errmsg_internal("bypassing archive command")));
+ return true;
+ }
+
snprintf(pathname, MAXPGPATH, XLOGDIR "/%s", xlog);
/*
On Mon, 2010-06-14 at 12:39 +0200, Dimitri Fontaine wrote: > I tend to consider it a bug that there's no known way under windows to > use the same trick as under Unix by using '/usr/bin/true' as your > archive command. And this Unix trick itself does feel like a hack. > > Also I'd very much like to be able to recommend (even if not change the > official defaults) to setup wal_level to archive, archive_mode=on and > archive_command=pg_archive_bypass, so that the day you have a HA budget > ain't the day you're going to restart the server to enable the fault > tolerance settings… > > So please find attached a very simple "let's see about it" patch to > implement an internal archive_command that just returns true and is > called pg_archive_bypass. It's missing documentation, which I'll provide > if needed (meaning there's some will to consider applying such a patch). ISTM like a good idea to have a couple of more obvious commands provided purely as "internal commands". If we do this on restore_command as well, we can skip pg_archivecleanup completely, for example. So, not for 9.0. But I like concept for 9.1. Would need to be coded as some kind of escape phrase, followed by other command. So not just a quick hack with this one exception. Something like archive_command = 'pg_archive_internal:true' Wouldn't rush to another patch though, needs agreement first. -- Simon Riggs www.2ndQuadrant.com
On 14/06/10 13:39, Dimitri Fontaine wrote: > I tend to consider it a bug that there's no known way under windows to > use the same trick as under Unix by using '/usr/bin/true' as your > archive command. And this Unix trick itself does feel like a hack. > > Also I'd very much like to be able to recommend (even if not change the > official defaults) to setup wal_level to archive, archive_mode=on and > archive_command=pg_archive_bypass, so that the day you have a HA budget > ain't the day you're going to restart the server to enable the fault > tolerance settings… That particular use case is better handled by making archive_mode changeable on-the-fly. Now that we have wal_level as a separate GUC, it shouldn't be too hard. I like internal commands as a solution in general, but it's still in search of a problem.. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com
Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> writes: > On 14/06/10 13:39, Dimitri Fontaine wrote: >> I tend to consider it a bug that there's no known way under windows to >> use the same trick as under Unix by using '/usr/bin/true' as your >> archive command. And this Unix trick itself does feel like a hack. >> >> Also I'd very much like to be able to recommend (even if not change the >> official defaults) to setup wal_level to archive, archive_mode=on and >> archive_command=pg_archive_bypass, so that the day you have a HA budget >> ain't the day you're going to restart the server to enable the fault >> tolerance settings… > > That particular use case is better handled by making archive_mode changeable > on-the-fly. Now that we have wal_level as a separate GUC, it shouldn't be > too hard. Ok. > I like internal commands as a solution in general, but it's still in search > of a problem.. What about /usr/bin/true, or a simple archive where you cp in a given location (which could happen to be a remote server thanks to unix network file systems or windows shares), etc. It seems to me those are existing problem that we solve poorly: let each user face the same pitfalls (error management). I also like Simon's approach to have options to internal commands too. What about this syntax : guc_name = "pg_internal function_name arg1 %r ... argn" Then function_name is any SQL callable function and you have the %x substitution still happening there for you. So you can mix hard-coded arguments (some path) and dynamic arguments. And you can script your commands in PL/WhateverU. It's more involved a patch, of course, but if we do that and provide some simple commands in -core, then most installations will only use that and stop bothering writing scripts to handle their simple cases. Which is what I'm after. Regards, -- dim
On Mon, Jun 14, 2010 at 1:55 PM, Dimitri Fontaine <dfontaine@hi-media.com> wrote: > What about /usr/bin/true, or a simple archive where you cp in a given > location (which could happen to be a remote server thanks to unix > network file systems or windows shares), etc. It seems to me those are > existing problem that we solve poorly: let each user face the same > pitfalls (error management). > I would like to see the case where the archive is just a mounted directory accessible through the filesystem be handled internally. Ideally if I could just specify the archive location on both the master and slave using the same parameter just containing a filesystem path then I could have the same configuration on both machines except for the actual parameter which decides whether they're a master or slave. That would make me much more confident I've configured both machines properly. All the nodes would have the same information and they would be deciding for themselves whether to push or pull archives based on whether they're the master or slave. -- greg
Dimitri Fontaine wrote: > Hi, > > I tend to consider it a bug that there's no known way under windows to > use the same trick as under Unix by using '/usr/bin/true' as your > archive command. And this Unix trick itself does feel like a hack. > > Also I'd very much like to be able to recommend (even if not change the > official defaults) to setup wal_level to archive, archive_mode=on and > archive_command=pg_archive_bypass, so that the day you have a HA budget > ain't the day you're going to restart the server to enable the fault > tolerance settings? > > So please find attached a very simple "let's see about it" patch to > implement an internal archive_command that just returns true and is > called pg_archive_bypass. It's missing documentation, which I'll provide > if needed (meaning there's some will to consider applying such a patch). Turn out 'REM' acts like /bin/true on Windows. I have documented that fact in the attached, applied patch. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + None of us is going to be here forever. + Index: doc/src/sgml/config.sgml =================================================================== RCS file: /cvsroot/pgsql/doc/src/sgml/config.sgml,v retrieving revision 1.287 diff -c -c -r1.287 config.sgml *** doc/src/sgml/config.sgml 29 Jun 2010 22:29:13 -0000 1.287 --- doc/src/sgml/config.sgml 30 Jun 2010 02:40:40 -0000 *************** *** 1793,1799 **** disabled, but the server continues to accumulate WAL segment files in the expectation that a command will soon be provided. Setting <varname>archive_command</> to a command that does nothing but ! return true, e.g. <literal>/bin/true</>, effectively disables archiving, but also breaks the chain of WAL files needed for archive recovery, so it should only be used in unusual circumstances. </para> --- 1793,1800 ---- disabled, but the server continues to accumulate WAL segment files in the expectation that a command will soon be provided. Setting <varname>archive_command</> to a command that does nothing but ! return true, e.g. <literal>/bin/true</> (<literal>REM</> on ! Windows), effectively disables archiving, but also breaks the chain of WAL files needed for archive recovery, so it should only be used in unusual circumstances. </para>
Bruce Momjian <bruce@momjian.us> writes: > Turn out 'REM' acts like /bin/true on Windows. I have documented that > fact in the attached, applied patch. I guess that kills it for the "pg_archive_bypass" internal command, but in a good way. Thanks! Regards, -- Dimitri Fontaine PostgreSQL DBA, Architecte Damn, I still baffled that we can escape the internal commands altogether.