Thread: DeArchiver process
Currently, the Startup process is responsible for running restore_command. So when the Startup process is busy or waiting, then no new WAL files arrive. That has these effects * Recovery must wait while the Startup process requests next WAL file. This reduces performance of archive recovery. * If replication is file-based then no new files can be downloaded while we are waiting. If the Startup process waits, it then is much slower to catch up than it could be if it had already downloaded the files from the archive. * We cannot run an archive_cleanup_command, so the archive keep growing. * Cascading from a standby that uses file based replication is not easily possible My solution is to create a new process called the DeArchiver. This will run restore_command in a tight loop until the number of files would exceed wal_keep_files, then sleep. Each time the DeArchiver executes restore_command it will set the return code and if rc=0 the new XLogRecPtr reached. If standby_mode = on it will continue to retry indefinitely. The Startup process will just read files from pg_xlog rather than from the archive, just as it does for streaming, so this will remove the special case code in xlog.c. (WALReciver and this process will still need to coordinate so they are not both simultaneously active at any point, as now). This proposal gives a performance gain because the DeArchiver can be restoring the next file while the Startup process is processing the current file, so they work together using pipeline parallelism. The DeArchiver would start when we are not in crash recovery and exit at the end of recovery. This would then allow restore_command to be set via reload rather than restart. Previously, we have given greater weight to files from the archive to files already in pg_xlog. To ensure that behaviour continues, if restore_command is set at the Startup process will read the files in the pg_xlog directory and remember which ones were there at startup. That way it will be able to tell the difference between files newly downloaded and those already in the directory. If a file is absent from the archive we will use the file from pg_xlog. This makes file-based and stream-based replication work in a similar way, which is neater, and it also means all required files are available in case of a crash, which means we can more easily get rid of shutdown checkpoints in case of failoiver (discussed on separate thread). Since more files are available, it allows cascading replication to have a sender which receives WAL data in files. Which do we prefer "DeArchiver", "Restore process", or "WALFileReceiver". Thoughts? -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
On Wed, Nov 2, 2011 at 11:56 AM, Simon Riggs <simon@2ndquadrant.com> wrote: > My solution is to create a new process called the DeArchiver. This > will run restore_command in a tight loop until the number of files > would exceed wal_keep_files, then sleep. Each time the DeArchiver > executes restore_command it will set the return code and if rc=0 the > new XLogRecPtr reached. If standby_mode = on it will continue to retry > indefinitely. Are you thinking of reusing the existing GUC wal_keep_segments (not wal_keep_files), or creating a new one? I'd suggest creating a new one, so as to avoid having a GUC that does one thing on the master and something quite different on the slave. > Which do we prefer "DeArchiver", "Restore process", or "WALFileReceiver". My personal preference would be restore process, since we already use the name restore_command. > Thoughts? +1. Great idea. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
Simon Riggs <simon@2ndQuadrant.com> writes: > My solution is to create a new process called the DeArchiver. This > will run restore_command in a tight loop until the number of files > would exceed wal_keep_files, then sleep. Each time the DeArchiver > executes restore_command it will set the return code and if rc=0 the > new XLogRecPtr reached. If standby_mode = on it will continue to retry > indefinitely. +1 I think you mean wal_keep_segments, a GUC that we already have. > Which do we prefer "DeArchiver", "Restore process", or "WALFileReceiver". The only part of your proposal that I don't like is the process name, that "deArchiver" thing. "wal restore process" or something like that would be better. We already have "wal writer process" and "wal sender process" and "wal receiver process". Regards, -- Dimitri Fontaine http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support
On Wed, Nov 2, 2011 at 4:14 PM, Robert Haas <robertmhaas@gmail.com> wrote: > On Wed, Nov 2, 2011 at 11:56 AM, Simon Riggs <simon@2ndquadrant.com> wrote: >> My solution is to create a new process called the DeArchiver. This >> will run restore_command in a tight loop until the number of files >> would exceed wal_keep_files, then sleep. Each time the DeArchiver >> executes restore_command it will set the return code and if rc=0 the >> new XLogRecPtr reached. If standby_mode = on it will continue to retry >> indefinitely. > > Are you thinking of reusing the existing GUC wal_keep_segments (not > wal_keep_files), or creating a new one? I'd suggest creating a new > one, so as to avoid having a GUC that does one thing on the master and > something quite different on the slave. Any standby can now become a sender node, so the meaning in that case would be the same. That takes a little time to get your head around, and I'm not used to it myself yet. I guess you might say that you know for certain that a node will never be a relay node, and so a different meaning is possible, but it seems easier to avoid adding a new parameter just for that case. >> Which do we prefer "DeArchiver", "Restore process", or "WALFileReceiver". > > My personal preference would be restore process, since we already use > the name restore_command. Restore process, with file called restore.c in src/backend/postmaster (or src/backend/replication?) We can change that again later, but its enough to be getting on with. >> Thoughts? > > +1. Great idea. Thanks. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
On Wed, Nov 2, 2011 at 12:42 PM, Simon Riggs <simon@2ndquadrant.com> wrote: > Any standby can now become a sender node, so the meaning in that case > would be the same. That takes a little time to get your head around, > and I'm not used to it myself yet. I think a new parameter will be more clear, even if in practice the difference is fairly thin. >>> Which do we prefer "DeArchiver", "Restore process", or "WALFileReceiver". >> >> My personal preference would be restore process, since we already use >> the name restore_command. > > Restore process, with file called restore.c in src/backend/postmaster > (or src/backend/replication?) Yeah, that works. I'd go for postmaster over replication, for parallelism. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
Dimitri Fontaine <dimitri@2ndQuadrant.fr> writes:
> The only part of your proposal that I don't like is the process name,
> that "deArchiver" thing. "wal restore process" or something like that
> would be better. We already have "wal writer process" and "wal sender
> process" and "wal receiver process".
+1, "restore" seems pretty vague in this context.
regards, tom lane
On Wed, Nov 2, 2011 at 5:20 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote: > Dimitri Fontaine <dimitri@2ndQuadrant.fr> writes: >> The only part of your proposal that I don't like is the process name, >> that "deArchiver" thing. "wal restore process" or something like that >> would be better. We already have "wal writer process" and "wal sender >> process" and "wal receiver process". > > +1, "restore" seems pretty vague in this context. Yeh, walrestore seems more natural than just "restore". -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
On Thu, Nov 3, 2011 at 2:52 AM, Simon Riggs <simon@2ndquadrant.com> wrote: > On Wed, Nov 2, 2011 at 5:20 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote: >> Dimitri Fontaine <dimitri@2ndQuadrant.fr> writes: >>> The only part of your proposal that I don't like is the process name, >>> that "deArchiver" thing. "wal restore process" or something like that >>> would be better. We already have "wal writer process" and "wal sender >>> process" and "wal receiver process". >> >> +1, "restore" seems pretty vague in this context. > > Yeh, walrestore seems more natural than just "restore". +1 with this name and whole idea. If we introduce "walrestore" process, pg_standby seems no longer useful. We should get rid of it? Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center
On Fri, Nov 4, 2011 at 2:36 AM, Fujii Masao <masao.fujii@gmail.com> wrote: > If we introduce "walrestore" process, pg_standby seems no longer useful. > We should get rid of it? Removing things too quickly can cause problems. There's no harm done by keeping it a while longer. I agree it should go, just want to be absolutely clear that its no longer needed for any use case. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
Fujii Masao <masao.fujii@gmail.com> writes: > If we introduce "walrestore" process, pg_standby seems no longer useful. pg_standby is one possible restore_command, right? I had understood that walrestore would be the process that cares for running that command, not another implementation of it. That said, I would really like us to provide a default restore command, so if you had any intend of handling the restoring command in the walrestore process, by all means, go ahead :) Regards, -- Dimitri Fontaine http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support
On Fri, Nov 4, 2011 at 11:08 AM, Dimitri Fontaine <dimitri@2ndquadrant.fr> wrote: > Fujii Masao <masao.fujii@gmail.com> writes: >> If we introduce "walrestore" process, pg_standby seems no longer useful. > > pg_standby is one possible restore_command, right? I had understood > that walrestore would be the process that cares for running that > command, not another implementation of it. Yes, that was the idea. > That said, I would really like us to provide a default restore command, > so if you had any intend of handling the restoring command in the > walrestore process, by all means, go ahead :) A different proposal, I think. Not no, just not here and now. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
On Fri, Nov 4, 2011 at 5:15 AM, Simon Riggs <simon@2ndquadrant.com> wrote: > On Fri, Nov 4, 2011 at 2:36 AM, Fujii Masao <masao.fujii@gmail.com> wrote: >> If we introduce "walrestore" process, pg_standby seems no longer useful. >> We should get rid of it? > > Removing things too quickly can cause problems. There's no harm done > by keeping it a while longer. > > I agree it should go, just want to be absolutely clear that its no > longer needed for any use case. I agree that it would be premature to remove pg_standby at this point.But how about changing the default value of standby_modefrom "off" to "on" in 9.2? I think most new installations are probably using that, rather than pg_standby, and changing the default would give people a gentle push in what now seems to be the preferred direction. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company