Thread: remove pg_standby?
While we're talking about removing old things, is there any use left for pg_standby?
On Wed, Nov 5, 2014 at 6:36 AM, Peter Eisentraut <peter_e@gmx.net> wrote: > While we're talking about removing old things, is there any use left for > pg_standby? +1 for this autumn cleanup. -- Michael
On Tue, Nov 4, 2014 at 10:36 PM, Peter Eisentraut <peter_e@gmx.net> wrote: > While we're talking about removing old things, is there any use left for > pg_standby? Was the original reason to keep it around anything other than backwards compatibility? If not, then it was backwards compatibility with a version that's not even supported anymore - so strong +1 for getting rid of it. -- Magnus HaganderMe: http://www.hagander.net/Work: http://www.redpill-linpro.com/
On Wed, Nov 5, 2014 at 6:36 AM, Peter Eisentraut <peter_e@gmx.net> wrote: > While we're talking about removing old things, is there any use left for > pg_standby? -1 for removing it. There is still the case where I'd like to use log-shipping rather than replication. For example, it's the case where I need to compress WAL files before streaming them via very thin network. We can set up log-shipping using standby_mode and without using pg_standby, but it keeps emitting the failure of restore_command while while there is no WAL activity, and which is bothersome. So I still need pg_standby for log-shipping. Regards, -- Fujii Masao
<p dir="ltr"><br /> On Nov 10, 2014 6:16 PM, "Fujii Masao" <<a href="mailto:masao.fujii@gmail.com">masao.fujii@gmail.com</a>>wrote:<br /> ><br /> > On Wed, Nov 5, 2014 at 6:36AM, Peter Eisentraut <<a href="mailto:peter_e@gmx.net">peter_e@gmx.net</a>> wrote:<br /> > > While we'retalking about removing old things, is there any use left for<br /> > > pg_standby?<br /> ><br /> > -1 forremoving it. There is still the case where I'd like to use log-shipping<br /> > rather than replication. For example,it's the case where I need to<br /> > compress WAL files before streaming them via very thin network.<br /> >We can set up log-shipping using standby_mode and without using<br /> > pg_standby, but it keeps emitting the failureof restore_command while<br /> > while there is no WAL activity, and which is bothersome. So I still need<br />> pg_standby for log-shipping.<p dir="ltr">I didn't realize that part, but maybe we should fix that instead of keepingpg_standby around? <p dir="ltr">(BTW, you can use streaming with compression as well using ssl of course, but it won'tget quite the same levels due to smaller block sizes. And there are certainly still reasons for file based standbysso we should definitely not remove that) <p dir="ltr">/Magnus <br />
On Tue, Nov 11, 2014 at 2:19 AM, Magnus Hagander <magnus@hagander.net> wrote: > > On Nov 10, 2014 6:16 PM, "Fujii Masao" <masao.fujii@gmail.com> wrote: >> >> On Wed, Nov 5, 2014 at 6:36 AM, Peter Eisentraut <peter_e@gmx.net> wrote: >> > While we're talking about removing old things, is there any use left for >> > pg_standby? >> >> -1 for removing it. There is still the case where I'd like to use >> log-shipping >> rather than replication. For example, it's the case where I need to >> compress WAL files before streaming them via very thin network. >> We can set up log-shipping using standby_mode and without using >> pg_standby, but it keeps emitting the failure of restore_command while >> while there is no WAL activity, and which is bothersome. So I still need >> pg_standby for log-shipping. > > I didn't realize that part, The log-shipping standby using standby_mode tries to execute the restore_command to restore new WAL file but it fails and the message meaning the failure is logged if there is no new WAL file. Then the standby tries to execure the same restore_command after five seconds. But it fails and the message is logged again. These happen continuously while no new WAL file is appearing in the standby's archival area. > but maybe we should fix that instead of keeping > pg_standby around? Yes. Even if we do that, we should announce pg_standby will be removed and wait for several releases before removing it? > (BTW, you can use streaming with compression as well using ssl of course, > but it won't get quite the same levels due to smaller block sizes. And there > are certainly still reasons for file based standbys so we should definitely > not remove that) Yes. Regards, -- Fujii Masao
On 11/04/2014 01:36 PM, Peter Eisentraut wrote: > > While we're talking about removing old things, is there any use left for > pg_standby? -1. A lot of people, a lot of customers use log shipping for various creative and business requirement setups. JD -- Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564 PostgreSQL Support, Training, Professional Services and Development High Availability, Oracle Conversion, @cmdpromptinc "If we send our children to Caesar for their education, we should not be surprised when they come back as Romans."
On 11/10/2014 07:50 PM, Joshua D. Drake wrote: > > On 11/04/2014 01:36 PM, Peter Eisentraut wrote: >> >> While we're talking about removing old things, is there any use left for >> pg_standby? > > -1. > > A lot of people, a lot of customers use log shipping for various > creative and business requirement setups. Yes, but do they use pg_standby to implement it? If they do, why? pg_standby is more configurable than the built-in standby_mode=on. You can set the sleep time, for example, while standby_mode=on uses a hard-coded delay of 5 s. And pg_standby has a configurable maximum wait time. And as Fujii pointed out, the built-in system will print an annoying message to the log every time it attempts to restore a file. Nevertheless, 99% of users would probably be happy with the built-in thing. - Heikki
On Mon, Nov 10, 2014 at 7:48 PM, Heikki Linnakangas <hlinnakangas@vmware.com> wrote: > On 11/10/2014 07:50 PM, Joshua D. Drake wrote: >> >> >> On 11/04/2014 01:36 PM, Peter Eisentraut wrote: >>> >>> >>> While we're talking about removing old things, is there any use left for >>> pg_standby? >> >> >> -1. >> >> A lot of people, a lot of customers use log shipping for various >> creative and business requirement setups. > > > Yes, but do they use pg_standby to implement it? If they do, why? > > pg_standby is more configurable than the built-in standby_mode=on. You can > set the sleep time, for example, while standby_mode=on uses a hard-coded > delay of 5 s. And pg_standby has a configurable maximum wait time. And as > Fujii pointed out, the built-in system will print an annoying message to the > log every time it attempts to restore a file. Nevertheless, 99% of users > would probably be happy with the built-in thing. As long as pg_standby has features that are actually useful and that are not in the built-in system, we shouldn't remove it. We should, however, try to fix those in the main system so we can get rid of it after that :) -- Magnus HaganderMe: http://www.hagander.net/Work: http://www.redpill-linpro.com/
On 11/10/2014 10:54 AM, Magnus Hagander wrote: > On Mon, Nov 10, 2014 at 7:48 PM, Heikki Linnakangas > <hlinnakangas@vmware.com> wrote: >> pg_standby is more configurable than the built-in standby_mode=on. You can >> set the sleep time, for example, while standby_mode=on uses a hard-coded >> delay of 5 s. And pg_standby has a configurable maximum wait time. And as >> Fujii pointed out, the built-in system will print an annoying message to the >> log every time it attempts to restore a file. Nevertheless, 99% of users >> would probably be happy with the built-in thing. > > As long as pg_standby has features that are actually useful and that > are not in the built-in system, we shouldn't remove it. We should, > however, try to fix those in the main system so we can get rid of it > after that :) As of current 9.5, we have configurable retries and standby delay in mainstream. Is there some reason we still need pg_standby? -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com
On Sat, Feb 28, 2015 at 5:00 AM, Josh Berkus <josh@agliodbs.com> wrote: > On 11/10/2014 10:54 AM, Magnus Hagander wrote: >> On Mon, Nov 10, 2014 at 7:48 PM, Heikki Linnakangas >> <hlinnakangas@vmware.com> wrote: >>> pg_standby is more configurable than the built-in standby_mode=on. You can >>> set the sleep time, for example, while standby_mode=on uses a hard-coded >>> delay of 5 s. And pg_standby has a configurable maximum wait time. And as >>> Fujii pointed out, the built-in system will print an annoying message to the >>> log every time it attempts to restore a file. Nevertheless, 99% of users >>> would probably be happy with the built-in thing. >> >> As long as pg_standby has features that are actually useful and that >> are not in the built-in system, we shouldn't remove it. We should, >> however, try to fix those in the main system so we can get rid of it >> after that :) > > As of current 9.5, we have configurable retries and standby delay in > mainstream. Is there some reason we still need pg_standby? Yes, it's not easy to perform "fast failover" without pg_standby for now. Regards, -- Fujii Masao
On 03/02/2015 11:53 AM, Fujii Masao wrote: > On Sat, Feb 28, 2015 at 5:00 AM, Josh Berkus <josh@agliodbs.com> wrote: >> On 11/10/2014 10:54 AM, Magnus Hagander wrote: >>> On Mon, Nov 10, 2014 at 7:48 PM, Heikki Linnakangas >>> <hlinnakangas@vmware.com> wrote: >>>> pg_standby is more configurable than the built-in standby_mode=on. You can >>>> set the sleep time, for example, while standby_mode=on uses a hard-coded >>>> delay of 5 s. And pg_standby has a configurable maximum wait time. And as >>>> Fujii pointed out, the built-in system will print an annoying message to the >>>> log every time it attempts to restore a file. Nevertheless, 99% of users >>>> would probably be happy with the built-in thing. >>> >>> As long as pg_standby has features that are actually useful and that >>> are not in the built-in system, we shouldn't remove it. We should, >>> however, try to fix those in the main system so we can get rid of it >>> after that :) >> >> As of current 9.5, we have configurable retries and standby delay in >> mainstream. Is there some reason we still need pg_standby? > > Yes, it's not easy to perform "fast failover" without pg_standby for now. What is "fast failover"? - Heikki
On Mon, Mar 2, 2015 at 8:37 PM, Heikki Linnakangas <hlinnaka@iki.fi> wrote: > On 03/02/2015 11:53 AM, Fujii Masao wrote: >> >> On Sat, Feb 28, 2015 at 5:00 AM, Josh Berkus <josh@agliodbs.com> wrote: >>> >>> On 11/10/2014 10:54 AM, Magnus Hagander wrote: >>>> >>>> On Mon, Nov 10, 2014 at 7:48 PM, Heikki Linnakangas >>>> <hlinnakangas@vmware.com> wrote: >>>>> >>>>> pg_standby is more configurable than the built-in standby_mode=on. You >>>>> can >>>>> set the sleep time, for example, while standby_mode=on uses a >>>>> hard-coded >>>>> delay of 5 s. And pg_standby has a configurable maximum wait time. And >>>>> as >>>>> Fujii pointed out, the built-in system will print an annoying message >>>>> to the >>>>> log every time it attempts to restore a file. Nevertheless, 99% of >>>>> users >>>>> would probably be happy with the built-in thing. >>>> >>>> >>>> As long as pg_standby has features that are actually useful and that >>>> are not in the built-in system, we shouldn't remove it. We should, >>>> however, try to fix those in the main system so we can get rid of it >>>> after that :) >>> >>> >>> As of current 9.5, we have configurable retries and standby delay in >>> mainstream. Is there some reason we still need pg_standby? >> >> >> Yes, it's not easy to perform "fast failover" without pg_standby for now. > > > What is "fast failover"? Per document, ------------------ In fast failover, the server is brought up immediately. Any WAL files in the archive that have not yet been applied will be ignored, and all transactions in those files are lost. To trigger a fast failover, create a trigger file and write the word fast into it. pg_standby can also be configured to execute a fast failover automatically if no new WAL file appears within a defined interval. ------------------ Regards, -- Fujii Masao
> Per document, > > ------------------ > In fast failover, the server is brought up immediately. Any WAL files > in the archive that have not yet been applied will be ignored, and all > transactions in those files are lost. To trigger a fast failover, > create a trigger file and write the word fast into it. pg_standby can > also be configured to execute a fast failover automatically if no new > WAL file appears within a defined interval. > ------------------ I thought we had that as a 9.4 feature, actually. Well wait ... that's for streaming. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com
On Tue, Mar 3, 2015 at 3:07 AM, Josh Berkus <josh@agliodbs.com> wrote: > >> Per document, >> >> ------------------ >> In fast failover, the server is brought up immediately. Any WAL files >> in the archive that have not yet been applied will be ignored, and all >> transactions in those files are lost. To trigger a fast failover, >> create a trigger file and write the word fast into it. pg_standby can >> also be configured to execute a fast failover automatically if no new >> WAL file appears within a defined interval. >> ------------------ > > I thought we had that as a 9.4 feature, actually. Well wait ... that's > for streaming. s/9.4/9.3? That's different from one we had in 9.3. Fast failover that pg_standby supports is something like the feature that I was proposing at http://www.postgresql.org/message-id/CAHGQGwHtvyDqKZaYWYA9zyyLEcAKiF5P0KpcpuNE_tsrGTFtQw@mail.gmail.com that is, the feature which allows us to give up replaying remaining WAL data for some reasons at the standby promotion. OTOH, fast failover that was supported in 9.3 enables us to skip an end-of-recovery checkpoint at the promotion and reduce the failover time. Regards, -- Fujii Masao
On Mon, Mar 2, 2015 at 6:22 PM, Fujii Masao <masao.fujii@gmail.com> wrote: > On Tue, Mar 3, 2015 at 3:07 AM, Josh Berkus <josh@agliodbs.com> wrote: >>> Per document, >>> >>> ------------------ >>> In fast failover, the server is brought up immediately. Any WAL files >>> in the archive that have not yet been applied will be ignored, and all >>> transactions in those files are lost. To trigger a fast failover, >>> create a trigger file and write the word fast into it. pg_standby can >>> also be configured to execute a fast failover automatically if no new >>> WAL file appears within a defined interval. >>> ------------------ >> >> I thought we had that as a 9.4 feature, actually. Well wait ... that's >> for streaming. > > s/9.4/9.3? > > That's different from one we had in 9.3. Fast failover that pg_standby > supports is something like the feature that I was proposing at > http://www.postgresql.org/message-id/CAHGQGwHtvyDqKZaYWYA9zyyLEcAKiF5P0KpcpuNE_tsrGTFtQw@mail.gmail.com > that is, the feature which allows us to give up replaying remaining > WAL data for some reasons at the standby promotion. OTOH, fast > failover that was supported in 9.3 enables us to skip an end-of-recovery > checkpoint at the promotion and reduce the failover time. Calling those things by the same name is very confusing. The data-losing feature ought to be called "immediate failover" or something else more dangerous-sounding than "fast". -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company