Thread: status/timeline of pglogical?
Folks, I'm wondering whether or not we should be promoting pglogical with the release as an external extension. Here's what that would hinge on: 1. is it easily installable as an external extension? if so, where can people download it? 2. are the issues raised on -hackers likely to be resolved by September? 3. is it ready for playing with now? can people get as far as test setups with it? -- -- Josh Berkus Red Hat OSAS (any opinions are my own)
On 05/09/2016 09:53 AM, Josh berkus wrote: > Folks, > > I'm wondering whether or not we should be promoting pglogical with the > release as an external extension. Here's what that would hinge on: > > 1. is it easily installable as an external extension? if so, where can > people download it? Yes, if you are running deb/ubuntu or RHEL/Cent. > > 2. are the issues raised on -hackers likely to be resolved by September? > Unknown as there is very little if any communication from 2Q about PgLogical. > 3. is it ready for playing with now? can people get as far as test > setups with it? Play with? Yes. I have been testing it since 1.0. 1.1 is on my list this week, it supposedly fixes a few things that i ran into but see my comment about communication. Whether or not it is production ready still remains to be seen, there are way too many, "It through this error, what does it mean" and "Why doesn't the documentation tell me X?" If we advocate PgLogical it needs to be opened up more or at least the message should be one of, "Look at what cool things companies can do with the Pg extension API" not of "Look at this new PostgreSQL project thing" Sincerely, JD -- Command Prompt, Inc. http://the.postgres.company/ +1-503-667-4564 PostgreSQL Centered full stack support, consulting and development. Everyone appreciates your honesty, until you are honest with them.
On 05/09/2016 10:02 AM, Joshua D. Drake wrote: > On 05/09/2016 09:53 AM, Josh berkus wrote: >> 3. is it ready for playing with now? can people get as far as test >> setups with it? > > Play with? Yes. I have been testing it since 1.0. 1.1 is on my list this > week, it supposedly fixes a few things that i ran into but see my > comment about communication. Whether or not it is production ready still > remains to be seen, there are way too many, "It through this error, what > does it mean" and "Why doesn't the documentation tell me X?" Sigh... not enough coffee: "threw" not "through" and of course a missing ? JD -- Command Prompt, Inc. http://the.postgres.company/ +1-503-667-4564 PostgreSQL Centered full stack support, consulting and development. Everyone appreciates your honesty, until you are honest with them.
Hi Josh, On 09/05/16 18:53, Josh berkus wrote: > Folks, > > I'm wondering whether or not we should be promoting pglogical with the > release as an external extension. Here's what that would hinge on: > > 1. is it easily installable as an external extension? if so, where can > people download it? Yes with debs and rpms + sources tarball as well for other systems. The project page is on 2ndQ website ( http://2ndquadrant.com/en/resources/pglogical/ ) at the moment, not sure if that's issue for community. The absolutely bare bones download page is at http://packages.2ndquadrant.com/pglogical/ . > > 2. are the issues raised on -hackers likely to be resolved by September? > Well afaik the the issues reported on -hackers are fixed in 1.1 which was released last week (unless I missed some). So in terms of extension it should be fine. > 3. is it ready for playing with now? can people get as far as test > setups with it? > Sure, there are people testing it for production usage, the regression tests have about 96% function coverage, etc. The 1.1 tarball builds fine with current head so will work with beta1 as well I assume. Is there any interest in us making packages for beta1? (We have packages for 9.4 and 9.5 so far) -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
On 05/09/2016 01:24 PM, Petr Jelinek wrote: > Is there any interest in us making packages for beta1? (We have packages > for 9.4 and 9.5 so far) > I'd say thats a prerequiste to announcing pglogical in the beta announcement ... -- -- Josh Berkus Red Hat OSAS (any opinions are my own)
On 09/05/16 22:38, Josh berkus wrote: > On 05/09/2016 01:24 PM, Petr Jelinek wrote: > >> Is there any interest in us making packages for beta1? (We have packages >> for 9.4 and 9.5 so far) >> > > I'd say thats a prerequiste to announcing pglogical in the beta > announcement ... > Okay, that should not be a problem (provided that debs and rpms are enough). But would be good to get early access to pgdg beta1 packages then so that we can build pglogical against them in time. I admit I have not much idea of how the release packaging is coordinated. -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
On 05/09/2016 03:39 PM, Petr Jelinek wrote: > On 09/05/16 22:38, Josh berkus wrote: >> On 05/09/2016 01:24 PM, Petr Jelinek wrote: >> >>> Is there any interest in us making packages for beta1? (We have packages >>> for 9.4 and 9.5 so far) >>> >> >> I'd say thats a prerequiste to announcing pglogical in the beta >> announcement ... >> > > Okay, that should not be a problem (provided that debs and rpms are > enough). But would be good to get early access to pgdg beta1 packages > then so that we can build pglogical against them in time. I admit I have > not much idea of how the release packaging is coordinated. I think we've left this until too late. Sorry about that. Let's maybe plan on announcing pglogical next week from pgCon? Will you or Craig be there? -- -- Josh Berkus Red Hat OSAS (any opinions are my own)
On 10/05/16 00:55, Josh berkus wrote: > On 05/09/2016 03:39 PM, Petr Jelinek wrote: >> On 09/05/16 22:38, Josh berkus wrote: >>> On 05/09/2016 01:24 PM, Petr Jelinek wrote: >>> >>>> Is there any interest in us making packages for beta1? (We have packages >>>> for 9.4 and 9.5 so far) >>>> >>> >>> I'd say thats a prerequiste to announcing pglogical in the beta >>> announcement ... >>> >> >> Okay, that should not be a problem (provided that debs and rpms are >> enough). But would be good to get early access to pgdg beta1 packages >> then so that we can build pglogical against them in time. I admit I have >> not much idea of how the release packaging is coordinated. > > I think we've left this until too late. Sorry about that. > > Let's maybe plan on announcing pglogical next week from pgCon? Will you > or Craig be there? > Nope, Simon will be there though. There is always beta2 etc in worst case. -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
On 10 May 2016 at 06:39, Petr Jelinek <petr@2ndquadrant.com> wrote:
Okay, that should not be a problem (provided that debs and rpms are enough). But would be good to get early access to pgdg beta1 packages then so that we can build pglogical against them in time. I admit I have not much idea of how the release packaging is coordinated.
I've prepped pglogical rpms for 9.6 and uploaded to the main pglogical repo Petr linked to earlier. I had to base them on temporary home-brewed Pg 9.6 rpms pending release of the official ones.
It looks like beta1 Debian/Ubuntu packages for Pg are still a WIP and we can't really build pglogical packages before they're ready.
RPM detail: There are no rpm tree changes in git yet, but I've prepped my own beta1 RPMs to build pglogical against. I've had to guess about the version scheme, so it's not ideal. Past beta RPM releases have used a scheme where the Version is "9.5" and the Release is "beta1_1PGDG" but I don't know if that'll be maintained since it's not really correct according to packaging guidelines (and I mailed Devrim about it).
Once PGDG 9.6beta1 rpms are live on yum.postgresql.org (or some private repo I can access) it takes about 20 mins to build pglogical rpms. I'll want rebuild them once the official packages go up no matter what, just to make 100% sure I'm building against exactly the same packages that'll be deployed, but it's not that important unless the version doesn't match.
Anyway, once the required Pg packages are on PGDG repos it's very quick to rebuild the debs and rpms for pglogical. If I'm not awake (+0800 Western Australia time) Petr will be so one of us can kick off a build and we'll have packages something like half an hour later.
On 05/10/2016 10:55 AM, Craig Ringer wrote: > On 10 May 2016 at 06:39, Petr Jelinek <petr@2ndquadrant.com > <mailto:petr@2ndquadrant.com>> wrote: > > > Okay, that should not be a problem (provided that debs and rpms are > enough). But would be good to get early access to pgdg beta1 > packages then so that we can build pglogical against them in time. I > admit I have not much idea of how the release packaging is coordinated. > > > > I've prepped pglogical rpms for 9.6 and uploaded to the main pglogical > repo Petr linked to earlier. I had to base them on temporary home-brewed > Pg 9.6 rpms pending release of the official ones. Wow, fast. > Anyway, once the required Pg packages are on PGDG repos it's very quick > to rebuild the debs and rpms for pglogical. If I'm not awake (+0800 > Western Australia time) Petr will be so one of us can kick off a build > and we'll have packages something like half an hour later. OK. Is there a page I can link to for pglogical which is not a 2ndQuadrant corporate page? Something on github, maybe? -- -- Josh Berkus Red Hat OSAS (any opinions are my own)
On 10/05/16 20:10, Josh berkus wrote: > On 05/10/2016 10:55 AM, Craig Ringer wrote: >> On 10 May 2016 at 06:39, Petr Jelinek <petr@2ndquadrant.com >> <mailto:petr@2ndquadrant.com>> wrote: >> >> >> Okay, that should not be a problem (provided that debs and rpms are >> enough). But would be good to get early access to pgdg beta1 >> packages then so that we can build pglogical against them in time. I >> admit I have not much idea of how the release packaging is coordinated. >> >> >> >> I've prepped pglogical rpms for 9.6 and uploaded to the main pglogical >> repo Petr linked to earlier. I had to base them on temporary home-brewed >> Pg 9.6 rpms pending release of the official ones. > > Wow, fast. > >> Anyway, once the required Pg packages are on PGDG repos it's very quick >> to rebuild the debs and rpms for pglogical. If I'm not awake (+0800 >> Western Australia time) Petr will be so one of us can kick off a build >> and we'll have packages something like half an hour later. > > OK. Is there a page I can link to for pglogical which is not a > 2ndQuadrant corporate page? Something on github, maybe? > Yes, there is a public github page with both pglogical and pglogical_output. https://github.com/2ndQuadrant/pglogical/ -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
On Mon, May 9, 2016 at 5:53 PM, Josh berkus <josh@agliodbs.com> wrote: > Folks, > > I'm wondering whether or not we should be promoting pglogical with the > release as an external extension. We have a long standing practice of not promoting external tools/utilities/add-ons in docs or with releases - as you know we went out of our way to remove such references from the docs years ago. This becomes especially problematic when such external tools/utilities/add-ons have been developed by one particular company, as Robert pointed out here http://www.postgresql.org/message-id/CA+TgmoZVBmTnUT419mS3pV9hESJh78gibD_2tviaoGMWvOTwtg@mail.gmail.com I strongly oppose recommending any non-core 'stuff' in the docs or press releases/announcements (including pgAdmin 4). -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On 05/11/2016 07:25 AM, Dave Page wrote: > On Mon, May 9, 2016 at 5:53 PM, Josh berkus <josh@agliodbs.com> wrote: >> Folks, >> >> I'm wondering whether or not we should be promoting pglogical with the >> release as an external extension. > > We have a long standing practice of not promoting external > tools/utilities/add-ons in docs or with releases - as you know we went > out of our way to remove such references from the docs years ago. > > This becomes especially problematic when such external > tools/utilities/add-ons have been developed by one particular company, > as Robert pointed out here > http://www.postgresql.org/message-id/CA+TgmoZVBmTnUT419mS3pV9hESJh78gibD_2tviaoGMWvOTwtg@mail.gmail.com > > I strongly oppose recommending any non-core 'stuff' in the docs or > press releases/announcements (including pgAdmin 4). > Agreed, if for no other reason that including them makes the project responsible for them. -- Adrian Klaver adrian.klaver@aklaver.com
On 05/11/2016 07:30 AM, Adrian Klaver wrote: > On 05/11/2016 07:25 AM, Dave Page wrote: >> On Mon, May 9, 2016 at 5:53 PM, Josh berkus <josh@agliodbs.com> wrote: > > Agreed, if for no other reason that including them makes the project > responsible for them. Not on any planet in reality is this true. JD -- Command Prompt, Inc. http://the.postgres.company/ +1-503-667-4564 PostgreSQL Centered full stack support, consulting and development. Everyone appreciates your honesty, until you are honest with them.
On 05/11/2016 07:25 AM, Dave Page wrote: > On Mon, May 9, 2016 at 5:53 PM, Josh berkus <josh@agliodbs.com> wrote: >> Folks, >> >> I'm wondering whether or not we should be promoting pglogical with the >> release as an external extension. > > We have a long standing practice of not promoting external > tools/utilities/add-ons in docs or with releases - as you know we went > out of our way to remove such references from the docs years ago. The idea that the concrete we poured 15 years ago never needs to be inspected for upgrade is a very poor way to insure the integrity and strength of the foundation. > > This becomes especially problematic when such external > tools/utilities/add-ons have been developed by one particular company, > as Robert pointed out here > http://www.postgresql.org/message-id/CA+TgmoZVBmTnUT419mS3pV9hESJh78gibD_2tviaoGMWvOTwtg@mail.gmail.com Correct. > > I strongly oppose recommending any non-core 'stuff' in the docs or > press releases/announcements (including pgAdmin 4). > We play favorites all the time we just don't like to admit it publicly. This actually comes back to the idea of having an official extensions project but we don't need to get into that in this thread. Sincerely, JD -- Command Prompt, Inc. http://the.postgres.company/ +1-503-667-4564 PostgreSQL Centered full stack support, consulting and development. Everyone appreciates your honesty, until you are honest with them.
On Wed, May 11, 2016 at 5:17 PM, Joshua D. Drake <jd@commandprompt.com> wrote: > On 05/11/2016 07:25 AM, Dave Page wrote: >> >> On Mon, May 9, 2016 at 5:53 PM, Josh berkus <josh@agliodbs.com> wrote: >>> >>> Folks, >>> >>> I'm wondering whether or not we should be promoting pglogical with the >>> release as an external extension. >> >> >> We have a long standing practice of not promoting external >> tools/utilities/add-ons in docs or with releases - as you know we went >> out of our way to remove such references from the docs years ago. > > > The idea that the concrete we poured 15 years ago never needs to be > inspected for upgrade is a very poor way to insure the integrity and > strength of the foundation. And the idea of redesigning the foundation without barely a nod to the thought process behind the original design is also not a good idea. >> This becomes especially problematic when such external >> tools/utilities/add-ons have been developed by one particular company, >> as Robert pointed out here >> >> http://www.postgresql.org/message-id/CA+TgmoZVBmTnUT419mS3pV9hESJh78gibD_2tviaoGMWvOTwtg@mail.gmail.com > > > Correct. > >> >> I strongly oppose recommending any non-core 'stuff' in the docs or >> press releases/announcements (including pgAdmin 4). >> > > We play favorites all the time we just don't like to admit it publicly. Right - and for good reason in my opinion (and I say that as someone who would probably get significant advantages if we did). > This actually comes back to the idea of having an official extensions > project but we don't need to get into that in this thread. True dat. -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On 05/11/2016 09:35 AM, Dave Page wrote: >>> I strongly oppose recommending any non-core 'stuff' in the docs or >>> press releases/announcements (including pgAdmin 4). Well, the line I was going to add was this: Version 9.6 Beta 1 also makes [changes to the binary backup API](http://www.postgresql.org/docs/devel/static/functions-admin.html#FUNCTIONS-ADMIN-BACKUP-TABLE). Users should test version 9.6 with PostgreSQL backup tools, including pgBackRest, Barman, WAL-E, and other packaged and in-house software. Users may also wish to test [pglogical](https://github.com/2ndQuadrant/pglogical/), the newest logical replication system for PostgreSQL, currently in beta. ... none of that is "recommending" anything; it's all about "please test these things", which is what a beta announcement is *for*. It also doesn't hurt us at all to show a lot of activity on the replication front. However, given that it's less than 24 hours before the beta release, I don't see that we can get consensus on this before it needs to go out, so taking that line out. I'll work on a separate beta announcement with the 2Q folks for later. -- -- Josh Berkus Red Hat OSAS (any opinions are my own)
On 11 May 2016 at 18:07, Josh berkus <josh@agliodbs.com> wrote:
--
On 05/11/2016 09:35 AM, Dave Page wrote:
>>> I strongly oppose recommending any non-core 'stuff' in the docs or
>>> press releases/announcements (including pgAdmin 4).
Well, the line I was going to add was this:
Version 9.6 Beta 1 also makes [changes to the binary backup
API](http://www.postgresql.org/docs/devel/static/functions-admin.html#FUNCTIONS-ADMIN-BACKUP-TABLE).
Users should test version 9.6 with PostgreSQL backup tools, including
pgBackRest, Barman, WAL-E, and other packaged and in-house software.
Users may also wish to test
[pglogical](https://github.com/2ndQuadrant/pglogical/), the newest
logical replication system for PostgreSQL, currently in beta.
... none of that is "recommending" anything; it's all about "please test
these things", which is what a beta announcement is *for*. It also
doesn't hurt us at all to show a lot of activity on the replication front.
However, given that it's less than 24 hours before the beta release, I
don't see that we can get consensus on this before it needs to go out,
so taking that line out.
Agreed, no problem.
I'll work on a separate beta announcement with the 2Q folks for later.
Thanks. Not trying to set a precedent on this.
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On 05/11/2016 09:35 AM, Dave Page wrote: > On Wed, May 11, 2016 at 5:17 PM, Joshua D. Drake <jd@commandprompt.com> wrote: >> On 05/11/2016 07:25 AM, Dave Page wrote: >> The idea that the concrete we poured 15 years ago never needs to be >> inspected for upgrade is a very poor way to insure the integrity and >> strength of the foundation. > > And the idea of redesigning the foundation without barely a nod to the > thought process behind the original design is also not a good idea. Well I certainly wasn't trying to do that and really I wasn't trying to "promote" one solution over another as much as build stronger relationships with our external community. Perhaps there is a better way to do that than acknowledgement in a press release. JD -- Command Prompt, Inc. http://the.postgres.company/ +1-503-667-4564 PostgreSQL Centered full stack support, consulting and development. Everyone appreciates your honesty, until you are honest with them.
On 05/11/2016 09:15 AM, Joshua D. Drake wrote: > On 05/11/2016 07:30 AM, Adrian Klaver wrote: >> On 05/11/2016 07:25 AM, Dave Page wrote: >>> On Mon, May 9, 2016 at 5:53 PM, Josh berkus <josh@agliodbs.com> wrote: > >> >> Agreed, if for no other reason that including them makes the project >> responsible for them. > > Not on any planet in reality is this true. On the planet that is user space this is true to varying degrees. Even as it stands now, no third party packages mentioned in official releases, folks hit the list thinking some or all are part of the core Postgres release. This leads to repeated explanations of where the source really resides for said packages and where to file issues. That is part of the chore of participating on the lists, but including third party packages in official releases will just increase that work load and increase the frustration level of users who have to be reeducated. I believe in growing the community, however I think there should be a clear distinction between what is the core release and what is contributed from outside sources, namely by keeping the announcements separate. This seems to work for other projects I use/follow; Django, Pandas, IPython, Sqlite to name a few. Release announcements stick to the code the project generates and it up to third parties to update their own announcements. From what I have seen that has not negatively impacted the growth of the affiliated communities. > > JD > > > -- Adrian Klaver adrian.klaver@aklaver.com
On 05/11/2016 04:25 PM, Dave Page wrote: > We have a long standing practice of not promoting external > tools/utilities/add-ons in docs or with releases No we don't. See 092c38a and f02b14f. -- Vik Fearing +33 6 46 75 15 36 http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support
On 05/11/2016 03:06 PM, Vik Fearing wrote: > On 05/11/2016 04:25 PM, Dave Page wrote: >> We have a long standing practice of not promoting external >> tools/utilities/add-ons in docs or with releases > > No we don't. See 092c38a and f02b14f. > For those that don't git everyday, is there a link to this? JD -- Command Prompt, Inc. http://the.postgres.company/ +1-503-667-4564 PostgreSQL Centered full stack support, consulting and development. Everyone appreciates your honesty, until you are honest with them.
On 05/12/2016 12:13 AM, Joshua D. Drake wrote: > On 05/11/2016 03:06 PM, Vik Fearing wrote: >> On 05/11/2016 04:25 PM, Dave Page wrote: >>> We have a long standing practice of not promoting external >>> tools/utilities/add-ons in docs or with releases >> >> No we don't. See 092c38a and f02b14f. > > For those that don't git everyday, is there a link to this? They're the creation and an update of the last paragraph on this page: http://www.postgresql.org/docs/9.5/static/logfile-maintenance.html It was added in 2010 by Bruce and updated in 2013 by Peter Eisentraut. -- Vik Fearing +33 6 46 75 15 36 http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support
Vik Fearing wrote: > On 05/12/2016 12:13 AM, Joshua D. Drake wrote: > > On 05/11/2016 03:06 PM, Vik Fearing wrote: > >> On 05/11/2016 04:25 PM, Dave Page wrote: > >>> We have a long standing practice of not promoting external > >>> tools/utilities/add-ons in docs or with releases > >> > >> No we don't. See 092c38a and f02b14f. > > > > For those that don't git everyday, is there a link to this? > > They're the creation and an update of the last paragraph on this page: > http://www.postgresql.org/docs/9.5/static/logfile-maintenance.html > > It was added in 2010 by Bruce and updated in 2013 by Peter Eisentraut. Ah, that reminded me of this other page http://www.postgresql.org/docs/9.5/static/different-replication-solutions.html where we list a few external products too, such as DRDB, Slony-I, pgpool-II, Continuent Tungsten, Bucardo, and PL/Proxy. It also fails to describe logical decoding. -- Álvaro Herrera http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On 11 May 2016 at 23:16, Vik Fearing <vik@2ndquadrant.fr> wrote:
On 05/12/2016 12:13 AM, Joshua D. Drake wrote:
> On 05/11/2016 03:06 PM, Vik Fearing wrote:
>> On 05/11/2016 04:25 PM, Dave Page wrote:
>>> We have a long standing practice of not promoting external
>>> tools/utilities/add-ons in docs or with releases
>>
>> No we don't. See 092c38a and f02b14f.
>
> For those that don't git everyday, is there a link to this?
They're the creation and an update of the last paragraph on this page:
http://www.postgresql.org/docs/9.5/static/logfile-maintenance.html
It was added in 2010 by Bruce and updated in 2013 by Peter Eisentraut.
Plus also these references to pgAdmin, all present since at least 9.1
"There are several administration tools available for PostgreSQL. The most popular is pgAdmin III, and there are several commercially available ones as well."
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Wed, May 11, 2016 at 11:36:24PM +0100, Simon Riggs wrote: > On 11 May 2016 at 23:16, Vik Fearing <vik@2ndquadrant.fr> wrote: > > On 05/12/2016 12:13 AM, Joshua D. Drake wrote: > > On 05/11/2016 03:06 PM, Vik Fearing wrote: > >> On 05/11/2016 04:25 PM, Dave Page wrote: > >>> We have a long standing practice of not promoting external > >>> tools/utilities/add-ons in docs or with releases > >> > >> No we don't. See 092c38a and f02b14f. > > > > For those that don't git everyday, is there a link to this? > > They're the creation and an update of the last paragraph on this page: > http://www.postgresql.org/docs/9.5/static/logfile-maintenance.html > > It was added in 2010 by Bruce and updated in 2013 by Peter Eisentraut. > > > Plus also these references to pgAdmin, all present since at least 9.1 > > http://www.postgresql.org/docs/9.5/static/tutorial-accessdb.html > http://www.postgresql.org/docs/9.5/static/plpgsql-development-tips.html > http://www.postgresql.org/docs/9.5/static/external-admin-tools.html > http://www.postgresql.org/docs/9.5/static/adminpack.html > > "There are several administration tools available for PostgreSQL. The most > popular is pgAdmin III, and there are several commercially available ones as > well." I encourage mentioning external tools in the right context --- we need more of that. I think the question is whether a beta announcement is the right place for external tool announcements. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On 11 May 2016 at 23:40, Bruce Momjian <bruce@momjian.us> wrote:
--
I encourage mentioning external tools in the right context --- we need
more of that.
Good to know.
I think the question is whether a beta announcement is
the right place for external tool announcements.
The only reason this has been brought up is that pglogical is a tool designed to reduce the impact of upgrades. No other tool has ever been available externally to do that, apart from pg_upgrade which was accepted into core in quite a poor state. So this discussion has not been about "external tools" it has been about the one and only external tool that provides an improved upgrade route.
If people don't want to mention it, fine, no problem, as I said I've not sought to create a new precedent.
We can add some mentions in the docs in appropriate locations, for example, the detailed release notes and upgrade pages.
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Thu, May 12, 2016 at 12:10:06AM +0100, Simon Riggs wrote: > The only reason this has been brought up is that pglogical is a tool designed > to reduce the impact of upgrades. No other tool has ever been available > externally to do that, apart from pg_upgrade which was accepted into core in > quite a poor state. So this discussion has not been about "external tools" it > has been about the one and only external tool that provides an improved upgrade > route. > > If people don't want to mention it, fine, no problem, as I said I've not sought > to create a new precedent. > > We can add some mentions in the docs in appropriate locations, for example, the > detailed release notes and upgrade pages. I don't have an opinion on whether to add it or not --- I was just saying it is different to mention it in the release notes rather in the appropriate place in the docs. You are right that is very similar to pg_upgrade, though I think you are also right that pg_upgrade wasn't mentioned until it was in contrib, because it was (perceived?) to be in such poor shape, or just a crazy idea. :-) -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On 05/11/2016 04:10 PM, Simon Riggs wrote: > On 11 May 2016 at 23:40, Bruce Momjian <bruce@momjian.us > <mailto:bruce@momjian.us>> wrote: > > I encourage mentioning external tools in the right context --- we need > more of that. > > > Good to know. > > I think the question is whether a beta announcement is > the right place for external tool announcements. > > > The only reason this has been brought up is that pglogical is a tool > designed to reduce the impact of upgrades. No other tool has ever been > available externally to do that, apart from pg_upgrade which was Slony, Londiste (Which are still more useful in some ways, although PgLogical has them beat from a performance standpoint hands down) > accepted into core in quite a poor state. So this discussion has not > been about "external tools" it has been about the one and only external > tool that provides an improved upgrade route. No it wasn't? In fact this whole thread started as: """ 1. is it easily installable as an external extension? if so, where can people download it? 2. are the issues raised on -hackers likely to be resolved by September? 3. is it ready for playing with now? can people get as far as test setups with it? """ Which was originated because I thought it might be good to mention tools in the Beta announcement draft I wrote with feedback from Haas. > > We can add some mentions in the docs in appropriate locations, for > example, the detailed release notes and upgrade pages. > If it becomes a proper community project (not necessarily a .Org project) then yes sir, let's do it! If not, then I suggest it go in the app listing on the website (just like any other product). Sincerely, JD > -- > Simon Riggs http://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> > PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Command Prompt, Inc. http://the.postgres.company/ +1-503-667-4564 PostgreSQL Centered full stack support, consulting and development. Everyone appreciates your honesty, until you are honest with them.
On 12 May 2016 at 00:17, Joshua D. Drake <jd@commandprompt.com> wrote:
On 05/11/2016 07:25 AM, Dave Page wrote:On Mon, May 9, 2016 at 5:53 PM, Josh berkus <josh@agliodbs.com> wrote:Folks,
I'm wondering whether or not we should be promoting pglogical with the
release as an external extension.
We have a long standing practice of not promoting external
tools/utilities/add-ons in docs or with releases - as you know we went
out of our way to remove such references from the docs years ago.
The idea that the concrete we poured 15 years ago never needs to be inspected for upgrade is a very poor way to insure the integrity and strength of the foundation.
Right. Most importantly, we fail to mention the backup utilities that every user should use and know about, and we fail to point users at connection poolers despite not incorporating one in core.
Whatever the outcome with pglogical I'd love to see that change. And no, I don't think "use the wiki" is good enough.
On 05/11/2016 05:51 PM, Craig Ringer wrote: > The idea that the concrete we poured 15 years ago never needs to be > inspected for upgrade is a very poor way to insure the integrity and > strength of the foundation. > > > Right. Most importantly, we fail to mention the backup utilities that > every user should use and know about, and we fail to point users at > connection poolers despite not incorporating one in core. > > Whatever the outcome with pglogical I'd love to see that change. And no, > I don't think "use the wiki" is good enough. You are absolutely right. J -- Command Prompt, Inc. http://the.postgres.company/ +1-503-667-4564 PostgreSQL Centered full stack support, consulting and development. Everyone appreciates your honesty, until you are honest with them.
On Wed, May 11, 2016 at 11:06 PM, Vik Fearing <vik@2ndquadrant.fr> wrote: > On 05/11/2016 04:25 PM, Dave Page wrote: >> We have a long standing practice of not promoting external >> tools/utilities/add-ons in docs or with releases > > No we don't. See 092c38a and f02b14f. Hmm, my bad. Seems like what I thought was a long-standing practice was long forgotten. FYI, here's where it was established: http://www.postgresql.org/message-id/flat/200007280252.WAA05977@candle.pha.pa.us#200007280252.WAA05977@candle.pha.pa.us (many, many years ago - man I feel old!). The resulting change to purge the docs (of the then single reference to an external app) was in 530dc73cd18f4df708c0c998521a20f5f93f729a. -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On 12 May 2016 at 00:52, Joshua D. Drake <jd@commandprompt.com> wrote:
--
If it becomes a proper community project (not necessarily a .Org project) then yes sir, let's do it! If not, then I suggest it go in the app listing on the website (just like any other product).
If you can give some criteria as to how we might judge these proper community projects, it will help.
And list which projects meet those criteria and which do not, so we can understand who is who.
Thanks
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On 5/12/16 4:23 AM, Dave Page wrote: > Hmm, my bad. Seems like what I thought was a long-standing practice > was long forgotten. FYI, here's where it was established: > http://www.postgresql.org/message-id/flat/200007280252.WAA05977@candle.pha.pa.us#200007280252.WAA05977@candle.pha.pa.us > (many, many years ago - man I feel old!). > > The resulting change to purge the docs (of the then single reference > to an external app) was in 530dc73cd18f4df708c0c998521a20f5f93f729a. What was removed then was the *man page* of pgadmin. Earlier in that thread, someone said: "It's certainly nice to mention related products, but maybe a reference page is not the most appropriate form" which someone later interpreted as an incentive to a purge. :-/ I think these principles still apply today, and most in this thread appear to agree: Mention tools in the right context. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Thu, May 12, 2016 at 12:12 PM, Peter Eisentraut <peter.eisentraut@2ndquadrant.com> wrote: > On 5/12/16 4:23 AM, Dave Page wrote: >> >> Hmm, my bad. Seems like what I thought was a long-standing practice >> was long forgotten. FYI, here's where it was established: >> >> http://www.postgresql.org/message-id/flat/200007280252.WAA05977@candle.pha.pa.us#200007280252.WAA05977@candle.pha.pa.us >> (many, many years ago - man I feel old!). >> >> The resulting change to purge the docs (of the then single reference >> to an external app) was in 530dc73cd18f4df708c0c998521a20f5f93f729a. > > > What was removed then was the *man page* of pgadmin. > > Earlier in that thread, someone said: > > "It's certainly nice to mention related products, but maybe a reference page > is not the most appropriate form" > > which someone later interpreted as an incentive to a purge. :-/ > > I think these principles still apply today, and most in this thread appear > to agree: Mention tools in the right context. Right, but if you follow the thread through, later discussion is on the docs in general. -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On 05/12/2016 01:31 AM, Simon Riggs wrote: > On 12 May 2016 at 00:52, Joshua D. Drake <jd@commandprompt.com > <mailto:jd@commandprompt.com>> wrote: > > If it becomes a proper community project (not necessarily a .Org > project) then yes sir, let's do it! If not, then I suggest it go in > the app listing on the website (just like any other product). > > > If you can give some criteria as to how we might judge these proper > community projects, it will help. I listed that here: http://www.postgresql.org/message-id/5733CBDC.3030306@commandprompt.com > > And list which projects meet those criteria and which do not, so we can > understand who is who. I left that out because I didn't want the entities to think I was picking on them because, I am not. Sincerely, JD -- Command Prompt, Inc. http://the.postgres.company/ +1-503-667-4564 PostgreSQL Centered full stack support, consulting and development. Everyone appreciates your honesty, until you are honest with them.
On Wed, May 11, 2016 at 7:10 PM, Simon Riggs <simon@2ndquadrant.com> wrote: > The only reason this has been brought up is that pglogical is a tool > designed to reduce the impact of upgrades. No other tool has ever been > available externally to do that, apart from pg_upgrade which was accepted > into core in quite a poor state. So this discussion has not been about > "external tools" it has been about the one and only external tool that > provides an improved upgrade route. I want to be clear that I'm not disputing this contention (well, open source tool, maybe). I also want to be clear that I agree with your assessment of pg_upgrade. Where we may disagree is on what that implies: 1. I think pglogical needs some additional work so that we don't again end up with a tool that should help with upgrades but actually isn't quite fully baked. 2. I also think pglogical should be integrated into the core distribution, because it makes little sense to me to have logical decoding in core but nothing that can take advantage of that capability in core. External tools can continue to exist to provide extra capabilities that aren't in core yet based on the same infrastructure, but the core capabilities should be in the core distribution. 3. I think we need to replace pg_upgrade with a real in-place upgrade scheme so that you just fire up the new version of the server on your old data directory, and it rejiggers things in place without needing to create a new cluster and migrate stuff over to it. I think that actually making this work is a huge engineering effort, and I have no plans to undertake it in the near term, but I think it has to be done. pg_upgrade isn't reliable enough, and using pglogical means you need a second machine. Maybe everybody should run with a standby, but not everyone does. 4. In the absence of a pg_upgrade replacement, more work out to be done to file down the rough edges of pg_upgrade. For example, Tom's work to make the server run in single-user mode via a pipe would have made pg_upgrade safer, but it got shouted down because it wasn't free ice cream and a pony. That is a shame. Of course, your conclusions may be different, and that's fine. This is just what I think. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On Thu, May 12, 2016 at 10:37:28AM -0400, Robert Haas wrote: > 3. I think we need to replace pg_upgrade with a real in-place upgrade > scheme so that you just fire up the new version of the server on your > old data directory, and it rejiggers things in place without needing > to create a new cluster and migrate stuff over to it. I think that > actually making this work is a huge engineering effort, and I have no > plans to undertake it in the near term, but I think it has to be done. > pg_upgrade isn't reliable enough, and using pglogical means you need a > second machine. Maybe everybody should run with a standby, but not > everyone does. I don't see why you can't have the pg_logical slave be on the same server as the master for an upgrade. It will double the write volume while it is active, but assuming it is setup only to perform a major version upgrade, it should be fine. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Thu, May 12, 2016 at 10:37:28AM -0400, Robert Haas wrote: > 3. I think we need to replace pg_upgrade with a real in-place upgrade > scheme so that you just fire up the new version of the server on your > old data directory, and it rejiggers things in place without needing > to create a new cluster and migrate stuff over to it. I think that > actually making this work is a huge engineering effort, and I have no > plans to undertake it in the near term, but I think it has to be done. Also, let me add that I smile every time pg_upgrade breaks and the fix is in pg_dump because it allows me to avoid the problem. :-) I am afraid that not relying on pg_dump is going to cause a lot of maintenance work for every server change --- right now, only pg_dump needs to be adjusted. So, the issue is not only getting in-place upgrade to work, but the future maintenance of keeping it working. OK, I will stop smiling now. ;-) -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On May 12, 2016 16:57, "Bruce Momjian" <bruce@momjian.us> wrote:
>
> On Thu, May 12, 2016 at 10:37:28AM -0400, Robert Haas wrote:
> > 3. I think we need to replace pg_upgrade with a real in-place upgrade
> > scheme so that you just fire up the new version of the server on your
> > old data directory, and it rejiggers things in place without needing
> > to create a new cluster and migrate stuff over to it. I think that
> > actually making this work is a huge engineering effort, and I have no
> > plans to undertake it in the near term, but I think it has to be done.
> > pg_upgrade isn't reliable enough, and using pglogical means you need a
> > second machine. Maybe everybody should run with a standby, but not
> > everyone does.
>
> I don't see why you can't have the pg_logical slave be on the same
> server as the master for an upgrade. It will double the write volume
> while it is active, but assuming it is setup only to perform a major
> version upgrade, it should be fine.
>
I think that's a pretty bad assumption. A lot, if not most, of the people who actually need zero downtime upgrades don't have 50% extra space and in particular not 50% extra performance on their servers to throw at that.
Can it be made to work? Sure. But I definitely agree with Robert that we need "real" in place upgrades at some point.
/Magnus
On 05/12/2016 07:37 AM, Robert Haas wrote: > On Wed, May 11, 2016 at 7:10 PM, Simon Riggs <simon@2ndquadrant.com> wrote: > 3. I think we need to replace pg_upgrade with a real in-place upgrade > scheme so that you just fire up the new version of the server on your > old data directory, and it rejiggers things in place without needing > to create a new cluster and migrate stuff over to it. I think that > actually making this work is a huge engineering effort, and I have no > plans to undertake it in the near term, but I think it has to be done. > pg_upgrade isn't reliable enough, and using pglogical means you need a > second machine. Maybe everybody should run with a standby, but not > everyone does. I am not arguing with you but... good luck. History has shown that the argument above is basically this: https://en.wikipedia.org/wiki/Sisyphus Sincerely, JD -- Command Prompt, Inc. http://the.postgres.company/ +1-503-667-4564 PostgreSQL Centered full stack support, consulting and development. Everyone appreciates your honesty, until you are honest with them.
On 13 May 2016 at 02:13, Joshua D. Drake <jd@commandprompt.com> wrote:
I am not arguing with you but... good luck.
Yeah.
While we could get away without replay of old-version WAL for in-place upgrade we'd need versioned catalog tuples, so we could read a catalog tuple in the old version, convert it on the fly and write it out in the new version when we modified it. And that's just for starters. I think it'd be great, but it's no small amount of work.
On Thu, May 12, 2016 at 2:13 PM, Joshua D. Drake <jd@commandprompt.com> wrote: > I am not arguing with you but... good luck. History has shown that the > argument above is basically this: > > https://en.wikipedia.org/wiki/Sisyphus Hey, parallel query only took three release cycles. :-p -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
El 12/05/16 a las 12:26, Magnus Hagander escribió: > > On May 12, 2016 16:57, "Bruce Momjian" <bruce@momjian.us > <mailto:bruce@momjian.us>> wrote: >> >> On Thu, May 12, 2016 at 10:37:28AM -0400, Robert Haas wrote: >> > 3. I think we need to replace pg_upgrade with a real in-place upgrade >> > scheme so that you just fire up the new version of the server on your >> > old data directory, and it rejiggers things in place without needing >> > to create a new cluster and migrate stuff over to it. I think that >> > actually making this work is a huge engineering effort, and I have no >> > plans to undertake it in the near term, but I think it has to be done. >> > pg_upgrade isn't reliable enough, and using pglogical means you need a >> > second machine. Maybe everybody should run with a standby, but not >> > everyone does. >> >> I don't see why you can't have the pg_logical slave be on the same >> server as the master for an upgrade. It will double the write volume >> while it is active, but assuming it is setup only to perform a major >> version upgrade, it should be fine. >> > > I think that's a pretty bad assumption. A lot, if not most, of the > people who actually need zero downtime upgrades don't have 50% extra > space and in particular not 50% extra performance on their servers to > throw at that. I have the feeling that people who actually *need* zero downtime, have at least 1 standbys they can use to perform the online upgrade. > Can it be made to work? Sure. But I definitely agree with Robert that we > need "real" in place upgrades at some point. It would be a neat feature. The question is if there will be such a solution and if it will be or not reliable. Saludos, -- Martín Marqués http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
On 13/05/16 18:48, Martín Marqués wrote: > El 12/05/16 a las 12:26, Magnus Hagander escribió: > >> Can it be made to work? Sure. But I definitely agree with Robert that we >> need "real" in place upgrades at some point. > > It would be a neat feature. The question is if there will be such a > solution and if it will be or not reliable. > FWIW I agreed with Robert that we need this also. The question is when that will be available (pglogical, londiste and slony are available now). I remember several discussions about this over the years and we still haven't even started on this so I am not going to hold my breath for it. Of course part of the reason why we didn't is that there are logical replication solutions available and pg_upgrade helps quite a bit to alleviate the upgrade pain as well. And honestly even if we had true inplace upgrades I would still prefer upgrade via logical replication for important systems just because it's easier to test it in production while everything is still running, to migrate gradually and also to revert the upgrade back to the original server if something goes wrong (because I needed to do all that in practice before). -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
On Fri, May 13, 2016 at 01:48:39PM -0300, Martín Marqués wrote: > El 12/05/16 a las 12:26, Magnus Hagander escribió: > > Can it be made to work? Sure. But I definitely agree with Robert that we > > need "real" in place upgrades at some point. > > It would be a neat feature. The question is if there will be such a > solution and if it will be or not reliable. My point is that it will require major changes/testing for _every_ major release because there would be substantial upgrade-only code needed for every major release. This is not true for pg_upgrade, and avoiding this is part of its design. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +