Thread: Synchronous replication & Hot standby patches
Hi, Could you please let me know what are the outstanding features that are still to be developed in the respective patches? I'am currently referring the wiki: "Todo and Claim" for NTT and for HotStandby, i see that almost all issues are closed.Are there any features / refactoring / bugs still need to be fixed. Thanks, Niranjan
K, Niranjan (NSN - IN/Bangalore) wrote: > Hi, > > Could you please let me know what are the outstanding features > that are still to be developed in the respective patches? > > I'am currently referring the wiki: "Todo and Claim" for NTT and > for HotStandby, i see that almost all issues are closed. Are > there any features / refactoring / bugs still need to be fixed. Heikki is reviewing the hot standby code; we are trying to figure out how to address the GIN patch, and the SE-Linux patch has gone through several modifications and is waiting on someone to review it again. The hash patch also is waiting for performance testing, I think. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Hi, On Tue, Feb 24, 2009 at 3:47 PM, K, Niranjan (NSN - IN/Bangalore) <niranjan.k@nsn.com> wrote: > Could you please let me know what are the outstanding features that are still to be developed in the respective patches? > > I'am currently referring the wiki: "Todo and Claim" for NTT and for HotStandby, i see that almost all issues are closed.Are there any features / refactoring / bugs still need to be fixed. At least I'm planning to work on the following two items of Synch Rep for v8.5. Of course, Synch Rep works fine without these features. - Address the problem which the signals cannot interrupt poll() and select() on some platforms. http://archives.postgresql.org/pgsql-hackers/2008-12/msg00438.php - Add new feature which transfers all WAL records via the direct connection between the primary and the standby. In otherwords, get rid of file-based log shipping part from the patch. http://archives.postgresql.org/message-id/496B9495.4010902@enterprisedb.com Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center
On Wed, 2009-02-25 at 00:51 +0900, Fujii Masao wrote: > On Tue, Feb 24, 2009 at 3:47 PM, K, Niranjan (NSN - IN/Bangalore) > <niranjan.k@nsn.com> wrote: > > Could you please let me know what are the outstanding features that are still to be developed in the respective patches? > > > > I'am currently referring the wiki: "Todo and Claim" for NTT and for HotStandby, i see that almost all issues are closed.Are there any features / refactoring / bugs still need to be fixed. > > At least I'm planning to work on the following two items of Synch Rep for v8.5. > Of course, Synch Rep works fine without these features. > - Add new feature which transfers all WAL records via the direct connection > between the primary and the standby. In other words, get rid of > file-based log shipping part from the patch. > http://archives.postgresql.org/message-id/496B9495.4010902@enterprisedb.com Please bear in mind my strong objection to this. Attempting to transfer all data via a single connection destroys VLDB usage of this feature. So for me its just additional code for ease-of-use in the simplest case, not code replacement. -- Simon Riggs www.2ndQuadrant.comPostgreSQL Training, Services and Support
Simon Riggs wrote: > On Wed, 2009-02-25 at 00:51 +0900, Fujii Masao wrote: > >> On Tue, Feb 24, 2009 at 3:47 PM, K, Niranjan (NSN - IN/Bangalore) >> <niranjan.k@nsn.com> wrote: >>> Could you please let me know what are the outstanding features that are still to be developed in the respective patches? >>> >>> I'am currently referring the wiki: "Todo and Claim" for NTT and for HotStandby, i see that almost all issues are closed.Are there any features / refactoring / bugs still need to be fixed. >> At least I'm planning to work on the following two items of Synch Rep for v8.5. >> Of course, Synch Rep works fine without these features. > >> - Add new feature which transfers all WAL records via the direct connection >> between the primary and the standby. In other words, get rid of >> file-based log shipping part from the patch. >> http://archives.postgresql.org/message-id/496B9495.4010902@enterprisedb.com > > Please bear in mind my strong objection to this. Attempting to transfer > all data via a single connection destroys VLDB usage of this feature. So > for me its just additional code for ease-of-use in the simplest case, > not code replacement. Presumably we'll keep the capability to restore from a backup and restore from WAL archive as well, when those are available. Keeping that capability shouldn't add many lines of code. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com
On Tue, 2009-02-24 at 17:36 +0000, Simon Riggs wrote: > On Wed, 2009-02-25 at 00:51 +0900, Fujii Masao wrote: > > > On Tue, Feb 24, 2009 at 3:47 PM, K, Niranjan (NSN - IN/Bangalore) > > <niranjan.k@nsn.com> wrote: > > > Could you please let me know what are the outstanding features that are still to be developed in the respective patches? > > > > > > I'am currently referring the wiki: "Todo and Claim" for NTT and for HotStandby, i see that almost all issues are closed.Are there any features / refactoring / bugs still need to be fixed. > > > > At least I'm planning to work on the following two items of Synch Rep for v8.5. > > Of course, Synch Rep works fine without these features. > > > - Add new feature which transfers all WAL records via the direct connection > > between the primary and the standby. In other words, get rid of > > file-based log shipping part from the patch. > > http://archives.postgresql.org/message-id/496B9495.4010902@enterprisedb.com > > Please bear in mind my strong objection to this. Attempting to transfer > all data via a single connection destroys VLDB usage of this feature. So > for me its just additional code for ease-of-use in the simplest case, > not code replacement. Well VLDB is like 2% of what we need. If the above will remove all the B.S. currently associated with actually doing PITR (rsync, scp, nfs, pg_standby pick your poison) then I am all for it. Log shipping should be: I am master, my slave is here. I am slave, I understand my master is here. Here is our mutual authentication love token. Let congress begin. Anything more and we are being difficult for the sake of being difficult. Joshua D. Drake -- PostgreSQL - XMPP: jdrake@jabber.postgresql.org Consulting, Development, Support, Training 503-667-4564 - http://www.commandprompt.com/ The PostgreSQL Company, serving since 1997
On Tue, 2009-02-24 at 10:34 -0800, Joshua D. Drake wrote: > Well VLDB is like 2% of what we need. I am against removing an existing capability that is important to some users. We shouldn't need to debate the exact percentage of users that would be affected, or how to count them. -- Simon Riggs www.2ndQuadrant.comPostgreSQL Training, Services and Support
On Tue, 2009-02-24 at 18:48 +0000, Simon Riggs wrote: > On Tue, 2009-02-24 at 10:34 -0800, Joshua D. Drake wrote: > > > Well VLDB is like 2% of what we need. > > I am against removing an existing capability that is important to some > users. We shouldn't need to debate the exact percentage of users that > would be affected, or how to count them. See Heikki's comment. Joshua D. Drake > > -- > Simon Riggs www.2ndQuadrant.com > PostgreSQL Training, Services and Support > > -- PostgreSQL - XMPP: jdrake@jabber.postgresql.org Consulting, Development, Support, Training 503-667-4564 - http://www.commandprompt.com/ The PostgreSQL Company, serving since 1997
Simon Riggs wrote: > On Tue, 2009-02-24 at 10:34 -0800, Joshua D. Drake wrote: > > >> Well VLDB is like 2% of what we need. >> > > I am against removing an existing capability that is important to some > users. We shouldn't need to debate the exact percentage of users that > would be affected, or how to count them. > > Perhaps so, but I would hope you would support what Heikki and others have been talking about as an option for replication. The 2% shouldn't hold back the remaining 98%. cheers andrew
>>> Andrew Dunstan <andrew@dunslane.net> wrote: > Simon Riggs wrote: >> On Tue, 2009-02-24 at 10:34 -0800, Joshua D. Drake wrote: >> >>> Well VLDB is like 2% of what we need. >> >> I am against removing an existing capability that is important to >> some users. We shouldn't need to debate the exact percentage of >> users that would be affected, or how to count them. > > Perhaps so, but I would hope you would support what Heikki and > others have been talking about as an option for replication. The 2% > shouldn't hold back the remaining 98%. Right. I was made a bit nervous by Joshua's comments, but somewhat reassured by his reference back to Heikki's comments. If we can make common cases simple to implement, that's great, as long as we don't lose functionality needed to cover the more complex cases. >>> Anything more and we are being difficult for the sake of being >>> difficult. Even in context, that came off as a bit user-hostile. It probably wasn't meant that way, but it sounded provocative to me. We do have better reasons than that for what I mentioned (in simplified form) here: http://archives.postgresql.org/pgsql-hackers/2009-01/msg00740.php If the point was that we should not require anything more for configuring a common, simple case then I'd agree. -Kevin
Hi, On Wed, Feb 25, 2009 at 2:37 AM, Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> wrote: > Presumably we'll keep the capability to restore from a backup and restore > from WAL archive as well, when those are available. Keeping that capability > shouldn't add many lines of code. Yes, I assume that only missing WAL files (probably including .backup file) which are not in archive area of the standby server need to be transferred. If there are many missing files, we can also copy them via multiple connection before starting the standby. Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center
On Tue, 2009-02-24 at 13:25 -0600, Kevin Grittner wrote: > Right. I was made a bit nervous by Joshua's comments, but somewhat > reassured by his reference back to Heikki's comments. If we can make > common cases simple to implement, that's great, as long as we don't > lose functionality needed to cover the more complex cases. Right. That is my only point. I am not looking to "dumb down" postgresql. I am saying that the "default" solution should be that simple. Autovacuum is the same way for example. The "default" solution works for 95% of the users out there without having to tweak any knob. The knobs are there *if* they break out of the 95%. > > >>> Anything more and we are being difficult for the sake of being > >>> difficult. > > Even in context, that came off as a bit user-hostile. It probably > wasn't meant that way, but it sounded provocative to me. Sorry I am in rare form today. I wasn't trying to be provocative at all but was trying to be succinctly clear on what my belief about this is. > > If the point was that we should not require anything more for > configuring a common, simple case then I'd agree. Right. Joshua D. Drake -- PostgreSQL - XMPP: jdrake@jabber.postgresql.org Consulting, Development, Support, Training 503-667-4564 - http://www.commandprompt.com/ The PostgreSQL Company, serving since 1997
On Tue, 2009-02-24 at 13:53 -0500, Andrew Dunstan wrote: > > Simon Riggs wrote: > > On Tue, 2009-02-24 at 10:34 -0800, Joshua D. Drake wrote: > > > > > >> Well VLDB is like 2% of what we need. > >> > > > > I am against removing an existing capability that is important to some > > users. We shouldn't need to debate the exact percentage of users that > > would be affected, or how to count them. > > > Perhaps so, but I would hope you would support what Heikki and others > have been talking about as an option for replication. The 2% shouldn't > hold back the remaining 98%. So far, everything has been couched in terms of remove the way it is now and put in its place something "better". Heikki and Josh have said that or similar, as has Robert Haas on another thread, and Fujii-san specifically said "get rid of" the existing functionality. I am completely against the removal of an existing capability that is critically important to many users. If we can add new functionality that is a nice-to-have for a large number of people without removing a feature that is critical to many users, bring it on. If we can't do that, then I would oppose. -- Simon Riggs www.2ndQuadrant.comPostgreSQL Training, Services and Support
Hi, On Wed, Feb 25, 2009 at 5:08 AM, Simon Riggs <simon@2ndquadrant.com> wrote: > So far, everything has been couched in terms of remove the way it is now > and put in its place something "better". Heikki and Josh have said that > or similar, as has Robert Haas on another thread, and Fujii-san > specifically said "get rid of" the existing functionality. I am > completely against the removal of an existing capability that is > critically important to many users. Though my description may be confusing, I don't mean to remove any existing features unless necessary. I just said get rid of file-based log shipping part only from my patch. > If we can add new functionality that is a nice-to-have for a large > number of people without removing a feature that is critical to many > users, bring it on. If we can't do that, then I would oppose. Yes, I also think so. Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center
On Tue, Feb 24, 2009 at 3:08 PM, Simon Riggs <simon@2ndquadrant.com> wrote: > On Tue, 2009-02-24 at 13:53 -0500, Andrew Dunstan wrote: >> Simon Riggs wrote: >> > On Tue, 2009-02-24 at 10:34 -0800, Joshua D. Drake wrote: >> >> Well VLDB is like 2% of what we need. >> > I am against removing an existing capability that is important to some >> > users. We shouldn't need to debate the exact percentage of users that >> > would be affected, or how to count them. >> Perhaps so, but I would hope you would support what Heikki and others >> have been talking about as an option for replication. The 2% shouldn't >> hold back the remaining 98%. > So far, everything has been couched in terms of remove the way it is now > and put in its place something "better". Heikki and Josh have said that > or similar, as has Robert Haas on another thread, and Fujii-san > specifically said "get rid of" the existing functionality. I am > completely against the removal of an existing capability that is > critically important to many users. I didn't think I had proposed any such thing, although maybe I'm just not remembering. I'm pretty confused as to what the current thread is all about. It seems to me that in previous discussions of Streaming Replication, Heikki put forward the proposition that the standby server should be able to connect to the primary and stream not only newly-generated WAL but also, if necessary, a base backup. As I recall, he argued that without this functionality Streaming Replication would be far too difficult to administer for the majority of users. Assuming I'm representing his position more or less accurately, I completely agree with it. I don't object to providing other mechanisms as well, but if it's not about as simple as pointing the secondary at the primary and saying "go", it's probably more complicated than I want to mess around with. ...Robert
Hi, On Wed, Feb 25, 2009 at 6:52 AM, Robert Haas <robertmhaas@gmail.com> wrote: > It seems to me that in previous discussions of Streaming Replication, > Heikki put forward the proposition that the standby server should be > able to connect to the primary and stream not only newly-generated WAL > but also, if necessary, a base backup. As I recall, he argued that > without this functionality Streaming Replication would be far too > difficult to administer for the majority of users. The automatic transfer of a base backup is useful, but not essential. So, I'm not planning to work on it until synch rep patch is committed at least. IIRC, nobody argues that such a feature is indispensable also to the first step. We should develop synch rep in stages? Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center
On Tue, 2009-02-24 at 16:52 -0500, Robert Haas wrote: > I didn't think I had proposed any such thing, although maybe I'm just > not remembering. I'm pretty confused as to what the current thread is > all about. http://archives.postgresql.org/pgsql-hackers/2009-01/msg00978.php I don't think anyone who argued in favour of removal of existing system was aware that we'd lose anything as a result. I think everybody supports the easier-if-possible sentiment that Heikki was expressing; I just don't want to let that be seen as agreement to remove, by default, at a later time. -- Simon Riggs www.2ndQuadrant.comPostgreSQL Training, Services and Support
Bruce Momjian wrote: > K, Niranjan (NSN - IN/Bangalore) wrote: >> Hi, >> >> Could you please let me know what are the outstanding features >> that are still to be developed in the respective patches? >> >> I'am currently referring the wiki: "Todo and Claim" for NTT and >> for HotStandby, i see that almost all issues are closed. Are >> there any features / refactoring / bugs still need to be fixed. > > Heikki is reviewing the hot standby code; we are trying to figure out > how to address the GIN patch, and the SE-Linux patch has gone through > several modifications and is waiting on someone to review it again. I have updated my patch set a few times to fix bugs since I submitted the revision without row-level facilities (r1522) at this Feb.06. Was it unconfortable for you? These changes are just only bugfixes, so it is not necessary whole of patches again. Or, does the "several modifications" mean the separation of row-level facilities and so on? Thanks, -- OSS Platform Development Division, NEC KaiGai Kohei <kaigai@ak.jp.nec.com>
On Tue, Feb 24, 2009 at 5:58 PM, Simon Riggs <simon@2ndquadrant.com> wrote: > On Tue, 2009-02-24 at 16:52 -0500, Robert Haas wrote: > >> I didn't think I had proposed any such thing, although maybe I'm just >> not remembering. I'm pretty confused as to what the current thread is >> all about. > > http://archives.postgresql.org/pgsql-hackers/2009-01/msg00978.php I still don't see where I suggested removing anything. What Heikki suggested, and I agreed with, was adding something: integrated base backup. > I don't think anyone who argued in favour of removal of existing system > was aware that we'd lose anything as a result. I think everybody > supports the easier-if-possible sentiment that Heikki was expressing; I > just don't want to let that be seen as agreement to remove, by default, > at a later time. I'm still totally unclear as to what you think anyone might, at some point in the future, propose to remove. I think the more relevant question right now is whether the work Fujii Masao is planning to do for 8.5 is reponsive to the following comment from Heikki: # IMHO, the synchronous replication isn't in such good shape, I'm afraid. I've said # this before, but I'm not happy with the "built from spare parts" nature of it. You # shouldn't have to configure an archive, file-based log shipping using rsync or # whatever, and pg_standby. All that is in addition to the direct connection between # master and slave. The slave really should be able to just connect to the master, and # download all the WAL it needs directly. That's a huge usability issue if left as is, # but requires very large architectural changes to fix. ...Robert
Robert Haas wrote: > I think the more relevant question right now is whether the work Fujii > Masao is planning to do for 8.5 is reponsive to the following comment > from Heikki: > > # IMHO, the synchronous replication isn't in such good shape, I'm > afraid. I've said > # this before, but I'm not happy with the "built from spare parts" > nature of it. You > # shouldn't have to configure an archive, file-based log shipping using rsync or > # whatever, and pg_standby. All that is in addition to the direct > connection between > # master and slave. The slave really should be able to just connect to > the master, and > # download all the WAL it needs directly. That's a huge usability > issue if left as is, > # but requires very large architectural changes to fix. I believe so, see second bullet point in: http://archives.postgresql.org/message-id/3f0b79eb0902240751t13231593g17fbef70664d4444@mail.gmail.com -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com
On Wed, 2009-02-25 at 22:45 +0200, Heikki Linnakangas wrote: > Robert Haas wrote: > > I think the more relevant question right now is whether the work Fujii > > Masao is planning to do for 8.5 is reponsive to the following comment > > from Heikki: > > > > # IMHO, the synchronous replication isn't in such good shape, I'm > > afraid. I've said > > # this before, but I'm not happy with the "built from spare parts" > > nature of it. You > > # shouldn't have to configure an archive, file-based log shipping using rsync or > > # whatever, and pg_standby. All that is in addition to the direct > > connection between > > # master and slave. The slave really should be able to just connect to > > the master, and > > # download all the WAL it needs directly. That's a huge usability > > issue if left as is, > > # but requires very large architectural changes to fix. > > I believe so, see second bullet point in: > > http://archives.postgresql.org/message-id/3f0b79eb0902240751t13231593g17fbef70664d4444@mail.gmail.com That is exactly what I am against. Note the words "get rid of". This prevents parallel data transfer, use of split mirrors and various other techniques. It sounds neater, but it implies removal of useful features. -- Simon Riggs www.2ndQuadrant.comPostgreSQL Training, Services and Support
Simon Riggs wrote: > On Wed, 2009-02-25 at 22:45 +0200, Heikki Linnakangas wrote: > >> Robert Haas wrote: >> >>> I think the more relevant question right now is whether the work Fujii >>> Masao is planning to do for 8.5 is reponsive to the following comment >>> from Heikki: >>> >>> # IMHO, the synchronous replication isn't in such good shape, I'm >>> afraid. I've said >>> # this before, but I'm not happy with the "built from spare parts" >>> nature of it. You >>> # shouldn't have to configure an archive, file-based log shipping using rsync or >>> # whatever, and pg_standby. All that is in addition to the direct >>> connection between >>> # master and slave. The slave really should be able to just connect to >>> the master, and >>> # download all the WAL it needs directly. That's a huge usability >>> issue if left as is, >>> # but requires very large architectural changes to fix. >>> >> I believe so, see second bullet point in: >> >> http://archives.postgresql.org/message-id/3f0b79eb0902240751t13231593g17fbef70664d4444@mail.gmail.com >> > > That is exactly what I am against. Note the words "get rid of". > > This prevents parallel data transfer, use of split mirrors and various > other techniques. It sounds neater, but it implies removal of useful > features. > OK, so let's assume that we'll provide an extra facility that doesn't take anything away but which provides for close to zero config setup for the simple case. Frankly, that's what the vast majority of people want, in my experience. cheers andrew
Hi, On Thu, Feb 26, 2009 at 6:03 AM, Simon Riggs <simon@2ndquadrant.com> wrote: > That is exactly what I am against. Note the words "get rid of". > > This prevents parallel data transfer, use of split mirrors and various > other techniques. It sounds neater, but it implies removal of useful > features. OK, ISTM that my description was confusing you, so I removed that statement from the TODO item on wiki. http://wiki.postgresql.org/wiki/NTT%27s_Development_Projects#Todo_and_Claim Again, I'm not planning to get rid of any existing capabilities unless necessary. Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center
On Thu, 2009-02-26 at 06:15 +0900, Fujii Masao wrote: > Hi, > > On Thu, Feb 26, 2009 at 6:03 AM, Simon Riggs <simon@2ndquadrant.com> wrote: > > That is exactly what I am against. Note the words "get rid of". > > > > This prevents parallel data transfer, use of split mirrors and various > > other techniques. It sounds neater, but it implies removal of useful > > features. > > OK, ISTM that my description was confusing you, so I removed that statement > from the TODO item on wiki. > http://wiki.postgresql.org/wiki/NTT%27s_Development_Projects#Todo_and_Claim > > Again, I'm not planning to get rid of any existing capabilities Good > unless necessary. That is not a caveat I will accept, a priori. -- Simon Riggs www.2ndQuadrant.comPostgreSQL Training, Services and Support
On Wed, 2009-02-25 at 16:11 -0500, Andrew Dunstan wrote: > OK, so let's assume that we'll provide an extra facility that doesn't > take anything away but which provides for close to zero config setup for > the simple case. Frankly, that's what the vast majority of people want, > in my experience. My experience also. Very much agreed. -- Simon Riggs www.2ndQuadrant.comPostgreSQL Training, Services and Support
On Wed, Feb 25, 2009 at 3:45 PM, Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> wrote: > I believe so, see second bullet point in: > > http://archives.postgresql.org/message-id/3f0b79eb0902240751t13231593g17fbef70664d4444@mail.gmail.com Cool. ...Robert
Simon Riggs wrote: > > On Thu, 2009-02-26 at 06:15 +0900, Fujii Masao wrote: > > Hi, > > > > On Thu, Feb 26, 2009 at 6:03 AM, Simon Riggs <simon@2ndquadrant.com> wrote: > > > That is exactly what I am against. Note the words "get rid of". > > > > > > This prevents parallel data transfer, use of split mirrors and various > > > other techniques. It sounds neater, but it implies removal of useful > > > features. > > > > OK, ISTM that my description was confusing you, so I removed that statement > > from the TODO item on wiki. > > http://wiki.postgresql.org/wiki/NTT%27s_Development_Projects#Todo_and_Claim > > > > Again, I'm not planning to get rid of any existing capabilities > > Good > > > unless necessary. > > That is not a caveat I will accept, a priori. What does "accept" mean above? Are you the sole acceptor for this feature? That is surprising to me. You can say you would vote against it but your wording above seems overly controlling. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
On Wed, 2009-02-25 at 17:50 -0500, Bruce Momjian wrote: > Simon Riggs wrote: > > On Thu, 2009-02-26 at 06:15 +0900, Fujii Masao wrote: > > > > > > Again, I'm not planning to get rid of any existing capabilities > > > > Good > > > > > unless necessary. > > > > That is not a caveat I will accept, a priori. > > What does "accept" mean above? Are you the sole acceptor for this > feature? That is surprising to me. > > You can say you would vote against it but your wording above seems > overly controlling. I would hope my words carry the same weight as others when people speak of what can and cannot be included, when backed by reasonable and logical technical reasons. Some things are important, some not, and I've done my best to use words that indicate my views on that. -- Simon Riggs www.2ndQuadrant.comPostgreSQL Training, Services and Support
Simon Riggs wrote: > > On Wed, 2009-02-25 at 17:50 -0500, Bruce Momjian wrote: > > Simon Riggs wrote: > > > On Thu, 2009-02-26 at 06:15 +0900, Fujii Masao wrote: > > > > > > > > Again, I'm not planning to get rid of any existing capabilities > > > > > > Good > > > > > > > unless necessary. > > > > > > That is not a caveat I will accept, a priori. > > > > What does "accept" mean above? Are you the sole acceptor for this > > feature? That is surprising to me. > > > > You can say you would vote against it but your wording above seems > > overly controlling. > > I would hope my words carry the same weight as others when people speak > of what can and cannot be included, when backed by reasonable and > logical technical reasons. > > Some things are important, some not, and I've done my best to use words > that indicate my views on that. I think you need to work on improving your word choice then. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
>>> Bruce Momjian <bruce@momjian.us> wrote: > Simon Riggs wrote: >> On Wed, 2009-02-25 at 17:50 -0500, Bruce Momjian wrote: >> > Simon Riggs wrote: >> > > On Thu, 2009-02-26 at 06:15 +0900, Fujii Masao wrote: >> > > > >> > > > Again, I'm not planning to get rid of any existing >> > > > capabilities >> > > >> > > Good >> > > >> > > > unless necessary. >> > > >> > > That is not a caveat I will accept, a priori. >> > >> > What does "accept" mean above? Are you the sole acceptor for >> > this feature? That is surprising to me. >> > >> > You can say you would vote against it but your wording above >> > seems overly controlling. >> >> I would hope my words carry the same weight as others when people >> speak of what can and cannot be included, when backed by reasonable >> and logical technical reasons. >> >> Some things are important, some not, and I've done my best to use >> words that indicate my views on that. > > I think you need to work on improving your word choice then. Anything which breaks existing techniques for log shipping, PITR recovery, or warm standby would present a significant obstacle to adoption in our environment. Based on off-list emails received when I mentioned what we do, I know we're not alone. Some comments have made me nervous on this count, but responses to questions about it have generally been reassuring. The "unless necessary" seemed a little ominous. I won't get too excited unless someone thinks a change is necessary and I see the nature of the change. As long as there is some way to ship base backups and WAL files to multiple targets, not all of which are running any PostgreSQL software, and the database can be restored from such copies, we're not looking a more than a few weeks of staff time to adjust our scripts and procedures. -Kevin
On Thu, 2009-02-26 at 11:02 -0500, Bruce Momjian wrote: > Simon Riggs wrote: > > > > On Wed, 2009-02-25 at 17:50 -0500, Bruce Momjian wrote: > > > Simon Riggs wrote: > > > > On Thu, 2009-02-26 at 06:15 +0900, Fujii Masao wrote: > > > > > > > > > > Again, I'm not planning to get rid of any existing capabilities > > > > > > > > Good > > > > > > > > > unless necessary. > > > > > > > > That is not a caveat I will accept, a priori. > > > > > > What does "accept" mean above? Are you the sole acceptor for this > > > feature? That is surprising to me. > > > > > > You can say you would vote against it but your wording above seems > > > overly controlling. > > > > I would hope my words carry the same weight as others when people speak > > of what can and cannot be included, when backed by reasonable and > > logical technical reasons. > > > > Some things are important, some not, and I've done my best to use words > > that indicate my views on that. > > I think you need to work on improving your word choice then. I vote against this proposal. -- Simon Riggs www.2ndQuadrant.comPostgreSQL Training, Services and Support
On Tue, 2009-02-24 at 10:34 -0800, Joshua D. Drake wrote: > On Tue, 2009-02-24 at 17:36 +0000, Simon Riggs wrote: > > On Wed, 2009-02-25 at 00:51 +0900, Fujii Masao wrote: > > > > > On Tue, Feb 24, 2009 at 3:47 PM, K, Niranjan (NSN - IN/Bangalore) > > > <niranjan.k@nsn.com> wrote: > > > > Could you please let me know what are the outstanding features that are still to be developed in the respective patches? > > > > > > > > I'am currently referring the wiki: "Todo and Claim" for NTT and for HotStandby, i see that almost all issues areclosed. Are there any features / refactoring / bugs still need to be fixed. > > > > > > At least I'm planning to work on the following two items of Synch Rep for v8.5. > > > Of course, Synch Rep works fine without these features. > > > > > - Add new feature which transfers all WAL records via the direct connection > > > between the primary and the standby. In other words, get rid of > > > file-based log shipping part from the patch. > > > http://archives.postgresql.org/message-id/496B9495.4010902@enterprisedb.com > > > > Please bear in mind my strong objection to this. Attempting to transfer > > all data via a single connection destroys VLDB usage of this feature. So > > for me its just additional code for ease-of-use in the simplest case, > > not code replacement. > > Well VLDB is like 2% of what we need. If the above will remove all the > B.S. currently associated with actually doing PITR (rsync, scp, nfs, > pg_standby pick your poison) then I am all for it. If you use walmgr.py, then all you need is writing a conf file and making sure that ssh and rsync work. Actually the best way to do Sync Rep would have been to just move to C what walmgr.py does. That the patch could have started off from a well-tested foundation. > Log shipping should be: > > I am master, my slave is here. > I am slave, I understand my master is here. > Here is our mutual authentication love token. > Let congress begin. > > Anything more and we are being difficult for the sake of being > difficult. Actually I'd leave out the first line, and start with just - I am slave, my master accepts me, start replicationg So there could be several slaves, of both hot standby postgresql, wal-file-store and store-and-forward-to-many types. -- Hannu Krosing http://www.2ndQuadrant.com PostgreSQL Scalability and Availability Services, Consulting and Training
On Fri, 2009-02-27 at 22:17 +0200, Hannu Krosing wrote: > > Well VLDB is like 2% of what we need. If the above will remove all the > > B.S. currently associated with actually doing PITR (rsync, scp, nfs, > > pg_standby pick your poison) then I am all for it. > > If you use walmgr.py, then all you need is writing a conf file and > making sure that ssh and rsync work. > > Actually the best way to do Sync Rep would have been to just move to C > what walmgr.py does. That the patch could have started off from a > well-tested foundation. > For sync? > > Log shipping should be: > > > > I am master, my slave is here. > > I am slave, I understand my master is here. > > Here is our mutual authentication love token. > > Let congress begin. > > > > Anything more and we are being difficult for the sake of being > > difficult. > > Actually I'd leave out the first line, and start with just > > - I am slave, my master accepts me, start replicationg > Heh fair enough. > So there could be several slaves, of both hot standby postgresql, > wal-file-store and store-and-forward-to-many types. THat should be optional not the default. Joshua D. Drake -- PostgreSQL - XMPP: jdrake@jabber.postgresql.org Consulting, Development, Support, Training 503-667-4564 - http://www.commandprompt.com/ The PostgreSQL Company, serving since 1997
On Fri, 2009-02-27 at 12:21 -0800, Joshua D. Drake wrote: > On Fri, 2009-02-27 at 22:17 +0200, Hannu Krosing wrote: > > > > Well VLDB is like 2% of what we need. If the above will remove all the > > > B.S. currently associated with actually doing PITR (rsync, scp, nfs, > > > pg_standby pick your poison) then I am all for it. > > > > If you use walmgr.py, then all you need is writing a conf file and > > making sure that ssh and rsync work. > > > > Actually the best way to do Sync Rep would have been to just move to C > > what walmgr.py does. That the patch could have started off from a > > well-tested foundation. > > > > For sync? For everything up to starting sync. It somehow feels like current patch concentrated mainly on doing the sync part and the need to get also existing stete over automatically became as an afterthought. That's why I'm proposing to start (at least conceptually) from an existing, fully automated and working solution. Currently walmgr.py is doing everything from setting up replica to getting up-to-last-second changes to slave's disk. > > > Log shipping should be: > > > > > > I am master, my slave is here. > > > I am slave, I understand my master is here. > > > Here is our mutual authentication love token. > > > Let congress begin. > > > > > > Anything more and we are being difficult for the sake of being > > > difficult. > > > > Actually I'd leave out the first line, and start with just > > > > - I am slave, my master accepts me, start replicationg > > > > Heh fair enough. > > > So there could be several slaves, of both hot standby postgresql, > > wal-file-store and store-and-forward-to-many types. > > That should be optional not the default. Of course. One slave is a sub-case of "any number of slaves" :) But doing 1-1 replica _only_ would make us laughing stock for everybody else. -- Hannu Krosing http://www.2ndQuadrant.com PostgreSQL Scalability and Availability Services, Consulting and Training
Hannu Krosing wrote: > Currently walmgr.py is doing everything from setting up replica to > getting up-to-last-second changes to slave's disk. > > If walmgr.py and its cousins had good documentation there would possibly be much greater acceptance of them. cheers andrew
On Sat, 2009-02-28 at 23:21 -0800, Josh Berkus wrote: > Fujii, > > >> Again, I'm not planning to get rid of any existing capabilities > > > > Good > > > >> unless necessary. > > > > That is not a caveat I will accept, a priori. > > While Simon stated it a bit strongly My intention was only to be clear about how important a technical point it was for me. My comments were aimed at avoiding a costly blind alley, not critically towards any individual. I am happy to apologise in case people thought my words rude. -- Simon Riggs www.2ndQuadrant.comPostgreSQL Training, Services and Support
Fujii, >> Again, I'm not planning to get rid of any existing capabilities > > Good > >> unless necessary. > > That is not a caveat I will accept, a priori. While Simon stated it a bit strongly, I think it's important that you alert people if you think you have to remove existing features in order to make easy standby possible. It's possible that features which seem trivial to you are used extensively by people with particular failover requirements. --Josh
Hi, On Sun, Mar 1, 2009 at 4:21 PM, Josh Berkus <josh@agliodbs.com> wrote: > While Simon stated it a bit strongly, I think it's important that you alert > people if you think you have to remove existing features in order to make > easy standby possible. Now, I think that any existing capabilities don't need to be removed for Synch Rep. > It's possible that features which seem trivial to > you are used extensively by people with particular failover requirements. Of course, I have no intention of ignoring such people. Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center