Thread: What's planned for 7.5?
Hi, Is this all that's planned for 7.5? (based on current TODO list) -Change factorial to return a numeric (Gavin) -COMMENT ON [ CAST | CONVERSION | OPERATOR CLASS | LARGE OBJECT | LANGUAGE ] (Christopher) -Have psql \dn show only visible temp schemas using current_schemas() -Have psql '\i ~/<tab><tab>' actually load files it displays from home dir -Allow psql \du to show groups, and add \dg for groups -Allow pg_dump to dump CREATE CONVERSION (Christopher) -Use dependency information to dump data in proper order -Use background process to write dirty shared buffers to disk Thanks __________________________________ Do you Yahoo!? Yahoo! Hotjobs: Enter the "Signing Bonus" Sweepstakes http://hotjobs.sweepstakes.yahoo.com/signingbonus
Native Win32 is planned for it (whether it makes it or not is another question, but it is the goal) ... On Mon, 12 Jan 2004, ow wrote: > Hi, > > Is this all that's planned for 7.5? (based on current TODO list) > > -Change factorial to return a numeric (Gavin) > -COMMENT ON [ CAST | CONVERSION | OPERATOR CLASS | LARGE OBJECT | LANGUAGE ] > (Christopher) > -Have psql \dn show only visible temp schemas using current_schemas() > -Have psql '\i ~/<tab><tab>' actually load files it displays from home dir > -Allow psql \du to show groups, and add \dg for groups > -Allow pg_dump to dump CREATE CONVERSION (Christopher) > -Use dependency information to dump data in proper order > -Use background process to write dirty shared buffers to disk > > Thanks > > > > > > > __________________________________ > Do you Yahoo!? > Yahoo! Hotjobs: Enter the "Signing Bonus" Sweepstakes > http://hotjobs.sweepstakes.yahoo.com/signingbonus > > ---------------------------(end of broadcast)--------------------------- > TIP 6: Have you searched our list archives? > > http://archives.postgresql.org > ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
Mensaje citado por "Marc G. Fournier" <scrappy@postgresql.org>: > > Native Win32 is planned for it (whether it makes it or not is another > question, but it is the goal) ... Replication wasn't another BIG one? -- select 'mmarques' || '@' || 'unl.edu.ar' AS email; --------------------------------------------------------- Martín Marqués | Programador, DBA Centro de Telemática | Administrador Universidad Nacional del Litoral ---------------------------------------------------------
Mensaje citado por ow <oneway_111@yahoo.com>: > Hi, > > Is this all that's planned for 7.5? (based on current TODO list) > > -Change factorial to return a numeric (Gavin) > -COMMENT ON [ CAST | CONVERSION | OPERATOR CLASS | LARGE OBJECT | LANGUAGE ] > (Christopher) > -Have psql \dn show only visible temp schemas using current_schemas() > -Have psql '\i ~/<tab><tab>' actually load files it displays from home dir > -Allow psql \du to show groups, and add \dg for groups > -Allow pg_dump to dump CREATE CONVERSION (Christopher) > -Use dependency information to dump data in proper order > -Use background process to write dirty shared buffers to disk For what I have just seen in Robert Treats "PostgreSQL Weekly News" most of these issues are already solved in the CVS, and I guess that in a bit less than a year that's left for the release of 7.5 many new things will appear. Anyway, from http://developer.postgresql.org/todo.php I see in the URGENT part this: # Add replication of distributed databases [replication] * Automatic failover * Load balancing * Master/slave replication * Multi-master replication * Partition data acrossservers * Queries across databases or servers (two-phase commit) * Allow replication over unreliable or non-persistentlinks * http://gborg.postgresql.org/project/pgreplication/projdisplay.php # Point-in-time data recovery using backup and write-ahead log # Create native Win32 port, * http://momjian.postgresql.org/main/writings/pgsql/win32.html The first isn't new, but as for what I know, it's a new aproach that Jan has for adding replication to the server. The other 2 didn't get in the 7.4 release. -- select 'mmarques' || '@' || 'unl.edu.ar' AS email; --------------------------------------------------------- Martín Marqués | Programador, DBA Centro de Telemática | Administrador Universidad Nacional del Litoral ---------------------------------------------------------
On Mon, 12 Jan 2004, Martin Marques wrote: > Mensaje citado por "Marc G. Fournier" <scrappy@postgresql.org>: > > > > > Native Win32 is planned for it (whether it makes it or not is another > > question, but it is the goal) ... > > Replication wasn't another BIG one? Actually, I think it was PITR (Point in Time Recovery). Of course, since that works by replaying log files into the database, it's a pretty good way of setting up replication, which would likely follow a release or so after PITR was implemented. OTOH, there are some replication solutions already available, just no as part of the core.
On Mon, 12 Jan 2004, Martin Marques wrote: > Mensaje citado por ow <oneway_111@yahoo.com>: > > > Hi, > > > > Is this all that's planned for 7.5? (based on current TODO list) > > > > -Change factorial to return a numeric (Gavin) > > -COMMENT ON [ CAST | CONVERSION | OPERATOR CLASS | LARGE OBJECT | LANGUAGE ] > > (Christopher) > > -Have psql \dn show only visible temp schemas using current_schemas() > > -Have psql '\i ~/<tab><tab>' actually load files it displays from home dir > > -Allow psql \du to show groups, and add \dg for groups > > -Allow pg_dump to dump CREATE CONVERSION (Christopher) > > -Use dependency information to dump data in proper order > > -Use background process to write dirty shared buffers to disk > > For what I have just seen in Robert Treats "PostgreSQL Weekly News" most of > these issues are already solved in the CVS, and I guess that in a bit less than > a year that's left for the release of 7.5 many new things will appear. > > Anyway, from http://developer.postgresql.org/todo.php I see in the URGENT part > this: > > # Add replication of distributed databases [replication] > > * Automatic failover > * Load balancing > * Master/slave replication > * Multi-master replication > * Partition data across servers > * Queries across databases or servers (two-phase commit) > * Allow replication over unreliable or non-persistent links > * http://gborg.postgresql.org/project/pgreplication/projdisplay.php > > # Point-in-time data recovery using backup and write-ahead log > # Create native Win32 port, > * http://momjian.postgresql.org/main/writings/pgsql/win32.html > > The first isn't new, but as for what I know, it's a new aproach that Jan has for > adding replication to the server. Nope, pgreplication has nothing to do with Jan ... he's working on Slonik-1 ... ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
martin@bugs.unl.edu.ar (Martin Marques) writes: > Mensaje citado por "Marc G. Fournier" <scrappy@postgresql.org>: >> Native Win32 is planned for it (whether it makes it or not is another >> question, but it is the goal) ... > > Replication wasn't another BIG one? Well, there are two "replication things" going on: 1. The eRserv Java server software is already available on gBorg. It isn't likely to go into the software distribution any time soon because of its dependence on Java. It really isan extra add-on. 2. Jan Wieck's work on SLONY-1 Many are keen to see the code, but it's not out yet. And it is not self-evident that it will necessarily be readyto release in conjunction with 7.5 (Not to say it _can't_ happen, but just that we'll see the code when we seeit...) It is planned to be implemented in C, so it would presumably be more suitable for inclusion in the main code thaneRserv. But it stands as "hope," and not certainty. And none of this is ... 3. PITR (Point In Time Recovery). Where a characteristic "hope" would be to archive WAL files and use them to bring a failed DB up to date. Jan has plans to do something similar but _much_ less obscure/opaque with SLONY-1. We'll see what happens... Another fairly important item not mentioned: Nested Transactions. -- let name="cbbrowne" and tld="libertyrms.info" in String.concat "@" [name;tld];; <http://dev6.int.libertyrms.com/> Christopher Browne (416) 646 3304 x124 (land)
ow <oneway_111@yahoo.com> writes: > Is this all that's planned for 7.5? (based on current TODO list) If you think the TODO list has *anything* to do with release planning, you fundamentally misunderstand the way things are done around here. The TODO list is a list of things that are widely agreed to be problems (though sometimes it only means "Bruce thinks this is a problem") and therefore will probably get worked on at some point in the future. What actually gets done for 7.5 will depend on which itches bother which developers enough to get scratched in the next few months. We do not have any central planning. It is a safe bet that 7.5 will include some things mentioned in the present TODO, as well as many things not mentioned in it; and that many items mentioned in the present TODO will still remain undone. regards, tom lane
"Stephen" <jleelim@xxxxxxx.com> writes: > Any chance we'll see the VACUUM delay patch (throttle) get into 7.5? I only > had the chance to try the first patch by Tom Lane and it was very good > already. I was hoping it gets into 7.4.1 but it didn't. :-( > > I really need the VACUUM delay patch because my servers are begging to die > every time VACUUM runs. Please let it be in 7.5. The hope, in 7.5, is to have ARC, which is the "super-duper-duper" version, working. The delay patch is simple enough to add in if you need it :-). -- let name="cbbrowne" and tld="libertyrms.info" in String.concat "@" [name;tld];; <http://dev6.int.libertyrms.com/> Christopher Browne (416) 646 3304 x124 (land)
Christopher Browne <cbbrowne@libertyrms.info> writes: > "Stephen" <jleelim@xxxxxxx.com> writes: >> Any chance we'll see the VACUUM delay patch (throttle) get into 7.5? > The hope, in 7.5, is to have ARC, which is the "super-duper-duper" > version, working. Actually, I'm not sure that ARC should be considered to supersede the usefulness of a per-page delay in VACUUM. ARC should prevent VACUUM from trashing the contents of Postgres' shared buffer arena, but it won't do much of anything to prevent VACUUM from trashing the kernel buffer contents. And it definitely won't do anything to help if the real problem is that you're short of disk bandwidth and VACUUM's extra I/O demand pushes your total load over the knee of the response-time curve. What you need then is a throttle. The original patch I posted was incomplete for a number of reasons, but I think it may still be worth working on. Jan, any comments? regards, tom lane
Tom Lane wrote: > Christopher Browne <cbbrowne@libertyrms.info> writes: >> "Stephen" <jleelim@xxxxxxx.com> writes: >>> Any chance we'll see the VACUUM delay patch (throttle) get into 7.5? > >> The hope, in 7.5, is to have ARC, which is the "super-duper-duper" >> version, working. > > Actually, I'm not sure that ARC should be considered to supersede the > usefulness of a per-page delay in VACUUM. ARC should prevent VACUUM > from trashing the contents of Postgres' shared buffer arena, but it > won't do much of anything to prevent VACUUM from trashing the kernel > buffer contents. And it definitely won't do anything to help if the > real problem is that you're short of disk bandwidth and VACUUM's extra > I/O demand pushes your total load over the knee of the response-time > curve. What you need then is a throttle. > > The original patch I posted was incomplete for a number of reasons, > but I think it may still be worth working on. Jan, any comments? I agree that there is considerable value in IO distribution. As such I already have the basics of the Background Writer in there. I left out the vacuum delay since I thought it was good enough to proove that there is low hanging fruit, but that it was far from what I'd call a solution. Ideally Vacuum would coordinate it's IO not only against some GUC variables, but also against the BGWriter+BufStrategy combo. Jan -- #======================================================================# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #================================================== JanWieck@Yahoo.com #
The vacuum delay patch is not the ideal solution but it worked like a charm on my servers. I really need the vacuum delay patch or a better solution in 7.5. I'm getting millions of requests a month and running VACUUM without the patch makes PostgreSQL useless for many consecutive hours. Not quite the 24/7 system I was hopping for. :-( Unfortunately, it's rather difficult to patch so many machines as my entire system runs on Redhat RPMs. I'm really hopping to see a solution to this VACUUM problem in 7.5. I've been waiting for this fix for over 3 years and now it's almost there. Will this problem get addressed in the not so official TODO list? Thanks and keep up the good work! Stephen "Jan Wieck" <JanWieck@Yahoo.com> wrote in message news:400544E3.8030709@Yahoo.com... > Tom Lane wrote: > > Christopher Browne <cbbrowne@libertyrms.info> writes: > >> "Stephen" <jleelim@xxxxxxx.com> writes: > >>> Any chance we'll see the VACUUM delay patch (throttle) get into 7.5? > > > >> The hope, in 7.5, is to have ARC, which is the "super-duper-duper" > >> version, working. > > > > Actually, I'm not sure that ARC should be considered to supersede the > > usefulness of a per-page delay in VACUUM. ARC should prevent VACUUM > > from trashing the contents of Postgres' shared buffer arena, but it > > won't do much of anything to prevent VACUUM from trashing the kernel > > buffer contents. And it definitely won't do anything to help if the > > real problem is that you're short of disk bandwidth and VACUUM's extra > > I/O demand pushes your total load over the knee of the response-time > > curve. What you need then is a throttle. > > > > The original patch I posted was incomplete for a number of reasons, > > but I think it may still be worth working on. Jan, any comments? > > I agree that there is considerable value in IO distribution. As such I > already have the basics of the Background Writer in there. > > I left out the vacuum delay since I thought it was good enough to proove > that there is low hanging fruit, but that it was far from what I'd call > a solution. Ideally Vacuum would coordinate it's IO not only against > some GUC variables, but also against the BGWriter+BufStrategy combo. > > > Jan > > -- > #======================================================================# > # It's easier to get forgiveness for being wrong than for being right. # > # Let's break this rule - forgive me. # > #================================================== JanWieck@Yahoo.com # > > > ---------------------------(end of broadcast)--------------------------- > TIP 6: Have you searched our list archives? > > http://archives.postgresql.org >
Any chance we'll see the VACUUM delay patch (throttle) get into 7.5? I only had the chance to try the first patch by Tom Lane and it was very good already. I was hoping it gets into 7.4.1 but it didn't. :-( I really need the VACUUM delay patch because my servers are begging to die every time VACUUM runs. Please let it be in 7.5. Thanks. Stephen "Tom Lane" <tgl@sss.pgh.pa.us> wrote in message news:5658.1073947557@sss.pgh.pa.us... > ow <oneway_111@yahoo.com> writes: > > Is this all that's planned for 7.5? (based on current TODO list) > > If you think the TODO list has *anything* to do with release planning, > you fundamentally misunderstand the way things are done around here. > > The TODO list is a list of things that are widely agreed to be problems > (though sometimes it only means "Bruce thinks this is a problem") and > therefore will probably get worked on at some point in the future. > > What actually gets done for 7.5 will depend on which itches bother which > developers enough to get scratched in the next few months. We do not > have any central planning. > > It is a safe bet that 7.5 will include some things mentioned in the > present TODO, as well as many things not mentioned in it; and that > many items mentioned in the present TODO will still remain undone. > > regards, tom lane > > ---------------------------(end of broadcast)--------------------------- > TIP 6: Have you searched our list archives? > > http://archives.postgresql.org >
Christopher Browne wrote: > 2. Jan Wieck's work on SLONY-1 > > Many are keen to see the code, but it's not out yet. And it is > not self-evident that it will necessarily be ready to release in > conjunction with 7.5 (Not to say it _can't_ happen, but just that > we'll see the code when we see it...) > > It is planned to be implemented in C, so it would presumably be > more suitable for inclusion in the main code than eRserv. But it > stands as "hope," and not certainty. I've just made one major step backward on that. Discovered that my first thread model was deadlock prone, so I better throw that away instead of building a lot of code on top of it. What I currently do is documenting the Slony-I ERD and the new thread model I have in mind. This will be a document in work, but I plan to have something readable by the end of this week, latest mid next week. I will create a mailing list for Slony-I on gborg so we can start discussing the implementation details. About the inclusion of a replication solution into the core distributon. The much I personally would be proud to see this one added, as a CORE member I do not see any of the replication "things" fit. All of them, and neither Slony-I nor Slony-II will be any different here, have pros and cons, none is the "one size that fits all" magic solution. To select one of the replication projects that high above all the others that it will be added to the core distribution, it should be really outstanding and general purpose. I think that Slony in the end will be outstanding, but only for what it was designed for. Jan -- #======================================================================# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #================================================== JanWieck@Yahoo.com #
Stephen wrote: > The vacuum delay patch is not the ideal solution but it worked like a charm > on my servers. I really need the vacuum delay patch or a better solution in > 7.5. I'm getting millions of requests a month and running VACUUM without the > patch makes PostgreSQL useless for many consecutive hours. Not quite the > 24/7 system I was hopping for. :-( > > Unfortunately, it's rather difficult to patch so many machines as my entire > system runs on Redhat RPMs. I'm really hopping to see a solution to this > VACUUM problem in 7.5. I've been waiting for this fix for over 3 years and > now it's almost there. > > Will this problem get addressed in the not so official TODO list? Well, I had a few different "versions" of vacuum delay stuff out as patches, together with ARC and the beginnings of the background writer. Instead of getting some numbers on those, the whole discussion got stuck in differences about how we actually let the background writer tell the kernel "do something" ... the whole sync(), fsync(), fdatasync(), fadvise() discussion. I don't have the time to make enough different attempts to find the one that pleases all. My argument still is that all this IO throttling and IO optimizing is mainly needed for dedicated servers, because I think that if you still run multiple services on one box you're not really in trouble yet. So in the first round a configurable sync() approach would do. So far nobody even agreed to that. I currently have better to do. We do not have a big IO problem, we have other problems, and I spend my time on solving them. If someone wants to pick up the IO throttle problem, I am allways here to help, but I will not waste my time with making patches nobody even gives a try. > > Thanks and keep up the good work! Sorry for the venting, but I needed that out. Jan -- #======================================================================# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #================================================== JanWieck@Yahoo.com #
On Mon, Jan 19, 2004 at 08:12:28AM -0500, Jan Wieck wrote: > and cons, none is the "one size that fits all" magic solution. To select Does anyone realy believe that there can be a one size fits all solution? Heck, even Oracle and IBM offer a couple of different systems, depending on what you need. (That also suggests that any replication system need not always be shipped with the "basic" distribution, but could instead be integrated into a larger, postgresql_plus_enterprise_features.tgz or something like that.) A -- Andrew Sullivan | ajs@crankycanuck.ca I remember when computers were frustrating because they *did* exactly what you told them to. That actually seems sort of quaint now. --J.D. Baldwin
On Mon, 2004-01-19 at 08:37, Jan Wieck wrote: > but I will not waste my time with making patches nobody even gives a try. I downloaded and tested your patches. I just didn't get results get results that were put together well enough to present to the group. I hope this doesn't fall by the wayside, it is IMHO, on of the critical problems that needs to be solved. > Sorry for the venting, but I needed that out. I understand. I'm sorry there wasn't more feedback as a result of your work.
Andrew Sullivan wrote: > On Mon, Jan 19, 2004 at 08:12:28AM -0500, Jan Wieck wrote: > >>and cons, none is the "one size that fits all" magic solution. To select > > Does anyone realy believe that there can be a one size fits all > solution? Heck, even Oracle and IBM offer a couple of different > systems, depending on what you need. (That also suggests that any > replication system need not always be shipped with the "basic" > distribution, but could instead be integrated into a larger, > postgresql_plus_enterprise_features.tgz or something like that.) I don't, but consider Linux which can be configured to run on devices as small as a wristwatch and as large as the the big irons... -- dave
ow wrote: > Hi, > > Is this all that's planned for 7.5? (based on current TODO list) > > -Change factorial to return a numeric (Gavin) > -COMMENT ON [ CAST | CONVERSION | OPERATOR CLASS | LARGE OBJECT | LANGUAGE ] > (Christopher) > -Have psql \dn show only visible temp schemas using current_schemas() > -Have psql '\i ~/<tab><tab>' actually load files it displays from home dir > -Allow psql \du to show groups, and add \dg for groups > -Allow pg_dump to dump CREATE CONVERSION (Christopher) > -Use dependency information to dump data in proper order > -Use background process to write dirty shared buffers to disk Those dashes mean the items are done and are already in CVS. As to what additional items we will have in 7.5, no one knows. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania19073
Tom Lane wrote: > ow <oneway_111@yahoo.com> writes: > > Is this all that's planned for 7.5? (based on current TODO list) > > If you think the TODO list has *anything* to do with release planning, > you fundamentally misunderstand the way things are done around here. > > The TODO list is a list of things that are widely agreed to be problems > (though sometimes it only means "Bruce thinks this is a problem") and > therefore will probably get worked on at some point in the future. > > What actually gets done for 7.5 will depend on which itches bother which > developers enough to get scratched in the next few months. We do not > have any central planning. > > It is a safe bet that 7.5 will include some things mentioned in the > present TODO, as well as many things not mentioned in it; and that > many items mentioned in the present TODO will still remain undone. Yes, the TODO is just a collection of items that could be done in 7.5. Some will be, and some new items not on the TODO list will be in 7.5. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania19073
>>-COMMENT ON [ CAST | CONVERSION | OPERATOR CLASS | LARGE OBJECT | LANGUAGE ] >>(Christopher) Hey Bruce, You probably should add 'Dump LOB comments in custom dump format' to the todo. That's the last part of that task above which I haven't done yet, and for various reasons probably won't have time to try for a while. Just so we don't forget it. Chris
Christopher Kings-Lynne wrote: > > >>-COMMENT ON [ CAST | CONVERSION | OPERATOR CLASS | LARGE OBJECT | LANGUAGE ] > >>(Christopher) > > Hey Bruce, > > You probably should add 'Dump LOB comments in custom dump format' to the > todo. That's the last part of that task above which I haven't done yet, > and for various reasons probably won't have time to try for a while. > Just so we don't forget it. Added: * Dump large object comments in custom dump format -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania19073