Thread: Not ready for 8.3
Based on our progress during this feature freeze, we will not complete the feature freeze until August/September. I think we need adjust expectations about an 8.3 release date, and decide if we want to radically change our work process. Basically, to make a release anywhere near July, we need 10x more progress than we have had, which is unlikely. We could ship what is in CVS now, but that just pushes the patches for 8.4, and isn't fair to patch submitters. We could shove what we have now into CVS, but that is unwise. I think the only other thing we _could_ do is to re-open normal 8.3 development, so we aren't hampering updates to trivial parts of the code. Many of the patches now in the queue had been developed for months before 8.3 started, so the hope is that we wouldn't have many more new large patches, but several small ones we could deal with while we whittle away at the larger patches during the next few months. The question is whether it is healthy for us to remain in feature freeze for months, and if it is unhealthy, what are our options? Patch status: http://developer.postgresql.org/index.php/Todo:PatchStatus -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
"Bruce Momjian" <bruce@momjian.us> writes: > I think the only other thing we _could_ do is to re-open normal 8.3 > development, so we aren't hampering updates to trivial parts of the > code. Many of the patches now in the queue had been developed for months > before 8.3 started, so the hope is that we wouldn't have many more new > large patches, but several small ones we could deal with while we > whittle away at the larger patches during the next few months. > > The question is whether it is healthy for us to remain in feature freeze > for months, and if it is unhealthy, what are our options? I don't see any reason development has to stop while the tree is in feature freeze. If it led to patches being ready for review and getting reviewed and committed early in the cycle rather than just before release I think it would actually be extremely healthy. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com
Bruce Momjian wrote: > Based on our progress during this feature freeze, we will not complete > the feature freeze until August/September. I think we need adjust > expectations about an 8.3 release date, and decide if we want to > radically change our work process. > > Basically, to make a release anywhere near July, we need 10x more > progress than we have had, which is unlikely. Or we have another short release cycle, basically accepting what we have now that can be worked through in the next 3 weeks and committed. If it can't be done in that time, it waits. Then we have beta etc... > > We could ship what is in CVS now, but that just pushes the patches for > 8.4, and isn't fair to patch submitters. We could shove what we have > now into CVS, but that is unwise. Sure it is, if we have a short release cycle. There are plenty of things out there that are not quite ready yet, that could be if we released again next January... > > I think the only other thing we _could_ do is to re-open normal 8.3 > development, so we aren't hampering updates to trivial parts of the > code. Many of the patches now in the queue had been developed for months Gah, that sounds like a horrible solution. Sorry Bruce. > > The question is whether it is healthy for us to remain in feature freeze > for months, and if it is unhealthy, what are our options? > > Patch status: > > http://developer.postgresql.org/index.php/Todo:PatchStatus If... this is actually a problem (I leave to other committers and reviewers to comment) then I suggest we push all patches without a reviewer as of now to 8.4. Leaving only those patches that have confirmed reviewers to be worked through. FYI, whoever did that Todo:Patch status, Bravo! That is easily one of the smallest but best improvements to the process I have seen in recent memory. Sincerely, Joshua D. Drake
Joshua D. Drake wrote: > Bruce Momjian wrote: > > Based on our progress during this feature freeze, we will not complete > > the feature freeze until August/September. I think we need adjust > > expectations about an 8.3 release date, and decide if we want to > > radically change our work process. > > > > Basically, to make a release anywhere near July, we need 10x more > > progress than we have had, which is unlikely. > > Or we have another short release cycle, basically accepting what we have > now that can be worked through in the next 3 weeks and committed. If it > can't be done in that time, it waits. Then we have beta etc... That is not fair to patch submitters, and pushes the problem to 8.4, where it will be no better. > > We could ship what is in CVS now, but that just pushes the patches for > > 8.4, and isn't fair to patch submitters. We could shove what we have > > now into CVS, but that is unwise. > > Sure it is, if we have a short release cycle. There are plenty of things > out there that are not quite ready yet, that could be if we released > again next January... Huh, you didn't answer my issue about unfair. > > The question is whether it is healthy for us to remain in feature freeze > > for months, and if it is unhealthy, what are our options? > > > > Patch status: > > > > http://developer.postgresql.org/index.php/Todo:PatchStatus > > If... this is actually a problem (I leave to other committers and > reviewers to comment) then I suggest we push all patches without a > reviewer as of now to 8.4. > > Leaving only those patches that have confirmed reviewers to be worked > through. > > FYI, whoever did that Todo:Patch status, Bravo! That is easily one of > the smallest but best improvements to the process I have seen in recent > memory. Fairness and not pushing our problems out to the future --- address those issues. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Joshua D. Drake wrote: > > Patch status: > > > > http://developer.postgresql.org/index.php/Todo:PatchStatus > > If... this is actually a problem (I leave to other committers and > reviewers to comment) then I suggest we push all patches without a > reviewer as of now to 8.4. > > Leaving only those patches that have confirmed reviewers to be worked > through. > > FYI, whoever did that Todo:Patch status, Bravo! That is easily one of > the smallest but best improvements to the process I have seen in recent > memory. I did one of those for previous releases. I guess you forgot. It was done by someone else this time only because I was going to be traveling. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
"Bruce Momjian" <bruce@momjian.us> writes: > Joshua D. Drake wrote: >> Bruce Momjian wrote: >> > Based on our progress during this feature freeze, we will not complete >> > the feature freeze until August/September. I think we need adjust >> > expectations about an 8.3 release date, and decide if we want to >> > radically change our work process. >> > >> > Basically, to make a release anywhere near July, we need 10x more >> > progress than we have had, which is unlikely. >> >> Or we have another short release cycle, basically accepting what we have >> now that can be worked through in the next 3 weeks and committed. If it >> can't be done in that time, it waits. Then we have beta etc... > > That is not fair to patch submitters, and pushes the problem to 8.4, > where it will be no better. It's nice to be fair but it's not really the goal here. However I think the latter point is key. This *was* a short release cycle. If we push off these patches to later there's no reason to think the situation will be any different in a few months. I suspect as much as Tom finds reviewing boring he would rather get it out of the way and then have a chance to do development before more major patches arrive than start the new cycle with major patches sitting in the review queue and spend the whole cycle reviewing them. As much as I hate that the patches sit in the queue until feature freeze there's nothing else that will really force us to make a yea or nay decision on them. If we start the next release with things in the patch queue they're liable to sit there with no feedback for months again. I think the key here is to give feedback for the authors but not allow the time to respond to that feedback to grow. If you review a patch and find problems with it and there's only three weeks for the author to deal with those problems and it's not enough time then he can deal with them and submit it for next release. But letting the patch ride over without giving any feedback just seems pointless. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com
Joshua D. Drake wrote: > FYI, whoever did that Todo:Patch status, Bravo! That is easily one of > the smallest but best improvements to the process I have seen in recent > memory. well bruce asked for something like that: http://archives.postgresql.org/pgsql-hackers/2007-05/msg00249.php and I simply went ahead and did it: http://archives.postgresql.org/pgsql-hackers/2007-05/msg00265.php Stefan
Bruce, Realistically I just don't see getting everything in the ToDo patch list in; my vote is that we start deferring stuff for 8.4 if it doesn't have a reviewer, except for items which were submitted early in the cycle (and to whom it would be unfair). If that means shortening the 8.4 cycle somewhat, I'm for that ... feature freeze in Feburary would be even better than April, because it means we could be in Beta for the May-June-July conferences, and increase our probability of being able to release at a major conference or PostgreSQL conference. Obviously for 8.4 reviewers need to start reviewing stuff from the first week of the development cycle. I also don't actually see anything wrong with a 3-month feature freeze if we can somehow branch development earlier. Easy for me to say, I know. -- Josh Berkus PostgreSQL @ Sun San Francisco
> That is not fair to patch submitters, and pushes the problem to 8.4, > where it will be no better. If it isn't done, it isn't done. We aren't dropping the patch. The patch has been accepted, just not reviewed. It is just delayed. >> Sure it is, if we have a short release cycle. There are plenty of things >> out there that are not quite ready yet, that could be if we released >> again next January... > > Huh, you didn't answer my issue about unfair. Sure I did see above :). Unfair would be to let go all together. We are just delaying. If we don't have reviewers, we don't have reviewers and as I look at the open list we have more reviewers than we did for 8.2 so I would suspect that 8.4 is going to go smoother as it is. >> Leaving only those patches that have confirmed reviewers to be worked >> through. >> >> FYI, whoever did that Todo:Patch status, Bravo! That is easily one of >> the smallest but best improvements to the process I have seen in recent >> memory. > > Fairness and not pushing our problems out to the future --- address > those issues. Life isn't fair. It isn't like we are telling these people to bugger off. We are just delaying the review to what is a reasonable workload for the current set of reviewers. Life... is a perpetual problem. We do what we can, when we can, and the best that we can. Taking my suggestion above for leaving patches that don't have reviewers let's look at some of them. Tsearch2 in core. I know that Tom has some reservations, he I and Treat all commented on how it was done and to my knowledge those reservations have not been resolved. HOT is a huge patch that has gone round and round and round. Lots of people want it, including me but it is invasive enough to where it may due it good to work through another cycle. Grouped Index Tuples. I like the result of this patch. I tested the performance and it did work as advertised but we didn't get much feedback as a whole from known hackers. Either there isn't much interest or we are too busy. Either way it is certainlynot a critical patch and can be pushed. Concurrent psql, nifty but still trying to decide on actual interface. Full Page Writes Improvement, doesn't actually do anything *yet* (as far as I can tell) it just makes it so something can be done in the future. It is also apparently a small patch. UTF8 text matching performance improvements (todo wiki link broke), so I don't have much comment. On Disk Bitmap indexes (todo wiki link broke): Doesn't support Vacuum (to my knowledge), community member tested, reported problems... no response yet from author. The author is known to be time constrained, boot it till 8.4. Posix shared memory, not usable in current state per Tom's comments and Apples, agreement. Dumped till 8.4 or further. Stream bitmaps, apparently needed more info from Gavin (see previous comment about author's time). No feed back since March 9th. Dumped till 8.4. Bitmapscan change, this one is unfortunate because at the time Heikki had resources and desire but was basically ignored. Sometimes we have to punt although Heikki is doing patch review right now and it is not unheard of for a patch reviewer to commit his own patch (in this case we would need a comitter to actually accept it.). Updateable cursors, apparently breaks explain and patch has been updated. Punt! Group Commit, Heikki has already suggested that it may be a good idea to push for 8.4 on this patch: http://momjian.us/mhonarc/patches/msg00127.html Index Advisor.. I think the link is wrong: http://momjian.us/mhonarc/patches/msg00119.html Ctid chain patch, apparently no discussion since last January even though requests had been made to change the patch to some degree. Punt. I will note that no one was negative about the patch, it just doesn't appear that the requests were ever finished. Error correct for n_dead_tuples, patch was requested for hold in Feb. No discussion since. Punt! DSM, apparently includes n_dead_tuples, please remove n_dead_tuples line on todo. Significant memory allocation enhancements are expected in 8.4 for this. Discussion died in April. Concerns were raised about how memory is allocated (fixed, shared) which author already admints to wanting to change for 8.4. PL/PSM, link wrong (goes to guc temp_tablespaces). This can certainly be developed outside of core. Don't get me wrong I like the feature but it can take advantage of facilities outside of core. So there ya go... thoughts, flames? Sincerely, Joshua D. Drake >
Gregory Stark wrote: > "Bruce Momjian" <bruce@momjian.us> writes: > > >> I think the only other thing we _could_ do is to re-open normal 8.3 >> development, so we aren't hampering updates to trivial parts of the >> code. Many of the patches now in the queue had been developed for months >> before 8.3 started, so the hope is that we wouldn't have many more new >> large patches, but several small ones we could deal with while we >> whittle away at the larger patches during the next few months. >> >> The question is whether it is healthy for us to remain in feature freeze >> for months, and if it is unhealthy, what are our options? >> > > I don't see any reason development has to stop while the tree is in feature > freeze. If it led to patches being ready for review and getting reviewed and > committed early in the cycle rather than just before release I think it would > actually be extremely healthy. > > If we had multiple dev branches it might be more possible. That has its own costs, of course - having a single dev branch makes management much easier, and we never have to worry about things like merging. Patches seem to be getting larger, more complex, and harder to review. That is stressing our processes more than somewhat. Short cycles would only make matters worse - the frictional cost of each dev cycle is growing. I think at least we have learned not to repeat this exercise. cheers andrew
We have new patch available http://www.sigaev.ru/misc/tsearch_core-0.47.gz to sync with CVS HEAD. Oleg On Tue, 15 May 2007, Joshua D. Drake wrote: > Bruce Momjian wrote: >> Based on our progress during this feature freeze, we will not complete >> the feature freeze until August/September. I think we need adjust >> expectations about an 8.3 release date, and decide if we want to >> radically change our work process. >> >> Basically, to make a release anywhere near July, we need 10x more >> progress than we have had, which is unlikely. > > Or we have another short release cycle, basically accepting what we have now > that can be worked through in the next 3 weeks and committed. If it can't be > done in that time, it waits. Then we have beta etc... > >> >> We could ship what is in CVS now, but that just pushes the patches for >> 8.4, and isn't fair to patch submitters. We could shove what we have >> now into CVS, but that is unwise. > > Sure it is, if we have a short release cycle. There are plenty of things out > there that are not quite ready yet, that could be if we released again next > January... > > >> >> I think the only other thing we _could_ do is to re-open normal 8.3 >> development, so we aren't hampering updates to trivial parts of the >> code. Many of the patches now in the queue had been developed for months > > Gah, that sounds like a horrible solution. Sorry Bruce. > >> >> The question is whether it is healthy for us to remain in feature freeze >> for months, and if it is unhealthy, what are our options? >> >> Patch status: >> >> http://developer.postgresql.org/index.php/Todo:PatchStatus > > If... this is actually a problem (I leave to other committers and reviewers > to comment) then I suggest we push all patches without a reviewer as of now > to 8.4. > > Leaving only those patches that have confirmed reviewers to be worked > through. > > FYI, whoever did that Todo:Patch status, Bravo! That is easily one of the > smallest but best improvements to the process I have seen in recent memory. > > Sincerely, > > Joshua D. Drake > > > > > ---------------------------(end of broadcast)--------------------------- > TIP 4: Have you searched our list archives? > > http://archives.postgresql.org > Regards, Oleg _____________________________________________________________ Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru), Sternberg Astronomical Institute, Moscow University, Russia Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(495)939-16-83, +007(495)939-23-83
josh@agliodbs.com (Josh Berkus) writes:
> Bruce,
>
> Realistically I just don't see getting everything in the ToDo patch
> list in; my vote is that we start deferring stuff for 8.4 if it
> doesn't have a reviewer, except for items which were submitted early
> in the cycle (and to whom it would be unfair).
>
> If that means shortening the 8.4 cycle somewhat, I'm for that
> ... feature freeze in Feburary would be even better than April,
> because it means we could be in Beta for the May-June-July
> conferences, and increase our probability of being able to release
> at a major conference or PostgreSQL conference.
If we're pushing stuff planned for 8.3 off into 8.4, then doesn't that
mean that we'd be inclined to _lengthen_ the 8.4 cycle?
If we were to shorten the 8.4 cycle, that seems likely to me to worsen
the problem, so that I'd expect to see even more stuff deferred for
8.5 than was the case with 8.3...
> Obviously for 8.4 reviewers need to start reviewing stuff from the
> first week of the development cycle. I also don't actually see
> anything wrong with a 3-month feature freeze if we can somehow
> branch development earlier. Easy for me to say, I know.
Well, if people are essentially already working on patches for 8.4
*now*, but just deferring merging until after 8.3 is committed +
released, then we've got 8.4 work underway already.
Unfortunately, that's also likely to worsen the problem of the
reviewers' queues being even more overflowing.
I'd sorta like to see the Tom Lanes and Bruce Momjians of the project
getting to have some time to work on things that *they'd* like add, as
opposed to turning things into a spiral cycle where they keep spending
more and more of their time reviewing patches, as opposed to doing
things they find neat and new. Too many iterations of that sort of
thing, and they'll not want to ever see a patch again...
--
output = ("cbbrowne" "@" "linuxdatabases.info")
http://cbbrowne.com/info/wp.html
"...It is also possible to post imbecilic articles with any browser,
especially when you toppost and omit snippage." -- CBFalconer
<cbfalconer@yahoo.com> - seen on comp.lang.c
On Tue, 15 May 2007, Joshua D. Drake wrote:
> Tsearch2 in core. I know that Tom has some reservations, he I and Treat all
> commented on how it was done and to my knowledge those reservations have not
> been resolved.
We'd like to know about these reservations. If I understand you mean there
are issues with the patch ? Our patch is several months old. We permanently
keep it in sync with CVS HEAD, latest version is 0.47.
We're really bored with the whole process of development. We're not
full-time developers, we just devote our spare time which we withdraw from
time we should spend on many other things. We have no time even to discuss
those very useful threads about community management, patches, etc. We just
rely on community decision.
Regards, Oleg
_____________________________________________________________
Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru),
Sternberg Astronomical Institute, Moscow University, Russia
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(495)939-16-83, +007(495)939-23-83
Joshua D. Drake wrote: [...] > Concurrent psql, nifty but still trying to decide on actual interface. > > Full Page Writes Improvement, doesn't actually do anything *yet* (as far > as I can tell) it just makes it so something can be done in the future. > It is also apparently a small patch. > > UTF8 text matching performance improvements (todo wiki link broke), so I > don't have much comment. url should be fixed now > > On Disk Bitmap indexes (todo wiki link broke): Doesn't support Vacuum > (to my knowledge), community member tested, reported problems... no > response yet from author. The author is known to be time constrained, > boot it till 8.4. url fixed too > > Bitmapscan change, this one is unfortunate because at the time Heikki > had resources and desire but was basically ignored. Sometimes we have to > punt although Heikki is doing patch review right now and it is not > unheard of for a patch reviewer to commit his own patch (in this case we > would need a comitter to actually accept it.). > > Updateable cursors, apparently breaks explain and patch has been > updated. Punt! > > Group Commit, Heikki has already suggested that it may be a good idea to > push for 8.4 on this patch: http://momjian.us/mhonarc/patches/msg00127.html > > Index Advisor.. I think the link is wrong: > > http://momjian.us/mhonarc/patches/msg00119.html url fixed > PL/PSM, link wrong (goes to guc temp_tablespaces). This can certainly be > developed outside of core. Don't get me wrong I like the feature but it > can take advantage of facilities outside of core. url fixed - I wonder why we had so much of them and all those pointing to the patch list bruce maintains - are the urls to the mails there not stable somehow ? Stefan
Oleg Bartunov wrote: > On Tue, 15 May 2007, Joshua D. Drake wrote: > >> Tsearch2 in core. I know that Tom has some reservations, he I and >> Treat all commented on how it was done and to my knowledge those >> reservations have not been resolved. > > We'd like to know about these reservations. If I understand you mean > there are issues with the patch ? Our patch is several months old. We > permanently > keep it in sync with CVS HEAD, latest version is 0.47. http://archives.postgresql.org/pgsql-hackers/2007-01/msg01172.php Sincerely, Joshua D. Drake
Stefan Kaltenbrunner wrote: > Joshua D. Drake wrote: > > PL/PSM, link wrong (goes to guc temp_tablespaces). This can certainly be > > developed outside of core. Don't get me wrong I like the feature but it > > can take advantage of facilities outside of core. > > url fixed - I wonder why we had so much of them and all those pointing > to the patch list bruce maintains - are the urls to the mails there not > stable somehow ? They are not stable. The items should point to the archives, which are supposedly more stable. (I had already fixed one item in PatchStatus this morning). Really it would be much nicer to have links using the Message-Id but I doubt that's at all doable. -- Alvaro Herrera http://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc.
Alvaro Herrera wrote: > Stefan Kaltenbrunner wrote: >> Joshua D. Drake wrote: > >>> PL/PSM, link wrong (goes to guc temp_tablespaces). This can certainly be >>> developed outside of core. Don't get me wrong I like the feature but it >>> can take advantage of facilities outside of core. >> url fixed - I wonder why we had so much of them and all those pointing >> to the patch list bruce maintains - are the urls to the mails there not >> stable somehow ? > > They are not stable. The items should point to the archives, which are > supposedly more stable. (I had already fixed one item in PatchStatus > this morning). Really it would be much nicer to have links using the > Message-Id but I doubt that's at all doable. I think we discussed it once... I don't remember the reason why we didn't go that direction. Joshua D. Drake >
Alvaro Herrera wrote: > Stefan Kaltenbrunner wrote: >> Joshua D. Drake wrote: > >>> PL/PSM, link wrong (goes to guc temp_tablespaces). This can certainly be >>> developed outside of core. Don't get me wrong I like the feature but it >>> can take advantage of facilities outside of core. >> url fixed - I wonder why we had so much of them and all those pointing >> to the patch list bruce maintains - are the urls to the mails there not >> stable somehow ? > > They are not stable. The items should point to the archives, which are > supposedly more stable. (I had already fixed one item in PatchStatus > this morning). Really it would be much nicer to have links using the > Message-Id but I doubt that's at all doable. hrm - I see so is there a particular reason for that behaviour ? I will work on pointing to the archives tomorrow ... Stefan
> They are not stable. The items should point to the archives, which are > supposedly more stable. (I had already fixed one item in PatchStatus > this morning). Really it would be much nicer to have links using the > Message-Id but I doubt that's at all doable. I use this all the time:http://news.gmane.org/find-root.php?message_id=$MSGID It uses non-PostgreSQL infrastructure, but hey, I think gmane does list archives better than PostgreSQL right now ;-) Try it:http://news.gmane.org/find-root.php?message_id=20070515201925.GL12731@alvh.no-ip.org a. -- Aidan Van Dyk Create like a god, aidan@highrise.ca command like a king, http://www.highrise.ca/ work like a slave.
Stefan Kaltenbrunner wrote: > Alvaro Herrera wrote: > > Stefan Kaltenbrunner wrote: > >> Joshua D. Drake wrote: > > > >>> PL/PSM, link wrong (goes to guc temp_tablespaces). This can certainly be > >>> developed outside of core. Don't get me wrong I like the feature but it > >>> can take advantage of facilities outside of core. > >> url fixed - I wonder why we had so much of them and all those pointing > >> to the patch list bruce maintains - are the urls to the mails there not > >> stable somehow ? > > > > They are not stable. The items should point to the archives, which are > > supposedly more stable. (I had already fixed one item in PatchStatus > > this morning). Really it would be much nicer to have links using the > > Message-Id but I doubt that's at all doable. > > hrm - I see so is there a particular reason for that behaviour ? > > I will work on pointing to the archives tomorrow ... Bruce adds and removes emails from the "pgpatches" inbox and I assume he regenerates the MHonarc archives when he does, which may change the URL for each specific item. I don't think it was ever intended that the URLs were to be stable. -- Alvaro Herrera http://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support
Stefan Kaltenbrunner wrote: >> They are not stable. The items should point to the archives, which are >> supposedly more stable. (I had already fixed one item in PatchStatus >> this morning). Really it would be much nicer to have links using the >> Message-Id but I doubt that's at all doable. > > hrm - I see so is there a particular reason for that behaviour ? They're stable until Bruce removes something from the queue. When something is removed, it's renumbered. It's how mhonarc works. It's the same with the archives - if we delete a mail, they get renumbered. So we never should delete, we should just blank out, but it has happened a couple of times. //Magnus
Aidan Van Dyk wrote: > > > They are not stable. The items should point to the archives, which are > > supposedly more stable. (I had already fixed one item in PatchStatus > > this morning). Really it would be much nicer to have links using the > > Message-Id but I doubt that's at all doable. > > I use this all the time: > http://news.gmane.org/find-root.php?message_id=$MSGID > > It uses non-PostgreSQL infrastructure, but hey, I think gmane does list > archives better than PostgreSQL right now ;-) > > Try it: > http://news.gmane.org/find-root.php?message_id=20070515201925.GL12731@alvh.no-ip.org Wow, this is really nice, thanks. -- Alvaro Herrera http://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support
On Tue, 15 May 2007, Joshua D. Drake wrote: > Oleg Bartunov wrote: >> On Tue, 15 May 2007, Joshua D. Drake wrote: >> >>> Tsearch2 in core. I know that Tom has some reservations, he I and Treat >>> all commented on how it was done and to my knowledge those reservations >>> have not been resolved. >> >> We'd like to know about these reservations. If I understand you mean there >> are issues with the patch ? Our patch is several months old. We permanently >> keep it in sync with CVS HEAD, latest version is 0.47. > > http://archives.postgresql.org/pgsql-hackers/2007-01/msg01172.php there are several others threads. well, we did answer all raised questions and I don't want to begin another wave. The only problem I see is that some people doesn't like introducing SQL syntax, they claim functions would be enough. SQL is a nice language to manipulate db objects, finally. Also, select to fts_create_dictionary(bla-bla....) looks pretty ugly, pretty artificial. CREATE FULLTEXT DICTIONARY dictname [ { INIT init_function | LEXIZE lexize_function | OPTION opt_text } [ ... ]] LIKE template_dictname; select > > Sincerely, > > Joshua D. Drake > Regards, Oleg _____________________________________________________________ Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru), Sternberg Astronomical Institute, Moscow University, Russia Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(495)939-16-83, +007(495)939-23-83
Gregory Stark wrote: > "Bruce Momjian" <bruce@momjian.us> writes: > > > I think the only other thing we _could_ do is to re-open normal 8.3 > > development, so we aren't hampering updates to trivial parts of the > > code. Many of the patches now in the queue had been developed for months > > before 8.3 started, so the hope is that we wouldn't have many more new > > large patches, but several small ones we could deal with while we > > whittle away at the larger patches during the next few months. > > > > The question is whether it is healthy for us to remain in feature freeze > > for months, and if it is unhealthy, what are our options? > > I don't see any reason development has to stop while the tree is in feature > freeze. If it led to patches being ready for review and getting reviewed and > committed early in the cycle rather than just before release I think it would > actually be extremely healthy. So you are saying just let people keep developing for 8.4 and we will use it as soon as we start for 8.4. OK. We might get to a point where we can just open development for 8.4, apply outstanding patches, and head for beta. ;-) -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Josh Berkus wrote: > Bruce, > > Realistically I just don't see getting everything in the ToDo patch list in; > my vote is that we start deferring stuff for 8.4 if it doesn't have a > reviewer, except for items which were submitted early in the cycle (and to > whom it would be unfair). It seems unfair to disinguish between early/last in cycle postings. > If that means shortening the 8.4 cycle somewhat, I'm for that ... feature > freeze in Feburary would be even better than April, because it means we could > be in Beta for the May-June-July conferences, and increase our probability of > being able to release at a major conference or PostgreSQL conference. > > Obviously for 8.4 reviewers need to start reviewing stuff from the first week > of the development cycle. I also don't actually see anything wrong with a > 3-month feature freeze if we can somehow branch development earlier. Easy > for me to say, I know. Yea, this is just pushing off work until later, which I don't support. There is no easy out here and I am afraid we will just have to accept a 3-month feature freeze. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Joshua D. Drake wrote: > > That is not fair to patch submitters, and pushes the problem to 8.4, > > where it will be no better. > > If it isn't done, it isn't done. We aren't dropping the patch. The patch > has been accepted, just not reviewed. It is just delayed. Delayed isn't rejected, but it isn't accepted either. I see that as unfair to patch submitters, and not making things any easier. It just pushes our problem into 8.4. > >> Sure it is, if we have a short release cycle. There are plenty of things > >> out there that are not quite ready yet, that could be if we released > >> again next January... > > > > Huh, you didn't answer my issue about unfair. > > Sure I did see above :). Unfair would be to let go all together. We are > just delaying. If we don't have reviewers, we don't have reviewers and > as I look at the open list we have more reviewers than we did for 8.2 so > I would suspect that 8.4 is going to go smoother as it is. And even more patches for 8.4. Given your list below, let me match it up with Tom's comments and see if I can defer some patches that are not ready for application. I think bitmapped indexes are one of them. --------------------------------------------------------------------------- > >> Leaving only those patches that have confirmed reviewers to be worked > >> through. > >> > >> FYI, whoever did that Todo:Patch status, Bravo! That is easily one of > >> the smallest but best improvements to the process I have seen in recent > >> memory. > > > > Fairness and not pushing our problems out to the future --- address > > those issues. > > Life isn't fair. It isn't like we are telling these people to bugger > off. We are just delaying the review to what is a reasonable workload > for the current set of reviewers. Again, it just pushed things off. I see just keeping going as a better approach. > Life... is a perpetual problem. We do what we can, when we can, and the > best that we can. Yea, but we have to do that best we can, not give up and hope something improves for 8.4, because I see no basis for such a belief. > Taking my suggestion above for leaving patches that don't have reviewers > let's look at some of them. > > Tsearch2 in core. I know that Tom has some reservations, he I and Treat > all commented on how it was done and to my knowledge those reservations > have not been resolved. > > HOT is a huge patch that has gone round and round and round. Lots of > people want it, including me but it is invasive enough to where it may > due it good to work through another cycle. > > Grouped Index Tuples. I like the result of this patch. I tested the > performance and it did work as advertised but we didn't get much > feedback as a whole from known hackers. Either there isn't much interest > or we are too busy. Either way it is certainly not a critical patch > and can be pushed. > > Concurrent psql, nifty but still trying to decide on actual interface. > > Full Page Writes Improvement, doesn't actually do anything *yet* (as far > as I can tell) it just makes it so something can be done in the future. > It is also apparently a small patch. > > UTF8 text matching performance improvements (todo wiki link broke), so I > don't have much comment. > > On Disk Bitmap indexes (todo wiki link broke): Doesn't support Vacuum > (to my knowledge), community member tested, reported problems... no > response yet from author. The author is known to be time constrained, > boot it till 8.4. > > Posix shared memory, not usable in current state per Tom's comments and > Apples, agreement. Dumped till 8.4 or further. > > Stream bitmaps, apparently needed more info from Gavin (see previous > comment about author's time). No feed back since March 9th. Dumped till 8.4. > > Bitmapscan change, this one is unfortunate because at the time Heikki > had resources and desire but was basically ignored. Sometimes we have to > punt although Heikki is doing patch review right now and it is not > unheard of for a patch reviewer to commit his own patch (in this case we > would need a comitter to actually accept it.). > > Updateable cursors, apparently breaks explain and patch has been > updated. Punt! > > Group Commit, Heikki has already suggested that it may be a good idea to > push for 8.4 on this patch: http://momjian.us/mhonarc/patches/msg00127.html > > Index Advisor.. I think the link is wrong: > > http://momjian.us/mhonarc/patches/msg00119.html > > Ctid chain patch, apparently no discussion since last January even > though requests had been made to change the patch to some degree. Punt. > I will note that no one was negative about the patch, it just doesn't > appear that the requests were ever finished. > > Error correct for n_dead_tuples, patch was requested for hold in Feb. No > discussion since. Punt! > > DSM, apparently includes n_dead_tuples, please remove n_dead_tuples line > on todo. Significant memory allocation enhancements are expected in 8.4 > for this. Discussion died in April. Concerns were raised about how > memory is allocated (fixed, shared) which author already admints to > wanting to change for 8.4. > > PL/PSM, link wrong (goes to guc temp_tablespaces). This can certainly be > developed outside of core. Don't get me wrong I like the feature but it > can take advantage of facilities outside of core. > > So there ya go... thoughts, flames? > > Sincerely, > > Joshua D. Drake > > > > > > > > > > > ---------------------------(end of broadcast)--------------------------- > TIP 1: if posting/reading through Usenet, please send an appropriate > subscribe-nomail command to majordomo@postgresql.org so that your > message can get through to the mailing list cleanly -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Andrew Dunstan wrote: > >> I think the only other thing we _could_ do is to re-open normal 8.3 > >> development, so we aren't hampering updates to trivial parts of the > >> code. Many of the patches now in the queue had been developed for months > >> before 8.3 started, so the hope is that we wouldn't have many more new > >> large patches, but several small ones we could deal with while we > >> whittle away at the larger patches during the next few months. > >> > >> The question is whether it is healthy for us to remain in feature freeze > >> for months, and if it is unhealthy, what are our options? > >> > > > > I don't see any reason development has to stop while the tree is in feature > > freeze. If it led to patches being ready for review and getting reviewed and > > committed early in the cycle rather than just before release I think it would > > actually be extremely healthy. > > > > > > If we had multiple dev branches it might be more possible. That has its > own costs, of course - having a single dev branch makes management much > easier, and we never have to worry about things like merging. > > Patches seem to be getting larger, more complex, and harder to review. > That is stressing our processes more than somewhat. > > Short cycles would only make matters worse - the frictional cost of each > dev cycle is growing. I think at least we have learned not to repeat > this exercise. Agreed. Good summary. Let's look at our problems honestly now and find a direction, rather than pushing them off for later. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Bruce Momjian wrote: > Gregory Stark wrote: > > "Bruce Momjian" <bruce@momjian.us> writes: > > > > > I think the only other thing we _could_ do is to re-open normal 8.3 > > > development, so we aren't hampering updates to trivial parts of the > > > code. Many of the patches now in the queue had been developed for months > > > before 8.3 started, so the hope is that we wouldn't have many more new > > > large patches, but several small ones we could deal with while we > > > whittle away at the larger patches during the next few months. > > > > > > The question is whether it is healthy for us to remain in feature freeze > > > for months, and if it is unhealthy, what are our options? > > > > I don't see any reason development has to stop while the tree is in feature > > freeze. If it led to patches being ready for review and getting reviewed and > > committed early in the cycle rather than just before release I think it would > > actually be extremely healthy. > > So you are saying just let people keep developing for 8.4 and we will > use it as soon as we start for 8.4. OK. We might get to a point where > we can just open development for 8.4, apply outstanding patches, and > head for beta. ;-) This is what happens with the Linux kernel. They have hundreds of developers getting their hands dirty during a previous period. Then 2.6.20 is released; the 2.6.21 "merge window" opens, and all sort of patches are flooded in. The merge window closes some time after that (it's something like 2 or 3 weeks), and the stabilization period follows (2 months?), during which 2.6.21-rc1, -rc2 etc are released. When stability is reached, 2.6.21 is released, and the cycle starts again. Sadly, we are missing the hundreds of developers. We are nowhere near the scale of Linux. -- Alvaro Herrera http://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc.
Chris Browne wrote: > josh@agliodbs.com (Josh Berkus) writes: > > Bruce, > > > > Realistically I just don't see getting everything in the ToDo patch > > list in; my vote is that we start deferring stuff for 8.4 if it > > doesn't have a reviewer, except for items which were submitted early > > in the cycle (and to whom it would be unfair). > > > > If that means shortening the 8.4 cycle somewhat, I'm for that > > ... feature freeze in Feburary would be even better than April, > > because it means we could be in Beta for the May-June-July > > conferences, and increase our probability of being able to release > > at a major conference or PostgreSQL conference. > > If we're pushing stuff planned for 8.3 off into 8.4, then doesn't that > mean that we'd be inclined to _lengthen_ the 8.4 cycle? > > If we were to shorten the 8.4 cycle, that seems likely to me to worsen > the problem, so that I'd expect to see even more stuff deferred for > 8.5 than was the case with 8.3... And, as I remember, we already deferred some stuff during 8.2 for 8.3. :-( > > Obviously for 8.4 reviewers need to start reviewing stuff from the > > first week of the development cycle. I also don't actually see > > anything wrong with a 3-month feature freeze if we can somehow > > branch development earlier. Easy for me to say, I know. > > Well, if people are essentially already working on patches for 8.4 > *now*, but just deferring merging until after 8.3 is committed + > released, then we've got 8.4 work underway already. > > Unfortunately, that's also likely to worsen the problem of the > reviewers' queues being even more overflowing. > > I'd sorta like to see the Tom Lanes and Bruce Momjians of the project > getting to have some time to work on things that *they'd* like add, as > opposed to turning things into a spiral cycle where they keep spending > more and more of their time reviewing patches, as opposed to doing > things they find neat and new. Too many iterations of that sort of > thing, and they'll not want to ever see a patch again... Another good summary. Myself, I have always done what no one wanted to do so that more people can do what they want to do, and I am OK with that. It helps the project move forward faster than if I did only what I wanted to do. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Oleg Bartunov wrote: > On Tue, 15 May 2007, Joshua D. Drake wrote: > >> Oleg Bartunov wrote: >>> On Tue, 15 May 2007, Joshua D. Drake wrote: >>> >>>> Tsearch2 in core. I know that Tom has some reservations, he I and >>>> Treat all commented on how it was done and to my knowledge those >>>> reservations have not been resolved. >>> >>> We'd like to know about these reservations. If I understand you mean >>> there are issues with the patch ? Our patch is several months old. We >>> permanently >>> keep it in sync with CVS HEAD, latest version is 0.47. >> >> http://archives.postgresql.org/pgsql-hackers/2007-01/msg01172.php > > there are several others threads. > > well, we did answer all raised questions and I don't want to begin > another wave. Did you address them? For example: http://archives.postgresql.org/pgsql-hackers/2007-03/msg00914.php Sincerely, Joshua d. drake The only problem I see is that some people doesn't like > introducing SQL syntax, they claim functions would be enough. SQL is a > nice language to manipulate db objects, finally. Also, select to > fts_create_dictionary(bla-bla....) looks pretty ugly, > pretty artificial. > > CREATE FULLTEXT DICTIONARY dictname [ { INIT init_function > | LEXIZE lexize_function > | OPTION opt_text } [ ... ]] LIKE template_dictname; > > select > >> >> Sincerely, >> >> Joshua D. Drake >> > > Regards, > Oleg > _____________________________________________________________ > Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru), > Sternberg Astronomical Institute, Moscow University, Russia > Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ > phone: +007(495)939-16-83, +007(495)939-23-83 > > ---------------------------(end of broadcast)--------------------------- > TIP 7: You can help support the PostgreSQL project by donating at > > http://www.postgresql.org/about/donate >
Joshua D. Drake wrote: > Oleg Bartunov wrote: > > On Tue, 15 May 2007, Joshua D. Drake wrote: > > > >> Tsearch2 in core. I know that Tom has some reservations, he I and > >> Treat all commented on how it was done and to my knowledge those > >> reservations have not been resolved. > > > > We'd like to know about these reservations. If I understand you mean > > there are issues with the patch ? Our patch is several months old. We > > permanently > > keep it in sync with CVS HEAD, latest version is 0.47. > > http://archives.postgresql.org/pgsql-hackers/2007-01/msg01172.php Oleg has been producing new versions of his patch, and no one has objected to any of it, so I assume all the issues were addressed. As far was whether tsearch2 should be in core, I think most agreed that SQL support for full-text would make it easier to use, and that it is a fundamental technology. As I remember, the only holdback was full multi-byte support, but that is done. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
On Tue, 15 May 2007, Joshua D. Drake wrote: > Oleg Bartunov wrote: >> On Tue, 15 May 2007, Joshua D. Drake wrote: >> >>> Oleg Bartunov wrote: >>>> On Tue, 15 May 2007, Joshua D. Drake wrote: >>>> >>>>> Tsearch2 in core. I know that Tom has some reservations, he I and Treat >>>>> all commented on how it was done and to my knowledge those reservations >>>>> have not been resolved. >>>> >>>> We'd like to know about these reservations. If I understand you mean >>>> there are issues with the patch ? Our patch is several months old. We >>>> permanently >>>> keep it in sync with CVS HEAD, latest version is 0.47. >>> >>> http://archives.postgresql.org/pgsql-hackers/2007-01/msg01172.php >> >> there are several others threads. >> >> well, we did answer all raised questions and I don't want to begin >> another wave. > > Did you address them? For example: > > http://archives.postgresql.org/pgsql-hackers/2007-03/msg00914.php we added support of gin index for text data type, so one could just create index fts_idx on TABLE using gin(body); and should be able to do full-text search. Of course, ranking is not available, since index doesn't store positional information. About this syntax see http://archives.postgresql.org/pgsql-hackers/2007-03/msg00936.php > > Sincerely, > > Joshua d. drake > > > The only problem I see is that some people doesn't like >> introducing SQL syntax, they claim functions would be enough. SQL is a >> nice language to manipulate db objects, finally. Also, select to >> fts_create_dictionary(bla-bla....) looks pretty ugly, >> pretty artificial. >> >> CREATE FULLTEXT DICTIONARY dictname [ { INIT init_function >> | LEXIZE lexize_function >> | OPTION opt_text } [ ... ]] LIKE template_dictname; >> >> select >> >>> >>> Sincerely, >>> >>> Joshua D. Drake >>> >> >> Regards, >> Oleg >> _____________________________________________________________ >> Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru), >> Sternberg Astronomical Institute, Moscow University, Russia >> Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ >> phone: +007(495)939-16-83, +007(495)939-23-83 >> >> ---------------------------(end of broadcast)--------------------------- >> TIP 7: You can help support the PostgreSQL project by donating at >> >> http://www.postgresql.org/about/donate >> > Regards, Oleg _____________________________________________________________ Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru), Sternberg Astronomical Institute, Moscow University, Russia Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(495)939-16-83, +007(495)939-23-83
Oleg Bartunov wrote: > On Tue, 15 May 2007, Joshua D. Drake wrote: > > > Tsearch2 in core. I know that Tom has some reservations, he I and Treat all > > commented on how it was done and to my knowledge those reservations have not > > been resolved. > > We'd like to know about these reservations. If I understand you mean there > are issues with the patch ? Our patch is several months old. We permanently > keep it in sync with CVS HEAD, latest version is 0.47. > > We're really bored with the whole process of development. We're not > full-time developers, we just devote our spare time which we withdraw from > time we should spend on many other things. We have no time even to discuss > those very useful threads about community management, patches, etc. We just > rely on community decision. This is a good example of how developers can get frustrated. Pushing a patch to 8.4 that was completed before 8.3 feature freeze is guaranteed to add to that --- and if we lose our developers, we might as well shut down the PostgreSQL project. One good thing is that we have community discussion this now, so at least we are focusing on it. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Stefan Kaltenbrunner wrote: > > PL/PSM, link wrong (goes to guc temp_tablespaces). This can certainly be > > developed outside of core. Don't get me wrong I like the feature but it > > can take advantage of facilities outside of core. > > url fixed - I wonder why we had so much of them and all those pointing > to the patch list bruce maintains - are the urls to the mails there not > stable somehow ? Yep, the URLs are unstable. I am using mhonarc and regenerate the page when the patches mbox file changes. When I append to the mailbox, the URLs don't change, but when I delete, they do. Does anyone know a way to make those URLs stable? -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Alvaro Herrera wrote: > Stefan Kaltenbrunner wrote: > > Alvaro Herrera wrote: > > > Stefan Kaltenbrunner wrote: > > >> Joshua D. Drake wrote: > > > > > >>> PL/PSM, link wrong (goes to guc temp_tablespaces). This can certainly be > > >>> developed outside of core. Don't get me wrong I like the feature but it > > >>> can take advantage of facilities outside of core. > > >> url fixed - I wonder why we had so much of them and all those pointing > > >> to the patch list bruce maintains - are the urls to the mails there not > > >> stable somehow ? > > > > > > They are not stable. The items should point to the archives, which are > > > supposedly more stable. (I had already fixed one item in PatchStatus > > > this morning). Really it would be much nicer to have links using the > > > Message-Id but I doubt that's at all doable. > > > > hrm - I see so is there a particular reason for that behaviour ? > > > > I will work on pointing to the archives tomorrow ... > > Bruce adds and removes emails from the "pgpatches" inbox and I assume he > regenerates the MHonarc archives when he does, which may change the URL > for each specific item. I don't think it was ever intended that the > URLs were to be stable. Right. Is there no way to make the mhonarc URLs stable? -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Bruce Momjian wrote: > Stefan Kaltenbrunner wrote: > > > PL/PSM, link wrong (goes to guc temp_tablespaces). This can certainly be > > > developed outside of core. Don't get me wrong I like the feature but it > > > can take advantage of facilities outside of core. > > > > url fixed - I wonder why we had so much of them and all those pointing > > to the patch list bruce maintains - are the urls to the mails there not > > stable somehow ? > > Yep, the URLs are unstable. I am using mhonarc and regenerate the page > when the patches mbox file changes. When I append to the mailbox, the > URLs don't change, but when I delete, they do. > > Does anyone know a way to make those URLs stable? Does it matter? I assume that we should not consider pgpatches to be a stable source anyway ... those emails are there just as a reminder for short-lived information anyway, so why bother? -- Alvaro Herrera http://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support
Bruce Momjian wrote: > Oleg Bartunov wrote: > This is a good example of how developers can get frustrated. Pushing a > patch to 8.4 that was completed before 8.3 feature freeze is guaranteed > to add to that --- and if we lose our developers, we might as well shut > down the PostgreSQL project. Let's not call the sky falling just yet. We are doing what we can. There are several sub projects afoot trying to eliminate some of this pain. If you recall, Lukas and I both spent time before feature freeze helping keep people in communication. Secondly and although you aren't 100% on board, there is also the tracker project. Lastly, more people are starting to review code. Even people that can't code are starting to at least participate in testing features. > > One good thing is that we have community discussion this now, so at > least we are focusing on it. > Agreed, but it certainly is not a critical mass problem either. We are starting to bounce off the wall, and are starting to take a step back to reflect what size ladder or rope we need to get over it. Joshua D. Drake
Alvaro Herrera wrote: > Bruce Momjian wrote: >> Stefan Kaltenbrunner wrote: >>>> PL/PSM, link wrong (goes to guc temp_tablespaces). This can certainly be >>>> developed outside of core. Don't get me wrong I like the feature but it >>>> can take advantage of facilities outside of core. >>> url fixed - I wonder why we had so much of them and all those pointing >>> to the patch list bruce maintains - are the urls to the mails there not >>> stable somehow ? >> Yep, the URLs are unstable. I am using mhonarc and regenerate the page >> when the patches mbox file changes. When I append to the mailbox, the >> URLs don't change, but when I delete, they do. >> >> Does anyone know a way to make those URLs stable? > > Does it matter? I assume that we should not consider pgpatches to be a > stable source anyway ... those emails are there just as a reminder for > short-lived information anyway, so why bother? Yeah let's just source archives, the urls are reasonable stable there. Joshua D. Drake >
Joshua D. Drake wrote: > Bruce Momjian wrote: > > Oleg Bartunov wrote: > > This is a good example of how developers can get frustrated. Pushing a > > patch to 8.4 that was completed before 8.3 feature freeze is guaranteed > > to add to that --- and if we lose our developers, we might as well shut > > down the PostgreSQL project. > > Let's not call the sky falling just yet. We are doing what we can. There > are several sub projects afoot trying to eliminate some of this pain. If > you recall, Lukas and I both spent time before feature freeze helping > keep people in communication. > > Secondly and although you aren't 100% on board, there is also the > tracker project. > > Lastly, more people are starting to review code. Even people that can't > code are starting to at least participate in testing features. > > > > > One good thing is that we have community discussion this now, so at > > least we are focusing on it. > > > > Agreed, but it certainly is not a critical mass problem either. We are > starting to bounce off the wall, and are starting to take a step back to > reflect what size ladder or rope we need to get over it. Right. I just don't want a short-term solution (push patches for 8.4) that avoids a long-term analysis. Lots of people complained we didn't have a patches status page. Someone has done that and it is current, but it is not moving us forward fast enough. What is our next move, or do we just keep moving and settle on August? -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Bruce Momjian wrote: > Joshua D. Drake wrote: >>> One good thing is that we have community discussion this now, so at >>> least we are focusing on it. >>> >> Agreed, but it certainly is not a critical mass problem either. We are >> starting to bounce off the wall, and are starting to take a step back to >> reflect what size ladder or rope we need to get over it. > > Right. I just don't want a short-term solution (push patches for 8.4) > that avoids a long-term analysis. > > Lots of people complained we didn't have a patches status page. Someone > has done that and it is current, but it is not moving us forward fast > enough. > > What is our next move, or do we just keep moving and settle on August? Well I think what you said about reviewing my list versus Tom's triage is a good start. If there are things we can push, let's push them. Plus based on this conversation I think we need reviewers to step up and say... "Hey! this is where I am at...". Sincerely, Joshua D. Drake
Joshua D. Drake wrote: > Bruce Momjian wrote: > > Joshua D. Drake wrote: > > >>> One good thing is that we have community discussion this now, so at > >>> least we are focusing on it. > >>> > >> Agreed, but it certainly is not a critical mass problem either. We are > >> starting to bounce off the wall, and are starting to take a step back to > >> reflect what size ladder or rope we need to get over it. > > > > Right. I just don't want a short-term solution (push patches for 8.4) > > that avoids a long-term analysis. > > > > Lots of people complained we didn't have a patches status page. Someone > > has done that and it is current, but it is not moving us forward fast > > enough. > > > > What is our next move, or do we just keep moving and settle on August? > > Well I think what you said about reviewing my list versus Tom's triage > is a good start. If there are things we can push, let's push them. Plus > based on this conversation I think we need reviewers to step up and > say... "Hey! this is where I am at...". Agreed. I have already contacted all reviewers individually to ask them for a status, and mentioned I was concerned. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
On Tue, May 15, 2007 at 12:42:28PM -0400, Bruce Momjian wrote: > Joshua D. Drake wrote: > > > Patch status: > > > > > > http://developer.postgresql.org/index.php/Todo:PatchStatus > > > > If... this is actually a problem (I leave to other committers and > > reviewers to comment) then I suggest we push all patches without a > > reviewer as of now to 8.4. > > > > Leaving only those patches that have confirmed reviewers to be worked > > through. > > > > FYI, whoever did that Todo:Patch status, Bravo! That is easily one of > > the smallest but best improvements to the process I have seen in recent > > memory. > > I did one of those for previous releases. I guess you forgot. It was > done by someone else this time only because I was going to be traveling. Unless you're really in love with doing that sort of thing it's really good that someone else did it. You're one of a handful of folks that can actually review patches, while there's any number of us that can update a wiki. Speaking of reviewers... should we put some thought into how we can increase the number of people who can review code? It seems that's one of our biggest bottlenecks... -- Jim Nasby decibel@decibel.org EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)
On Tue, May 15, 2007 at 10:32:14PM +0200, Magnus Hagander wrote: > Stefan Kaltenbrunner wrote: > >> They are not stable. The items should point to the archives, which are > >> supposedly more stable. (I had already fixed one item in PatchStatus > >> this morning). Really it would be much nicer to have links using the > >> Message-Id but I doubt that's at all doable. > > > > hrm - I see so is there a particular reason for that behaviour ? > > They're stable until Bruce removes something from the queue. When > something is removed, it's renumbered. > > It's how mhonarc works. It's the same with the archives - if we delete a > mail, they get renumbered. So we never should delete, we should just > blank out, but it has happened a couple of times. Isn't there any other archiver we could use? The lack of URL stability in mhonarc is bad enough, but the cross-month issue is just horrible. -- Jim Nasby decibel@decibel.org EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)
Jim C. Nasby wrote: > On Tue, May 15, 2007 at 12:42:28PM -0400, Bruce Momjian wrote: > > Joshua D. Drake wrote: > > > > Patch status: > > > > > > > > http://developer.postgresql.org/index.php/Todo:PatchStatus > > > > > > If... this is actually a problem (I leave to other committers and > > > reviewers to comment) then I suggest we push all patches without a > > > reviewer as of now to 8.4. > > > > > > Leaving only those patches that have confirmed reviewers to be worked > > > through. > > > > > > FYI, whoever did that Todo:Patch status, Bravo! That is easily one of > > > the smallest but best improvements to the process I have seen in recent > > > memory. > > > > I did one of those for previous releases. I guess you forgot. It was > > done by someone else this time only because I was going to be traveling. > > Unless you're really in love with doing that sort of thing it's really > good that someone else did it. You're one of a handful of folks that can > actually review patches, while there's any number of us that can update > a wiki. Sure. Good idea. All I had to do was ask (10 times, though). ;-) > Speaking of reviewers... should we put some thought into how we can > increase the number of people who can review code? It seems that's one > of our biggest bottlenecks... Sure. That is going to move us forward. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Jim C. Nasby wrote: > On Tue, May 15, 2007 at 10:32:14PM +0200, Magnus Hagander wrote: > > Stefan Kaltenbrunner wrote: > > >> They are not stable. The items should point to the archives, which are > > >> supposedly more stable. (I had already fixed one item in PatchStatus > > >> this morning). Really it would be much nicer to have links using the > > >> Message-Id but I doubt that's at all doable. > > > > > > hrm - I see so is there a particular reason for that behaviour ? > > > > They're stable until Bruce removes something from the queue. When > > something is removed, it's renumbered. > > > > It's how mhonarc works. It's the same with the archives - if we delete a > > mail, they get renumbered. So we never should delete, we should just > > blank out, but it has happened a couple of times. > > Isn't there any other archiver we could use? The lack of URL stability > in mhonarc is bad enough, but the cross-month issue is just horrible. Agreed. It is just crippled. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
On Tue, May 15, 2007 at 07:01:39PM -0400, Bruce Momjian wrote: > > Unless you're really in love with doing that sort of thing it's really > > good that someone else did it. You're one of a handful of folks that can > > actually review patches, while there's any number of us that can update > > a wiki. > > Sure. Good idea. All I had to do was ask (10 times, though). ;-) Even having to ask 10 times undoubtedly was a lot faster than doing it yourself though. :) > > Speaking of reviewers... should we put some thought into how we can > > increase the number of people who can review code? It seems that's one > > of our biggest bottlenecks... > > Sure. That is going to move us forward. So... any ideas? :) -- Jim Nasby decibel@decibel.org EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)
Jim C. Nasby wrote: > On Tue, May 15, 2007 at 07:01:39PM -0400, Bruce Momjian wrote: > > > Unless you're really in love with doing that sort of thing it's really > > > good that someone else did it. You're one of a handful of folks that can > > > actually review patches, while there's any number of us that can update > > > a wiki. > > > > Sure. Good idea. All I had to do was ask (10 times, though). ;-) > > Even having to ask 10 times undoubtedly was a lot faster than doing it > yourself though. :) Very true. > > > Speaking of reviewers... should we put some thought into how we can > > > increase the number of people who can review code? It seems that's one > > > of our biggest bottlenecks... > > > > Sure. That is going to move us forward. > > So... any ideas? :) No, that is my problem. I was hoping others would come up with some. I think communicating the problem is part of the solution. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
> Stefan Kaltenbrunner wrote: > >> They are not stable. The items should point to the archives, which are > >> supposedly more stable. (I had already fixed one item in PatchStatus > >> this morning). Really it would be much nicer to have links using the > >> Message-Id but I doubt that's at all doable. > > > > hrm - I see so is there a particular reason for that behaviour ? > > They're stable until Bruce removes something from the queue. When > something is removed, it's renumbered. > > It's how mhonarc works. It's the same with the archives - if we delete a > mail, they get renumbered. So we never should delete, we should just > blank out, but it has happened a couple of times. As I proposed for many times, why don't we add message number to each subject line in mail? For example like this: [HACKERS: 12345] Re: Not ready for 8.3 This way, we could always obtain stable (logical) pointer, without reling on particular archival infrastructure. -- Tatsuo Ishii SRA OSS, Inc. Japan
On 5/15/07, Tatsuo Ishii <ishii@postgresql.org> wrote:
This sounds like a good idea to me - though I'm just a lurker on the list.
Nathan
> Stefan Kaltenbrunner wrote:
> >> They are not stable. The items should point to the archives, which are
> >> supposedly more stable. (I had already fixed one item in PatchStatus
> >> this morning). Really it would be much nicer to have links using the
> >> Message-Id but I doubt that's at all doable.
> >
> > hrm - I see so is there a particular reason for that behaviour ?
>
> They're stable until Bruce removes something from the queue. When
> something is removed, it's renumbered.
>
> It's how mhonarc works. It's the same with the archives - if we delete a
> mail, they get renumbered. So we never should delete, we should just
> blank out, but it has happened a couple of times.
As I proposed for many times, why don't we add message number to each
subject line in mail? For example like this:
[HACKERS: 12345] Re: Not ready for 8.3
This way, we could always obtain stable (logical) pointer, without
reling on particular archival infrastructure.
This sounds like a good idea to me - though I'm just a lurker on the list.
Nathan
Tatsuo Ishii wrote: > > Stefan Kaltenbrunner wrote: > > >> They are not stable. The items should point to the archives, which are > > >> supposedly more stable. (I had already fixed one item in PatchStatus > > >> this morning). Really it would be much nicer to have links using the > > >> Message-Id but I doubt that's at all doable. > > > > > > hrm - I see so is there a particular reason for that behaviour ? > > > > They're stable until Bruce removes something from the queue. When > > something is removed, it's renumbered. > > > > It's how mhonarc works. It's the same with the archives - if we delete a > > mail, they get renumbered. So we never should delete, we should just > > blank out, but it has happened a couple of times. > > As I proposed for many times, why don't we add message number to each > subject line in mail? For example like this: > > [HACKERS: 12345] Re: Not ready for 8.3 > > This way, we could always obtain stable (logical) pointer, without > reling on particular archival infrastructure. While that helps, it does not actually allow software to link the threads together, i.e. it can't tell which is the head and tail of the thread. The actual software needs to improve because the thread information is already part of the email headers. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
On Tuesday 15 May 2007 16:48, Bruce Momjian wrote:
> Joshua D. Drake wrote:
> > > That is not fair to patch submitters, and pushes the problem to 8.4,
> > > where it will be no better.
> >
> > If it isn't done, it isn't done. We aren't dropping the patch. The patch
> > has been accepted, just not reviewed. It is just delayed.
>
> Delayed isn't rejected, but it isn't accepted either. I see that as
> unfair to patch submitters, and not making things any easier. It just
> pushes our problem into 8.4.
>
So, I'm not sure that these are actual rules, but it's how I have always seen
the process working:
a) if we cannot resolve a submitted patch of technical issues within a
reasonable time, it gets pushed to the next release.
it is at the discression of -core/reviewer to determine both what are
unresolvable issues and what is an unreasonable time... but I think patches
that are not making progress within a period of months probably qualify on
both counts.
b) the development branch for the next release is not opened until delayed
patches are dealt with
dealing with a patch may not mean committing it, if it determined that the
flaws are enough that it just isnt ready, or the author decides they aren't
ready to devote time to it.
i think everyone understands we all want everyones patches to get in, but it
isn't realistic to think that will happen every release.
one of the reasons we have a feature freeze is because the community has
decided that holding up a release for feature X isn't desirable; aiui all of
the patches in the current queue have been given a once over, and some more
than that; i'm sure that cycle could go on quite some time for some patches,
so as unfortunate as it is, I think at some point you have to just suck it up
and call it day. all imho.
--
Robert Treat
Build A Brighter LAMP :: Linux Apache {middleware} PostgreSQL
Bruce, > It seems unfair to disinguish between early/last in cycle postings. I think it's fair. A patch which was submitted for 8.2 and didn't get in should get consideration over a patch which was still in prototype form a week before feature freeze. Also, I really don't think it's a crime to push back patches which are under active redevelopment (seqscan) if that redevelopment isn't a result of review neglect. > > Obviously for 8.4 reviewers need to start reviewing stuff from the first > > week of the development cycle. I also don't actually see anything wrong > > with a 3-month feature freeze if we can somehow branch development > > earlier. Easy for me to say, I know. > > Yea, this is just pushing off work until later, which I don't support. > There is no easy out here and I am afraid we will just have to accept a > 3-month feature freeze. I think that may be where we're heading. In that case, we may need to talk about branching earlier so that developers can work on the new version who are frozen out of the in-process one. > Oleg has been producing new versions of his patch, and no one has > objected to any of it, so I assume all the issues were addressed. > > As far was whether tsearch2 should be in core, I think most agreed that > SQL support for full-text would make it easier to use, and that it is a > fundamental technology. As I remember, the only holdback was full > multi-byte support, but that is done. I'm going to echo Bruce on this; I've mentioned that TSearch was going into Core at conferences and the reaction from existing users has been very enthusiastic, ranging from "yippee!" to "about time!". Our users hate the fact that FTS is a separate module. -- Josh Berkus PostgreSQL @ Sun San Francisco
> Tatsuo Ishii wrote: > > > Stefan Kaltenbrunner wrote: > > > >> They are not stable. The items should point to the archives, which are > > > >> supposedly more stable. (I had already fixed one item in PatchStatus > > > >> this morning). Really it would be much nicer to have links using the > > > >> Message-Id but I doubt that's at all doable. > > > > > > > > hrm - I see so is there a particular reason for that behaviour ? > > > > > > They're stable until Bruce removes something from the queue. When > > > something is removed, it's renumbered. > > > > > > It's how mhonarc works. It's the same with the archives - if we delete a > > > mail, they get renumbered. So we never should delete, we should just > > > blank out, but it has happened a couple of times. > > > > As I proposed for many times, why don't we add message number to each > > subject line in mail? For example like this: > > > > [HACKERS: 12345] Re: Not ready for 8.3 > > > > This way, we could always obtain stable (logical) pointer, without > > reling on particular archival infrastructure. > > While that helps, it does not actually allow software to link the > threads together, i.e. it can't tell which is the head and tail of the > thread. The actual software needs to improve because the thread > information is already part of the email headers. I know. My point is, subject-with-sequence will guarantee at least we do not lose the pointer completely. Also it will help us searching particular message in the private mail box. -- Tatsuo Ishii SRA OSS, Inc. Japan
Josh Berkus wrote: >> Yea, this is just pushing off work until later, which I don't support. >> There is no easy out here and I am afraid we will just have to accept a >> 3-month feature freeze. >> > > I think that may be where we're heading. In that case, we may need to talk > about branching earlier so that developers can work on the new version who > are frozen out of the in-process one. > > I've argued this in the past. But be aware that it will make more work for committers. It is very far from cost-free. cheers andrew
Jim C. Nasby wrote: > On Tue, May 15, 2007 at 10:32:14PM +0200, Magnus Hagander wrote: >> Stefan Kaltenbrunner wrote: >>>> They are not stable. The items should point to the archives, which are >>>> supposedly more stable. (I had already fixed one item in PatchStatus >>>> this morning). Really it would be much nicer to have links using the >>>> Message-Id but I doubt that's at all doable. >>> hrm - I see so is there a particular reason for that behaviour ? >> They're stable until Bruce removes something from the queue. When >> something is removed, it's renumbered. >> >> It's how mhonarc works. It's the same with the archives - if we delete a >> mail, they get renumbered. So we never should delete, we should just >> blank out, but it has happened a couple of times. > > Isn't there any other archiver we could use? The lack of URL stability > in mhonarc is bad enough, but the cross-month issue is just horrible. Nothing useful last time I looked (a year or two back admittedley). I have a design for one in mind that I was looking to prototype - there are some php classes that would make it quite simple to get messages into a database either via procmail, or from an mbox. the stumbling block I was running into was rewriting the old archives URLs to the new ones. Regards, Dave
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wed, May 16, 2007 at 10:16:43AM +0900, Tatsuo Ishii wrote: > > Stefan Kaltenbrunner wrote: > > >> They are not stable. [...] > As I proposed for many times, why don't we add message number to each > subject line in mail? For example like this: > > [HACKERS: 12345] Re: Not ready for 8.3 What I don't understand is why mailing list software doesn't use message ID as primary index. Granted, it's ugly, but it is globally stable (and hopefully unique) _across all mailboxes_ Sorry for the random rant - -- tomás -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) iD8DBQFGSrYGBcgs9XrR2kYRAlY2AJ9Vq5JZZEqX/y/DkN/HJ4Wg47RMyQCfbdgh Z6KnR4eJHh/oDHr7GI/OiPU= =Nxev -----END PGP SIGNATURE-----
* Tatsuo Ishii <ishii@postgresql.org> [070515 21:19]: > As I proposed for many times, why don't we add message number to each > subject line in mail? For example like this: > > [HACKERS: 12345] Re: Not ready for 8.3 > > This way, we could always obtain stable (logical) pointer, without > reling on particular archival infrastructure. Isn't that what the "Message-Id" field is for? http://news.gmane.org/find-root.php?message_id=20070516.101643.94564776.t-ishii@sraoss.co.jp a. -- Aidan Van Dyk Create like a god, aidan@highrise.ca command like a king, http://www.highrise.ca/ work like a slave.
> * Tatsuo Ishii <ishii@postgresql.org> [070515 21:19]: > > > As I proposed for many times, why don't we add message number to each > > subject line in mail? For example like this: > > > > [HACKERS: 12345] Re: Not ready for 8.3 > > > > This way, we could always obtain stable (logical) pointer, without > > reling on particular archival infrastructure. > > Isn't that what the "Message-Id" field is for? > > http://news.gmane.org/find-root.php?message_id=20070516.101643.94564776.t-ishii@sraoss.co.jp > a. Maybe. However I think "subject-sequence" has some advantages over Message-Id: - Easy to identify. Message-Id may not appear on some MUA with default setting - More handy than lengthy message Id - Easy to detect messages not delivered, by knowing that the sequence number was skipped -- Tatsuo Ishii SRA OSS, Inc. Japan
> I'm going to echo Bruce on this; I've mentioned that TSearch was going > into Core at conferences and the reaction from existing users has been > very enthusiastic, ranging from "yippee!" to "about time!". Our users > hate the fact that FTS is a separate module. Here here. And with respect to the debate about syntax, who cares? I think I prefer introducing real SQL-ish syntax over a bunch of pg_* functions, but doing it either way is IMHO better than doing nothing. ...Robert
Am Mittwoch, 16. Mai 2007 05:37 schrieb Josh Berkus: > In that case, we may need to talk > about branching earlier so that developers can work on the new version who > are frozen out of the in-process one. Well, we could branch right now, but who is going to commit anything into that new head branch? The committers are already claimed to be falling behind. -- Peter Eisentraut http://developer.postgresql.org/~petere/
Am Mittwoch, 16. Mai 2007 05:37 schrieb Josh Berkus: > I'm going to echo Bruce on this; I've mentioned that TSearch was going into > Core at conferences and the reaction from existing users has been very > enthusiastic, ranging from "yippee!" to "about time!". Our users hate the > fact that FTS is a separate module. At the same time they are subconsciously counting on us to make decisions based on rationality, not on enthusiasm. -- Peter Eisentraut http://developer.postgresql.org/~petere/
Robert Haas wrote: >> Our users hate the fact that FTS is a separate module. >> > > Here here. > Where? Where? Oh, you mean "Hear Hear." (sorry - one of my pet peeves) > And with respect to the debate about syntax, who cares? I think I > prefer introducing real SQL-ish syntax over a bunch of pg_* functions, > but doing it either way is IMHO better than doing nothing. > > > We do have a responsibility, I think, to keep the grammar fairly clean, so the answer to your question "who cares?" is "we do." That said, last time I looked most of the warts seemed to have been knocked off, IIRC, and the functional syntax would have been sufficiently ugly and cumbersome to weigh against it. So, like most others, I'm in favor of going with this unless there is some overwhelming reason I haven't heard of yet not to. cheers andrew
tomas@tuxteam.de wrote: > On Wed, May 16, 2007 at 10:16:43AM +0900, Tatsuo Ishii wrote: > > > Stefan Kaltenbrunner wrote: > > > >> They are not stable. [...] > > > As I proposed for many times, why don't we add message number to each > > subject line in mail? For example like this: > > > > [HACKERS: 12345] Re: Not ready for 8.3 > > What I don't understand is why mailing list software doesn't use message > ID as primary index. Granted, it's ugly, but it is globally stable (and > hopefully unique) _across all mailboxes_ It does. The problem is that it only considers messages posted in the last calendar month. So on the 1st of each month, all threads "start again" as far as the archive goes. There is just one remaining problem: Outlook and derivatives don't set the In-Reply-To: nor References: headers. This breaks the threads (the best the software can do is group the messages by subject, but it works only partially). I've tried a coupld of times to decode how the Thread-Id stuff works, which is what Outlook seems to use, but I haven't been able to detect a pattern. -- Alvaro Herrera http://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc.
Tatsuo Ishii wrote: > > * Tatsuo Ishii <ishii@postgresql.org> [070515 21:19]: > > > > > As I proposed for many times, why don't we add message number to each > > > subject line in mail? For example like this: > > > > > > [HACKERS: 12345] Re: Not ready for 8.3 > > > > > > This way, we could always obtain stable (logical) pointer, without > > > reling on particular archival infrastructure. > > > > Isn't that what the "Message-Id" field is for? > > > > http://news.gmane.org/find-root.php?message_id=20070516.101643.94564776.t-ishii@sraoss.co.jp > > a. > > Maybe. However I think "subject-sequence" has some advantages over > Message-Id: > > - Easy to identify. Message-Id may not appear on some MUA with default > setting Message-Ids are present in all messages. When the MUA doesn't set it, the MTA does. The problem starts when the MUA doesn't set the In-Reply-To header. > - More handy than lengthy message Id True. > - Easy to detect messages not delivered, by knowing that the sequence > number was skipped The problem is that the number would be possibly set at a later stage of email delivery by the list software, so it doesn't help if the message is skipped in an earlier stage (spam filter, etc). -- Alvaro Herrera http://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support
* Tatsuo Ishii <ishii@postgresql.org> [070516 07:23]: > Maybe. However I think "subject-sequence" has some advantages over > Message-Id: > > - Easy to identify. Message-Id may not appear on some MUA with default > setting > > - More handy than lengthy message Id > > - Easy to detect messages not delivered, by knowing that the sequence > number was skipped IMNSHO, we should be encouraging lists to move *away* from subject munging. Adding more characters into a subject line which can already be quite long is just making the situation worse. And of course, once you realize that subject munging is wrong, putting the "list-sequence" into a header of no real value, unless you believe your MTA/MUA filtering to be somehow dropping list messages, and your sequence numbers will prove to you if that's true or not. Oh - but wait, don't we all, as PostgreSQL users realize that sequences aren't generally guaranteed to be gapless anyways (sure, there are work "solutions", but I'm not about to audit any MTA/list software for that...) ;-) -- Aidan Van Dyk Create like a god, aidan@highrise.ca command like a king, http://www.highrise.ca/ work like a slave.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wed, May 16, 2007 at 10:03:47AM -0400, Alvaro Herrera wrote: > tomas@tuxteam.de wrote: [...] > There is just one remaining problem: Outlook and derivatives don't set > the In-Reply-To: nor References: headers. This breaks the threads (the > best the software can do is group the messages by subject, but it works > only partially). > > I've tried a coupld of times to decode how the Thread-Id stuff works, > which is what Outlook seems to use, but I haven't been able to detect a > pattern. Heh. Seems to be a well-known problem: <http://osdir.com/ml/discuss/2003-10/threads.html#00194> Ackthpttp. Regards - -- tomás -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) iD8DBQFGSxVVBcgs9XrR2kYRAsRrAJ9GC7fHJgaS424yoylnzB/9YPtE/wCbBCRE C62mzkj1BVXKlIau0TKlOS4= =GVAM -----END PGP SIGNATURE-----
> > Here here. > > > Where? Where? Oh, you mean "Hear Hear." (sorry - one of my pet peeves) Oops. Of course since it's in written form perhaps I should be writing "Read! Read!" instead. > We do have a responsibility, I think, to keep the grammar fairly clean, > so the answer to your question "who cares?" is "we do." Sure. I'm asking that rhetorical question under the assumption that none of the options on the table are so bad as to constitute an abrogation of that responsibility. If that's not the case, then it's an issue; otherwise, we shouldn't let differences of opinion over a question that may not have only one right answer prevent us from doing anything at all. ...Robert
On Wed, May 16, 2007 at 08:58:44AM +0100, Dave Page wrote: > Jim C. Nasby wrote: > > On Tue, May 15, 2007 at 10:32:14PM +0200, Magnus Hagander wrote: > >> Stefan Kaltenbrunner wrote: > >>>> They are not stable. The items should point to the archives, which are > >>>> supposedly more stable. (I had already fixed one item in PatchStatus > >>>> this morning). Really it would be much nicer to have links using the > >>>> Message-Id but I doubt that's at all doable. > >>> hrm - I see so is there a particular reason for that behaviour ? > >> They're stable until Bruce removes something from the queue. When > >> something is removed, it's renumbered. > >> > >> It's how mhonarc works. It's the same with the archives - if we delete a > >> mail, they get renumbered. So we never should delete, we should just > >> blank out, but it has happened a couple of times. > > > > Isn't there any other archiver we could use? The lack of URL stability > > in mhonarc is bad enough, but the cross-month issue is just horrible. > > Nothing useful last time I looked (a year or two back admittedley). I > have a design for one in mind that I was looking to prototype - there > are some php classes that would make it quite simple to get messages > into a database either via procmail, or from an mbox. > > the stumbling block I was running into was rewriting the old archives > URLs to the new ones. How much visibility do we have into the mhonarc database? We should be able to come up with a simple redirector that would point the old mhonarc URLs to URLs for the new system... -- Jim Nasby decibel@decibel.org EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)
Jim C. Nasby wrote: > How much visibility do we have into the mhonarc database? We should be > able to come up with a simple redirector that would point the old > mhonarc URLs to URLs for the new system... <cough>database?</cough> It's a file system. It simply generates HTML (or in our case) PHP files from each message in an mbox. That's one of the reasons for the monthly break - without it, the directories would be unusably full of files. I the current URLs represent the month, and the ID of the message as it comes out of the mbox I believe. We could probably write a script to dump a list of message IDs, directories and mbox positions I imagine, and then import that into a new database. It's been on my list to rewrite the whole archive system for a while for various reasons. There is quite a bit of crossover with the patch tracker I proposed so I was hoping to look at both together. Regards, Dave
Aidan Van Dyk wrote: > * Tatsuo Ishii <ishii@postgresql.org> [070516 07:23]: > > > Maybe. However I think "subject-sequence" has some advantages over > > Message-Id: > > > > - Easy to identify. Message-Id may not appear on some MUA with default > > setting > > > > - More handy than lengthy message Id > > > > - Easy to detect messages not delivered, by knowing that the sequence > > number was skipped > > IMNSHO, we should be encouraging lists to move *away* from subject > munging. Adding more characters into a subject line which can already > be quite long is just making the situation worse. +1 on that. It gets worse when messages are crossposted to multiple lists and so multiple [FOO] strings are put into the subject. On the other hand I know there is people who remove the [FOO] strings in their local machines! (I don't do it just because I have been too lazy to write a procmail rule about it). -- Alvaro Herrera http://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support
On Wed, May 16, 2007 at 04:34:56PM +0100, Dave Page wrote: > >How much visibility do we have into the mhonarc database? We should be > >able to come up with a simple redirector that would point the old > >mhonarc URLs to URLs for the new system... > > <cough>database?</cough> And here I thought the reason we used tho POS was because it was the only archiver that used PostgreSQL as it's backend... > It's a file system. It simply generates HTML (or in our case) PHP files > from each message in an mbox. That's one of the reasons for the monthly > break - without it, the directories would be unusably full of files. > > I the current URLs represent the month, and the ID of the message as it > comes out of the mbox I believe. We could probably write a script to > dump a list of message IDs, directories and mbox positions I imagine, > and then import that into a new database. Yeah, if the files still resemble real emails then we can probably come up with a way to pull the data in. > It's been on my list to rewrite the whole archive system for a while for > various reasons. There is quite a bit of crossover with the patch > tracker I proposed so I was hoping to look at both together. Let me know when you start on that... -- Jim Nasby decibel@decibel.org EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)
aidan@highrise.ca (Aidan Van Dyk) writes:
> * Tatsuo Ishii <ishii@postgresql.org> [070516 07:23]:
>
>> Maybe. However I think "subject-sequence" has some advantages over
>> Message-Id:
>>
>> - Easy to identify. Message-Id may not appear on some MUA with default
>> setting
>>
>> - More handy than lengthy message Id
>>
>> - Easy to detect messages not delivered, by knowing that the sequence
>> number was skipped
>
> IMNSHO, we should be encouraging lists to move *away* from subject
> munging. Adding more characters into a subject line which can already
> be quite long is just making the situation worse.
>
> And of course, once you realize that subject munging is wrong, putting
> the "list-sequence" into a header of no real value, unless you believe
> your MTA/MUA filtering to be somehow dropping list messages, and your
> sequence numbers will prove to you if that's true or not. Oh - but
> wait, don't we all, as PostgreSQL users realize that sequences aren't
> generally guaranteed to be gapless anyways (sure, there are work
> "solutions", but I'm not about to audit any MTA/list software for
> that...)
>
> ;-)
The message ID *ought* to be usable for at least some of the desired
purposes; having a web client that uses message IDs as a query
mechanism, and where switching months doesn't ludicrously break
things, would seem to be one part of the solution.
Adding some sort of (I don't know...) database that associates bug
tracking system items with message IDs would seem like an "along-side"
way of linking in relevant information.
Adding x-???: tags might be a way of adding bug tracking information
into messages; it wouldn't help with original copies, but could be a
reasonable change to make in a given message repository.
--
output = ("cbbrowne" "@" "acm.org")
http://cbbrowne.com/info/rdbms.html
If nothing ever sticks to Teflon, how do they make Teflon stick to the
pan?
Robert Haas wrote: > >> hate the fact that FTS is a separate module. > > Here here. > > And with respect to the debate about syntax, who cares? I think I > prefer introducing real SQL-ish syntax over a bunch of pg_* functions, > but doing it either way is IMHO better than doing nothing. > I care. I want a professional easy to understand and easy to maintain that doesn't cause potential conflict with future and past development syntax. Or should we be writing SQL in assembly? Joshua D. Drake > ...Robert >
> I care. I want a professional easy to understand and easy to maintain > that doesn't cause potential conflict with future and past development > syntax. I agree with this. The point of my comment was that ISTM that an arbitrary amount of time can be consumed determining the optimal syntax, during which Oleg is obligated to continually update his patch without any guarantee that it will be accepted in anything like its current form, or at all. If people have strong opinions about the syntax, why were they not ALL expressed when the proposal was originally laid on the table? Sure, some people offered opinions, but it doesn't appear that there was any real consensus, and there certainly wasn't any clear guidance of the form "this is what you need to do to get your patch accepted", which leaves everything in limbo and Oleg doing a lot of extra work to keep updating his patch. I haven't studied the proposed syntaxes in detail, but ISTM that if there is no consensus then probably all of the alternatives being advocated are reasonable. Again, if that's not the case, then let's eliminate the unreasonable ones and pick one of the remaining choices. But if committer A thinks that X is the only reasonable options and committer B thinks that Y is the only reasonable option, does that mean that the patch just sits there forever, or do we find a way to move forward? ...Robert
Jim C. Nasby wrote: > On Wed, May 16, 2007 at 04:34:56PM +0100, Dave Page wrote: >>> How much visibility do we have into the mhonarc database? We should be >>> able to come up with a simple redirector that would point the old >>> mhonarc URLs to URLs for the new system... >> <cough>database?</cough> > > And here I thought the reason we used tho POS was because it was the > only archiver that used PostgreSQL as it's backend... You're thinking of the old search engine. >> It's a file system. It simply generates HTML (or in our case) PHP files >> from each message in an mbox. That's one of the reasons for the monthly >> break - without it, the directories would be unusably full of files. >> >> I the current URLs represent the month, and the ID of the message as it >> comes out of the mbox I believe. We could probably write a script to >> dump a list of message IDs, directories and mbox positions I imagine, >> and then import that into a new database. > > Yeah, if the files still resemble real emails then we can probably come > up with a way to pull the data in. We have all the mbox files, so we can import them from there as raw messages. >> It's been on my list to rewrite the whole archive system for a while for >> various reasons. There is quite a bit of crossover with the patch >> tracker I proposed so I was hoping to look at both together. > > Let me know when you start on that... Roger. /D
Dave Page wrote: >>> I the current URLs represent the month, and the ID of the message as >>> it comes out of the mbox I believe. We could probably write a script >>> to dump a list of message IDs, directories and mbox positions I >>> imagine, and then import that into a new database. >> >> Yeah, if the files still resemble real emails then we can probably come >> up with a way to pull the data in. > > We have all the mbox files, so we can import them from there as raw > messages. yeah, that's clearly the best source to work from. It's *possible* work from the mhonarc files (I've done it before), but it's more work. >>> It's been on my list to rewrite the whole archive system for a while >>> for various reasons. There is quite a bit of crossover with the patch >>> tracker I proposed so I was hoping to look at both together. >> >> Let me know when you start on that... > > Roger. Same here - I've done something similar (off mhonarc files and in much smaller scale) before, and I'm definitely interested in helping out. //Magnus
Andrew Dunstan wrote: > Josh Berkus wrote: >> I think that may be where we're heading. In that case, we may need to >> talk about branching earlier so that developers can work on the new >> version who are frozen out of the in-process one. > > I've argued this in the past. But be aware that it will make more work > for committers. It is very far from cost-free. I hate to bring up the CMS flamewar again, but IMHO the one advantage the distributed systems (Git, Monotone, Darcs, etc) have over CVS & subversion is that they push this branch management cost down to the developer who wants a branch - rather from the guys maintaining the official CMS.
Ron Mayer wrote: > Andrew Dunstan wrote: >> Josh Berkus wrote: >>> I think that may be where we're heading. In that case, we may need to >>> talk about branching earlier so that developers can work on the new >>> version who are frozen out of the in-process one. >> I've argued this in the past. But be aware that it will make more work >> for committers. It is very far from cost-free. > > I hate to bring up the CMS flamewar again, but IMHO the one advantage > the distributed systems (Git, Monotone, Darcs, etc) have over CVS & > subversion is that they push this branch management cost down to the > developer who wants a branch - rather from the guys maintaining the > official CMS. One step at a time :). We are now married, and have just begun to have have children. We need to buy a house in the next year, and figure out how to pay off some debt without going into hock up to our ears with a long term low interest loan. Joshua D. Drake > > ---------------------------(end of broadcast)--------------------------- > TIP 4: Have you searched our list archives? > > http://archives.postgresql.org >
On 5/16/07, Joshua D. Drake <jd@commandprompt.com> wrote: > I care. I want a professional easy to understand and easy to maintain > that doesn't cause potential conflict with future and past development > syntax. You've just tempted me to create embedded SQL in assembly :) -- Jonah H. Harris, Software Architect | phone: 732.331.1324 EnterpriseDB Corporation | fax: 732.331.1301 33 Wood Ave S, 3rd Floor | jharris@enterprisedb.com Iselin, New Jersey 08830 | http://www.enterprisedb.com/
Magnus Hagander wrote: >>>> It's been on my list to rewrite the whole archive system for a while >>>> for various reasons. There is quite a bit of crossover with the patch >>>> tracker I proposed so I was hoping to look at both together. >>> Let me know when you start on that... >> Roger. > > Same here - I've done something similar (off mhonarc files and in much > smaller scale) before, and I'm definitely interested in helping out. Is everyone aware of this system that runs on a well-known open-source database? http://www.archiveopteryx.org/ I've used it in a small way, and while I don't claim to have looked at it in detail it seems to pretty much do what it claims to. -- Richard Huxton Archonet Ltd
Richard Huxton wrote: > Magnus Hagander wrote: >>>>> It's been on my list to rewrite the whole archive system for a while >>>>> for various reasons. There is quite a bit of crossover with the patch >>>>> tracker I proposed so I was hoping to look at both together. >>>> Let me know when you start on that... >>> Roger. >> >> Same here - I've done something similar (off mhonarc files and in much >> smaller scale) before, and I'm definitely interested in helping out. > > Is everyone aware of this system that runs on a well-known open-source > database? > http://www.archiveopteryx.org/ > I've used it in a small way, and while I don't claim to have looked at > it in detail it seems to pretty much do what it claims to. > Yeah, I looked at it in the past. The database storage part is actually pretty simple - it's the web front end that's going to take more effort, and thats what that product doesn't do (or if it does, it's a secondary function they don't shout about). Regards, Dave.
Dave Page wrote: > Richard Huxton wrote: >> Magnus Hagander wrote: >>>>>> It's been on my list to rewrite the whole archive system for a while >>>>>> for various reasons. There is quite a bit of crossover with the patch >>>>>> tracker I proposed so I was hoping to look at both together. >>>>> Let me know when you start on that... >>>> Roger. >>> Same here - I've done something similar (off mhonarc files and in much >>> smaller scale) before, and I'm definitely interested in helping out. >> Is everyone aware of this system that runs on a well-known open-source >> database? >> http://www.archiveopteryx.org/ >> I've used it in a small way, and while I don't claim to have looked at >> it in detail it seems to pretty much do what it claims to. >> > > Yeah, I looked at it in the past. The database storage part is actually > pretty simple - it's the web front end that's going to take more effort, > and thats what that product doesn't do (or if it does, it's a secondary > function they don't shout about). It's supposed to have something in the latest version, I think. I used it as backing store for a small workflow app, so I've got some simple views/functions I added and PHP code (cake-php framework) for displaying messages if it'll be of any use. My one concern with the schema was that there didn't seem to be a way to partition archives (e.g. by year) to make maintenance a little simpler for large databases. -- Richard Huxton Archonet Ltd
On Wed, May 16, 2007 at 07:48:10PM +0200, Magnus Hagander wrote: > Dave Page wrote: > >>> I the current URLs represent the month, and the ID of the message as > >>> it comes out of the mbox I believe. We could probably write a script > >>> to dump a list of message IDs, directories and mbox positions I > >>> imagine, and then import that into a new database. > >> > >> Yeah, if the files still resemble real emails then we can probably come > >> up with a way to pull the data in. > > > > We have all the mbox files, so we can import them from there as raw > > messages. > > yeah, that's clearly the best source to work from. It's *possible* work > from the mhonarc files (I've done it before), but it's more work. We'd want the old URLs to be redirected too, so at some point we'll have to deal with mhonarc. -- Jim Nasby decibel@decibel.org EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)
On Wed, May 16, 2007 at 09:32:44PM +0100, Richard Huxton wrote: > Dave Page wrote: > >Richard Huxton wrote: > >>Magnus Hagander wrote: > >>>>>>It's been on my list to rewrite the whole archive system for a while > >>>>>>for various reasons. There is quite a bit of crossover with the patch > >>>>>>tracker I proposed so I was hoping to look at both together. > >>>>>Let me know when you start on that... > >>>>Roger. > >>>Same here - I've done something similar (off mhonarc files and in much > >>>smaller scale) before, and I'm definitely interested in helping out. > >>Is everyone aware of this system that runs on a well-known open-source > >>database? > >> http://www.archiveopteryx.org/ > >>I've used it in a small way, and while I don't claim to have looked at > >>it in detail it seems to pretty much do what it claims to. > >> > > > >Yeah, I looked at it in the past. The database storage part is actually > >pretty simple - it's the web front end that's going to take more effort, > >and thats what that product doesn't do (or if it does, it's a secondary > >function they don't shout about). > > It's supposed to have something in the latest version, I think. I used > it as backing store for a small workflow app, so I've got some simple > views/functions I added and PHP code (cake-php framework) for displaying > messages if it'll be of any use. > > My one concern with the schema was that there didn't seem to be a way to > partition archives (e.g. by year) to make maintenance a little simpler > for large databases. Luckily I happen to know of a database that would make that transparent to the app... But I tend to agree with Dave; the storage part is pretty easy. If we've still got to write our own front-end ISTM it'd be better to just make it exactly what we want... -- Jim Nasby decibel@decibel.org EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)
On Tue, 15 May 2007, Jim C. Nasby wrote: > Speaking of reviewers... should we put some thought into how we can > increase the number of people who can review code? It seems that's one > of our biggest bottlenecks... Having recently dragged myself from never seeing the code before to being able to provide an early review to help the committers out, I can tell you a few things that would have sped up that process and therefore improve the odds more people will navigate the trail. My main difficulty was figuring out a workable way to build a local mirror of the code (so I could get off-line diffs), keep it in sync with the development tree, while branching out working areas to evaluate patches or do new development in. A good example walkthrough of how to do that using standard tools would be a big help. Hint: if you suggest CVsup I'll smack you. Overall, though, I don't think there would have been any substitute for what I learned by putting together my own small patches, so in some respects thinking about how to lower the bar for doing that would also ultimately expand the review pool. The main issues I had as a newcomer to the codebase was getting data in/out of the new code I added that was buried inside the system. Here are some examples of what I kept hoping to find documentation on: -Examples and usage guidelines for eLog and eReport (the stuff in the FAQ needs some more meat) -How to add a GUC variable. Once I've got that, now I can adjust the code in real-time just by updating it. -How to add to the statistics for some part of the system that track a new variable and follow how that moves through the collector process. If you put people into a position where they easily can poke data in at one end, see how it rattles around inside the engine by logging, and get some statistics on the result, now you've got someone who can build their own simple patches without being so frustrated. -- * Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD
Jim C. Nasby wrote: > On Wed, May 16, 2007 at 07:48:10PM +0200, Magnus Hagander wrote: >> Dave Page wrote: >>>>> I the current URLs represent the month, and the ID of the message as >>>>> it comes out of the mbox I believe. We could probably write a script >>>>> to dump a list of message IDs, directories and mbox positions I >>>>> imagine, and then import that into a new database. >>>> >>>> Yeah, if the files still resemble real emails then we can probably come >>>> up with a way to pull the data in. >>> We have all the mbox files, so we can import them from there as raw >>> messages. >> yeah, that's clearly the best source to work from. It's *possible* work >> from the mhonarc files (I've done it before), but it's more work. > > We'd want the old URLs to be redirected too, so at some point we'll have > to deal with mhonarc. Right. Grabbing the msgid alone from them shouldn't be too hard though. It's included in the "meta-headers" mhonarc sticks in each file, so it should be a simple regex to find it. //Magnus
On Wed, May 16, 2007 at 11:50:26PM -0400, Greg Smith wrote: > On Tue, 15 May 2007, Jim C. Nasby wrote: > > >Speaking of reviewers... should we put some thought into how we can > >increase the number of people who can review code? It seems that's one > >of our biggest bottlenecks... > > Having recently dragged myself from never seeing the code before to being > able to provide an early review to help the committers out, I can tell you > a few things that would have sped up that process and therefore improve > the odds more people will navigate the trail. > > My main difficulty was figuring out a workable way to build a local mirror > of the code (so I could get off-line diffs), keep it in sync with the > development tree, while branching out working areas to evaluate patches or > do new development in. A good example walkthrough of how to do that using > standard tools would be a big help. Hint: if you suggest CVsup I'll > smack you. > > Overall, though, I don't think there would have been any substitute for > what I learned by putting together my own small patches, so in some > respects thinking about how to lower the bar for doing that would also > ultimately expand the review pool. The main issues I had as a newcomer to > the codebase was getting data in/out of the new code I added that was > buried inside the system. Here are some examples of what I kept > hoping to find documentation on: > > -Examples and usage guidelines for eLog and eReport (the stuff in the FAQ > needs some more meat) > > -How to add a GUC variable. Once I've got that, now I can adjust the code > in real-time just by updating it. > > -How to add to the statistics for some part of the system that track a new > variable and follow how that moves through the collector process. > > If you put people into a position where they easily can poke data in at > one end, see how it rattles around inside the engine by logging, and get > some statistics on the result, now you've got someone who can build their > own simple patches without being so frustrated. Would you be interested in providing this meat? You're uniquely qualified because your shins still smart from all the things you barked them on :) Cheers, D -- David Fetter <david@fetter.org> http://fetter.org/ phone: +1 415 235 3778 AIM: dfetter666 Skype: davidfetter Remember to vote! Consider donating to PostgreSQL: http://www.postgresql.org/about/donate
Magnus Hagander wrote: > Right. Grabbing the msgid alone from them shouldn't be too hard though. > It's included in the "meta-headers" mhonarc sticks in each file, so it > should be a simple regex to find it. Should be easier than that - when we import the existing messages from the mbox files we should be able to figure out the current URL for each message based on the year/month/list of the mbox file, and the message number within the mbox. /D
David Fetter wrote: >> >> My main difficulty was figuring out a workable way to build a local mirror >> of the code (so I could get off-line diffs), keep it in sync with the >> development tree, while branching out working areas to evaluate patches or >> do new development in. A good example walkthrough of how to do that using >> standard tools would be a big help. Hint: if you suggest CVsup I'll >> smack you. >> >> > Would you be interested in providing this meat? You're uniquely > qualified because your shins still smart from all the things you > barked them on :) > > > For this item at least the work has been done in showing how to set up a local mirror using rsync instead of CVSup. It's mentioned in the dev version of the docs at http://developer.postgresql.org/pgdocs/postgres/rsync.html - although frankly it would be better to import the information rather than just refer to the buildfarm HOWTO. cheers andredw
On Wednesday 16 May 2007 13:02, Robert Haas wrote:
> > I care. I want a professional easy to understand and easy to maintain
> > that doesn't cause potential conflict with future and past development
> >
> > syntax.
>
> <snip>
> If people have strong opinions about the syntax, why
> were they not ALL expressed when the proposal was originally laid on the
> table?
<snip>
> I haven't studied the proposed syntaxes in detail,
<snip>
>
lol... just thought this was ironic. :-)
--
Robert Treat
Build A Brighter LAMP :: Linux Apache {middleware} PostgreSQL
On Thursday 17 May 2007 08:16, Andrew Dunstan wrote: > David Fetter wrote: > >> My main difficulty was figuring out a workable way to build a local > >> mirror of the code (so I could get off-line diffs), keep it in sync with > >> the development tree, while branching out working areas to evaluate > >> patches or do new development in. A good example walkthrough of how to > >> do that using standard tools would be a big help. Hint: if you suggest > >> CVsup I'll smack you. > > > > Would you be interested in providing this meat? You're uniquely > > qualified because your shins still smart from all the things you > > barked them on :) > > For this item at least the work has been done in showing how to set up a > local mirror using rsync instead of CVSup. It's mentioned in the dev > version of the docs at > http://developer.postgresql.org/pgdocs/postgres/rsync.html - although > frankly it would be better to import the information rather than just > refer to the buildfarm HOWTO. > And really should probably be written up into FAQ section or a full on guide for "how do i get started hacking on postgresql?" -- Robert Treat Build A Brighter LAMP :: Linux Apache {middleware} PostgreSQL
On Thu, 17 May 2007, Andrew Dunstan wrote: > For this item at least the work has been done in showing how to set up a > local mirror using rsync instead of CVSup...although frankly it would be > better to import the information rather than just refer to the buildfarm > HOWTO. The information in the buildfarm HOWTO has maybe 1/2 of what you need to know--you get a local mirror out of it but no idea how to then use that on the client side to branch and generate patches. And what it does provide is less helpful than it might be because it includes diversions specific to the buildfarm application I found confusing. -- * Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD
On Thu, 17 May 2007, Robert Treat wrote: > And really should probably be written up into FAQ section or a full on guide > for "how do i get started hacking on postgresql?" To be clear here: while there are guides for new hackers out there, they focus on the big picture. I walked from those with a good idea of how the major pieces fit together and was able to navigate the source code. It was the little details of how to actually work with the code and the repository that were the biggest drag on my progress. I used to wonder if it was just me, but watching the discussion that went along with the "OS/X startup scripts" thread recently made me realize how many other people struggle(d) with this as well. -- * Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD
Greg Smith wrote: > On Thu, 17 May 2007, Andrew Dunstan wrote: > >> For this item at least the work has been done in showing how to set >> up a local mirror using rsync instead of CVSup...although frankly it >> would be better to import the information rather than just refer to >> the buildfarm HOWTO. > > The information in the buildfarm HOWTO has maybe 1/2 of what you need > to know--you get a local mirror out of it but no idea how to then use > that on the client side to branch and generate patches. And what it > does provide is less helpful than it might be because it includes > diversions specific to the buildfarm application I found confusing. > > Sure. And of course the creation of branches and generation of patches is irrelevant to the buildfarm. All this just reinforces my point that we should remove that reference and fill in the blanks on the docs, FAQs etc. Incidentally, I don't use a repo mirror for online or offline work, although I do for buildfarm work. What I do is to have a pristine checkout of each live stable branch plus HEAD. I have a cron job that keeps these moderately up to date. When I'm working on a feature or patch, I make a copy (using cp -a) of the relevant branch and then go to work on that. If the work lasts a while, every so often I run 'cvs update' in that copy. And of course I run 'cvs update' before cutting a patch or doing a commit. When the work is done I blow away the work directory. If the work is large and I need to checkpoint certain files locally so I can roll back, I occasionally use RCS. There's no right answer on how to work - everyone uses tools they feel comfortable with. cheers andrew
On Tue, May 15, 2007 at 04:52:16PM -0400, Alvaro Herrera wrote: > This is what happens with the Linux kernel. They have hundreds of > developers getting their hands dirty during a previous period. Then > 2.6.20 is released; the 2.6.21 "merge window" opens, and all sort of > patches are flooded in. I hasten to point out that the Linux kernel has also had several "stable" releases with huge bugs -- things like massive filesystem corruption, bizarre failure cases, and nasty compatibility problems with modules across versions. I am not entirely sure that the Linux model is the one to ape. PostgreSQL has a history with remarkably few of those blunders, and I'd hate to give that up. A -- Andrew Sullivan | ajs@crankycanuck.ca If they don't do anything, we don't need their acronym. --Josh Hamilton, on the US FEMA
Andrew Sullivan wrote:
> On Tue, May 15, 2007 at 04:52:16PM -0400, Alvaro Herrera wrote:
>
>> This is what happens with the Linux kernel. They have hundreds of
>> developers getting their hands dirty during a previous period. Then
>> 2.6.20 is released; the 2.6.21 "merge window" opens, and all sort of
>> patches are flooded in.
>
> I hasten to point out that the Linux kernel has also had several
> "stable" releases with huge bugs --
/me fondly remembers kernel 2.4.
things like massive filesystem
> corruption, bizarre failure cases, and nasty compatibility problems
> with modules across versions. I am not entirely sure that the Linux
> model is the one to ape. PostgreSQL has a history with remarkably
> few of those blunders, and I'd hate to give that up.
>
> A
>
--
=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/
Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL Replication: http://www.commandprompt.com/products/
Joshua D. Drake wrote: > Andrew Sullivan wrote: >> On Tue, May 15, 2007 at 04:52:16PM -0400, Alvaro Herrera wrote: >> >>> This is what happens with the Linux kernel. They have hundreds of >>> developers getting their hands dirty during a previous period. Then >>> 2.6.20 is released; the 2.6.21 "merge window" opens, and all sort of >>> patches are flooded in. >> >> I hasten to point out that the Linux kernel has also had several >> "stable" releases with huge bugs -- > > /me fondly remembers kernel 2.4. > > We keep focusing on process. I am on record as saying we can improve our processes, but the fact is our major immediate problem is person-power, not process. We need more qualified reviewers. Qualified means (to me, at least) you have to have done enough visible PostgreSQL hacking that a committer can reasonably place some level of trust in your review, thereby saving some time. That's not to say that others can't or shouldn't do reviews - every little bit helps, but if Freda Bloggs comes along with a review of some new, large, feature, she isn't helping to make the process shorter, although she might be helping to make it more robust. cheers andrew
Andrew Sullivan wrote: > On Tue, May 15, 2007 at 04:52:16PM -0400, Alvaro Herrera wrote: > > > This is what happens with the Linux kernel. They have hundreds of > > developers getting their hands dirty during a previous period. Then > > 2.6.20 is released; the 2.6.21 "merge window" opens, and all sort of > > patches are flooded in. > > I hasten to point out that the Linux kernel has also had several > "stable" releases with huge bugs -- things like massive filesystem > corruption, bizarre failure cases, and nasty compatibility problems > with modules across versions. I am not entirely sure that the Linux > model is the one to ape. PostgreSQL has a history with remarkably > few of those blunders, and I'd hate to give that up. Sorry, I wasn't trying to say that we should follow the Linux model all that closely. I know there are regressions in the "point zero" releases, and that there are bugs. But then, we're nowhere near the scale of Linux. In the press release for 8.0 we mentioned something about "hundreds of developers". This was true -- but it was not in the same league as Linux's "hundreds of developers". We're nowhere near the manpower that Linux has, nowhere near the amount of code these guys change. I think the figure is around 9000 lines of code changed per day, _every day_[1]. They have new drivers all the time, internal interfaces are cleaned up, new facilities are invented to support new kinds of hardware, performance improvements are made all over the place. So there are a hundred of strange machines where the thing does not boot; yes, but those bugs are fixed in 2.6.2x.1 or subsequent "stable branch" releases. I dare not think of the 2.2 or 2.4 disasters, where distributions were fond of backporting huge patches from the 2.3 or 2.5 development branches. I compiled my own kernel from Linus' tree back then, which worked without the strange behavior the other kernels had (maybe the distro kernels stabilized at some point, but I didn't try later in the game -- I was too used to compiling my own). Now with 2.6 I don't do that anymore (of course I don't run 2.6.22 as soon as it is out either). In my opinion Linux 2.6 is much better than Linux 2.4. I don't think we should follow either model though. We have different problems and different people. [1] http://lwn.net/Articles/232379/ -- Alvaro Herrera http://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support
On Thu, 17 May 2007, David Fetter wrote: > Would you be interested in providing this meat? You're uniquely > qualified because your shins still smart from all the things you > barked them on :) Unfortunately I'm temporarily on the other side of this problem; all the time I have to spare right now is going to into reviewing other people's patches. Even more unfortunately, I'm not 100% happy with what I'm doing, and certainly wouldn't want to present the shoddy techniques I'm using as recommended. If anyone else has useful info on this subject (keeping a local repository in sync with HEAD via rsync while working on branches) they'd like to pass along, please drop me a message with whatever level of documentation you might have--even rough notes would be a help. Once I'm finished with the reviews I'm trying to contribute, I will sort through any suggestions I get and turn that into formal documentation. This has left enough shin damage that I'd enjoy completely slaying the cause. -- * Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD
Greg Smith wrote: > On Thu, 17 May 2007, David Fetter wrote: > >> Would you be interested in providing this meat? You're uniquely >> qualified because your shins still smart from all the things you >> barked them on :) > > Unfortunately I'm temporarily on the other side of this problem; all the > time I have to spare right now is going to into reviewing other people's > patches. > > Even more unfortunately, I'm not 100% happy with what I'm doing, and > certainly wouldn't want to present the shoddy techniques I'm using as > recommended. If anyone else has useful info on this subject (keeping a > local repository in sync with HEAD via rsync while working on branches) > they'd like to pass along, please drop me a message with whatever level > of documentation you might have--even rough notes would be a help. Once > I'm finished with the reviews I'm trying to contribute, I will sort > through any suggestions I get and turn that into formal documentation. > This has left enough shin damage that I'd enjoy completely slaying the > cause. Let me describe my method. It's not ideal by any means, but I've been satisfied: First of all, having a local CVS repository is a must. I used to use cvsup until a couple of months ago, but it suddenly stopped working, so I switched to rsync and I haven't looked back. I have a one line shell script to update it: rsync --progress -avzCH --delete anoncvs.postgresql.org::pgsql-cvs /home/hlinnaka/pgcvsrepository When I start working on a subject, I make a new checkout from the local repository to a designated directory under "pg_sandbox"-directory. For example, when I started working on the mvcc-safe cluster patch: ~/pg_sandbox$ cvs -d /home/hlinnaka/pgcvsrepository cvs co pgsql ~/pg_sandbox$ mv pgsql pgsql.cluster ~/pg_sandbox$ cd pgsql.cluster ~/pg_sandbox/pgsql.cluster$ cvs update -dP # Remove empty dirs. There's a checkout-flag for that as well, but I never remember it... ~/pg_sandbox/pgsql.cluster$ ./configure --enable-depend --enable-cassert --enable-debug --prefix=/home/hlinnaka/pgsql.cluster For primitive version control, I make a diff after any significant changes: ~/pg_sandbox/pgsql.cluster$ cvs diff -cN > cluster-mvcc-1.patch The last number is a revision number, increment by one after each diff. That's it! -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com
On 5/18/07, Heikki Linnakangas <heikki@enterprisedb.com> wrote:
You can use cvs co -d pgsql.cluster to save the above step.
I find configuring with CFLAGS="-g -O0" more useful.
I usually commit each version and tag the tree. That helps me to get
diff between two versions as well.
Thanks,
When I start working on a subject, I make a new checkout from the local
repository to a designated directory under "pg_sandbox"-directory. For
example, when I started working on the mvcc-safe cluster patch:
~/pg_sandbox$ cvs -d /home/hlinnaka/pgcvsrepository cvs co pgsql
~/pg_sandbox$ mv pgsql pgsql.cluster
You can use cvs co -d pgsql.cluster to save the above step.
~/pg_sandbox$ cd pgsql.cluster
~/pg_sandbox/pgsql.cluster$ cvs update -dP # Remove empty dirs.
There's a checkout-flag for that as well, but I never remember it...
~/pg_sandbox/pgsql.cluster$ ./configure --enable-depend --enable-cassert
--enable-debug --prefix=/home/hlinnaka/pgsql.cluster
I find configuring with CFLAGS="-g -O0" more useful.
For primitive version control, I make a diff after any significant changes:
~/pg_sandbox/pgsql.cluster$ cvs diff -cN > cluster-mvcc-1.patch
I usually commit each version and tag the tree. That helps me to get
diff between two versions as well.
Pavan
--
Pavan Deolasee
EnterpriseDB http://www.enterprisedb.com
Pavan Deolasee wrote: > For primitive version control, I make a diff after any significant changes: >> >> ~/pg_sandbox/pgsql.cluster$ cvs diff -cN > cluster-mvcc-1.patch > > I usually commit each version and tag the tree. That helps me to get > diff between two versions as well. Doesn't that confuse rsync? I use "diff patch-1 patch-2" for diffs between versions, but it's really hard to read. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com
On 5/18/07, Heikki Linnakangas <heikki@enterprisedb.com> wrote:
I use cvsup. But thats not relevant here. Yes, cvsup would delete any
local tags/branches and sync the tree. But then I don't sync the tree very
often. And when I need to sync my work with CVS HEAD, I create
all the required diffs, sync the tree and reapply the patches (and commit
and tag) the tree again. Thats some work, but is I am OK with that
because as I said we don't need to sync with head that often.
ThanksPavan Deolasee wrote:
> For primitive version control, I make a diff after any significant changes:
>>
>> ~/pg_sandbox/pgsql.cluster$ cvs diff -cN > cluster-mvcc-1.patch
>
> I usually commit each version and tag the tree. That helps me to get
> diff between two versions as well.
Doesn't that confuse rsync?
I use cvsup. But thats not relevant here. Yes, cvsup would delete any
local tags/branches and sync the tree. But then I don't sync the tree very
often. And when I need to sync my work with CVS HEAD, I create
all the required diffs, sync the tree and reapply the patches (and commit
and tag) the tree again. Thats some work, but is I am OK with that
because as I said we don't need to sync with head that often.
--
Pavan Deolasee
EnterpriseDB http://www.enterprisedb.com
On Thu, May 17, 2007 at 11:02:31PM -0400, Greg Smith wrote: > On Thu, 17 May 2007, David Fetter wrote: > > >Would you be interested in providing this meat? You're uniquely > >qualified because your shins still smart from all the things you > >barked them on :) > > Unfortunately I'm temporarily on the other side of this problem; all the > time I have to spare right now is going to into reviewing other people's > patches. > > Even more unfortunately, I'm not 100% happy with what I'm doing, and > certainly wouldn't want to present the shoddy techniques I'm using as > recommended. If anyone else has useful info on this subject (keeping a > local repository in sync with HEAD via rsync while working on branches) > they'd like to pass along, please drop me a message with whatever level of > documentation you might have--even rough notes would be a help. Once I'm > finished with the reviews I'm trying to contribute, I will sort through > any suggestions I get and turn that into formal documentation. This has > left enough shin damage that I'd enjoy completely slaying the cause. Rather than trying to get something "FAQ-perfect" right now, can you at least brain-dump what your current process is so that others can learn? I've created a wiki page for this and added Heikki's tips: http://developer.postgresql.org/index.php/Working_with_CVS -- Jim Nasby decibel@decibel.org EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)
On Fri, 18 May 2007, Jim C. Nasby wrote: > Rather than trying to get something "FAQ-perfect" right now, can you at > least brain-dump what your current process is so that others can learn? > I've created a wiki page for this and added Heikki's tips: > http://developer.postgresql.org/index.php/Working_with_CVS Heikki's great summary helps (I think the one piece I was screwing up is covered there), and Pavan's comments adds some useful bits. The still missing part is how to make a real branch to work in, which is much easier to work with once you figure out how to do it than either using just diffs or tagging. I agree the wiki is the right place to hash this out; when that's converged to something useful, then take a snapshot and convert for the official manual. I'll work on it. -- * Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD
Greg Smith wrote: > > Heikki's great summary helps (I think the one piece I was screwing up > is covered there), and Pavan's comments adds some useful bits. The > still missing part is how to make a real branch to work in, which is > much easier to work with once you figure out how to do it than either > using just diffs or tagging. > > What would making a branch actually do for you? The only advantage I can see is that it will give you a way of checkpointing your files. As I remarked upthread, I occasionally use RCS for that. But mostly I don't actually bother. I don't see how you can do it reasonably off a local cvs mirror - rsync will just blow away any changes you have checked in next time you sync with the master. I don't think we can make CVS behave like a distributed SCM system, and ability to create local branches seems to me one of the fundamental points of such systems. If that's what the demand is for, then we should look again at moving to something like Mercurial. cheers andrew
Andrew Dunstan wrote: > What would making a branch actually do for you? The only advantage I can > see is that it will give you a way of checkpointing your files. As I > remarked upthread, I occasionally use RCS for that. But mostly I don't > actually bother. I don't see how you can do it reasonably off a local > cvs mirror - rsync will just blow away any changes you have checked in > next time you sync with the master. > > I don't think we can make CVS behave like a distributed SCM system, and > ability to create local branches seems to me one of the fundamental > points of such systems. If that's what the demand is for, then we should > look again at moving to something like Mercurial. I think the great thing about DCVS systems is that not everybody necessarily needs to use the *same* system. And it doesn't really matter what the central repository runs on - I think they are gateway from/to nearly everything available... I currently use GIT for my SoC project, and it works quite well - I can create an abitrary number of local branches, and syncing the currently active branch with CVS is archived by just doing "cg-update pgsql-head". greetings, Florian Pflug
On Sat, 19 May 2007, Andrew Dunstan wrote: > What would making a branch actually do for you? The only advantage I can see > is that it will give you a way of checkpointing your files. Exactly. It's not as bad now, but when I was getting started there were lots of times I got something working and I wanted a clean way to save it in that state before I started messing with anything else--such that I could backtrack what I did if I later broke it. This style of work has some advantages for working on other people's patches as well, especially if you're trying to review two at once and it takes you a while to finish--the situation I always find myself in. > I don't see how you can do it reasonably off a local cvs mirror - rsync > will just blow away any changes you have checked in next time you sync > with the master. It's certainly not easy. I try not to let the fact that what I'd like to do may actually be impossible ever discourage me. There are at least three ways to approach this general problem just counting the rsync/CVS variations, each with some obvious and some subtle advantages/disadvantages. I'm still in the stage where I'm mapping out the options. -- * Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD